February 12, 2009
Communications of the ACM
Women in computing --- take 2
By M. Klawe,T. Whitney,C. Simard
This article appeared in the February 2009 issue of Communications of the ACM. Read the full story below or view it at the the ACM web site.
"Women in computing: Where Are We Now?"—an article by Maria Klawe and Nancy Leveson in the January 1995 issue of Communications—addressed women's representation at the time, as undergraduate and graduate students and in the work force, in computing fields. That article, part of the issue's special section on Women and Computing, described successful activities and offered recommendations for future programs.
In this article, 14 years later, we assess the changes that have since occurred, including both positive and negative trends; we present strategies shown to be successful for the recruitment, retention, and advancement of women in computing; and we explore promising new initiatives for further increasing women's participation. While the 1995 article focused on the U.S. and Canada, as does the present one, we now also include data from other parts of the world.
Why should computing professionals be concerned about women and other groups underrepresented in our field? In large part, out of self-interest. Diversity often leads to enhanced abilities to perform tasks, greater creativity, and better decisions and outcomes.17 Sadly, bias and stereotyping—often unconscious, but nevertheless pervasive—continue to affect the gender and ethnic composition of our talent pool and thus limit the possibilities of technological innovation around the world. Meanwhile, demand for computer scientists and computer engineers in the U.S. is expected to grow 37% between 2006 and 2016,4 despite the overall economy's present travails. Clearly, society requires the contributions of women as well as men to computing.
On the Plus Side
Around the world, women have made some progress in the field of computing over the past decade. Women now play a heightened role in technology leadership, and they have gained representation at many important points in organizational hierarchies.
- The number of women earning U.S. undergraduate computer science (CS) degrees increased from 7,063 in 1995 to 11,235 in 2005.25
- Some countries are making gains in the numbers of women majoring in math or CS, but because data is often unavailable for computer science alone, related percentages are not exactly comparable to U.S. figures. Indeed, the percentage of U.S. female bachelor's degree recipients in math is much higher than that of CS—44.6% versus 22.2%.25 Thus grouping math with CS may be masking lower participation in CS.
- In Asia (including only those countries for which data is available), women earned 43% of first university degrees in math and CS in 2004.23 Women's representation in technical fields is growing in India—the percentage of female engineers graduating from ITT Bombay has grown from 1.8% in 1972 to 8% in 2005. In the Middle East, women earned 43% of first-time math and CS degrees.23 In Western Europe, while the overall percentage of math and CS undergraduate degrees going to women is just 30%, some countries have been doing significantly better—Portugal was at 41% in 2004, Finland 42%, Greece 40%, and Italy 43%. In North America, Mexico also fares reasonably well, with 38% of math and CS undergraduate degrees awarded to women.23
- The total number of female CS graduate students in the U.S. grew from 9,881 in 1997 to 12,061 in 2005. The proportion of women awarded CS master's degrees rose from 26.4% in 1995 to 28.5% in 2005, and the proportion of women awarded CS doctoral degrees rose from 16.5% in 1997 to 19.8% in 2005,25 pointing to some graduate-level progress.
- The proportion of newly hired women in U.S. and Canadian CS faculty increased from 18% in 1995 to 24% in 2006–2007.6
- The proportion of women in full CS professorships more than doubled between 1995 and 2007, from 5% to 10.9%.6
- The number of women in significant academic leadership positions has increased. For example, the proportion of female university presidents in the U.S. rose from 18% in 199531 to 23% in 2007.1 In recent years, some high-profile research institutions—including Brown, Harvard, Michigan, MIT, Princeton, Penn, RPI, and several University of California campuses—named their first woman presidents.
- The percentage of U.S. information-technology patents obtained by female inventors rose from 4.4% in 1995 to 6.1% in 2005.22
The Bad News
The gains listed here, while encouraging, stop short of achieving equal representation and point to the fact that much work has yet to be done.
"My slogan is: Computing is too important to be left to men." Karen Sparck-Jones: Pioneer in Information Retrieval and Natural Language Processing. 1935–2007
The proportion of undergraduate CS degrees received by women has declined sharply—from 37% in 1985 to 22% in 2005.25 In research-intensive CS departments that participate in the annual Taulbee Survey conducted by the Computing Research Association (CRA), the number dropped from 19% in 2001 to 11.8% in 2006–2007.6
- Interest in CS as a major is at an all-time low both for men and women. In a 2007 teacher survey, a lack of student interest at the high-school level was cited as the number-one challenge.5 Intention of women freshmen to major in computer science dropped from 2.8% in 1985 to 1.3% in 1995 and to 0.4% in 200623,25
- Since 1995, the representation of African-American and Hispanic women among CS degree recipients has remained flat.6 Across genders, the proportion of African-American Ph.D. recipients in the United States and Canada has remained unchanged at 1–2% since 1995, and Hispanic representation has dropped from 3% to 2%.23
- The proportion of female CS graduate students in the U.S. remained flat at 27% from 1997 to 2004 and declined to 25% in 2005.23 Similarly in the European Union, the proportion of women earning math and CS doctorates has stood at 24%.31
- Women in CS faculty positions at U.S. four-year institutions remain underrepresented, at just 15.8% of all faculty and 11% of full professors.6 Ethnic minority women are doubly underrepresented on faculties, with Asian and African-American women holding just 3% of faculty positions. Hispanic and Native American women are virtually nonexistent among CS faculty.25 Disparity in faculty salaries across all disciplines has remained unchanged since the 1970s—women faculty earn 81% of men's salary for equivalent qualifications.34
- The proportion of women employed in math and CS occupations in the work force declined from 33% in 1984 to 27% in 2004.25
- The proportion of technology-industry women in top leadership positions is quite low—for example, only 5% of chief technology officers in Fortune 100 IT companies are women. Their representation on technology-company boards remains low as well, with 13% of board seats of Fortune 100 IT companies going to women, compared to 17% for Fortune 100 organizations across sectors.21
- The wage differential between men and women holding computer science degrees persists. Women with undergraduate CS degrees earn a median of $44K compared to $46K for men and $40K for underrepresented minorities.25
Many initiatives are currently under way to counter such negative trends, and they have shown promise in helping to turn the tide. While it is not possible to review all such efforts in one article, we do highlight some encouraging programs at the K-12 level, in academia (undergraduate, graduate, and faculty levels), and in industry.
K-12: Appealing to Girls and Their Influencers. It is widely recognized that declining interest in technical disciplines among female students starts at a young age. Therefore early-intervention efforts are important to ensure future increases in representation.
Successful approaches at the K-12 level include:
- Expose girls to positive role models in the technology sphere, given that the absence of such models has proven to be a deterrent.
- Dispel computing-career myths and stereotypes; for example, the notion that computing is a "white male profession" discourages girls and minorities from entering the field.2 The Image of Computing Task Force (www.imageofcomputing.com), comprised of global technology companies, professional associations, nonprofit organizations, and others, focuses its efforts on creating and disseminating positive images of computing designed to appeal to girls.
- Provide accurate information to key influencers of girls. Because parents and teachers with unconscious biases will subtly discourage girls from pursuing computer-related activities,16 providing these influencers with information and resources is vital not only to igniting their daughters' and students' interest in technology at a young age but also to retaining it. Resources include the Girls Scouts' Girls Go Tech initiative booklet "It's Her Future." One educational program that touches on multiple audiences (girls, parents, and teachers) is Computer Mania Day—hosted by the Center for Women and Information Technology at the University of Maryland, Baltimore County—during which participants learn about pertinent issues and explore technology career for girls. This effort helps to create or strengthen positive attitudes about women's involvement in technology.20
- Provide girls with age-appropriate, hands-on technology activities; examples can be found at the Girls Go Tech (www.girlsgotech.org) and Sally Ride Science (sallyridescience.com) Web sites.
- Enroll girls in summer computer programs in which they have immersive experiences with technology.
- Motivate girls and women through the potential social impact of technology.29 Several excellent programs are built around socially relevant themes and involve teamwork, collaboration, and hands-on learning—pedagogical approaches shown to be highly effective for girls.18 The Edge Summer Engineering Workshop for High School Girls, offered yearly by Union College, is a two-week summer residential workshop devoted to this purpose (antipasto.union.edu/edge/). Similarly, Purdue University's Engineering Projects In Community Service (EPICS) program, launched in 1995, is a learning approach in which undergraduate teams come together to apply technology solutions to an identified community problem. While EPICS does not specifically target women, it has significantly enhanced women's participation in computer science and engineering.7
Encouraged by this result, EPICS launched a high-school summer program, based on the same model.
- Engage students and faculty of university CS departments to work with local middle schools and high schools while encouraging companies to implement their own outreach programs of this type.
Academia: Attracting and Retaining Students and Faculty. Over more than a decade, a host of initiatives has evolved to increase and sustain the participation of women, at all levels of academia, in computing. Many of these programs are projects of organizations specifically devoted to this purpose—for example, ACM's Committee on Women in Computing (ACM-W),a the Anita Borg Institute for Women and Technology (ABI), the CRA Committee on the Status of Women in Computing Research (CRA-W), MentorNet, and the National Center for Women & Information Technology (NCWIT). Readers are encouraged to visit the organizations' Web sites for more information.
Other such programs have been initiated by funding agencies. These initiatives include the Increasing the Participation and Advancement of Women in Academic Science and Engineering Careers (ADVANCE) grants and the Broadening Participation in Computing (BPC) grants from the National Science Foundation (NSF), as well as Canada's NSERC-Industry Chairs for Women in Science and Engineering. (NSERC is the Natural Sciences and Engineering Research Council of Canada.) Still other programs, such as Google's Anita Borg scholarships and Microsoft's New Faculty Fellowships, are supported by industry.
"Today's computing is not your father's computing. Interaction design, empirical studies of user experience, project management, understanding social impacts of technology, and much more are new faces of academic computing. Check them out." Bonnie A. Nardi: Professor, Donald Bren School of Information and Computer Science, University of California, Irvine.
At the undergraduate level, three approaches have been the most successful:
- Redesign "Introduction to CS" courses to emphasize applications in areas of interest to females. Excellent examples of such redesigned courses include those at Georgia Tech and the University of British Columbia (UBC), which respectively emphasize applications in digital media and in psychology, fine arts, and biology.27 Harvey Mudd College and Princeton each redesigned their introductory CS course to focus on science applications.9
- Require students to do assignments in their introductory CS course using "pair programming," which provides benefits (demonstrated by research conducted at UC, Santa Cruz19) for female and male students alike. A result of this approach is that more women are likely to complete the course, obtain a higher grade, take another CS course, and do better in the second CS course. Thus a number of institutions now impose a pair-programming requirement for their introductory CS classes.
- Make a computing-related course a requirement, or a highly recommended option, for all students in majors that have many females (arts and education, for example). While introducing a new general-curriculum requirement for a wide range of undergraduate majors is often politically difficult, Arizona State University has managed to do it. Otherwise, as demonstrated at UBC, a simple statement in the undergraduate handbook that "the Dean recommends that all [such] majors take at least one computer science course" can dramatically increase the number of female students taking CS courses.
Some of the successful approaches for attracting and retaining more female computer majors include:
- Create or publicize majors that combine computer science with another area. Examples include media computation at Georgia Tech; majors at Cornell cosponsored by the Faculty of Computing and Information Science and either the College of Engineering, the College of Arts and Sciences, or the College of Agriculture and Life Sciences; and informatics and business information systems at UC Irvine. At UBC, over a third of the students in double majors involving CS are female.
- Train instructors of introductory CS courses to encourage high-performing women to take a second course and consider majoring in computing. If the institution offers an introductory course aimed at non-CS majors, it should ensure that students who do well in it are able to become CS majors without losing credit for the introductory course.
- Provide and publicize opportunities for science students to enter CS majors after completing their second year. Many female students start majoring in biology or chemistry with the intention of going to medical school but realize during the second year that their choice is unlikely or undesirable.
- Provide bachelor's or master's programs in CS for people who already have a bachelor's degree in another field.13
- Encourage female students to participate in mentoring programs at their institution in person or by email. See, for example, Mentornet (www.mentornet.org).
- Encourage female students to attend computing conferences, and help to finance the excursion. For example, the Grace Hopper Celebration of Women in Computing has an outstanding track record of improving recruitment, retention, and advancement. Positive outcomes reported by attendees include inspiration, decreased feelings of isolation, renewed commitment to a computer science or technology degree, and the establishment of a professional network that aids in career advancement. ACM-W has spearheaded and supported a number of regional conferences modeled on the Hopper conference. In addition, ACM-W recently launched a program that provides scholarships to female students so that they can attend research conferences.
- Form an ACM-W chapter (see women.acm.org/activities.html).
- Engage female students in computing research during the summer after their first or second year.
"Illegitimi non carborundum, which is mock-Latin for 'don't let the bastards grind you down'. (See Wikipedia.) It's helped me a lot over the years!" Professor Dame Wendy Hall: ACM President; School of Electronics and Computer Science, University of Southampton.
Much has been written about ways to enroll more women in CS programs at the graduate level and retain them. Basic approaches to enhancing enrollment include the funding of visits by accepted students to the department, recruitment visits by female graduate students to their undergraduate institutions, and departmental delegations to conferences, such as Hopper, attended by many women undergraduate and graduate students majoring in computing fields. But in our view, three kinds of experiences make undergraduate females most likely to commence graduate work in CS: encouragement by a faculty member; research experience as an undergraduate; and sustained interaction with graduate students.
In the U.S., many universities and colleges offer Research Experiences for Undergraduate programs (REUs, often funded by NSF) during the summer. For over a decade, CRA-W has run its Distributed Mentor Program (DMP), which matches female undergraduates with female faculty members (usually at a different institution) for the purpose of doing research together, and the program provides funding for the effort. A research study by Harrod12 demonstrated that students participating in DMP were significantly more likely to enter a graduate program later on. Similarly, many programs connect female undergraduates with counterparts in graduate school. Over the past few years, the Women in Computing Society program at Carnegie Mellon has sent groups of female graduate students to several academic institutions in order to talk to female undergraduates about graduate school. MentorNet provides email mentoring for undergraduates by graduate students, faculty, and computing professionals; and many departmental mentoring programs pair undergraduates with graduate students or conduct tri-mentoring programs that group an undergraduate, a graduate student, and a computing professional.
Retention initiatives fall into two groups: those conducted within the institution, usually at the departmental or school (faculty, college) level; and regional, national, or international programs that bring together women graduate students from more than one institution. Most of the within-institution initiatives are designed to build a sense of community among the students and provide mentoring, especially at critical retention points in the graduate programs. An example of the second group of initiatives is CRA-W's long history of offering graduate-student programs—beginning with academic career workshops at computing conferences and more recently at the annual Grad Cohort symposia—that bring together hundreds of women graduate students from across the U.S.
At the faculty level, the primary goals are to recruit more women faculty and ensure that they ultimately achieve tenure and promotion.
Significant efforts has been made over the last decade, supported by ADVANCE and other NSF grants, to establish best practices that achieve more diversity—that is, the recruitment of more women and underrepresented minorities—in science and engineering faculties.26 Here we cite a few of these efforts' key characteristics:
- Provide training for faculty search committees on best practices. Unconscious bias in academic hiring and evaluating remains widespread,33 but raising awareness of such bias helps reduce its influence. Moreover, training can help each committee build a more diverse pool of candidates, design an effective interview schedule, avoid common pitfalls when interviewing women candidates, and ensure that committee members have the answers to often-asked questions.
- Identify potential women candidates and build proactive relationships with them, even during years when the department is not conducting a search. For example, invite promising women graduate students and postdocs to give presentations or conduct seminars. Invite female researchers from industry to visit the department for, say, a week. Invite untenured women faculty from institutions (especially those with records of often denying tenure) to give colloquia.
- Be prepared to help find jobs for women candidates' spouses or partners,28 many of whom are academics themselves, often in science or engineering disciplines. Some academic institutions in fact have programs to assist departments interested in hiring the spouse or partner of an especially talented faculty candidate.
- Establish parent-friendly practices in the department and institution. For example, do not schedule department meetings after 5 P.M. and encourage the institution to provide paid parental leave for faculty members with newborn children.
"Though female leaders have the same technical challenges and are expected to produce the same kind of results as male leaders, there is often a cultural context that influences their approach and a different interpretation of their performance that ups the ante." Francine Berman: Director, San Diego Supercomputer Center; High Performance Computing Endowed Chair, Jacobs School of Engineering, University of California, San Diego.
For the last 15 years, CRA-W, through its Cohort of Associate Professors Project, has maintained programs to help young women CS faculty progress successfully; and it has also conducted workshops for older female faculty further along in their careers. Significant support/programs are also offered through the NSERC-Industry WISE chairs, ADVANCE grants, and more recently the ABI TechLeaders workshops for senior academic women.
"If we want young girls to choose to learn how to program computers, we need to deeply understand the kinds of programs girls will be motivated to create and design programming environments that make those programs readily achievable." Caitlin Kelleher: Assistant Professor Computer Science and Engineering; As A Ph.D. Student Working with Randy Pausch, Created "Storytelling Alice" to Inspire Middle School Girls to Learn Programming.
For beginning untenured faculty, important actions by the department include:
- Provide a lighter teaching load for the first two years and limit the number of different courses she teaches during the first four years. Encourage each new faculty member to participate in professional-development courses offered by the institution and to invite other faculty members to observe her teaching. In addition, perform informal midterm teaching evaluations in all of her courses.
- At research-intensive institutions: Ensure that new faculty members receive enough startup funding to support two or more graduate students for at least two years. Provide significant help in writing grant proposals.
- Make certain that the new faculty member understands what is expected in order to gain tenure. Provide clear and constructive feedback annually on achievements she should focus on in the coming year.
- Match the new faculty member with a senior faculty member who is a compatible and effective mentor.
- Proactively engage the new faculty member in departmental gatherings, both formal and informal.
- Encourage the new faculty member to attend an Academic Career Workshop (CRA-W), Hopper conference, or similar events, and provide some of the associated funding.
- At research-intensive institutions, nominate deserving new faculty members for prestigious fellowships.
Key actions to take before the tenure decision:
- If there is a significant chance that the faculty member will not receive tenure, make sure she is aware of it as soon as possible.
- At research-intensive institutions: In the year before the tenure decision, encourage the faculty member to give seminars at several of the top departments in her research area and send copies of her key publications to those departments' leading researchers.
After a faculty member receives tenure, continue mentoring and provide annual feedback to ensure that she stays on track for promotion to full professor. Encourage her to assume leadership roles in the department, institution, or professional community.
"The best advice I've ever heard about how women should compete in the workplace was spoken by Betty Snyder Holberton, the first of my three favorite work partners: 'Look like a girl. Act like a lady. Think like a man. Work like a dog.'" Jean Bartik: Programmer for The Groundbreaking Eniac Computer.
Industry: Cultivating Its Most Critical Assets. Some of the successful approaches that companies may use for recruiting, retaining, and advancing more women in computing, as well as in other technical professions, include:
- Senior managers should be aware of the unconscious biases that often permeate industrial settings. Even when they offer competence and qualifications equivalent to those of their male counterparts, women are perceived less favorably, which leaves them at a significant disadvantage for advancement.10 Thus company leaders should familiarize themselves with this phenomenon—for example, by attending pertinent workshops—and then revise their evaluation and promotion practices accordingly.
- Cast a broad net to recruit female computer scientists. Indeed, many women in technical positions in the work force earned degrees in other disciplines.32
- Address technical women's isolation, in part by developing a network to address their specific needs. Networking is paramount to career advancement, yet women in entry- to mid-level technical positions have fewer opportunities to participate in it outside their immediate department.14
- Implement a mentoring program. Indeed, make mentoring, which positively impacts career advancement and satisfaction, a basic part of the organizational culture. Sun Microsystems' SEED program, for example, is regarded as a major step in this direction.8
- Join organizations, such as the Anita Borg Institute, the Workforce Alliance of NCWIT, and Catalyst, that are actively working on solutions to problems facing women in the technical work force.
- Send the company's women in computing to some of the field's conferences so that they can expand their professional network. Company recruiters should attend such conferences as well.
- Implement "best practices" —those shown to be most successful at increasing women's representation in the technical work force (see resources provided at www.ncwit.org/resources.res.practices.php).
- Make an active effort to place more women in senior leadership positions. This policy is not only rewarding in its own right but it also increases the company's ability to recruit and retain other female talent.15 Provide leadership-development opportunities through programs such as the Anita Borg Institute's TechLeaders.
- Correct the company's gender-related wage differential, thereby sending a strong signal to its work force that women are deemed to be critical assets.3
"Seek inspiration from mentors—family, friends, teachers, and/or prominent people—to create careers combining your education, talents, interests, and dreams." Maxine D. Brown: Associate Director, Electronic Visualization Laboratory, University of Illinois at Chicago; Co-Authored The 1987 NSF Report, Visualization In Scientific Computing, Which Defined The Field of Scientific Visualization.
Consider the exemplary efforts of IBM. Over the past 15 years, the company has effected dramatic and systemic cultural changes, resulting in a 370% increase in its women executives and a 233% increase in ethnic minority executives (as of 2004).30 These changes occurred in four main ways: demonstrating leadership support, engaging employees as partners, integrating diversity goals with management practices, and linking diversity goals to business goals.
When Lou Gerstner assumed control of the company in 1993, he deliberately set out to change IBM's culture by uncovering, and endeavoring to understand, differences between underrepresented population groups (including women and minorities). The first step was to establish task forces, composed of executives and employees alike, for each group. Once group needs were better understood, management implemented practices to establish and sustain diversity—in creating pools of high-potential candidates for recruitment or of outstanding employees for advancement, women and minorities had to be well represented. These changes, though company-wide, were particularly focused on IBM's technical workforce, which it considered to be one of its most critical assets.
"The theoretical and practical knowledge embodied in CS is interesting as standalone study. But the real opportunity lies in equipping oneself to partner with scientists or business experts, to learn what they know and, together, to change how research or business is conducted." Adele Goldberg: Former Research Laboratory Manager, Xerox Parc; Founding Chairman and CEO, Parcplace Systems; General Partner, Pharma Capital Partners.
From the beginning, IBM's diversity efforts were driven by the desire not only to do the right thing but also to broaden its customer base. Gerstner created plans to embrace group differences in order to appeal to broader sets both of employees and customers. As a result, the company extended its reach into women-owned businesses, for example, as well as into new market segments.
Government Has a Role to Play
Improving women's representation in computing must also entail more enlightened governmental institutions and policies. The following agency practices have been shown to be particularly effective:
- Rigorously adhere to evaluating Criterion 2. In 1997, the NSF's board approved a reformulated merit review policy that included two criteria: (1) intellectual merit and quality of research, and (2) broader impacts of the proposed activity. Criterion 2 required that proposals address areas of societal concern, including the broadening of underrepresented groups' participation in the science, technology, engineering, and mathematics (STEM) work force.24 But although NSF continuously improves enforcement of Criterion 1 under its grants, it has yet to establish a method for ensuring observance of Criterion 2.
- Hold academic institutions accountable for measuring diversity among their employees through enforcement of Title IX. Four federal science agencies have made efforts to ensure that grantees comply with Title IX by performing several compliance activities, such as investigating complaints and providing technical assistance, but most agencies have not conducted all the monitoring activities required.11
- Publicly advertise the agency's diversity programs and its proportion of women in leadership positions.
- Stay aware of new policy and legislative initiatives, even in advance of their demonstrated impacts.
- The House Diversity and Innovation Caucus, whose mission is to help generate policy ideas for addressing the underrepresentation of women and minorities in the STEM fields, has held a number of briefings.
- The America COMPETES Act of 2007 (Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science) seeks to strengthen research, provide technical training for 21st-century occupations, and attract the best and brightest workers.
- NSF's ADVANCE program, in place for about a decade, contributes to the development of a more diverse technical work force (see www.nsf.gov/funding/pgm_summ.jsp?pims_id=5383).
- The BPC program, while relatively new, is having significant impact on many computing educators and professionals. Its largest initiative is NCWIT, whose organizational structure includes a set of alliances—within participating communities in K-12, academia, and industry—that have been very successful. For example, the Stars Alliance is a partnership of over 20 southeastern universities that share best practices for recruiting students to computing and retaining them. The goal of the Alliance for Access to Computing Careers is to increase the field's representation of people with disabilities.
Every computing professional, male and female alike, can contribute to the increased participation of women in the field. At the very least, each of us should do more to encourage women with whom we daily interact. For those readers not well informed about practices and programs that help attract women to our profession and retain them, we hope this article has provided useful information and indicated actionable steps pertinent to one's particular circumstances. By way of encouragement, know that institutions that have already made decisions to implement these kinds of practices are seeing significant increases in the participation of women in computing at all levels. Thus we encourage our colleagues to work to effect positive change, both locally—in individual institutions—and globally. Long-term success depends on our entire community taking responsibility for making computing a broadly supportive and inclusive discipline.
Women in computing --- take 2