Tracy Chou wanted to see the numbers.
In 2013, fresh off the Grace Hopper Celebration of Women in Computing, an annual conference focused on the advancement and professional development of women in the field, Chou wrote a post on Medium asking for numbers on just how many women were in technical roles in the industry. Then a software engineer for Pinterest, Chou set up a crowdsourced repository where startup employees could catalog the gender breakdown of technical teams.
"In raw numbers, are there actually more technical women in industry now than before? Is the percentage of women in engineering going up?" she asked. "What's working? Is anything? Does anybody know?"
Chou helped catalyze the current practice of tech firms releasing diversity reports. Her call for numbers elevated the idea that data meant transparency and transparency could lead to change.
In the intervening three years, we've seen a wave of tech companies release employee base demographics. We know the tech industry is made up largely of white men. Few women hold technical roles.
Change seems to happen a percentage point at a time. Sometimes, it doesn't happen at all.
Or, companies face diversity-related scandal, like Google's flap last week with James Damore, an engineer who wrote a 10-page document containing such opinions as women aren't biologically suited for tech and leadership.
But even with the increasing noise in favor of diversity and the release of these reports from Apple, Facebook, Microsoft and many others, there are still lots of things we don't know about the current state of diversity in tech. Diversity is most commonly defined by gender and race, but advocates say that's an incomplete picture and that building efforts around it could hurt the push for diversity going forward.
What's in a diversity report
After eight years as an international human rights and labor lawyer working in places like Nicaragua among sugarcane workers, Y-Vonne Hutchinson was alarmed by tech's big diversity problem. She felt it was the next big issue that needed attention.
So Hutchinson founded diversity consultancy firm ReadySet.
She sees diversity reports helping establish two important points: transparency and answerability. "Without those measurements, it's really hard to establish accountability," Hutchinson said. "Those measurements also form the basis for goal-setting, they help track progress over time."
The problem is, the reports aren't really all that transparent.
Gender and race aren't the only two ways to define diversity. One of the more significant metrics that diversity reports typically don't include is intersectionality, which is the idea that people can have multiple social identities. In other words, you can be a woman and also Hispanic, and that's a unique experience from being one or the other.
It also means companies can't end up with a workforce loaded with white women and call it a day.
We know the numbers of women of color in tech are low. A 2016 report from the National Center for Women & Information Technology said black and Hispanic women hold about 3 percent and 1 percent of computing occupations. In comparison, white women come in at about 25 percent.
"We don't really see the intersection of those [groups] and how those groups may be progressing or not progressing," said Elizabeth Ames, senior vice president of programs, marketing and alliances at the Anita Borg Institute.
Chou thinks nuance is getting lost.
"There's still somewhat of a tendency for people to assume that diversity is gender diversity, and it looks very different for white women than it does for black women, for example," she said.
Even if a company is increasing the number of women in its ranks, it could still be turning away or turning off women of color -- or failing to retain them. An April study from the Kapor Center for Social Impact on why people leave the tech industry found that, as a group, women of color were most likely to get passed over for a promotion. One-third reported it.
So how do you address an issue you might not even totally know you have?
Well, large tech companies do have some sense of this. Private employers with 100 or more employees that are subject to Title VII of the 1964 Civil Rights Act are legally required to fill out a form every year with employee demographics. The form asks for breakdowns of men and women by role, as well as by race/ethnicity.
The thing is, companies don't have to make that form, the EEO-1, publicly available. Many do and will link to it from their latest diversity report. (Check out the most recent ones from Google and Facebook.) But the way they count their employee base for the Equal Employment Opportunity Commission might not be the same way they do on the report. And if you're looking for a percentage for women of color, you're going to have to crunch the numbers yourself. This makes it difficult to get clear numbers.
To track or not to track
After a period of tumult following a public scandal involving a former employee who was sexually harassed at Github, the code repository went through a restructuring and changed its leadership. In 2016, it released its first diversity report.
The company decided it could get more detailed than just race and gender, said Merritt Anderson, GitHub's vice president of employee experience and engagement.
The report encompassed areas you don't usually see in similar reports -- age, parental status, where employees live (San Francisco, elsewhere in the US or internationally). Gender categories included transgender and queergender, and the report also broke down how many women of color work at Github.
"We do this as much for external accountability as we do for our own Hubbers, who appreciate the extra level of transparency," Anderson said. Future reports might include employees with physical disabilities and veterans.
While the EEO-1 doesn't ask for demographics like LGBTQ identification, it's up to companies to decide what they want to tell the public about whom they employ. After the release of its latest diversity report in early August, we asked Facebook about its employee makeup and a spokeswoman said it tracks other metrics internally but couldn't say whether those would show up in a future report.
This isn't to say tracking all these other demographics is easy.
When Chou first published her crowdsourced repository, she got requests to create a similar one for race and ethnicity. But she didn't partly because of how problematic it would be to try to look around an office and assume people's racial or ethnic backgrounds.
"It gets harder and requires self-identification, which is harder to do without some kind of HR-level buy-in from the company," Chou said.
Hutchinson thinks companies are only now just starting to realize the importance of tracking more than just race and gender.
Both she and Chou recommend companies use data to establish goals around hiring and retention. Pinterest did this for 2016. While it set a goal of having 30 percent of engineers be women, it fell short at 22 percent. However, the report also included some analysis of why that might have been. The company suggested it had put more emphasis on hiring senior women engineers, and that took longer to accomplish.
"You need people to know what's happening in your companies," Hutchinson said, "and then you need the public and your employees to hold you accountable."
The story was originally published on August 17 at 5 a.m. PT.
Update, 6:05 a.m. on August 18: Adds background on ex-Googler James Damore.
Solving for XX: The tech industry seeks to overcome outdated ideas about "women in tech."
Tech Enabled: CNET chronicles tech's role in providing new kinds of accessibility.