Who’s a CEO? Google image results can shift gender biases

Jennifer Langston UW News

Getty Images last year created a new online image catalog of women in the workplace – one that countered visual stereotypes on the Internet of moms as frazzled caregivers rather than powerful CEOs.

A new University of Washington study adds to those efforts by assessing how accurately gender representations in online image search results for 45 different occupations match reality.

In a few jobs — including CEO — women were significantly underrepresented in Google image search results, the study found, and that can change searchers’ worldviews. Across all the professions, women were slightly underrepresented on average.

The study also answers a key question: Does the gender ratio in images that pop up when we type “author,” “receptionist” or “chef” influence people’s perceptions about how many men or women actually hold those jobs?

Read about another of Munson’s CHI 2015 papers on making public exercise commitments on Facebook

In a paper to be presented in April at the Association for Computing Machinery’s CHI 2015 conference in South Korea, researchers from the UW and the University of Maryland, Baltimore County found that manipulated image search results could determine, on average, 7 percent of a study participant’s subsequent opinion about how many men and women work in a particular field, compared with earlier estimates.

“You need to know whether gender stereotyping in search image results actually shifts people’s perceptions before you can say whether this is a problem. And, in fact, it does — at least in the short term,” said co-author Sean Munson, UW assistant professor of human centered design and engineering.

The study first compared the percentages of women who appeared in the top 100 Google image search results in July 2013 for different occupations — from bartender to chemist to welder — with 2012 U.S. Bureau of Labor statistics showing how many women actually worked in that field.

In some jobs, the discrepancies were pronounced, the study found. In a Google image search for CEO, 11 percent of the people depicted were women, compared with 27 percent of U.S. CEOs who are women. Twenty-five percent of people depicted in image search results for authors are women, compared with 56 percent of actual U.S. authors.

By contrast, 64 percent of the telemarketers depicted in image search results were female, while that occupation is evenly split between men and women.

Yet for nearly half of the professions – such as nurse practitioner (86 percent women), engineer (13 percent women), and pharmacist (54 percent women) — those two numbers were within five percentage points.

“I was actually surprised at how good the image search results were, just in terms of numbers,” said co-author Matt Kay, a UW doctoral student in computer science and engineering. “They might slightly underrepresent women and they might slightly exaggerate gender stereotypes, but it’s not going to be totally divorced from reality.”

When the researchers asked people to rate the professionalism of the people depicted in top image search results, though, other inequities emerged. Images that showed a person matching the majority gender for a profession tended to be ranked by study participants as more competent, professional and trustworthy. They were also more likely to choose them to illustrate that profession in a hypothetical business presentation.

By contrast, the image search results depicting a person whose gender didn’t match an occupational stereotype were more likely to be rated as provocative or inappropriate.

“A number of the top hits depicting women as construction workers are models in skimpy little costumes with a hard hat posing suggestively on a jackhammer. You get things that nobody would take as professional,” said co-author Cynthia Matuszek, a former UW doctoral student who is now an assistant professor of computer science and electrical engineering at University of Maryland, Baltimore County.

Most importantly, researchers wanted to explore whether gender biases in image search results actually affected how people perceived those occupations.

They asked study volunteers a series of questions about a particular job, including how many men and women worked in that field. Two weeks later, they showed them a set of manipulated image search results and asked the same questions.

Exposure to the skewed image search results did shift their estimates slightly, accounting for 7 percent of those second opinions. The study did not test long-term changes in perception, but other research suggests that many small exposures to biased information over time can have a lasting effect on everything from personal preconceptions to hiring practices.

The measured effect raises interesting questions, the researchers say, about whether search image algorithms should be changed to help counter occupational stereotypes.

“Our hope is that this will become a question

The University of Washington just released a preview of a study that claims search engine results can influence people's perceptions about how many men or women hold certain jobs. One figure quoted in the preview is that in a Google image search for CEO, only 11 percent of the people returned were women (by comparison, the university says 27 percent of CEOs in the US are women). That's pretty crazy, so I decided to fire up incognito mode in Chrome and search Google Images as an Anonymous Internet Person to see the authentic, natural results. And, uh, the results are insane. I haven't seen this many white men in suits since the last time I was in church.

Here's a look, zoomed way out to 50 percent:

No women to be found. Oh, wait. See that thumbnail in the bottom row to the right?

CEO Barbie. This is the first woman who appears in a Google Image search for "CEO," rows and rows and rows below the top results — and she's not even a real Barbie doll! This image is actually from a 2005 Onion article which sarcastically observed that "women don't run companies," they just "work behind the scenes to bring a man's vision to light." This may be the most meta Onion joke of all time.

Stuff gets even crazier if you broaden the search terms.

Oh hey this CEO image search thing gets even better @chillmage pic.twitter.com/RxiHGeENPB — Eileen Ridge (@notbangalore) April 9, 2015

Search for "male CEO" or "man CEO" and you just get pictures of dudes who run companies. Search for "woman CEO" or "female CEO" and you get a whole bunch of wacky, gendered suggested search categories, including:

Outfit

Attire

Glasses

Successful Business Woman Profile

Business Woman Silhouette

Google's search engine intelligence may be mostly artificial, but it's bootstrapped by the garbage expectations that many of its users feed into the system over time. Getting rid of nasty occupational stereotypes will require better human intelligence.

Oh, and in case Bing and Yahoo thought they'd get off easy, they're not any better:

The Ellen Pao-Kleiner Perkins trial shone a light on discrimination in the tech industry, but for a more immediate look at the challenges women face in corporate America, look no further than a Google Images search.

Doing a search at the site for “CEO” reveals just one female face in the top results: CEO Barbie. The doll (which may not even be a real Barbie product) appears way down in the results, under a sea of male, mostly white faces.

It’s not really the fault of Google, whose algorithms in many ways reflect the pervasive culture: Most of the top images labeled CEO at popular sites apparently are men. But it’s an indication of how under-represented woman are at the top of the corporate ladder.

Nor is Google alone in its results, noticed earlier by The Verge. Search for “CEO” on Bing and the service offers to refine your search to “women CEO”—using the same picture of Barbie. Much lower down, Bing shows an image of former Burberry CEO Angela Ahrendts.

The results offer a glimpse into a male-dominated corporate culture, but they are also skewed against women. For the aforementioned Google search, 11 percent of the faces depicted in the top 100 results are women, but 27 percent of U.S. CEOs are female, according to researchers at the University of Washington, who published results of a study on the issue Thursday.

It works the other way, too. Do a Google Images search for “telemarketer,” and women dominate the top results, even though that occupation is split evenly between men and women, the researchers said.

The search results aren’t inconsequential, according to the study: They influence how people view the real world.

“In a few jobs—including CEO—women were significantly underrepresented in Google image search results ... and that can change searchers’ worldviews,” the researchers said.

Google is a modern oracle, and a miraculous one at that. It can lead you to the Perfect Strangers theme song lyrics, or to a satellite image of your childhood neighborhood, or to a blueprint for building a quantum computer. But for as much as it is a portal to the world's knowledge, and despite its inherently aspirational functionality, Google searches are also a reflection of the status quo.

Do an image search for "CEO," for instance, and Google's algorithm returns a mosaic of mostly white, male faces.

Screenshot from Google

Which makes sense: Only two dozen Fortune 500 companies have women as top bosses—that's less than 5 percent of overall Fortune 500 CEOs. And the 10 best paid CEOs in America are all white and male, according to The Guardian.

There is, however, one female face that pops up among the first few dozen male CEOS, though.

Can you spot it?

Screenshot from Google

Let's zoom in a little.

Screenshot from Google

That's right. It's CEO Barbie.

Screenshot from Google

Here's the thing, though: Google image searches don't just reflect the sad state of diversity in corporate leadership; they actually influence the ways in which people think about what it means to be a CEO. That's according to researchers at the University of Washington and the University of Maryland, who determined that Google images measurably sway a person's opinion about how many men and women work in a particular field—compared with what that person thought before conducting the search. The effect is small but significant. "We find that

people’s existing perceptions of gender ratios in occupations are quite accurate," the researchers wrote. "But that manipulated search results can... [shift] estimations on average [by] 7 percent."

In today's modern professional world men can be doctors, investment bankers, and professors, while women, of course, can be nurses, secretaries, and sexy Halloween costume models—at least according to Google Image Search.

Why did we spend all day Googling? Yesterday, T.C. Scottek, a senior editor for The Verge, tweeted that an Image Search for "CEO" resulted in all male photos—save for, yes, CEO Barbie. Which made us curious: How were other professions gendered?

We conducted a similar search experiment using Google's incognito feature, which essentially allowed us to search as an anonymous, non-gendered person. The results, sadly, are exactly what we expected. Here's a play-by-play of what we found.

"Doctor"

Half of new doctors in America are women, yet when you search the term, only three female doctors appear in the first 26 photos (and one appears twice). Also, check out the "clip art" option, which shows only men in the thumbnail—to find a woman, you must explicitly search "woman doctor clipart." Awesome.

Advertisement

"Cop"

When searching "cop," no women appear until the fourth row, and that woman is … wait for it … wait for it … a sexy Halloween cop! In fact, we don't get to see a real woman in law enforcement until the eighth row, which features a lady wearing a vest that says "sheriff." If you follow that image, it takes you to a Pinterest page titled "Female cop," which also includes donut earrings and a sexy woman showering.

Advertisement

"Professor"

How many female professors did you have in college? None, amirite? Kidding. According to the American Association of University Professors, women make up 42 percent of full-time faculty positions. Yet according to Google Image Search, professors IRL are white, male, or illustrated. In fact, the one human female shown in the first four rows isn't even a professor. The board behind her reads "5 + 1 = __". Pretty sure second grade is not a college course these days.

Advertisement

"Boss"

A recent report revealed that women now make up 52 percent of all management and professional positions. Well, Google Image Search scoffs at that number—it knows that when searching for "boss," you're really looking for a man. Check out this return:

Advertisement

"Oscar winner"

I had hoped that Hollywood's gender inequality wouldn't also be reflected in Google's Image Search, but alas, it was. Only five women appear in the first 23 photos. One of whom was not a winner, but a host. On the bright side, the search thumbnail for "crying" yields plenty of female results, so there's that.

Advertisement

"Engineer"

It's pretty well known that STEM fields need more women. Well, if little girls who dream of becoming engineers Google their chosen profession, they're not likely to see many female role models to emulate. Google Image Search returned all men in hard hats, save for two women.

Advertisement

"Investment Banker"

This really needs no explanation. Everyone knows you can't trust women with investment decisions. For reals, though, in 2013 women made up 18.2 percent of executive/senior-level officials in companies involving "securities, commodity contracts and other financial investments and related activities" and 16.1 percent of executive/senior-level officials in companies that dealt with "investment banking and securities dealing," according to the non-profit Catalyst. Both those numbers are higher than zero.

Advertisement

"Nurse"

Of course, when you think "doctor" you think "man"—but switch that search to "nurse," and it's ovary city. Google returned all female images, save for one ambiguously ethnic guy, because white men sure as hell aren't nurses (they're surgeons, duh).

Advertisement

"Teacher"

Teacher is also an acceptable career for women. In fact, 75 percent of teachers from kindergarten through high school are female. So it makes sense that Google returned this:

Advertisement

"Secretary"

Of course. And FYI, searching the word "assistant" didn't change the gendered results.

Advertisement

For one last experiment, we typed in "average woman" and "average man." The results suggest that, for women, we're all about comparing faces and body sizes—the images consist largely of women in their underwear. For men, however, life's more about just being a dude. The guys are almost all fully clothed and seem pretty content.

"Average Woman"

Advertisement

"Average Man"

In case you're wondering, Google searches photos based on how images are named, labeled, and linked to, as well as the context of the page in which the image appears. So more than simply blaming the engine, we might take a hard look at who is choosing to represent these careers with these photos (ahem, the media, editors, websites, blogs) and the library of photos they have to choose from (stock images), which can be limiting.

Advertisement

Taryn Hillin is Fusion's love and sex writer, with a large focus on the science of relationships. She also loves dogs, Bourbon barrel-aged beers and popcorn — not necessarily in t

Not all doctors or CEOs are men. Not all nurses are women. But you might think otherwise if you searched for these professions in Google images.

It turns out that there's a noticeable gender bias in the image search results for some jobs, according to a new study by researchers at the University of Washington and the University of Maryland. This underrepresentation of women in image search results actually affects people's ideas about professional gender ratios in the real world, the study found.

See for yourself...

This is what happened when I searched "nurse" in Google images:

This is what happened when I searched "doctor" in Google images:

This is what happened when I searched "CEO" in Google images:

That's all white men in that last one, in case you couldn't tell.

Though they're in the minority, there are some chief executives who are women and who are not white.

Researchers looked at the top 100 image search results for 45 different jobs. They then compared the gender breakdown of those results to actual gender data from the Bureau of Labor Statistics for each job.

It turns out that for some search terms, like "chief executive officer," the gender imbalance on Google is a lot worse than it is in real life. The study found that only 11 percent of the people shown in an image search for "chief executive officer" were women, while BLS data indicates that 27 percent of CEOs are women. Similarly, a search for telemarketers showed 64 percent women, even though in reality it's about a 50-50 gender split. Nearly 60 percent of bartenders are female, but in the image search results, only 23 percent were.

Not all jobs demonstrated such a gulf between search results and reality. But women were slightly underrepresented on average across all jobs the researchers looked at.

“I was actually surprised at how good the image search results were, just in terms of numbers,” said Matt Kay, a co-author of the study. “They might slightly underrepresent women and they might slightly exaggerate gender stereotypes, but it’s not going to be totally divorced from reality.”

And try searching a term like "female construction worker." (According to BLS data, 2.9 percent of construction workers are women.)

“A number of the top hits depicting women as construction workers are models in skimpy little costumes with a hard hat posing suggestively on a jackhammer,” said Cynthia Matuszek, a co-author of the study. "You get things that nobody would take as professional."

Google image search, like Google's many other search tools, mines the Web for content based on algorithms that are designed to show you the results it thinks you'd most want to see. Google's searches take into account a number of factors, like keywords, timeliness of the content, where you're searching from and PageRank, which is Google's way of analyzing how important a page on the Internet is. Google declined to comment about the study's findings.

The sad thing is, the gender biases displayed in search actually do have an effect on how people perceive gender breakdowns in the real world, according to the study. After being shown skewed image results for a certain occupation, study participants did report a slight change in their perceptions of how male-dominated a field was.

Of course, none of this is exactly Google's fault. Google is a reflection of what's on the Internet. You can also see similar search results on Yahoo and Bing.

But the researchers suggested that search engine designers could actually use these study results to develop algorithms that would help work against gender stereotypes.

Try this: Google image "CEO." Notice anything? The first female Google image search result for "CEO" appears TWELVE rows down—and it's Barbie.

A recent study conducted at the University of Washington sought to examine how well female representation in online search results reflected reality—and, subsequently, how impressions in search results could, in turn, impact the way that people perceive how women are represented in industry, in real life.

"You need to know whether gender stereotyping in search image results actually shifts people's perceptions before you can say whether this is a problem," UW assistant professor of human-centered design and engineering says. "And, in fact, it does—at least in the short term."

How skewed—and how much does it skew perceptions, then, that the first female image result in Google for "CEO" is Barbie, appearing 12 rows (and two page scrolls) down?

The study first compared the percentages of women who appeared in the top 100 Google image search results in July 2013 for different occupations—from bartender to chemist to welder—with 2012 U.S. Bureau of Labor statistics showing how many women actually worked in that field.

In some jobs, the discrepancies were pronounced, the study found. In a Google image search for CEO, 11 percent of the people depicted were women, compared with 27 percent of U.S. CEOs who are women.

Twenty-five percent of people depicted in image search results for authors are women, compared with 56 percent of actual U.S. authors.

By contrast, 64 percent of the telemarketers depicted in image search results were female, while that occupation is evenly split between men and women.

Researchers found that around seven percent of test subjects shown skewed search results shifted their estimates of how many women and men are employed in different industries—"Small exposures to biased information over time can have a lasting effect on everything from personal preconceptions to hiring practices," the study reports.

What do you think about this? Are there other industries and/or job titles you've noticed this happen in? Tell us in the comments below.

Search the term "CEO" in Google Images and the first picture of woman you get is a picture of Barbie in a suit.

This "gender bias" has become apparent after a paper was published showing that many image searches for specific occupations favour men or women.

The authors of the report say they have found "stereotype exaggeration and systematic underrepresentation of women in search results".

They have also identified the "sexy construction worker problem".

This is where female construction workers in their search results "tended to be sexualised caricatures of construction workers" who the researches said were "almost certainly not engaged in the profession they portrayed".

Although the paper focuses on gender representations, the same search also shows that most of the men, and Barbie, are white.

The image of Barbie that appears is actually one from spoof news site The Onion, which jokingly reports on Mattel being criticised for encouraging "young girls to set impractical career goals".

Other search engines, such as Bing, Yahoo and DuckDuckGo also return all-male results on the first page - as first uncovered by The Verge.

The BBC's own picture archive, which is used by journalists across the organisation, can show results in a variety of different orders.

With the search prioritising newest picture first, multiple images of the CEO of McDonald's Japan - Sarah Casanova - appear.

Obviously the priorities for this search a different, with recency being the focus, however in the first 20 pages of results, you also see PepsiCo CEO Indra Nooyi, CEO of the International Culinary Center, Dorothy Cann Hamilton and international leaders such as Angela Merkel.

The report's authors, Matthew Kay and Sean A Munson from the University of Washington and Cynthia Matusek from the University of Maryland Baltimore County, say that the minority gender for any given occupation, whether that is a man or a woman, are usually portrayed "less professionally" than the majority gender for that role.

They claim this fits with how people also perceive results to be better when they match a stereotype they already hold - for example men making for better construction workers.

Although fairer representation may improve the real-world balance of genders in different occupations, the report recommends "balance" when it comes to the algorithms search engines use.

The authors say results which support "socially desirable outcomes" may not accurately represent either the available images, or the "real-world" numbers of men and women in different careers.

Google has declined to comment, while Mattel has yet to respond to Newsbeat's request.

Follow @BBCNewsbeat on Twitter, BBCNewsbeat on Instagram, Radio1Newsbeat on YouTube and you can now follow BBC_Newsbeat on Snapchat

Just when you thought biases were a completely human construct, more evidence suggests that both algorithms and interfaces could be biased, too.

ADVERTISEMENT

The latest example of this is from a study conducted by researchers from University of Washington and University of Maryland and reveals how a gender bias is working its way through web searches when people look for images to represent careers and jobs.

First, they did a comparative analysis to see if the prevalence of men and women in image search results for professions actually correspond to their representation in actual professions. The researchers did this by comparing the number of women who appeared in the top 100 Google image search results in July 2013 for 45 different occupations, which ranged from bartender to chemist to welder, with 2012 U.S. Bureau of Labor statistics of how many women actually worked in those fields. Then, they did a qualitative analysis to see how men and women are portrayed in the image results.

The goal was to answer some compelling questions:

Are there systemic over- or under-representations of women in preferred results?

Do biased image search results lead people to perpetuate a bias in image search results when they choose images to represent a profession (i.e. through stereotype exaggeration)?

Do differences in representation in image search results affect viewers’ perceptions of the prevalence of men and women in that occupation?

Can we shift those opinions by manipulating results?

The answers were equally compelling. For instance, according to their study, more than half of U.S. authors are women (56%), yet the image search shows only about 25% women authors.

On the flip side is telemarketing, an industry where men and women are equally represented, but the Google image results would have you believe that 64% of telemarketers are female.

Not all the results were so skewed. The research uncovered that, in nearly half of the professions, the actual gender representation and the image search numbers were within 5 percentage points of each other.

How men and women looked in those images was another story. When the researchers asked participants to rate professionalism, images showing a person who matched the majority gender for the job was viewed as more competent, professional, and trustworthy. Those who didn’t match were rated provocative or inappropriate.

“A number of the top hits depicting women as construction workers are models in skimpy little costumes with a hard hat posing suggestively on a jackhammer. You get things that nobody would take as professional,” says Cynthia Matuszek, a co-author of the study.

None of this would matter if people wouldn’t then be nudged into making assumptions about men and women in particular roles in the real world. However, when the researchers manipulated the search results, not surprisingly, participants’ opinions changed to conform with stereotypes. Though they stressed that this was just a short-term observation, other research bears out that incremental exposure to these images over time will contribute to unconscious bias.

It has also already been revealed that Wikipedia’s entries–a supposed bastion of diversity and editorial neutrality–skew heavily towards men in both actual articles as well as within links. Articles about women tended to be linked to those about men.

Part of this is due to Wikipedia’s community, the preponderance of which are educated men, who are English-speaking and hail from mostly Christian countries.

WHAT GOOGLE IS DOING ABOUT UNCONSCIOUS BIAS

In addition to the image searches being gender biased in some cases, Google’s also been taken to task for lack of diversity within its ranks and even disproportionately using white men in its doodles.

While Google may not be aware of the results of this latest study and the researchers’ recommendation, the search giant did recognize that its tough for anyone–even its own cadre of emotionally intelligent staff–to process the 11 million bits of information that we are bombarded with at any given moment and focus instead on finding out what biases might spring from them.

As such, Google offers a workshop focused on unconscious biases that might sabotage the workplace dynamics or upend the equality of the hiring process.

The researchers of this study hope that the information will influence designers of search engines to create algorithms that more accurately represent reality. Sean Munson, UW assistant professor of human-centered design and engineering and a coauthor of the study says: “[Search engine designers] may come to a range of conclusions, but I would feel better if people are at least aware of the consequences and are making conscious choices around them.”

Fresh off the revelation that Google image searches for “CEO” only turn up pictures of white men, there’s new evidence that algorithmic bias is, alas, at it again. In a paper published in April, a team of researchers from Carnegie Mellon University claim Google displays far fewer ads for high-paying executive jobs…

Follow the latest on Election 2020

… if you’re a woman.

“I think our findings suggest that there are parts of the ad ecosystem where kinds of discrimination are beginning to emerge and there is a lack of transparency,” Carnegie Mellon professor Annupam Datta told Technology Review. “This is concerning from a societal standpoint.”

To come to those conclusions, Datta and his colleagues basically built a tool, called Ad Fisher, that tracks how user behavior on Google influences the personalized Google ads that each user sees. Because that relationship is complicated and based on a lot of factors, the researchers used a series of fake accounts: theoretical job-seekers whose behavior they could track closely.

[Google Maps’ White House glitch, Flickr auto-tag and the case of the racist algorithm]

That online behavior — visiting job sites and nothing else — was the same for all the fake accounts. But some listed their sex as men and some as women.

The Ad Fisher team found that when Google presumed users to be male job seekers, they were much more likely to be shown ads for high-paying executive jobs. Google showed the ads 1,852 times to the male group — but just 318 times to the female group.

This isn’t the first time that algorithm systems have appeared to be sexist — or racist, for that matter. When Flickr debuted image recognition tools in May, users noticed the tool sometimes tagged black people as “apes” or “animals.” A landmark study at Harvard previously found serious discrimination in online ad delivery, like when searching ethnic names on Google turned up more results around arrest records. Algorithms have hired by voice inflection. The list goes on and on.

[The uncomfortable truth about how we view working women, in one simple Google search]

But how much of this is us and how much of this is baked into the algorithm? It’s a question that a lot of people are struggling to answer.

After all, algorithmic personalization systems, like the ones behind Google’s ad platform, don’t operate in a vacuum: They’re programmed by humans and taught to learn from user behavior. So the more we click or search or generally Internet in sexist, racist ways, the algorithms learn to generate those results and ads (supposedly the results we would expect to see).

“It’s part of a cycle: How people perceive things affects the search results, which affect how people perceive things,” Cynthia Matuszek, a computer ethics professor at University of Maryland and co-author of a study on gender bias in Google image search results, told The Washington Post in April.

[What we can learn from these less-than-legit Google Knowledge Graph results]

Google cautions that some other things could be going on here, too. The advertiser in question could have specified that the ad only been shown to certain users for a whole host of reasons, or the advertiser could have specified that the ad only show on certain third-party sites.

“Advertisers can choose to target the audience they want to reach, and we have policies that guide the type of interest-based ads that are allowed,” reads a statement from Google.

The interesting thing about the fake users in the Ad Fisher study, however, is that they had entirely fresh search histories: In fact, the accounts used were more or less identical, except for their listed gender identity. That would seem to indicate either that advertisers are requesting that high-paying job ads only display to men (and that Google is honoring that request) or that some type of bias has been programmed, if inadvertently, into Google’s ad-personalization system.

In either case, Datta, the Carnegie Mellon researcher, says there’s room for much more scholarship, and scrutiny, here.

“Many important decisions in society these days are being made by algorithms,” he said. “These algorithms run inside of boxes that we don’t have access to the internal details of. The genesis of this project was that we wanted to peek inside this box a little to see if there are more undesirable consequences of this activity going on.”

“You can’t be what you can’t see,” Marie Wilson of the White House Project said back in 2010. According to a new study, Google Images may not be helping to improve the situation.

AdView analyzed employment data to determine the number of women in various jobs (baker, call-center worker, CEO), and then calculated the number of women who showed up in Google Image search results for the same roles. While women make up about 28% of chief executives (at least, according to data from the U.S. Bureau of Labor Statistics), they made up only 11% of the CEOs that show up in a Google Image search, according to AdView. (When we tried the search, at least one of those “CEOs” was Gal Gadot, who is a Wonder Woman, but not a CEO, as far as we know.)

Granted, Google Image search results are not frozen in time (they change based on things such as the news cycle, for instance), but the lack of women CEOs in these results raises important questions about the lack of representation of women leaders in media, and how that relates to their lack of representation in the real world.

The number of women in chief executive roles at major companies actually declined 25% this year, according to Fortune’s 2018 list of female CEOs of Fortune 500 companies. There were 24 women, down from 32 the year before. Part of the reason there are so few women in those roles is that women are 18% less likely to be promoted to manager than their male peers. Perhaps more women–and men–need to be reminded that women are CEO material, too. And that’s something Google Images could help with by changing the perception of CEOs to include more women in search results.

Reached for comment, a Google spokesperson did not provide an on-the-record statement about the study.