Photo credit: SvetaZi / iStock
The year is 2022 and women in science are still being hired and promoted less frequently than their male colleagues. Women are less mentored by outstanding faculty, they publish in less prestigious journals, have fewer staff, are underrepresented among journal reviewers and editors, and their articles are cited less.
As. Is this. event?!
Information Science Institute of USC (ISI) Principal Scientist Kristina Lemann and her team used AI to search for answers to this question. The resulting paper was published in the prestigious peer-reviewed, multidisciplinary scientific journal Proceedings of the National Academy of Sciences (PNAS) on September 26, 2022.
As a woman in science, Lerman knows the world she works in, but even she was shocked by stats she recently learned: Only 2 percent of Nobel Prize winners in physics were women (until a few years ago that was 1 percent) and this Numbers are similar in many scientific fields. Lerman said: “Only seven percent of the Nobel Prize winners in chemistry were women! Women have been working in chemistry for so long, how come? We were curious about this discrepancy.”
Right dates, right time
Lerman had the right data set for the problem. Since 2019 she has been working with her team on a large project that used AI to predict the reproducibility of research papers. Funded by DARPA (The Defense Advanced Research Projects Agency), the ISI team used AI to analyze many aspects of scientific papers, including citations to predict reproducibility. They published the newspaper “Evaluate scientific research with Knowledge Graphs” at ACM SIGIR 22 (The Association for Computing Machinery’s Special Interest Group on Information Retrieval) in July 2022, describing their novel method and promising results.
To conduct this reproducibility research, Lerman’s team collected a vast amount of data from scientific papers. your co-author JayPujaraDirector of the Center on Knowledge Graphs at ISI, said: “We collected this very large citation graph – the network of articles, authors, citations, references, collaborations, author institutions, where they publish etc. They turned this data into a huge knowledge graph ( a “knowledge graph” is a representation of a network of real-world entities illustrating the relationships between them).
The team looked at the shapes or “structures” that emerged in the Knowledge Graph. They wondered if there is some kind of natural phenomenon that causes the different structures in the citation networks. Additionally, they wanted to ensure that the data used in their reproducibility predictions were not affected by bias in the data. Pujara said, “Kristina [Lerman] came up with the idea of examining covariates like gender or prestige.” And with that idea, the research team set out to see if there’s a difference in a network depending on whether the author is male or female and whether he is at a high-ranking university or a lower-ranking university.
The who, what and why of quotes
Before we go any further, a little info on how citation works in scientific research. There are usually three reasons why an author might cite another author’s work. First, as background – in order to understand his work, an author will cite other works that provide the necessary background information. Second, to explain a method – if an author uses a method that is similar, a version of, or comparable to a method in another article, he will cite the article that explains that method. And third, results – an author will explain their results but may cite other papers that have examined the same but obtained different results.
Gathering information from quotations
“Trying to study the citation network for every researcher out there is really difficult, so why don’t we pick the crème de la crème?” said Pujara. The team studied scientists who were elected to the US National Academy of Sciences (NAS), one of the oldest and best known professional science organizations. New members of the NAS are elected by the current members on the basis of scientific excellence, which means that in theory they have all achieved the same level of recognition. The ISI team studied 766 NAS researchers, 120 of whom were women, and hypothesized that complex gender differences would be visible within this elite group of researchers.
Their hypothesis turned out to be correct.
They constructed citation networks that captured the structure of peer discovery for each NAS member. These structures differed significantly between male and female NAS members. Women’s networks were much more densely clustered, suggesting that a female scientist needs to be more socially embedded and have a stronger support network than their male counterparts. The differences were systemic enough to allow the member’s gender to be accurately classified based solely on its citation network.
Lerman said, “We could write an AI algorithm that just looks at the citation networks and predicts whether that is a woman’s citation network or a man’s citation network. That was quite shocking and disappointing for us.”
As a control study, the team also considered the covariate prestige. Similar to women, NAS members who belong to less respected institutions are in the minority in the NAS. Lerman said, “We would have imagined that perhaps women’s citation networks would look like those of members of non-reputable universities.” But that wasn’t the case. They observed no differences based on the prestige of a member’s institutional affiliation.
Conclusion: Based solely on a scientist’s citation network, gender can be accurately determined, but not the reputation of the university to which the scientist belongs. This suggests that gender continues to influence career success in science, according to the ISI team.
How to stop being so shortCited
Why is this happening? Pujara said, “We don’t know. It could be because there is an aspect of gender that alters cooperative behavior. Or it could be something in society that shapes researchers and their paths due to social prejudice. So we don’t really know the answer to that. What we do know is that there is a difference.”
The real question is: How can we change this? How can we make science a less hostile climate for women, removing barriers for women and creating an environment that allows women to rise to the top of their field?
The ISI team hopes that their methods and results can help. First of all, this study could serve to help researchers understand what their networks look like. Additionally, it could help policymakers understand whether programs to improve gender equity in science are working.
Finally, and importantly, we can learn from these differences in citation structures between men and women. “For a woman to be recognized, she needs to be well embedded and have a strong support network,” Lerman said. “Mentoring young women and telling them that they really need to build these networks of social support and be very conscious of them” seems like a way to change the shape of those structures… and the shape of science.
This work was supported in part by the Defense Advanced Research Projects Agency (contract W911NF192027) and the Air Force Office of Scientific Research (contract FA9550-17-1-0327).
Published on 09/26/2022
Last updated on September 26, 2022
