Selected Studies on Bias in Science and the Humanities and in Evaluation and Decision-Making Processes
In order for decisions to be made in a way that is free of prejudice and equity-oriented and therefore science-led, it is necessary to avoid (implicit) prejudice.
In the film “Unbiased review”, the following studies are presented in infographics, showing how bias can potentially impact on review, assessment and decision-making processes.
In addition to studies on possible bias and its impact in evaluation procedures, other studies look at areas such as personnel selection. These studies on the tendency towards prejudice show how selection procedures can also be biased.
- Prominence as an Advantage in Peer Revie
- Homogeneity as an Advantage in Peer Revie
- Stereotypes in Letters of Recommendatio
- Disadvantages in Application
- "Motherhood Penalty" and "Maybe-Baby Effect
- Success in Application Procedures as a Result of "Higher-Class Hobbies
- Gender Bias in Science and the Humanitie
- Bias in the Decision-Making Procedures of Research Funding Organisation
Prominence as an Advantage in Peer Review
Taking the example of a behavioural economics journal, Huber et al. showed that the prominence of an author can influence how they are assessed in peer review. Reviewers assessed identical manuscripts differently, depending on who the stated author was. In the study, a paper was passed on to reviewers with the name of the prominent author, with the name of the lesser-known author (both men) and without any name, i.e. with the author remaining anonymous. The degree of approval between the prominent and lesser-known author ranged from 77 to 35 percent. When anonymised, the article received 52 percent approval.
Huber, J., Inoua, S., Kerschbamer, R., König-Kersting, C., Palan, S. & Smith, L. S. (2022).
Nobel and novice: Author prominence affects peer review, PNAS 119 (41).
Homogeneity as an Advantage in Peer Review
Murray et al. were able to show for the call for full submissions of articles in a life science journal that acceptance rates were higher, the greater the degree of homogeneity between submitters and reviewers. For example, the chance of receiving a call for articles from US corresponding authors was 39.2 percent, compared to 29.3 percent and 12.6 percent for German and Chinese corresponding authors respectively.
Murray, D., Siler, K., Larivière, V., Chan, W. M., Collings, A. M., Raymond, J. & Sugimoto, C. R. (2019). Author-Reviewer Homophily in Peer Review.
Stereotypes in Letters of Recommendation
Turrentine et al. found for US letters of recommendation in the field of surgery that the adjectives used to describe applicants reflected gender stereotypes. Male researchers tended to be described as "exceptional", while female researchers were more frequently said to be "hard-working" or "delightful".
Turrentine, F. E., Dreisbach, C. N., St Ivany, A. R., Hanks, J. B., & Schroen, A. T. (2019).
Influence of gender on surgical residency applicants' recommendation letters. Journal of the American College of Surgeons, 228(4), 356-365.
Disadvantages in Applications
In their analysis of English-language studies, Lippens et al. found that people with disabilities, elderly individuals, and those with recognisable “ethnic characteristics” were significantly less likely to receive a positive response to their application, with the probability decreasing from 41 percent to 37 percent and to 31 percent.
Lippens, L., Vermeiren, S. & Baert, S. (2023). The state of hiring discrimination: A meta-analysis of (almost) all recent correspondence experiments. European Economic Review, 151 (2023) 104315.
"Motherhood Penalty" and "Maybe-Baby Effect"
In connection with part-time jobs in German-speaking countries, Becker et al. found that married women with young children had worse chances of being invited to job interviews than women with children who were older, for whom it was assumed that family planning had been completed. The interview invitation rates differ by about seven percentage points (20.7% versus 27.18%).
At the same time, married childless women considered to be more likely to become pregnant were disadvantaged compared to childless single women (interview invitation rates of 13.11% versus 18.77%).
On the one hand there is evidence of the so-called “motherhood penalty” (reduced employability is assumed for women who have children in need of care) – in this case for women with younger children. On the other hand, childless women of childbearing age are subject to the “maybe-baby effect” (a possible pregnancy is assumed).
Becker, S. O., Fernandes, A. & Weichselbaumer, D. (2019), Discrimination in Hiring Based on Potential and Realized Fertility: Evidence from a Large-Scale Field Experiment (2019). IZA Discussion Paper No. 12308.
Lutter, M. & Schröder, M. (2020). Is there a motherhood penalty in academia? The gendered effect of children on academic publications in German sociology. European Sociological Review, 36(3), 442-459.
Peterson Gloor, J. L., Okimoto, T. G. & King, E. B. (2022). "Maybe baby?" The employment risk of potential parenthood. Journal of Applied Social Psychology, 52(8), 623-642.
Success in Application Procedures as a Result of "Higher-Class Hobbies"
Rivera & Tilcsik showed for US job application procedures that people with hobbies which were supposedly associated with the “higher class” had significantly more success than those with hobbies associated with the “lower class”. For example, 16 percent of applicants who sailed, played polo or listened to classical music received an invitation to an interview. By contrast, only one percent of those with hobbies such as football, athletics and country music were called in for interview. Within the supposedly higher class group, it was found that men were rated better, while women were subject to negative stereotypes. Apart from the hobbies and sex of the applicant, the résumés used in the study were identical.
Rivera, L. A. & Tilcsik, A. (2016). Class Advantage, Commitment Penalty: The Gendered Effect of Social Class Signals in an Elite Labor Market, American Sociological Review 2016, Vol. 81(6) 1097–1131.
Gender Bias in Science and the Humanities
According to Roper’s review, recent studies continue to show the prevalence of gender bias in science and the humanities, and especially in medicine. This is said to negatively impact on the grading of female students, career advancement, hiring and tenure, the success of funding proposals, and the remuneration received by women in research. Sexual harassment was also identified as a significant barrier to women in research.
Roper, R.L. (2019). Does Gender Bias Still Affect Women in Science? Microbiology and Molecular Biology Reviews 83(3).
Bias in the Decision-Making Procedures of Research Funding Organisations
In its synthesis report on inequality in the allocation of third-party funding on the part of research funding organisations in Europe, the project GRANted concluded that, despite some positive developments, research careers clearly show gendered patterns in all countries studied. In the context of ERC funding, gender bias was also found to favour men more often than women in the life sciences, for example.
GRANteD 2022, D.5.1 Synthesis report on contextual factors, gender equality policy analysis and gender bias risk analysis.
Van den Besselaar, P. & Schiffbaenker, H. & Sandström, U. & Mom, Ch. (2018). Explaining gender bias in ERC grant selection – Life Sciences case.