Mitigating Bias in Scientific Evaluation and Decision-Making Processes

Further Information

Equity and Diversity Applicants / Funding Recipients Reviewers / Committee Members

No person is to be prevented or excluded from pursuing an academic career because of non-scientific factors.

In order to ensure the best research projects receive funding, it is important to make the most of all the potential and talent available and to arrive at funding decisions in a fact-based and non-discriminatory manner. For this reason, evaluation and decision-making processes such as when assessing funding proposals submitted to the DFG must be based solely on scientific criteria. At the same time, unavoidable, exceptional personal circumstances – where they apply – are to be taken into account exclusively to the benefit of the researcher.

Both positive and negative bias can potentially come into play in the process, for example with regard to sex/gender, origin, health status, family responsibilities or an applicant’s research institution. Every individual is biased in different ways and to differing degrees. This also applies to research and to researchers who strive for objectivity and evidence. In the course of the review, evaluation and decision-making process through which proposals submitted to the DFG pass, the stakeholders also receive information about the applicant.

In order to ensure that assessment is based purely on scientific criteria, a short film is provided along with recommendations for action and more background material. The following notes and information offer insights into the above-mentioned complex processes and can be used as a basis for reflection:

DFG form 10.60 - Guidelines on Mitigating Bias in Scientific Evaluation and Decision-Making Processes with concrete recommendations for action on p. 2

DFG film "Unbiased Review"

Note: The film is also available in an accessible version with a spoken audio description of the visual content

To the accessible version of the film with a spoken audio description

General Framework Conditions Established by the DFG to Mitigate Bias

Transparent procedures and the observance of predefined standards help reduce bias. In particular, the defined assessment criteria are to be applied equally to all individuals involved.

The DFG attaches particular importance to ensuring that its processes leading up to funding decisions do not discriminate based on non-scientific factors. Accordingly, the assessment of project proposals and research achievements is to be science-led. When conducting a review, the individual life situation (where applicable) is appropriately taken into account, and this is done exclusively in the applicant’s favour: unavoidable periods of absence or periods of restricted research activity should not impact negatively on the review, evaluation or decision. Applicants are advised to notify the DFG of special personal circumstances.

Details provided in Connection with Special Personal Circumstances

The DFG’s procedural steps of review, evaluation and decision are designed to be carried out separately and independently of each other: this in itself ensures that the review and evaluation undertaken by the DFG’s various statutory bodies is subject to quality assurance.

Before a decision is made, the DFG Head Office also carries out a quality assurance of approvals and rejections so as to ensure procedures have been followed correctly. If bias or discrimination is identified here, the appropriate action can be taken.

Decision making process

The general instructions and the specific instructions for the review process lay down the assessment criteria for project proposals; they also emphasise the confidential nature of the proposal documents to be reviewed and describe how to avoid the use of non-scientific criteria. Votes are to be based on the content of the proposal documents, the curriculum vitae, and also – in the case of on-site and panel reviews – on the presentation of the applicants and their qualifications for the project.

There is also a set procedure to rule out conflicts of interest. The Framework Rules of Procedure for Review Boards set down uniform working principles for each review board.

The categories of the DFG’s CV template clearly convey that research achievements in their entire breadth (without reference to metrics and without a photograph) are relevant to the review and that an applicant’s individual life situation should also be taken into account in their favour where relevant.

Good research practice includes being aware of and dealing with bias. Any potential factors that may give rise to the impression of conflicts of interest must be reported to the relevant department of the DFG Head Office (see guideline "Guidelines for Avoiding Conflicts of Interest").

The DFG’s philosophy states the following: “The DFG’s core task is to competitively select the best research projects undertaken by researchers at universities and research institutes and provide funding for them”. The votes obtained on the proposals are evaluated “exclusively according to scientific criteria” by reviewers working in an honorary capacity. But how does the DFG Head Office proceed when non-scientific criteria are mentioned in externally obtained reviews and these are included in the evaluation?

Further Information

Researchers in the German research system ultimately need to adhere to the DFG Code of Conduct “Guidelines for Safeguarding Good Research Practice”. The guidelines set out standards for research work and research integrity addressed at the entire German research system.

Mitigating bias is central to the Code of Conduct as an expression of good research practice and is relevant in different contexts.

For research work units, processes need to be implemented that help to avoid “implicit bias” (Guideline 3). In general, precautions must be taken to mitigate bias (examples can be found in the film and Guidelines on Mitigating Bias in Scientific Evaluation and Decision-Making Processes above on this web page).

In the context of the review of funding proposals, universally valid regulations for avoiding conflicts of interest ensure the neutrality of reviewers, which is central to a fair review process (Guideline 16). Any (appearance of a) conflict of interest must be disclosed. The avoidance of conflicts of interest on the part of reviewers and committee members is ensured on an individual basis by the Rules of Procedure for Dealing with Scientific Misconduct. Any failure to disclose circumstances that could give rise to the appearance of a conflict of interest can be sanctioned as scientific misconduct.

Furthermore, assessing performance adequately requires drawing solely on an appropriate scope of objective assessment criteria that are adequate for scientific purposes, in addition to considering individual career paths and personal circumstances (Guideline 5). .

Studies on Bias in Science and the Humanities and in Evaluation and Decision-Making Processes

There are numerous studies on the effects of prejudice in review, evaluation and decision-making processes as well as other areas – such as personnel selection. Examples of such studies are presented below.

To the Studies