We recently undertook a global thematic evaluation for UN Women. The evaluation process was conducted in accordance with the United Nations Evaluation Group Ethical Guidelines and Code of Conduct for Evaluation in the UN System. It was also based on gender and human rights principles.
To meet these standards and principles we applied human-centred design. This included rethinking ways of obtaining consent from evaluation participants – many of whom were in a position of powerlessness and/or might be expected to face challenges with literacy.
The most recognised standards for ethical conduct are derived from bioethics. These were codified in the Belmont Report (1979), which provides the principles of:
- maximising good and minimising risk,
- respect for participants autonomy, and
- justice, or fair distribution of risks and benefits.
The Belmont principles are derived from a utilitarian philosophy, which privileges individual autonomy. The practice of an individual giving their consent through a social contract is grounded in this worldview, along with the political-economic assumption that an individual will always act in their best interest. John Rawls’ maximin principle proposes that if it is to be considered morally fair, this social contract must maximise the position of the people who are least well-off. This is not easy to achieve, however, as the consideration of what is fair will always reflect the principles of justice that are imbued in culture of the person who has the power to take the decision.
As a result, the notion of Free, Prior and Informed Consent to take part in a preconceived project is liable to collapse a complex issue into a political technology that simply requires an optimal answer. Indeed, the very act of gaining written consent can compound power imbalances by projecting the legitimacy of a study and transferring the power of interpretation to the researcher. Privileging documentation can also undermine the traditional process for gaining trust in oral societies.
By contrast, Feminist and Afrocentric (decolonised) ethics emphasise our relationship with the Other and our relationship with society. These worldviews acknowledge human interdependency and the cogeneration of knowledge. In advocating for social justice, they highlight the need for fairer power relations.
Evaluators are “knowledge brokers, people who have the power to construct legitimating arguments for or against ideas, theories or practices.” (Cram et al 2004). The legitimising power of evaluators is derived from the application of scientific standards, which under the dominant western paradigm are considered fair (and thus ethical) because of their objectivity. Hence the importance placed on independence and economic language in evaluation quality standards.
Such standards preclude a relationship between the evaluator and the evaluated – heightening the risk of misrepresenting the Other. This has important consequences for how the legitimising criteria for success (effectiveness, efficiency, relevance) are defined. In reality, the meaning and value of these measures is contested between stakeholders, and is a negotiated outcome of a social process (Hedgecoe, 2004).
Mary Brydon-Miller (2009) proposes that a feminist approach to ethics should more appropriately be covenantal (grounded in trust) rather than contractual (grounded in mistrust). She also argues that participation of the least powerful in evaluations without compensation is a form of ‘scientific colonialism’ – extracting, exporting and commercialising a population’s data.
An ethical approach to any evaluation must therefore take into account the different identities and roles of the evaluators and programme staff as hosts. In addition to ensuring that instruments are culturally appropriate and compensation (including in kind) is appropriate, the evaluation must differentiate between the worldviews of people from different backgrounds and offer preferential options for the marginalised that can overcome the power difference between evaluator and evaluated.
The evaluation was, therefore, conducted using the following approaches:
- The data given to the evaluation team remained the property of the person giving it. All evaluation participants were provided with contact details so that they could request the removal of their data from the evaluation at any time. Whilst in safekeeping, all data was held in accordance with the UK Data Protection Act on secure password protected servers and computers that were only accessible to the evaluation team;
- The power of interpretation of individual stories remained with the person who provided the story. Evaluators asked contributors why they felt the story was important to them;
- Potential participants were extended an invitation to attend a meeting with the evaluation team to avoid a sense of obligation to participate;
- Before collecting any data, an explanation of the purpose and the intention of the evaluation team was given and explicit oral consent was sought. People who choose to participate were provided with two cards. One card had the contact details of the evaluation team. The other card had a smiley face…
- Participants were invited to submit the card with the smiley face to one of the evaluators to explicitly signal her consent for the data collected to be included in the evaluation; and
- At the end of each meeting, each participant was offered a small item of stationary in recognition of her time and contribution.
- United Nations Evaluation Group, UNEG Ethical Guidelines
- UNEG Code of Conduct for Evaluation in the UN system
- United Nations Evaluation Group, Norms and Standards for evaluation in the UN System
- Integrating Human Rights and Gender Equality in Evaluation – Towards UNEG Guidance