The importance of indicators

Last year a story in the Guardian emphasised the importance of being sensitive to how we collect monitoring data – and the impact of receiving a “would you recommend our service” SMS on a couple who had just lost their baby.

More recently, Maria and I had our own experience that emphasised the importance of the indicators that we choose. It started in very much the same way as the story in the Guardian, with the loss of our baby in a late miscarriage in November 2014.

As a result of this, our subsequent pregnancy has been heavily monitored under the direct supervision of a specialist consultant in fetal medicine. This supervision included the decision on whether Maria was to give birth through an elective caesarian or naturally.

Given that her first birthing experience was an emergency caesarian, and second was to ‘complete the miscarriage’ (to use the terminology), her strong preference was for an elective caesarian. At least once, she wanted to determine the circumstances of her birth.

However, securing an elective caesarian ended up being a long fight with the health system. A fight that was, ultimately, driven by the indicators we choose to measure.

Whilst the European Court of Justice established that women have a right to determine the circumstances in which they give birth, their is no explicit right to a caesarian birth. The (non-binding) guidance to the medical profession is that a woman’s preference should be respected unless it is in the medical interest of the woman and the baby to do otherwise.

Our local hospital is one that has a policy of strongly preferring natural births as a follow-on from a first caesarian birth. Natural birth after a first caesarian is associated with small but significant reductions in maternal morbidity among the general population.

We experienced this policy first hand. After expressing her preference for an elective caesarian, Maria was forcefully lobbied by the fetal medicine team to change her preference to attempt a natural birth. This created an enormous level of anxiety in a pregnancy that was already riddled with stress because of our previous loss.

We soon discovered that Maria’s experience was not unique, and came across multiple stories of women who were persuaded to abandon their choice for an elective caesarian, and to attempt a natural birth. Among our unrepresentative sample, at least 80% of the women seemed to regret this decision and resent the experience of the birth that they had. The numerous online forums speak of a similar trend.

It was puzzling to us that such an outcome should be the intentional policy of a group of highly expert and dedicated professionals. As we learned more, however, it became apparent why: indicators.

The indicators currently used in the medical debate on post-caesarian birth include:

  1. whether a natural birth was ‘successful’ or not. Success is defined simply as whether the baby came out the birth canal, and nothing more;
  2. whether the mother or baby died;
  3. whether the mother or baby suffered morbidity;
  4. whether the mother had post natal depression; and
  5. some long term health effects.

We found that data on these indicators leads to a logical recommendation by the medical profession: to attempt a natural birth after caesarian. Rates of mortality and depression are the same for both groups, morbidity is lower for natural birth, and there is some evidence that there are long term health benefits to the child from a natural birth.

So what is missing? Why did we come across so many unhappy women?

There is no patient-centred indicator. In other words, nobody measures women’s own perspective on their birthing experience, and whether they were happy with it or not. Whilst depression is measured, there exists a large and invisible spectrum of experiences between happy and clinically depressed.

Women’s own perceptions of their experiences are not counted. As a result, birthing plans that seem to be in their own best interest are not experienced this way. And, in arriving at those birthing plans, many women felt their preferences and the value of their opinions counted for very little.

When we dug into this issue more with our medical team, we found they acknowledged that, in their qualitative experience, women tend to be happier about their birth when they get the plan they want. Without an indicator to measure it, however, it does not figure in the hospital’s policy.

The whole experience brought home how important indicators can be – both the ones we choose and the one’s that we don’t.

Very often we hear complaints that international development management is not ‘results-based’ enough: that management decisions are not based on performance indicators. Perhaps, however, this looseness is also a form of protection. In my practice, I have rarely come across monitoring and evaluation frameworks with intentional, precise and comprehensive sets of indicators that I would feel confident in being the only data used to make management decisions. Yet, ultimately, this is exactly what results-based management is.

Our personal experience reflects what happens, and what some of the unintended consequences are, when a highly effective professional team does make decisions based on indicators. Fortunately, we fought to have Maria’s preference recognised, she had a positive experience of elective caesarian, and we now have a healthy baby boy in our family. But, I will never again make light of the outcome indicators we choose.

Comments(1)

  • 20th October 2015, 4:47 pm  Reply

    I totally agree with the lesson that participant reaction should be taken into account in evaluation and indicators of this must be included in a proper M&E design. Another indicator that was not included is cost. from a public health perspective a procedure that is lower cost and yields the same or better outcomes, is preferable.

Leave a Comment