Defining Health Campaigns and Health Campaign Effectiveness

WHO defines effectiveness as
…the extent to which the programme/initiative’s objectives were achieved or are expected to be achieved, taking into account their relative importance. Effectiveness is also used as an aggregate measure of (or judgement about) the merit of worth of an activity – i.e. the extent to which a programme has achieved, or is expected to achieve, its major relevant objectives and have a positive institutional impact (22).

Metrics to measure effectiveness vary across diseases, strategies, and countries but can be understood at the level of programmatic output (e.g., drug doses delivered), outcome (e.g., campaign coverage), and health impact (e.g., decline in the incidence of disease after the campaign) (1,17). Measures are the broad indicators of program performance, such as coverage or equity; whereas, a metric is a core indicator that quantifies a measure and is comparable between populations (18). Effectiveness measures are centered around programmatic outcomes or health impact:

Programmatic Outcomes

• Coverage—the proportion of the target population receiving the intervention (19)
• Equity—provision of high-quality interventions uniformly and in a fair and impartial manner to all target populations, including underserved groups (19)
• Access—the ease in reaching the health intervention in terms of location, time, and ease of approach (20)
• Efficiency—higher levels of performance (coverage, access, or equity) relative to the inputs (resources, time, money) (19). As an example, cost-effectiveness analysis is the ability to reach populations at the best outcome per dollar spent (1).
• Awareness—increases in awareness, community engagement, and/or change in behavior (21)

Health Impact Indicators

• Disease incidence—changes in the rate of individuals who develop a specific disease or experience a specific health-related event within a period of time (e.g., month or year) (22) 
• Morbidity and mortality related to the disease. These may be age-specific rates that relate to a specific time period (2).

The appropriate choice of these measures is determined by the 1) goals of the campaign, 2) available resources for data collection, and 3) political needs (1,23). Table 2 provides examples of the ways that health domains measure campaign effectiveness.

Across health domains, effectiveness is most often measured from the supply side. Coverage is the most frequently used measure of campaign effectiveness. However, utilizing coverage as the sole indicator of campaign effectiveness may not take into account the full complexity of campaign design, and valuable information might be overlooked that would account for contextual elements of a campaign that contribute to, hinder, or are extraneous to success. For example, many campaigns still leave coverage gaps. These gaps may be a result of poor data quality or inadequate monitoring systems due to weak health systems, or may be in relation to another measure of effectiveness not evaluated (1). For example, in 2017 only 31% of mass drug administration campaigns for NTDs reached their coverage targets (24). In studies of insecticide-treated bed net campaigns in Ethiopia, gaps in coverage were associated with both access and behavior change relating to use (25). Meanwhile, many polio campaigns have discordantly reported reaching more than 100% of their target population, demonstrating the difficulty in accurately assessing the numerator and/or the denominator (1). Numerators may be inflated by counting doses administered to persons who were not targeted (e.g., siblings outside of the target age group), and denominators may be either too high or too low because they are often based on outdated data that don’t take into account migration. Additionally, discrepancies may be due to inadequate recording, such as using the count of doses administered without recording which individuals received the intervention (e.g., using tally sheets to count doses delivered versus using home-based records) (26). These data quality issues can divert resources and draw attention away from disease programs, such as happened in Burkina Faso where a large measles outbreak occurred in 2009 despite vaccination coverage reported at greater than 95% (27). Data on coverage are often nationally aggregated, which limits a country’s ability to accurately monitor progress (28). This may mask subnational differences in campaign performance.

While coverage is the measure commonly used across all health domains, gaps that are not addressed by coverage indicators may be better understood through other measures of effectiveness. These include those related to the supply side such as equity, efficiency, and health worker acceptability, and to the demand side in terms of beneficiary acceptability, accessibility, awareness, and satisfaction. Measures to consider may include the perceived quality of the care provided, as well as other social determinants contributing to the access of an intervention including affordability, accessibility, and availability. These measures are important for determining systemic issues that hinder service delivery and thus contribute to lower coverage and equity (29). Additional measures that may be important for understanding the extent to which a campaign is or is not reaching coverage targets include awareness and acceptability, especially for campaigns that rely on a high level of behavior change or community engagement. This is influenced in part by perceived need by the target population, as individuals who do not think an intervention is needed are unlikely to partake in it, even if it is free (29). Community health workers may address some of these gaps by increasing the intervention’s acceptability by and access to the population, though more research is needed to determine if this is true in all campaign contexts (30). While the aforementioned issues are important to consider, this is not an exhaustive list of measures to be explored.

The complexity and context of a health campaign should be considered when selecting measures of effectiveness (31). To best understand what aspects of a campaign directly impact the outcomes, campaigns must identify “which elements belong to the intervention (and therefore participate in its effects and can be transferred), which ones belong to the context and interact with the former to influence results (and therefore must be taken into account when transferring the intervention) and which contextual elements are irrelevant to the intervention” (32). Because weak health systems often have inadequate monitoring systems leading to difficulties collecting reliable data, effectiveness measures for health interventions often lack comprehensive metrics (31).