Monitoring and Evaluation
Overview
The main health sector reforms over the last two decades in the African Region have centered on i) broadening financing options for the health sector e.g.introduction of user fees in the public sector, community financing mechanisms ii) organization and management of health services e.g.granting autonomy to national and regional referral hospitals, promotion of public/private partnerships.Social insurance has been implemented in fewer countries e.g. in Kenya and Nigeria.
- The WHO AFRO Regional Committee resolution AFR/RC49/R1 provides for the development of a one - stop, comprehensive monitoring and evaluation framework for health sector reforms in the region.
Objectives
- To provide a common conceptual framework for the analysis of health sector reforms in the region
- To provide a generic framework for identifying key indicators for monitoring and evaluating health sector reforms in the region
Purpose
- These guidelines are principally intended for national professional officers, policymakers and planners in Ministries of Health at both national and sub - national level in the African region.Other evaluators and researchers in collaborating NGO and academic institutions may find them of use in monitoring and evaluation of health sector reforms.
INDICATORS FOR MONITORING AND EVALUATING HEALTH SECTOR REFORMS
Monitoring and Evaluation of health sector reform
- Monitoring is a continuous process intended to ensure that an activity occurs as planned.Therefore monitoring a health sector reform is about as assessment of the level of inputs(financial, human and logistics), the dynamics, processes and the outputs of implementing the reform.
- Evaluation assesses the extent to which an activity has achieved the outcome and impact objectives.Therefore evaluating a health sector reform is about assessing the extent to which the reforms have helped to improve levels of equity, effectiveness, quality, sustainability.
Monitoring components
Dynamics – monitoring the dynamics of a health sector reform is related to analyzing the context and design and how they relate to inputs, processes and outputs.It therefore maps the different stakeholders, the roles they played and the mandates that they had, including the potential to(re -)direct progress of the reform.
Input indicators assess whether the prerequisite resources are being supplied to effect the change e.g. in granting autonomy to hospitals one might like to see if budgets meet the creation of the new management level
Process indicators determine if the ‘control knob’ is moving in the right direction in terms of timing of inputs, participation of the appropriate stakeholders.
Output indicators determine whether a process was accomplished.
Evaluation components
Indicators for evaluating a health sector reform will include both outcome and impact indicators:
The evaluation of a health sector reform establishes trends in equity, efficiency and quality of health service delivery or population health that can be attributed to the changes by reform implementation.
Source of data and frequency of assessment
Data for both monitoring and evaluating a health sector reform may be obtained from a routine source such as the health information systems.Data may also be obtained from regular national household, demographic or census surveys.Some aspects of monitoring and evaluation may require special studies.In principle, special studies should be organized only after one has fully explored the utility of available information.
The frequency of monitoring and evaluating reform will to a large extent depend on the availability of data.Usually for monitoring, if the routine data sources can produce information at least on a quarterly basis, this should be sufficient to inform the assessment process. Evaluation is suited for measurement of the long term effect of a reform so information from biennial household surveys and demographic household surveys which take place almost every five years should meet the information needs. Special studies are accordingly arranged if the content of the survey is insufficient to meet evaluation needs.
Data Quality
In many countries, existing data are often incomplete or even inaccurate. Even when adequate and relevant data exist, these merit intense scrutiny.Attention must be paid to factors that might affect the validity and reliability of data for each of the selected indicators.
Evaluative questions and indicators for health sector reform
Countries already have in place monitoring and evaluation tools for performance of the health sector that in many cases include indicators that are relevant to assessment of health sector reforms.This table serves as an illustration for the selection of evaluative questions and indicators in monitoring and evaluating health reforms.Countries can therefore adapt their own to this framework or identify new evaluative questions and indicators depending on the context.Examples are taken from “Decentralization of health services to district / sub - national levels” but the framework should assist in the identification of the key indicators for monitoring and evaluating any of the other reforms.Establishing the rationale for the evaluative questions and indicators will guide in their selection and definition.
BUILDING CAPACITY FOR MONITORING AND EVALUATION OF HEALTH SECTOR REFORM
This guideline for monitoring and evaluation assumes the following capacities will be available to the teams undertaking the task:
Human resource skills
- Health informatics – technical, organizational, and managerial skills related to sound processes of data gathering, analysis, storage, dissemination(appropriate package and communication) and use.
- Research Methods
- Applied epidemiology
- Management and leadershi
- Economic evaluation
- Policy and planning
The training to be conducted to improve the capacity for monitoring and evaluation of health sector reform will depend on the human resource training needs assessment for these skills. Capacity building for improving the monitoring and evaluation of health sector reform is not limited to training alone and should include other aspects such as:
