| Accountability | responsibility for the use of resources and the decisions made, as well as the obligation to demonstrate that work has been done in compliance with agreed-upon rules and standards and to report fairly and accurately on performance results. |
| Activity | actions taken to produce specific outputs from inputs such as funds, technical assistance, and other resources. |
| Adequacy Evaluation | measures how well program activities have met the expected objectives, but does not causally link program activities to observed changes. Adequacy evaluation often can be carried out by cross-sectional, one-time surveys among beneficiaries. |
| Assumptions | hypotheses about factors that could affect the progress or success of an intervention. Achieving results depends on whether or not the assumptions made prove to be true. Incorrect assumptions at any stage can become an obstacle to the validity of the expected result or achieving it. |
| Attribution | the causal link of one event with another, or the ascription of a causal link between observed changes and a specific intervention. |
| Baseline | the status of services and outcome-related measures, such as knowledge, attitudes, norms, behaviors, and conditions before intervention. |
| Benchmark | a reference point or standard against which progress or achievements can be assessed. A benchmark refers to the performance that has been achieved in the recent past by other comparable organizations, or what can be reasonably inferred to have been achieved in similar circumstances. |
| Capacity | the knowledge, organization, and resources needed to perform a function. |
| Case Study | a methodological approach that describes a situation, individual, or the like and that typically incorporates the data-gathering activities (e.g., interviews, observations, questionnaires) at selected sites or programs. Case studies are characterized by purposive selection of sites, or small samples, and the expectation of generalizability is less than that in many other forms of research. The findings are used to report to stakeholders, make recommendations for program improvement, and share lessons with other countries. |
| Conclusion | a sound judgment deducted from empirical findings or factual statements corresponding to a specific circumstance. |
| Country response information system (CRIS) | an information system for monitoring and evaluating national responses to HIV/AIDS. CRIS includes integrated indicator, project/resources tracking, and research modules. It facilitates the development of a clearinghouse for indicator data to enable indicator exchange between the United Nations and other partner applications. |
| Coverage | the extent to which a program reaches its intended target population, institution, or geographical area. |
| Data | specific quantitative and qualitative information or facts that are collected. |
| Effectiveness | the extent to which a program or project has achieved its objectives under normal conditions in a field setting. |
| Efficacy | the extent to which an intervention produces the expected results under ideal implementation conditions in a controlled environment. |
| Efficiency | a measure of how well inputs (resources such as funds, expertise, and time) are converted into outputs. This term is also used more specifically in economic evaluation to mean the cost value of producing a given product or service. |
| Epidemic | an infectious disease’s rapid spread through a demographic segment of a population. In the context of HIV, a generalized epidemic is characterized by an HIV prevalence higher than 1 percent in the total population; in a concentrated epidemic, the HIV prevalence is higher than 5 percent in any subpopulation at higher risk of HIV infection, but less than 1 percent in the total population. |
| Epidemiology | the study of how often diseases occur in different groups of people and why. Epidemiological information is used to plan and evaluate strategies to prevent illness. |
| Evaluability assessment | a study to determine whether or not a program or project can be evaluated. |
| Evaluation | the systematic collection of information about program activities, characteristics, and outcomes that determines the merit or worth of a specific program. Evaluation studies provide credible information for use in improving programs, identifying lessons learned, and informing decisions about future resource allocation. An evaluation can use a quantitative approach (e.g., structured or standardized approaches for collecting numeric or categorical data, such as surveys, questionnaires, and checklists, using experimental or quasi-experimental design), a qualitative approach (e.g., semistructured data collection, such as interviews, focus groups, and observation), or a mix of both approaches. |
| Exploratory Study | a preliminary study to provide information on the topic of the intervention to understand the problem better. |
| Facility survey | a survey of a representative sample of facilities that generally aims to assess the readiness of all elements required to provide services and other aspects of quality of care (e.g., basic infrastructure, drugs, equipment, test kits, registers, staff trained in the delivery of services). The units of observation are facilities of various types and levels in the same health system. The exact content of the survey can vary, but it typically includes a facility inventory and, sometimes, health worker interviews, client exit interviews, and client-provider observation. Depending on the objective of the survey, both public and private facilities may be included in the sample frame of sites; the Service Provision Assessment is one example. The term “health facility assessment” is sometimes used as a broader term than “facility survey.” A health facility assessment includes facility surveys, but it also includes facility censuses, such as the World Health Organization’s Service Availability Mapping. |
| Feasability | the coherence and quality of a program or project strategy that makes successful implementation likely. |
| Formative evaluation | an evaluation intended to improve the performance of a program or project. A formative evaluation is usually undertaken during the design and pretesting of the project or program, but it can also be conducted early in the implementation phase, particularly if implementation activities are not going as expected. |
| Generalizability | the extent to which findings can be assumed to be true for the entire target population, not just the sample. To ensure generalizability, the sample procedure and the data need to meet certain methodological standards. |
| Goals | the higher order aims of the program or project, to which the intervention is intended to contribute. |
| Health information systems (HIS) | a data system, usually computerized, that routinely collects and reports information about the delivery of services, costs, demographic and health information, and results status. The terms “routine health information systems” (RHIS) and “health management information systems” (HMIS) are also sometimes used. |
| Impact | the longer range, cumulative effect of programs over time on what they ultimately aim to change. Often, this effect will be a population-level health outcome, such as a change in HIV infection, morbidity, and mortality. Impacts are rarely, if ever, attributable to a single program, but a program may, with other programs, contribute to impacts on a population. Impact can also be used in the context of a specific program. In this case, it implies a much closer link to attribution of the program and a conceptual model underlying it. |
| Impact evaluation | a scientifically rigorous methodology to establish a causal association between programs and what they aimed to achieve beyond the outcomes on individuals targeted by the program(s). Impact evaluation looks at the rise and fall of impacts, such as disease incidence and prevalence or quality of life as a function of HIV/AIDS programs. The effects (impacts) on the entire populations seldom can be attributed to a single program or even several programs; therefore evaluations of impact on populations usually entail an evaluation design that includes the combined effects of a number of programs for at-risk populations. |
| Impact monitoring | in the field of public health, a process that is usually referred to as “disease surveillance” (defined above) and is concerned with the monitoring of disease prevalence or incidence. With this type of monitoring, data are collected at the jurisdictional, regional, and national levels.
|
| Incidence | the number of new cases of a disease that occur in a specified population during a specified time period. |
| Indicator | a quantitative or qualitative variable that provides simple and reliable means to measure achievement, monitor performance, or to reflect changes connected to an intervention. |
| Input | a resource used in a program, including monetary and personnel resources from a variety of sources, as well as curricula and materials. |
| Inputs and outputs monitoring | the basic tracking of information about program inputs, or resources that go into a program, and about outputs of the program activities. Data sources for monitoring inputs and outputs usually exist in program documentation (e.g., activity reports, logs) and client records, which offer details about the time, place, and amount of services delivered, as well as the types of clients receiving services.
|
| Internal evaluation | an evaluation of the intervention conducted by a unit reporting to the donors, partners, and/or implementing organization. |
| Intervention | a specific activity (or set of activities) intended to bring about change in some aspect of the status of the target population (e.g., HIV risk reduction, improving the quality of services) using a common strategy. An intervention has distinct process and outcome objectives and a protocol outlining the steps of the intervention.
|
| Joint evaluation | an evaluation of program or project where different partners or donors participate. |
| Lessons learned | learning from experience that is applicable to a generic situation, not just to a specific situation. Generalizations are based on evaluation experiences from programs, projects or policies. |
| Meta-evaluation | an evaluation that aggregates findings from a series of evaluations. A meta-evaluation can also be an evaluation of an evaluation to assess the performance of the evaluators.
|
| Monitoring | routine tracking and reporting of priority information about a program and its intended outputs and outcomes. |
| Monitoring and evaluation (M&E) plan | a comprehensive planning document for all M&E activities. An M&E plan documents the key M&E questions to be addressed, including what indicators are collected; how, how often, from where, and why they will be collected; what baselines, targets, and assumptions will be included; how the indicators are going to be analyzed or interpreted; and how or how often reports will be developed and distributed on these indicators. |
| Objective | a statement of desired program results. A good objective meets the criteria of being specific, measurable, achievable, realistic, and time based (SMART). |
| Operational research | the application of systematic research and evaluation techniques to improve programs and service delivery. This application analyzes only factors that are under the control of program managers, including indicators of program success, such as improving the quality of services, increasing training and supervision of staff members, and adding new service components. It is designed to assess the accessibility, availability, quality, and sustainability of programs. |
| Outcome | the changes that a program aims to effect on target audiences or populations, such as change in knowledge, attitudes, beliefs, skills, behaviors, access to services, policies, and environmental conditions.
|
| Outcome evaluation | a type of evaluation that is concerned with determining if, and by how much, program activities or services achieved their intended outcomes among the targeted population. Whereas outcome monitoring is helpful and necessary in knowing whether outcomes were attained, outcome evaluation attempts to attribute observed changes among the targeted population to the intervention tested, describe the extent or scope of program outcomes, and indicate what might happen in the absence of the program. An outcome evaluation is methodologically rigorous and generally requires a comparative element in design, such as a control or comparison group, although it is possible to use statistical techniques in some instances when control groups are not available (e.g., for a national program). |
| Outcome monitoring | the basic tracking of variables that have been adopted as measures or “indicators” of the desired program outcomes. Outcome monitoring does not infer causality; changes in outcome could be attributable to multiple factors, not just the program. With national AIDS programs, outcome monitoring is typically conducted through population-based surveys (representative of the target population, not necessarily the general population) to track whether desired outcomes have been reached; it may also track information directly related to program clients, such as change in knowledge, attitudes, beliefs, skills, behaviors, access to services, policies, and environmental conditions. |
| Outputs | the results of program activities. This term relates to the direct products or deliverables of program activities, such as the number of counseling sessions completed, the number of people reached, and the number of materials distributed. |
| Performance | the degree to which an intervention operates according to specific criteria, standards, or guidelines, or achieves results in accordance with stated plans. |
| Plausibility evaluation | a way to demonstrate with a certain level of certainty that impact is due to an intervention program. Plausibility evaluation includes the use of control groups and requires baseline and postintervention statistics, as well as multivariate analyses.
|
| Population-based surveys | large-scale national health surveys, such as Demographic and Health Surveys. Population-based surveys are statistically representative of their target populations. Usually, surveys that are population based imply representation of the general population of a given age and sex in a given geographic area, but they do not have to be national in scope or even of a large scale. National surveys can also be conducted in such a way so that they are not population based. |