What is monitoring and evaluation?
The online platform Sportanddev.org (2013) defines monitoring as the systematic and routine collection of information from projects and programmes for four main purposes:
- To learn from experiences to improve practices and activities in the future;
- To have internal and external accountability of the resources used and the results obtained;
- To take informed decisions on the future of the programme;
- To promote empowerment of beneficiaries of the programme.
Monitoring is defined as a periodically recurring task which begins in the planning stage of a programme. Monitoring allows results, processes and experiences to be documented and used as a basis to steer decision-making and learning processes. Monitoring is checking progress against plans. The data acquired through monitoring is used for evaluation.
Evaluation is systematically assessing a
completed or phase of an ongoing programme. Evaluations appraise the collected data and information, informing strategic decisions and thus improving the programme and outcomes in the future. Evaluation is made up of a number of progressive steps, the most important of which is the collection of appropriate data that is subsequently used to make a judgment about the value of a programme (Dugdill & Stratton, 2007).
Sportanddev.org (2013) states an evaluation should help to draw conclusions about five main aspects of the programme:
- Relevance of the programme to predetermined goals.
- Effectiveness of the programme.
- Efficiency of the implementation of the programme.
- Impact of the programme.
- Sustainability of running the programme.
Information gathered during the monitoring process provides the basis for the evaluation. The evaluation process is an analysis of the collected information which looks at the relationships between the results, the effects and overall impact of the programme (“Sportanddev.org”, 2013).
The World Health Organisation recommends that between 10-20% of the total intervention costs should be spent on evaluation (Dugdill & Stratton, 2007). However, evaluation can be seen as problematic, time consuming and can take lower priority compared to delivery of the programme (Stratton et al., 2005).
A recent review (Sport England, 2017) assessed the evidence to date supporting the impact of sports programmes on 5 key areas:
- Physical Wellbeing
- Mental Wellbeing
- Individual development
- Social and community development
- Economic development
The review outlined the evidence supporting the positive benefits of sports and physical activity programmes in each of these areas, including:
Good evidence for the prevention of illness, increased therapeutic and management effects in rehabilitation, improvements in strength, balance, gait and motor skills, and maintaining a healthy weight. Other outcomes include improved sleep, increased energy, healthy early year’s development, reduced engagement in risk behaviours such as smoking, reduced mortality, effective pain management and improved quality of life in ageing.
There is a strong association between taking part in sport and physical activity and positive mental wellbeing outcomes, but the causal mechanisms are less well understood. There are challenges around the varied definitions used in the field, and the subjective nature of measures. There is much evidence that sport and physical activity contributes to enjoyment, happiness, and life satisfaction. Social interaction appears to be central to this. Self-esteem and confidence can increase through the opportunity to develop new skills and relationships. There is potential for sports and physical activity programmes to reduce the symptoms of anxiety and depression in participants. Other outcomes include improved cognitive functioning, benefits for people with dementia, and impacts around emotion regulation.
To date substantial evidence indicates the potential for positive outcomes from taking part or volunteering, particularly for young people, but the wider set of circumstances around an individual will determine effectiveness in relation to these outcomes. Evidence was identified for improved educational attainment, either directly (improved grades and behaviour) or indirectly (enhanced skills like concentration and teamwork).
There are positive impacts on employability (employment opportunities, earnings, job performance and satisfaction), including (limited) evidence for younger people ‘not in education, employment or training’ (NEETS). Sport can promote self-efficacy (motivation and commitment), for groups including elderly people and disaffected young people. Other outcomes are an increased willingness to volunteer and the development of soft skills (such as integrity, responsibility and leadership).
Social and Community Development
Outcomes can be defined in relation to a range of concepts that are often challenging to evidence (such as social capital, trust and networks). Many of which are positively associated with participation or volunteering, but the effectiveness of any intervention is dependent on a broader set of conditions. Sport acts as a conduit for people of different backgrounds to interact, can bridge divides between groups such as men and women and people with different employment backgrounds, and play a key role in the integration of migrants. A small body of literature on bonding capital suggests that sport helps to build relationships within communities. For volunteers, motivations and outcomes overlap at the personal and the community level (for example, bonding with others increases a sense of community and citizenship).
The sources reviewed use a wide variety of techniques to calculate economic value and it was not possible to accurately assess the strength of the evidence base without a critically appraising the full range of these methods.
There was some evidence on the direct impact of the sport sector on the economy, largely in terms of gross value added and job creation. There was more evidence on the indirect impacts, including reduced healthcare costs due to a healthier population, reduced crime and improved employability.
This review concluded that:
The evidence base is strongest for the physical and mental well-being outcomes, then the individual development outcome. It is weaker for
the social and community development and economic development outcomes.
More longitudinal studies could help strengthen the evidence base by identifying the longer-term effects on mental well-being, individual development and social and community development.
The main focus of this review was examining
the evidence of the impact of sports programmes, rather than how to design programmes to effectively achieve outcomes. A number of other working groups have focused on developing a framework for identifying, monitoring and evaluating outcomes of sports programmes, including The Sport for Development Collation (2013).