Project/Program evaluation and monitoring

Keston Technologies has extensive experience of the evaluation of projects from proposal through delivery to impact and program evaluation. This experience therefore spans the lifecycle of project/proposal development through assessment of proposals through monitoring of live projects to evaluation of programs and impact analyses.

Keston Technologies undertakes evaluation and monitoring activities for a range of government-based organisations, including the UK’s Innovate UK (formerly the Technology Strategy Board (TSB)), the European Commission (EC), the Czech Ministry of Education, Innovation Fund Denmark (IFD) and the Government of Western Australia, including:

  • The evaluation/assessment of proposals submitted to publicly-funded programs.
  • The monitoring and evaluation of publicly-funded projects and programs.
  • The development of strategic regional development activities and initiatives.

 

Proposal Evaluation

The evaluation or assessment of proposals is a key step in the implementation of publicly funded programs. Experience in evaluating proposals and business cases is also very useful for undertaking subsequent monitoring and evaluation exercises. The evaluation of proposals provides a valuable insight into the underlying objectives of a funding scheme and an understanding of the requirements for robust needs assessments and feasibility studies, informing well-argued business cases at the pre-commencement (proposal) stage. When it then comes to monitoring a funded project, this experience is valuable in identifying gaps in the baseline plans (project plans, exploitation plans, risk management, financial) that need to be addressed at an early stage in order to avoid potential problems later on.

 

Project Monitoring

In all of the monitoring and evaluation work that Keston Technologies undertakes, there are three guiding principles:

  • Objectivity – each project is evaluated as on an objective, “arms length” basis.
  • Accuracy – judgment is made against the official evaluation criteria, and nothing else.
  • Consistency – the same standard of judgment is applied to each project.

The scope of evaluation services is extremely broad and encompasses a wide range of parameters, including:

  • The size and complexity of the project.
  • The location of the project.
  • The stage of the project – from planning to delivery to completion.
  • The requirements of the evaluation – from monitoring compliance to mentoring to assessment of impacts and benefits.
  • Evaluation of a single project to a portfolio of projects to the high level program.

Effective project monitoring enables the funding agency to assess each project’s progress, identify and address problems, and reassess the project’s relevance and priority. Project monitoring requires sound management structures to ensure all projects are appropriately monitored, and key dimensions considered. Effective project monitoring requires:

  • Appropriate management processes and program review cycles.
  • Accurate and accessible information on project progress against milestones.
  • Astructured approach to monitoring and project review.

This will allow the funding agency to:

  • Identify problems and take corrective action in a timely way.
  • Provide assurance that projects are proceeding appropriately.
  • Support project managers in resolving any difficult issues.
  • Intervene to change project direction, if required.

 

Evaluation of Project Portfolios

Evaluation of projects across a particular program usually requires a portfolio approach to the analysis of the project reviews and the projects overall. The portfolio analysis will need to be tailored to the specific requirements/objectives of the group of projects or the program to which they relate.

 

Evaluation of Programs

Separate to monitoring or evaluation of projects and project portfolios is the evaluation of entire programs. A useful working definition of monitoring/evaluation as:

“An objective and considered assessment of the efficiency, effectiveness, and/or appropriateness of the program within its strategic context.”

 

Evaluation of efficiency will seek to answer the questions:

  • How efficiently are the inputs being converted into outputs?
  • How do the costs and benefits compare?
  • How has the program concept evolved, such as operational structures and strategies and the evolution of priorities and activities, and has this improved efficiency?

Evaluation of effectiveness will seek to answer the following questions:

  • Do the outcomes of the program meet the program objectives?
  • Does the program actually achieve stakeholder satisfaction?
  • Does the program make an effective contribution to state priorities and policies?

Evaluation of appropriateness will seek to answer the following questions:

  • How appropriate are the program objectives?
  • Do the program objectives accurately address stakeholder needs?
  • Are higher level state priorities addressed by the program? Examples would be the role of the program in emerging policy initiatives.

 

Typical evaluation methodology:

 

Keston technologies has undertaken a range of monitoring activities for research and technology development projects, both as individual projects and at the portfolio level, including acting as Monitoring Officer for the UK Government for seven Micro and Nano-Technologies (MNT) Capital Facility projects – multi-year large projects covering R&D and technical facilities in the areas of materials and nano-materials and metrology (since 2007). Relevant assignments in recent years include:

  • (2011 - ): Monitoring officer for the UK Government’s Innovate UK (formerly the Technology Strategy Board (TSB)) for industrial and collaborative R&D projects.
  • (2016-): Monitor for the European Commission (EC) for projects funded under Horizon 2020.
  • (2012- 2016): Project Technical Advisor (PTA) for the European Commission (EC) on eight multi-national, multi-partner collaborative R&D projects.
  • (2007 – 2012): Monitoring Officer for the UK Government’s Technology Strategy Board (TSB) for seven Micro and Nano-Technologies Capital Facility projects – five year large projects covering R&D and technical facilities in the areas of materials and nano-materials and metrology.
  • (2013): international expert for the evaluation of the project “Centre for Research and Utilization of Renewable Energy - CVVOZE” on behalf of the Czech Ministry of Education, Czech Republic.

 

 Click here to see our recent projects in this service area.