Keston Technologies Services

Monitoring and Evaluation of Projects and Programs

Recent examples include: 

 

Independent Review of the Pilot Technology Developer Accelerator Programme (2015-2016).

The Advanced Propulsion Centre (APC) developed and implemented the Technology Developer Accelerator Programme (TDAP), a pilot programme aimed at small and medium sized technology developers (TDs) with low carbon propulsion technologies that can be applied to the automotive sector. Working with Beta Technology, Keston Technologies undertook an independent review of the TDAP programme, including the economic impact of TDAP along with opinion on the design, implementation and success of the programme.

 

The objective of the review was to assess the value of the TDAP programme to the TDs and evaluate how the programme has benefitted overall economic performance of the TDs, and ultimately provided a return on the programme intervention (investment). The review was undertaken through collecting and collating evidence of the effectiveness of the interventions. This included quantitative results, such as leverage of research funding and jobs created/safeguarded, demonstrating accelerated routes to the adoption of new technologies and value added. The review also included qualitative results, capturing an opinion on the programme design and implementation.

 

Monitoring of publicly-funded innovation development projects in the UK (2012 – on-going)

On behalf of the UK Government’s Innovate UK (formerly the Technology Strategy Board (TSB)), Richard Ball currently acts as a monitoring officer for publicly-funded innovation projects in the UK. The projects last between 9 months and 2 years and range in size and complexity. This work requires project visits on a quarterly basis, with reviews undertaken against a set of criteria from which judgements against progress and compliance can be made. This also involves providing advice and support to the projects and escalating issues that require Innovate UK resolution. The key aspect to these monitoring activities is establishing a solid baseline at the beginning of the project

 

Project Technical Advisor (PTA) for the European Commission on multi-partner collaborative R&D projects in various technical areas (2012 – on-going)

On behalf of the European Commission, and through Beta Technology, Richard Ball currently acts as a project monitor for six European-funded projects. These are large, multi-partner, multi-national projects lasting between three and four years and with budgets ranging from $5m to $15m each. The responsibilities of this activity is explained in more detail below. Most of the activity is undertaken remotely but does entail meetings with the project consortia on a six monthly basis.

 

Development of the PTA monitoring framework for the European Commission (2011)

Working with Beta Technology, Keston Technologies assisted in the development of the framework through which a large number of European funded projects are monitoring and evaluated. The requirement was to provide a framework to assist directorate RTD.G’s to monitor the progress on projects funded in Theme 4 – Nanosciences, Nanotechnologies, Materials and new Production Technologies - of the specific programme “Co-operation” within the 7thFramework Programme of the European Community (EC). The framework would ensure that the participants in the projects perform correctly their activities to the best possible program requirements.

In summary, the framework is now operating to:

  • Provide advice to the EC Project Officers, including assistance with contract preparation or amendment and ad hoc advice;
  • Monitor the progress of the projects on the basis of progress reports, on-site visits, and project meetings; pro-active follow-up of the work. This is achieved through regular contact with the project’s coordinator and the project participants;
  • Assess periodic and final reports of the projects, and make recommendations for contract related decisions to be taken by the Commission services. This could require the participation in reviewing panels or in technical audits;
  • Propose workable solutions to any contractual and/or managerial matters, as they arise, ensuring that administrative and financial reporting for all assigned projects is maintained on schedule. Also, as required, facing crisis management related to projects implementation;
  • Develop links between projects of a similar nature. This could involve activities with clusters of projects;
  • Provide assistance to projects in the areas of project management and planning to ensure that the results are exploited at the earliest opportunity;
  • Provide assistance to the EC Project Officers in the areas of dissemination and exploitation of project results. This could involve participating in the organisation of meetings, workshops and conferences, and in the preparation of related documents and publications;
  • Provide support to the analysis and assessment of the results, the impact of the project, and to promoting the dissemination of noteworthy technological achievements.

 

Development of a pilot environmental technology evaluation scheme as part of the European Environmental Technologies Action Plan (ETAP) (2006-2009)

Through Beta Technology, development of an evaluation scheme for environmental technologies on behalf of the European Commission. A total of 15 technologies were put through the pilot scheme in three areas – waste water, soil remediation and energy technologies (renewables and energy efficiency). The project results have fed into the process of developing a proposed mechanism and recommendations for a European-wide evaluation scheme. The project has involved collaboration at international level with other evaluation schemes in the USA and Canada. Recommendations from the pilot was used to draft a policy proposal presented to the European Council and Parliament.

 

Evaluation of the UK’s Micro and Nano Technologies (MNT) Facilities Network (2012)

The UK Government established a range of facilities (projects) offering world class MNT technology. Richard Ball formed part of the team that undertook an overall evaluation of this program. Program Level Benefits were evaluated based on the aggregated achievement of individual projects through their delivery of capability.  Benefits were the positive measured changes to a system which were directly related to the Program Objectives.  There were two principal types of benefits:

  • Direct benefits where a project could claim a direct contribution to a Program Objective through its activities (e.g., efficiency savings).
  • Indirect Benefits where the impact was not directly measureable or would have involved disproportionate effort to measure. The contributions of the overall projects to the program objectives were contrasted against organisation’s relevant KPIs (e.g., staff satisfaction).

A further indicative measure of the success of the program were the measures put in place for communications effectiveness, which prepared stakeholders for the depth and extent of change thereby improving the impact of the program.

The evaluation of the program was conducted through:

  • Interviews with the projects and their stakeholders to cover specific questions defined in an agreed structured interview questionnaire, leading to a rating of the effect of TSB funding and supporting information,
  • Analysis of the results of the interviews to support broader impact assessments
  • An assessment of critical success factors by monitors.

 

Monitoring and Evaluation of the UK’s Micro and Nano Technologies Facilities Network (2007-2012)

The UK Government established a range of facilities offering world class MNT technology. A key emphasis of these facilities was to understand and stimulate their markets to generate growth as well as developing technology. Richard Ball was assigned as a monitor to a number of these projects for a six year period, with the monitoring having both a strategic and business focus. The monitoring focused on the following aspects of the projects:

  • Impact and additionality, particularly that the facilities demonstrate that investment has created new innovative projects through the exploring of new technologies and markets.
  • Business Planning
  • Management and Governance
  • Business Development

The contract was successful in supporting UK Government, via the Technology Strategy Board, in monitoring and supporting the facilities projects.