Quis custodiet ipsos custodes? The EU’s energy efficiency policies scrutinised by the European Court of Auditors

Webinar #17 of the Energy Evaluation Academy. Speaker: Peter Welch, Director “Sustainable use of natural resources” at the European Court of Auditors .

View the recordings of the webinar here.

Download the slides here.

The European Court of Auditors (ECA) promotes accountability and transparency, and acts as the independent guardian of the financial interests of the EU citizens. ECA recently published two reports about EU energy efficiency policies (about appliances and buildings) (see links below).

This webinar presented ECA’s role in the EU policy process, how it relates to evaluation, and the main conclusions from the two recent reports.

The webinar ended with a discussion between Peter Welch, Charles Michaelis (chair of Energy Evaluation) and Sarah Rieseberg (project manager at the Secretariat of the International Climate Initiative (IKI) ) about differences and possible interactions between the work of Courts of Auditors and the work of evaluators.

FURTHER READINGS

Recent publications of the European Court of Auditors related to EU energy efficiency policies:

– “EU action on Ecodesign and Energy Labelling: important contribution to greater energy efficiency reduced by significant delays and non-compliance” (published in January 2020) : https://www.eca.europa.eu/Lists/ECADocuments/SR20_01/SR_Ecodesign_and_energy_labels_EN.pdf

– “Energy efficiency in buildings: greater focus on cost-effectiveness still needed” (published in April 2020): https://www.eca.europa.eu/Lists/ECADocuments/SR20_11/SR_Energy_efficiency_in_buildings_EN.pdf

The Energy Evaluation Academy is jointly organised by Energy Evaluation and Leonardo Energy.

Evaluating the UK’s Energy Savings Opportunity Scheme

Webinar #16 of the Energy Evaluation Academy. Speakers: Laura Edwards and Gary Shanahan (Department for Business, Energy and Industrial Strategy, UK) .

View the recordings of the webinar here.

Download the slides here.

In 2014, the UK introduced the Energy Savings Opportunity Scheme (ESOS) to implement requirements under the Article 8 of the Energy Efficiency Directive (mandatory energy audits for large companies/non-SMEs).

This webinar presented the methodology used in the evaluation and its key findings. It then ended with a broader discussion on policies to expand business energy efficiency.

FURTHER READINGS

Fore more details about the evaluation of the UK’s Energy Savings Opportunity Scheme:

– latest evaluation report: https://www.gov.uk/government/publications/energy-audits-and-reporting-research-including-the-energy-savings-opportunity-scheme

– interim evaluation report: https://www.gov.uk/government/publications/energy-savings-opportunity-scheme-esos-evaluation-of-the-scheme

The Energy Evaluation Academy is jointly organised by Energy Evaluation and Leonardo Energy.

US Department of Energy’s Uniform Methods Project

Webinar #15 of the Energy Evaluation Academy. Speakers: Charles “Chuck” Kurnik (US DOE National Renewable Energy Laboratory – NREL).

View the recordings of the webinar here.

Download the slides here.

This webinar provided an overview of the US Department of Energy’s Uniform Methods Project that develops protocols for determining energy savings from energy efficiency measures and programs.

The webinar will discuss the motivations of this project, the development process, measures, and how they are used.

For more details about the Uniform Methods Project, visit the website: https://www.energy.gov/eere/about-us/ump-protocols

The Energy Evaluation Academy is jointly organised by Energy Evaluation and Leonardo Energy.

Lessons learnt about evaluation practices for energy efficiency policies in Europe

Speakers:

• Gregor Thenius, Austrian Energy Agency
• Jean-Sébastien Broc, IEECP (Institute for a European Energy & Climate Policy)

View the recordings of the webinar here.

Download the slides here.

While we enter a transition in the European framework for energy efficiency, ex-post impact evaluations can bring facts and figures about current achievements towards the 2020 targets, and useful feedback to improve policies in view of the 2030 targets.

The Horizon 2020 EPATEE project analysed evaluation practices in EU countries and developed resources to help enhancing them. After an overview of these resources, this webinar presented the main lessons learnt from exchanging with evaluation customers and evaluators, with a focus on comparing estimated and metered energy savings, and how to integrate evaluation into the policy cycle.

Webinar joinlty organised by and

The EPATEE project has received funding from the European Union’s Horizon 2020 Research and innovation programme under grant agreement No 746265.

Disclaimer: Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of this webinar and related contents. The views expressed in this webinar are the sole responsibility of the speakers and do not necessarily reflect the views of the European Commission

How to improve the evaluation of complex systems to improve policy-making

Webinar #14 of the Energy Evaluation Academy. Speaker: Sara Giorgi (Research Consultant & Evaluator)

View the recordings of the webinar here.

Download the slides here.

Evaluation is intended to provide policymakers and practitioners with feedback and recommendations to improve policy making and implementation. The success of an evaluation and the impact of its findings hinges on the way policymakers and practitioners are involved in and perceive the evaluation process. The more complex the policy, the more challenging this relationship becomes.

Energy Program and Policy Evaluation Capacity Building in the Asia Pacific

Webinar #13 of the Energy Evaluation Academy. Speaker: Edward Vine (Lawrence Berkeley National Laboratory)

View the recordings of the webinar here.

Download the slides here.

The Asia Pacific region is a center of interest and activity for energy efficiency and renewable energy technology, programs and policies. Evaluation will play an essential role to developing effective policy and to understanding the impacts of policy and program implementation.

Energy Evaluation Asia-Pacific (EEAP) is a non-profit organization that is taking a leadership role in expanding the practice of, and capacity for, objective evaluation in the energy efficiency and renewable energy fields.

This presentation highlights evaluation capacity building, EEAP, and key topics of EEAP’s Second Evaluation Conference (held in Bangkok in October 2019).

How and what can we learn from verifying energy savings first estimated with engineering calculations? (part 2)

Experience sharing webinar of the EPATEE project.

Speakers:

Pr. Laure Itard, Chair Building Energy Epidemiology, Delft University of Technology, the Netherlands

Dr. Alex Summerfiel, Energy Institute, University College of London, UK

View the recordings of the webinar here.

Download the slides here.

The EPATEE project dedicated two experience sharing webinars to the comparison of energy savings based on engineering calculations with energy savings determined from metered energy consumption. The first part presented two examples of ex-post evaluations that tackled this issue. For more details about this topic, see also the dedicated EPATEE case study.

This webinar provided a complementary experience from studies using large databases of metered energy consumption by presenting the following cases:

> Case 1: Energy savings from renovations in the Dutch non-profit housing sector, with a statistical analysis of the differences between results based on the Energy Performance Certificates, and results based on metered energy consumption.

For more details about this case, see: Filippidou, F., Nieboer, N., and H. Visscher, 2017. Effectiveness of energy renovations: a reassessment based on actual consumption savings. Proceedings of the ECEEE 2017 Summer Study, 1737-1746.

> Case 2: Analysis of the differences between energy consumption from building energy stock models, Energy Performance Certificates, and the impact of efficiency measures using metered data from the UK National Energy Efficiency Data-framework (NEED)

For more details about this case, see: Summerfield, A.J., Oreszczyn T., Palmer, J., and I.G. Hamilton, 2018. Caveats for Policy Development when Combining Energy Ratings, National Building Energy Models, and Empirical Statistics. Proceedings of IEPPEC 2018.

The webinar dealt with the following questions:

• What data and methods could be used to assess energy savings?
• How could the “metered” energy savings be compared with the “estimated” energy savings?
• What difficulties were encountered?
• What can be learnt in terms of evaluation practices for future evaluations?

The EPATEE project has received funding from the European Union’s Horizon 2020 Research and innovation programme under grant agreement No 746265.

Disclaimer: Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of this webinar and related contents. The views expressed in this webinar are the sole responsibility of the speakers and do not necessarily reflect the views of the European Commission

How and what can we learn from verifying energy savings first estimated with engineering calculations? (part 1)

Experience sharing webinar of the EPATEE project.

Speakers:

Dr. Michaela Valentova, Czech Technical University, Czech Republic

Pr. Andy Gouldson, University of Leeds, UK

View the recordings of the webinar here.

Download the slides here.

The analysis of the EPATEE case studies showed that engineering calculations (including deemed savings, but also more detailed engineering calculations) are the methods the most frequently used to evaluate energy savings for regular reporting. This result is similar to the conclusions of Wade and Eyre (2015) and Labanca and Bertoldi (2016). This can be explained because methods based on measured (direct measurements) or metered (billing analysis) data need more time to provide results, as pointed in some case studies (see e.g., case on Warm Front). Whereas engineering calculations can be applied directly to data collected along the monitoring of the policy measure. Therefore, the need to report annually (or even more frequently) about energy savings often leads to choose to use engineering calculations.

At the opposite, billing analysis is the method the most frequently used for ex-post evaluations when the objective is to verify actual energy savings. The results from the billing analysis can then be compared to the results from the regular reporting, to improve the data or assumptions used in the engineering calculations. This is for example the approach that has been used in Ireland (Better Energy Homes), UK (Supplier Obligations) or in the US (Weatherization Assistance Program).

This experience sharing webinar aims at providing a practical feedback from two ex-post evaluations that compared energy savings based on engineering calculations with energy savings determined from metered energy consumption:

> Case 1: Green Investment Scheme in the Czech Republic

> Case 2: Kirklees Warm Zone Scheme in UK

Both cases were analysed in scientific papers, respectively :

Valentová M, Karásek J, & Knápek J. (2018). Ex post evaluation of energy efficiency programs: Case study of Czech Green Investment Scheme. WIREs Energy and Environment, 2018;e323.

Webber, P., Gouldson, A., & Kerr, N. (2015). The impacts of household retrofit and domestic energy efficiency schemes: A large scale, ex post evaluation. Energy Policy, 84, 35-43.

This webinar is suitable for anyone with an interest in evaluation of energy efficiency policies and programmes.

The EPATEE project has received funding from the European Union’s Horizon 2020 Research and innovation programme under grant agreement No 746265.

Disclaimer: Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of this webinar and related contents. The views expressed in this webinar are the sole responsibility of the speakers and do not necessarily reflect the views of the European Commission

Evaluation of Compliance Framework of Labeling Program in India

Webinar #12 of the Energy Evaluation Academy. Speaker: Neha Dhingra (CLASP).

View the recordings of the webinar here.

Download the slides here.

Standards and Labeling programs have proven to be highly effective in stimulating the development of cost-effective, energy-efficient technologies, offer enormous carbon reduction potential, save consumers money and reduce power demand.

The Bureau of Energy Efficiency (BEE) is responsible for monitoring, verification and enforcement (MVE) of India’s labeling program. A comprehensive evaluation of BEE’s MVE framework helped identify several gaps and a need to strengthen BEE’s procedures and processes.

The presentation discusses several of the concrete measures that were implemented by BEE as a result of the evaluation.

Intersections of energy efficiency and health and wellbeing: findings from the UK

Webinar #11 of the Energy Evaluation Academy. Speaker: Ian Hamilton (UCL Energy Institute)

View the recordings of the webinar here.

Download the slides here.

The presentation covers the intersections of energy efficiency and health and wellbeing in the residential housing sector in the UK.

It provides a summary of findings from recent energy performance and retrofit programme health impact evaluations in the UK and the impact of future retrofit programme to meet the UK’s climate change targets in the residential sector.