How to improve the evaluation of complex systems to improve policy-making

Webinar #14 of the Energy Evaluation Academy. Speaker: Sara Giorgi (Research Consultant & Evaluator)

View the recordings of the webinar here.

Download the slides here.

Evaluation is intended to provide policymakers and practitioners with feedback and recommendations to improve policy making and implementation. The success of an evaluation and the impact of its findings hinges on the way policymakers and practitioners are involved in and perceive the evaluation process. The more complex the policy, the more challenging this relationship becomes.

Energy Program and Policy Evaluation Capacity Building in the Asia Pacific

Webinar #13 of the Energy Evaluation Academy. Speaker: Edward Vine (Lawrence Berkeley National Laboratory)

View the recordings of the webinar here.

Download the slides here.

The Asia Pacific region is a center of interest and activity for energy efficiency and renewable energy technology, programs and policies. Evaluation will play an essential role to developing effective policy and to understanding the impacts of policy and program implementation.

Energy Evaluation Asia-Pacific (EEAP) is a non-profit organization that is taking a leadership role in expanding the practice of, and capacity for, objective evaluation in the energy efficiency and renewable energy fields.

This presentation highlights evaluation capacity building, EEAP, and key topics of EEAP’s Second Evaluation Conference (held in Bangkok in October 2019).

How and what can we learn from verifying energy savings first estimated with engineering calculations? (part 2)

Experience sharing webinar of the EPATEE project.

Speakers:

Pr. Laure Itard, Chair Building Energy Epidemiology, Delft University of Technology, the Netherlands

Dr. Alex Summerfiel, Energy Institute, University College of London, UK

View the recordings of the webinar here.

Download the slides here.

The EPATEE project dedicated two experience sharing webinars to the comparison of energy savings based on engineering calculations with energy savings determined from metered energy consumption. The first part presented two examples of ex-post evaluations that tackled this issue. For more details about this topic, see also the dedicated EPATEE case study.

This webinar provided a complementary experience from studies using large databases of metered energy consumption by presenting the following cases:

> Case 1: Energy savings from renovations in the Dutch non-profit housing sector, with a statistical analysis of the differences between results based on the Energy Performance Certificates, and results based on metered energy consumption.

For more details about this case, see: Filippidou, F., Nieboer, N., and H. Visscher, 2017. Effectiveness of energy renovations: a reassessment based on actual consumption savings. Proceedings of the ECEEE 2017 Summer Study, 1737-1746.

> Case 2: Analysis of the differences between energy consumption from building energy stock models, Energy Performance Certificates, and the impact of efficiency measures using metered data from the UK National Energy Efficiency Data-framework (NEED)

For more details about this case, see: Summerfield, A.J., Oreszczyn T., Palmer, J., and I.G. Hamilton, 2018. Caveats for Policy Development when Combining Energy Ratings, National Building Energy Models, and Empirical Statistics. Proceedings of IEPPEC 2018.

The webinar dealt with the following questions:

• What data and methods could be used to assess energy savings?
• How could the “metered” energy savings be compared with the “estimated” energy savings?
• What difficulties were encountered?
• What can be learnt in terms of evaluation practices for future evaluations?

The EPATEE project has received funding from the European Union’s Horizon 2020 Research and innovation programme under grant agreement No 746265.

Disclaimer: Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of this webinar and related contents. The views expressed in this webinar are the sole responsibility of the speakers and do not necessarily reflect the views of the European Commission

How and what can we learn from verifying energy savings first estimated with engineering calculations? (part 1)

Experience sharing webinar of the EPATEE project.

Speakers:

Dr. Michaela Valentova, Czech Technical University, Czech Republic

Pr. Andy Gouldson, University of Leeds, UK

View the recordings of the webinar here.

Download the slides here.

The analysis of the EPATEE case studies showed that engineering calculations (including deemed savings, but also more detailed engineering calculations) are the methods the most frequently used to evaluate energy savings for regular reporting. This result is similar to the conclusions of Wade and Eyre (2015) and Labanca and Bertoldi (2016). This can be explained because methods based on measured (direct measurements) or metered (billing analysis) data need more time to provide results, as pointed in some case studies (see e.g., case on Warm Front). Whereas engineering calculations can be applied directly to data collected along the monitoring of the policy measure. Therefore, the need to report annually (or even more frequently) about energy savings often leads to choose to use engineering calculations.

At the opposite, billing analysis is the method the most frequently used for ex-post evaluations when the objective is to verify actual energy savings. The results from the billing analysis can then be compared to the results from the regular reporting, to improve the data or assumptions used in the engineering calculations. This is for example the approach that has been used in Ireland (Better Energy Homes), UK (Supplier Obligations) or in the US (Weatherization Assistance Program).

This experience sharing webinar aims at providing a practical feedback from two ex-post evaluations that compared energy savings based on engineering calculations with energy savings determined from metered energy consumption:

> Case 1: Green Investment Scheme in the Czech Republic

> Case 2: Kirklees Warm Zone Scheme in UK

Both cases were analysed in scientific papers, respectively :

Valentová M, Karásek J, & Knápek J. (2018). Ex post evaluation of energy efficiency programs: Case study of Czech Green Investment Scheme. WIREs Energy and Environment, 2018;e323.

Webber, P., Gouldson, A., & Kerr, N. (2015). The impacts of household retrofit and domestic energy efficiency schemes: A large scale, ex post evaluation. Energy Policy, 84, 35-43.

This webinar is suitable for anyone with an interest in evaluation of energy efficiency policies and programmes.

The EPATEE project has received funding from the European Union’s Horizon 2020 Research and innovation programme under grant agreement No 746265.

Disclaimer: Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of this webinar and related contents. The views expressed in this webinar are the sole responsibility of the speakers and do not necessarily reflect the views of the European Commission

Evaluation of Compliance Framework of Labeling Program in India

Webinar #12 of the Energy Evaluation Academy. Speaker: Neha Dhingra (CLASP).

View the recordings of the webinar here.

Download the slides here.

Standards and Labeling programs have proven to be highly effective in stimulating the development of cost-effective, energy-efficient technologies, offer enormous carbon reduction potential, save consumers money and reduce power demand.

The Bureau of Energy Efficiency (BEE) is responsible for monitoring, verification and enforcement (MVE) of India’s labeling program. A comprehensive evaluation of BEE’s MVE framework helped identify several gaps and a need to strengthen BEE’s procedures and processes.

The presentation discusses several of the concrete measures that were implemented by BEE as a result of the evaluation.

Intersections of energy efficiency and health and wellbeing: findings from the UK

Webinar #11 of the Energy Evaluation Academy. Speaker: Ian Hamilton (UCL Energy Institute)

View the recordings of the webinar here.

Download the slides here.

The presentation covers the intersections of energy efficiency and health and wellbeing in the residential housing sector in the UK.

It provides a summary of findings from recent energy performance and retrofit programme health impact evaluations in the UK and the impact of future retrofit programme to meet the UK’s climate change targets in the residential sector.

Multiple impacts of energy efficiency: approaches, results and insights from the COMBI project

Webinar #10 of the Energy Evaluation Academy. Speakers: Johannes Thema, Felix Suerkemper (Wuppertal Institut)

View the recordings of the webinar here.

Download the slides here.

There have long been around: co-, non-energy or multiple benefits of energy efficiency. Since 2014 with the IEA report Capturing the Multiple Benefits of Energy Efficiency, they also arrived at the heart of political discourse in Europe. What do we know on those multiple impacts at European level, which quantifications are there? What methods can be applied for assessing multiple impacts of future policies, in order to take better informed decisions? Do we have an idea of the size of multiple impacts?

This webinar gives the first answers to these questions and introduces participants to the online tool containing all quantifications from the COMBI project (“Calculating and Operationalising the Multiple Benefits of Energy Efficiency in Europe”)

Plugging the gap between energy poverty management and the lived experience: five principles

Webinar #9 of the Energy Evaluation Academy. Speakers: Koen Straver (ECN part of TNO), Dr Lucie Middlemiss (Sustainability Research Institute, University of Leeds) .

View the recordings of the webinar here.

Download the slides here.

As an introduction we will elaborate on the current policy and activities in the Netherlands and the UK, to show differences and promising examples of new ideas on how to tackle energy poverty. Accordingly, we will articulate a new approach to energy poverty policy, based on bringing insights from a multi-disciplinary understanding of the lived experience of energy poverty into policy design.

We argue that understanding the lived experience of energy poverty is critical in designing appropriate policies, which are both effective and aligned with people’s day-to-day lives. In addition, the range of disciplines that examine the lived experience of energy poverty (housing, employment, education, social policy, health, energy etc.) help to give breadth to our understanding of this challenging condition.

We propose five principles for policy design, informed by a multi-disciplinary understanding of the lived experience. These principles can be applied at a range of scales (local, regional, national and super-national).

Evaluating energy efficiency policies: new online resources to share knowledge and experience

Webinar #8 of the Energy Evaluation Academy. Speakers: Barbara Schlomann, Barbara Breitschopf (Fraunhofer ISI), Jean-Sébastien Broc (IEECP) .

View the recordings of the webinar here.

Download the slides here.

Experience sharing about evaluation practices for energy efficiency policies is limited due to lack of time for stakeholders to disseminate or document evaluation activities. Front-runners publish their work in journals or conferences, but most of evaluation activities remain in national language and are difficult to find.

This webinar presents new online resources developed by the European project EPATEE to make knowledge and experience more easily available: a Knowledge Base including and a set of case studies.

The webinar also discusses stakeholders’ needs about evaluation practices.

How energy efficiency policy evaluation can produce benefits and add value to policy makers? (part 2)

Experience sharing webinar of the EPATEE project.

Speakers:

Peter Bach, Danish Energy Agency

Heikki Väisänen, Energy Authority, Finland

View the recordings of the webinar here.

Download the slides here.

According to numerous experiences and evidences, evaluation of energy efficiency policy can produce many benefits for policy makers and contribute to improve existing policies and design new ones that are more effective. The EPATEE project has contributed to this topic by sharing some interesting experiences on how evaluation has been used and has produced an added value for policy makers, besides giving some suggestions on the way to make a better use of it and to overcome possible barriers. This was the objective of this webinar, with experiences from Denmark and Finland.

This webinar is suitable for anyone with an interest in evaluation of energy efficiency policies and programmes.

The EPATEE project has received funding from the European Union’s Horizon 2020 Research and innovation programme under grant agreement No 746265.

Disclaimer: Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of this webinar and related contents. The views expressed in this webinar are the sole responsibility of the speakers and do not necessarily reflect the views of the European Commission