EEAP organized a Workshop on Energy Evaluation 101 during Youth-in-Evaluation Week
EEAP organized a Workshop on Energy Evaluation 101 during Youth-in-Evaluation Week
The second annual Youth in Evaluation Week was organized from July 8 – 12, 2024, to unite young and senior evaluation professionals worldwide. This intergenerational event aimed at generating greater commitment and action towards meaningful youth engagement in evaluation, aligning with the UN Summit of the Future 2024.
To mark the youth-in-evaluation week, Energy Evaluation Asia Pacific organized an interactive workshop on Energy Evaluation 101 on July 10th. The objective of the workshop was to increase the interest of youth in the Evaluation of Energy programs and Policies and increase their knowledge and understanding of the field.
This interactive session, expertly facilitated by the experienced steering committee members of EEAP, was packed with insights and vibrant discussions. The workshop included in-depth exploration of key energy evaluation concepts, discussion on practical case studies and real-world applications and also offered networking opportunities for young professionals with peers and industry leaders.
It was inspiring to see the next generation of evaluators and energy experts so passionate and engaged!
The workshop covered the following key topics:
- What is Energy Evaluation?
- What Methods, Indicators, and Metrics are Commonly Used in Energy Evaluation?
- Who are the Key Stakeholders in Energy Evaluations?
- What are the Typical Issues and Challenges Addressed in Energy Evaluations?
- What are the Emerging Trends and Innovations in Energy Evaluation?
- What Role Can Young and Emerging Evaluators Play in the Field of Energy Evaluation?
- How is Energy Evaluation Relevant to Your Work?
Presentations:
Presentation Slides from Energy Evaluation 101 Workshop
Recording:
Access the Workshop Recording here.
EEAP Webinar 11 Summary Notes
On June 7, 2024, Energy Evaluation Asia Pacific organized its 11th webinar during the gLOCAL2024 evaluation week, focusing on the important topics of Transforming the Way We Measure Clean Energy Transitions. The webinar featured two speakers; Michael Williamson, Section Chief Energy Division, United Nations ESCAP and Divya Gaur, Program Lead, Council on Energy, Environment and Water (CEEW), India.
Michael provided an overview of the Asia-Pacific region’s progress on SDG7 for clean and affordable energy access. He provided an update on how we are tracking against SDG7 targets, and offered insights on approaches to high-level measurement, the emerging opportunities and the challenges in evaluating SDG7 targets. Divya shared an innovative project from India evaluating renewable energy’s impact on livelihoods. The project used indicators for affordability, productivity, and gender equality across large, medium, and micro-scale projects.
The discussions underscored the importance of measuring the broader impact of clean energy transitions, going beyond energy and carbon savings, and assessing the social, economic, and environmental benefits and challenges of clean energy initiatives.
Presentations:
Summary Notes
Download Summary notes here.
Recording:
Access the recording here.
EEAP Webinar 10: Summary Note
Energy Evaluation in Asia Pacific: Latest Updates
On May 7, 2024, the Energy Evaluation Asia Pacific (EEAP) hosted its 10th webinar focusing on the crucial topic of ‘Measuring Progress on Energy Wellbeing in New Zealand’. This insightful webinar featured two notable speakers: Dr. Phoebe Taptiklis, Research Fellow, Motu Economic Public Policy Research, New Zealand, and Dr. Caroline Fyfe, Post-doctoral Researcher, Karolinska Institute, Stockholm, Sweden. The webinar served to provide a guided tour of two important evaluations which measured the impact of New Zealand’s recent energy hardship policies.
Presentations:
EEAP Webinar 9: Summary Note
Energy Evaluation in Asia Pacific: Latest Updates
On April 18, 2024, the Energy Evaluation Asia Pacific (EEAP) hosted the 4th session of its webinar series, dedicated to the “Evaluation of SDG7 Policies and Programs in Asia Pacific.” This enlightening webinar featured two distinguished speakers: Ada Ocampo, President of IDEAS – International Development Evaluation Association, and Henry Adams Co-founder and Director at Common Capital, Australia.
Henry provided valuable insights into the challenges and opportunities inherent in evaluating and adapting clean energy affordability programs in the context of evolving climate policy objectives, stressing the importance of transformative thinking and data-driven approaches. Ada provided a comprehensive overview of the rationale, challenges, and importance of country-led evaluations in assessing progress towards the SDGs and contributing to transformative change.
The discussions underscored the need for transformative approaches, such as rethinking program design and adopting a systems perspective, to effectively address challenges and maximize benefits, whether in clean energy affordability programs or SDG evaluations. Additionally, they emphasized the importance of equity, data-driven analysis, and active participation to ensure inclusive and sustainable outcomes. This document summarizes the key discussion points from the webinar.
Presentations:
EEAP Webinar 8: Summary Note
Energy Evaluation in Asia Pacific: Latest Updates
On March 20, 2024, Energy Evaluation Asia Pacific (EEAP) hosted its 3rd webinar session, focusing on “Evaluation of SDG7 Policies and Programs in Asia Pacific.” This engaging webinar featured two esteemed speakers from South Asia: Fayyaz Ahmed, MEL Coordinator at Mercy Corps from Pakistan, and Sadeep Acharya, Managing Director of Jalpadevi Auto Repairing Center Pvt. Ltd from Nepal.
Fayyaz’s presentation delved into evaluating SDG 7 at the policy level, drawing insights from policy briefs supporting the UN High-Level Political Forum 2023. Sadeep shed light on the challenges and opportunities in tracking energy access in remote areas of Nepal.
The webinar served as a valuable platform for knowledge-sharing on policymaking to achieve the SDG7 goal of universal access to affordable, reliable, sustainable, and modern energy by 2030. It also underscored the importance of balancing innovation, collaboration, and continuous evaluation in the journey towards sustainable energy solutions. This document encapsulates the key discussion points from the webinar.
Presentations:
EEAP Webinar 7: Summary Note
Energy Evaluation in Asia Pacific: Latest Updates
On January 31, 2024, the Energy Evaluation Asia Pacific (EEAP) hosted the second session of its webinar series, dedicated to the “Evaluation of SDG7 Policies and Programs in Asia Pacific.” This enlightening webinar featured two distinguished speakers: Dayyan Shayani, a Statistician at the United Nations Economic and Social Commission for Asia and the Pacific (UNESCAP), and Sohini Mookherjee, Lead at Abdul Latif Jameel Poverty Action Lab,( J-PAL) and CLEAR South Asia.
Dayyan Shayani presented an overview of the progress toward the SDG7 goal in the Asia Pacific since 2015, drawing on available data for the six SDG7 indicators. Sohini’s presentation delved into evidence from ongoing Randomized Control Trials (RCTs) related to energy access in India, offering key takeaways for policymakers involved in implementing energy policies.
The webinar provided a valuable platform for knowledge-sharing and learning about the advancement of SDG7 in the Asia Pacific region. It underscored the importance of employing robust evaluation methods to generate evidence on the impact of policies and programs. This document summarizes the key discussion points from the webinar.
Presentations:
EEAP Webinar 6: Summary Note
Energy Evaluation in Asia Pacific: Latest Updates
On January 31, 2024, Energy Evaluation Asia Pacific (EEAP) organized an insightful webinar on the topic of “Evaluation of SDG7 Policies and Programs”. This engaging virtual event featured two keynote speakers: Michael Williamson, Section Chief, Energy Division in the United Nations Economic and Social Commission for Asia and the Pacific (UNESCAP) and Navin Kumar Jha, Assistant Professor, Tribhuvan University, Nepal.
Michael Williamson provided an overview of SDG7 roadmaps by the Economic and Social Commission for Asia and the Pacific member states and highlighted evaluation challenges for evaluating the impact of SDG7 roadmaps. Mr. Navin’s presentation focused on the pathway to clean cooking transition in Nepal, examining the policies governing household biodigesters in Nepal. The webinar served as an opportunity for sharing knowledge on SDG7 policies and programs and highlighting the importance and challenges of evaluating such initiatives.
Presentations:
SDG7 Roadmaps: Identifying and Addressing Evaluation Challenges
Pathway to clean cooking transition in Nepal: Policies governing household biodigesters in Nepal
EEAP WEBINAR 6_Summary Notes
EEAP Webinar 5: Summary Note
Energy Evaluation in Asia Pacific: Latest Updates
On Thursday 12th October 2023, Energy Evaluation Asia Pacific (EEAP) held the fifth webinar of the EEAP Webinar Series. The webinar focused on the Evaluation of Energy Policies and Programs in Chile, featuring two distinguished speakers from the Energy Sustainability Agency of Chile. Raúl Riveros discussed the evaluation of energy programs in Chile, showcasing the agency’s work, while Javier Rojas shared insights gained from monitoring electric vehicles under the Mi Taxi Eléctrico Program.
Presentations:
EEAP Webinar 2: Summary Note
Energy Evaluation in Asia Pacific: Latest Updates
On Wednesday 16th June 2021, Energy Evaluation Asia Pacific (EEAP) held the second webinar of the 2021 series. It was titled ‘Energy Evaluation in Asia Pacific: Latest Updates’ and featured a keynote speech from Ed Vine from Lawrence Berkley National Laboratory (LBNL), followed by a lightening session with 15 speakers from countries across Asia Pacific. After these presentations, Energy Evaluation Indonesia was launched by Jon Respati and Benedictus Dwiagus Stepantoro.
The recording of the webinar is available to view on YouTube here
Presentations:
EEAP Webinar 1: Summary Note
EEAP Webinar 1
The recording of the webinar is available to view on YouTube here
Summary Note: Energy Efficiency for Developing Countries: Pivoting the Focus from Savings to Increased Outputs
On Wednesday 28th April, Energy Evaluation Asia Pacific (EEAP) held its first webinar of our 2021 series. It was titled ‘Energy Efficiency for Developing Countries: Pivoting the Focus from Savings to Increased Outputs’ and featured a presentation from Philippe Benoit from Global infrastructure Advisory Services 2050 and Columbia University’s Center on Global Energy Policy, and Joeri De Wit, an Energy Economist at the World Bank. A rich presentation of an innovative impact assessment approach was followed by reflections and reactions from a panel of experts including Amandine Gal from Econoler, Melanie Slade from the International Energy Agency (IEA) and Shingle Sebastien from SELCO foundation.
Introduction to EEAP
EEAP is a non-profit organisation established in 2018 with a mission to take a leadership role in expanding the practice of, and capacity for, objective evaluation in the energy efficiency and renewable energy program and policy arena. EEAP are using workshops, conferences, webinars, websites and other web-based tools to foster the development of self-sustaining evaluation communities in the Asia-Pacific region. We are led by a planning committee and 32 evaluation ambassadors from 21 countries, and are always open to new members of our expanding community (see our website or contact us for more details).
Presentation
Philippe Benoit, from Global infrastructure Advisory Services 2050 and Columbia University’s Center on Global Energy Policy, began by outlining the problem at hand, the traditional assumption that when you do an energy efficiency intervention, input or project, your objective is to generate energy savings. The calculation of benefits is based on a valuation of the energy savings relative to outputs, supporting the common notion that energy efficiency is designed to do more with less.
Philippe outlined that in developing countries, governments’ objectives are very much focused on raising standards of living and therefore the emphasis is on generating more goods and services, rather than on doing with less. He pointed out, for example, that may often be more important to consider expanding energy-using infrastructure, rather than just replacement situations.
That requires flipping the emphasis from consuming less energy, to the value of the additional goods and services energy efficiency can enable access to; changing the focus from less input to more outputs from energy.
Philippe outlined constraints to the current narrative, which is dominated by the OECD-orientated perspectives of energy savings and not necessarily fit for purpose in developing economies. Philippe pointed to figures from the IEA showing that while we can expect a reduction in overall energy consumption in developed countries, the opposite is expected to happen across developing countries where more energy will be used in the future.
Philippe pointed to the lack of energy evaluation methodologies to support output-orientated energy efficiency projects and the negative connotations attached to the rebound effect. In a “doing-with-less” narrative, the rebound effect (when energy demand goes up as energy efficiency improves) is generally viewed as a negative outcome, whereas in an “increased outputs” narrative, rebound in energy demand is actually typically related to an increase in access to energy services and an associated improvement in living standards.
Philippe then discussed a range of different project examples and outlined that this new increased outputs narrative and methodology is most relevant to what he defined as category 2 projects, where the aim is to increase goods and services (see table below).
Philippe ended by emphasising that maybe when we are looking at energy efficiency measures like street lighting or air conditioning units, maybe it is important to not just limit ourselves to replacement situations but also consider whether another project expanding the infrastructure base would be relevant (what he referred to as “greenfield” EE projects).
Joeri de Wit, an Energy Economist at the World Bank then took the floor, outlining the standard guidance provided for economic analysis of projects funded by the World Bank. This research found room for expansion in the standard approach, and proposed an addendum to existing guidance. The addendum broadens the framework of economic analysis of the costs and benefits of energy efficiency projects, offering a methodology for projects that increase goods and services, which enables the benefits in terms of increased outputs and energy access to be taken into account. An additional objective to this method is dispelling the idea that the rebound effect always reduces benefits and providing guidance on steps for valuing this positively in the economic analysis where appropriate.
Joeri explained the importance of the project counterfactual (typically the next best alternative), before providing an example of increasing goods/services with an alternative project counterfactual. The choice of the counterfactual can be important to the ‘increased outputs’ context, as energy consumption and GHG may increase in actual terms, but could be classed as a saving if the counterfactual is based on an alternative energy project, as opposed to no intervention.
Joeri then took the audience through examples of the status quo counterfactual, using a water supply project, and the installation of efficient street lights in an underserved area. Accurately capturing the benefits here requires cross-sector collaboration, and Joeri highlighted the need for energy economists to talk to water economists to do this type of analysis and benefit estimations.
Joeri built on Philippe’s argument that a rebound may be reflective of greater consumption of goods and services, offering the example of an LED that is left on longer after replacing an incandescent bulb, providing additional lighting benefits. If the economic model is quantifying the value of project goods and services, it is able to quantify the rebound. If not, one can make assumptions about the minimum value associated with the direct rebound effect. In either case, the import of that rebound needs to be properly accounted for in terms of the benefits it may represent.
Joeri used the case study example of an energy efficiency project in a government building in Ghana, sharing the calculations and an example summary of the economic analysis. He showed how to combine different types of information and then shared a decision matrix from this information to compare investment decisions. The matrix showed that with a high willingness to pay for building air conditioning, a greenfield project is most suited, as those projects are providing more goods and services with a much greater value than brownfield retrofit where are the benefits are derived from energy savings.
Joeri also took us through analysis conducted on the remaining lifespan of current AC’s, concluding that, in fast growing developing countries with high social discount rates, if you have existing infrastructure that’s providing services then very quickly providing new services and expanding the infrastructure base can give you higher economic returns than replacing existing infrastructure.
Overall, the conclusions of the presentation were as follows:
- Economic analysis methods for energy efficiency projects at Multi-lateral Development Banks like the World Bank and similar organisations, tend to focus almost exclusively on energy savings.
- This is in part because (i) it is often difficult to quantify the value of goods/services and (ii) the energy internal rate of return (EIRR) based on energy savings are high enough anyway.
- Where energy efficiency investments yield additional goods/services, the value of these benefits can often be estimated based on energy savings, provided you are using a realistic counterfactual that yields the same goods/services less efficiently.
- However, where the project isn’t designed to provide the same level of goods/services, the additional value of those would need to be estimated with cross-sector expertise.
- A perverse outcome of status quo analysis of energy efficiency opportunities based on energy savings and GHG emissions alone, is that it can undermine the business case for viable investment opportunities that efficiently produce more goods/services from consideration, especially in developing countries.
- This work aimed to shift the prevailing narrative, by recognizing energy efficiency as means to increase production of goods/services, broadening guidance on the economic analysis of EE projects and revisiting the treatment of the rebound effect to focus on outputs, where appropriate.
- You can read more about this work in the following
publications:
- Energy efficiency for more goods and services in developing countries | EEG (energyeconomicgrowth.org)
- Energy Efficiency for Developing Countries: Pivoting from Fewer Inputs to More Outputs | Inter Press Service (ipsnews.net)
Panel Discussion
After the main presentation, our moderator, Nina Campbell of EECA (Energy Efficiency & Conservation Authority of New Zealand),thanked the speakers,and reflected that the issues around the conflict between saving carbon and energy on the one hand, and improving living standards, quality of living and supporting development on the other are very topical and very much of interest to the EEAP community. She pointed out that having the tools to measure these trade-offs better could be the game-changer we have been waiting for. She supported the call for cross-sector collaboration and noted that as these techniques are used more widely and evolve further, there may also be a role for inter-disciplinary collaboration too – bringing in social and behavioural scientists to support deeper analysis of wellbeing improvements for example.
Nina invited the panel to provide comments on the presentation. Amandine Gal from Econoler reflected that it sounded very interesting and logical to approach and capture not only energy savings but also goods that projects can produce. Amandine reflected from her own experience analysing benefits, and reflected that it will need some structure and guidance on how to implement this research in practise.
In response, Nina reflected that, as evaluators, we usually come in at the end stage of a project, and that we need to consider the implications this may have for data collection plans and how to cater for these in project design stages. She asked the panellists what sort of data do we need to think about gathering and whether they had any advice to people on how to plan this up front? Joeri responded that it is a chicken and egg problem. It’s a challenge which sometimes stops us moving a step in the right direction because we don’t have that data we need. The authors of this work are recommending to make that step and do our best with the data we have as well as making the case for collecting richer data. Philippe added that this work is captured in a document available on the web here. He also highlighted that a lot of funding is climate finance, which is focused on absolute emissions reductions to get to net-zero emissions, and this type of pivot to outputs, recognising there may be an increase in emissions, doesn’t always fit well with this type of financing. Philippe said we need to get over this, as in order to promote development, we need these greenfield projects.
Mel Slade, Head of the Emerging Economies programme at the IEA, highlighted why this research is so important – because we are keen to see energy efficiency reach its potential in terms of supporting achievement of climate goals, but this is not happening. The rate of progress on energy efficiency is quite slow, about half what it needs to be. Mel gave two examples, one of which was India which has made huge progress in energy efficiency and renewables, deploying solar power generation in particular. The reason behind this was India’s objective to increase its energy security, and to reduce their costs of importing oil and vulnerability to international oil prices. China is another great example, who have made most progress not because of energy efficiency or climate goals, but driven by air pollution issues, and the potential to make savings in public health costs by improving people’s health. Mel talked about the biggest motivator now, in the post-COVID19 world, being economic development and jobs, and the risk of energy efficiency being scaled back rather than going forward. We need to get better as an evaluation community at valuing jobs, and how we can create jobs in the energy transition. We must take women into account, and those under 30 in this transition. It’s a complicated job collecting the evidence we need, but we must kep working to improve this so energy efficiency doesn’t lose ground whilst economic recovery is the focus for governments.
Shingle Sebastien from the SELCO foundation said the presentation rang true with what the SELCO foundation did on the ground. He used the example of the sewing machine, in the villages which is mostly done by women. The traditional pedal machine requires continuous pedalling throughout the day and can cause health issues for users. The immediate remedy is adding a locally available motor with the machine, which increases electricity consumption (and cost for consumers). Hence more energy efficient motors could bring down the drudgery and increase the value addition with lesser cost for customers. Shingle outlined the three main areas where the energy efficiency drive is needed in the Indian context; heaters, compressors and motors.
Nina then summarised the discussion and closed with the remarks encouraging everyone to keep up to date with Energy Evaluation Asia Pacific by signing up to our mailing list here.
