Calculating software emissions: learnings and limitations

Context

Earlier this year, we developed the sustainability calculator versions 1 and 2 at Theodo UK (version 2 can be downloaded here). The aim of the first version was to have a proof of concept that would allow us to calculate the CO2eq emissions of a website. With the initial version done on a spreadsheet, with formulas based on our research, we then kicked off the second version, which is a lot more automated. The second version exists as a chrome plugin that lets users record a user flow on their website, and get a value for the emissions associated with that flow. The goal of all this is to be able to keep an eye on the environmental impact of the features we develop.

Research

We found out during research that a large community already exists and has looked into these issues, this is where the first lead on how to calculate software carbon emissions appeared. The green software foundation published documentation software carbon intensity SCI. In this documentation, we can find the formula: SCI = (Energy used * carbon produced to generate this energy) + Embodied emissions (manufacturing of device) / unit.

Finding the embodied emissions of different devices, as well as the CO2e produced to generate energy, depending on the user’s location was not too complex as we had the manufacturing documents, and official documents of energy production for each country.

The hard part was to measure the energy consumed by the device. Although ideally we would measure different things (such as data transfer, battery usage associated with the software etc) and combine them into this energy usage, we did not find a good way of isolating the energy used by the website and quantifying it. The second best solution, was to estimate the energy usage of the page based on its network activity. By researching this, we found that it is possible to estimate this as 0.81 kWh per gigabyte of data transferred. It is controversial how accurate this is but it is a good way of getting a general idea.

Learnings

In our attempts to quantify the carbon emissions associated with software, we were able to extract some key learnings. The main theme from these learnings was that so many factors play a role in the total CO2e emissions, and a lot of them can be hard to quantify. We were able to quantify the impact of data transfer into energy used, as well as the manufacturing and transport of devices. When quantifying the hardware’s footprint, we had to compare the length of the measured flow with the full device lifecycle to weigh it in a way that doesn’t eclipse the software’s impact. With this in mind we realised the importance of developing mobile first and making sure the software we build is compatible with less powerful devices. This is due to mobile devices having a much smaller environmental impact than a laptop for example, manufacturing and transporting the average phone will emit ~70kg of CO2e, this figure for a computer is ~280kg of CO2e.

We observed that the efficiency and performance of software directly influence its environmental friendliness. The main thing to keep in mind when developing for sustainability is therefore to optimise everything as much as possible. This can go from optimising image size depending on the size of the screen (no need to load a huge image for a small mobile phone), to optimising API calls, caching and the data we exchange with our server. This falls back into learnings from “Designing for sustainability” by Tim Frick, which explores good practices for architecturing and developing sustainable software. With all this in mind, we were still unable to quantify all this into an accurate energy consumption. In order to make this more accurate, we’d need to be able to isolate processes running on the machine associated with the software we want to test, as well as the battery usage of the device. While considering these limitations, we identified some ways to work around them.

Limitations

The sustainability calculator provides a tool for assessing carbon emissions, but the score, in isolation, lacks meaning and does not provide much value. To extract practical insights, we need to focus on the differences in scores, by comparing iterations of the same flow, or similar flows on different websites. The tool is also limited to web based applications, rendering it useless for non-web based software.

Additionally, we encountered challenges in isolating the energy consumption directly attributable to the website. Despite efforts to estimate energy usage based on network activity, we are unable to get measurements that accurately represent real life impact. We also found edge cases when looking at the network activity. One of them being content streaming, the data transferred might be low when executing the flow but will grow overtime as content is streamed on the webpage. This can lead to confusing results as optimising a video for example, although beneficial, will only show results if we run the page for enough time to make the streaming noticeable in the results.

The tool’s primary use should not be to provide a standalone score, but to facilitate continuous improvement, for example, within a Kaizen framework. Measuring relative changes in emissions allows for a more practical understanding of the impact of our work.

Conclusion

In the future and with a potential third version of the calculator, the main aim would be to refine the calculations, and find ways to take into account different processes running on the machine, as well as battery usage. This would allow for more meaningful results that are closer to real world impact.

After that, being able to quantify the correlation between performance and emissions would allow us to get a score for mobile apps, using a tool like BAM’s Flashlight for example.