Estimating the carbon footprint of deep learning algorithms

Image: maciek905/Istock.com via AFP Relaxnews

Information technology (IT) students in Denmark have created a software program that can determine the energy consumption and the amount of carbon dioxide generated by the development of deep learning algorithms. According to their estimates, hardware used to train a deep learning algorithm can use worrying amounts of energy from an environmental standpoint.

Whether browsing movies suggested by Netflix based on your viewing history, asking your voice assistant a question or interacting with a chatbot on an e-commerce website, all of these everyday online processes rely on deep learning algorithms.

However, developing algorithms contributes to digital pollution. And it’s precisely this environmental impact that students from the IT department of the University of Copenhagen have sought to quantify, using their Carbontracker software program.

Developed by Lasse F. Wolff Anthony and Benjamin Kanding, with assistant professor Raghavendra Selvan, the program can calculate and predict the energy consumption and CO2 generated by training deep learning models.

According to the Carbontracker creators, artificial intelligence, in particular the subfield of deep learning, could become a significant climate culprit if current industry trends continue. In just six years, from 2012 to 2018, the compute needed for deep learning has grown 300,000%.

The GPT-3 algorithm, a year’s energy consumption of 126 Danish homes

The students explain that the training sessions for the algorithms involved in deep learning processes require specialist hardware that is particularly power hungry, and which runs 24 hours a day.

“As datasets grow larger by the day, the problems that algorithms need to solve become more and more complex,” explains Benjamin Kanding.

One of the biggest deep learning models developed to date is the GPT-3 advanced language model. According to the Carbontracker creators, a single training session for this model is estimated to use the equivalent of a year’s energy consumption of 126 Danish homes, and emit the same amount of carbon dioxide as driving 700,000 kilometers.

“Within a few years, there will probably be several models that are many times larger,” states Lasse F. Wolff Anthony.

As a result, Carbontracker aims to provide the sector with a free program offering a foundation for reducing the climate impact of such models. “It is possible to reduce the climate impact significantly,” says Wolff Anthony.

“For example, it is relevant if one opts to train their model in Estonia or Sweden, where the carbon footprint of a model training can be reduced by more than 60 times thanks to greener energy supplies. Algorithms also vary greatly in their energy efficiency. Some require less compute, and thereby less energy, to achieve similar results. If one can tune these types of parameters, things can change considerably,” the student concludes. JB

RELATED STORIES:

Elementary schools kids, as young as first graders, to learn coding in Canada

Apple to pay up to $1.5 million to those who can hack an iPhone or Mac

Algorithm used by thousands of US hospitals biased against blacks