Sustainability does not play a major role among companies implementing AI. A small minority is aware of the environmental impact of their AI use, although it is many times greater than that of traditional technology solutions. When selecting solutions, impact barely even plays a role.
Only 12 percent of companies actively measure the environmental impact of their use of generative AI. That’s according to a Capgemini survey of 2,000 executives at large companies with more than $1 billion in revenue in North America, Europe and Asia. However, 80 percent of organizations have ramped up their investments in generative AI over the past year.
Major impact
Generative AI has a major impact on the environment. GPT-4 training was estimated to consume between 51,772 MWh and 62,319 MWh, or the equivalent of the annual energy consumption of 5,000 American families. Inference is not much better, given the sheer scale of it. Already in 2022, 60 percent of Google’s energy consumption related to machine learning went to inference. The International Energy Agency previously calculated that one query to ChatGPT consumes about 2.9 Wh of electricity: 10 times more than a Google search.
The impact is making itself felt. 42 percent of companies surveyed are revisiting their climate goals spurred by the growing importance of generative AI. Just 31 percent of organizations are taking steps to effectively reduce the impact of GenAI as well.
Transparency and increase
A primary challenge here is transparency. 74 percent of organizations point the finger at big tech companies like Microsoft and Google for not being transparent enough about the carbon footprint of their AI models. 64 percent themselves struggle to measure impact linked to applications.
According to the report, generative AI’s share of companies’ total emissions will increase from 2.6 percent to 4.8 percent over the next two years. That percentage is based on figures from the few companies that effectively measure the impact enough to make an estimate.
No priority
Barely 20 percent indicate that sustainability is a criterion today when choosing a vendor or a generative AI model. In other words, the impact of a generative AI solution does not play a significant role in choosing a solution. 57 percent confirm that cost and quality are the priority, and a model’s carbon footprint is only a minor consideration.
The report shows that large companies worldwide see the potential of generative AI, and are especially terrified of not innovating fast enough. Promises of sustainability and environmental trade-offs fade to the back burner when efficiency gains are on the table.
Solutions
However, there are ways to innovate sustainably today with generative AI. Capgemini’s report highlights some possibilities. They show that sustainability can be part of the decision-making process about the entire AI implementation, from the choice of models and partners to infrastructure.
For example, Capgemini points to the use of smaller AI models (SLMs) in place of large-scale LLMs to reduce energy consumption. Specialized SLMs should not be inferior to LLMs when deployed correctly.
There are further various optimization techniques such as pruning, quantization, and knowledge distillation, which reduce energy consumption without loss of performance. Furthermore, it is smart to optimize prompts as much as possible.
For cloud-based solutions, organizations can look at where partners get their power, and what hardware models are running on. The more recent the chips, the less energy they consume. Capgemini points to the advantages of Nvidia’s Blackwell chips, though they immediately illustrate the complexity of the issue. Blackwell is 25 times more efficient than its predecessors, but the chips contain bugs and mass production hit, but not started.
Dead letter
Either way, sustainable implementation of generative AI remains dead letter when large organizations do not consider impact a priority. The report thus shows how companies quickly forget their environmental ambitions when tempted by innovations. The fact that so few large companies measure their impact, and sustainability does not factor into the selection of genAI solutions, speaks volumes in this regard.