Building Sustainable Algorithms: Energy-Efficient Python Programming

6 techniques for reducing the computational cost of Python algorithms

TLDR: Most Python code is wasteful, and inefficient algorithms come with hidden costs—not just slower runtimes but also higher energy consumption. Computational demands are growing faster than the computer chip industry, and sustainability consultants are constantly knocking on corporate doors. Writing energy-efficient code is therefore becoming essential for sustainability and performance. Luckily, several techniques can dramatically reduce computing cost. This includes, but is not limited to, smarter loops and data structures to profiling tools that pinpoint inefficiencies.

You can get your Python performing better by using these techniques. Image generated with Leonardo AI

A junior software developer shall be forgiven for being happy when their code works. If that’s you, I do not judge you.

However, if you are ready to get to the next level of building software with Python, your code should not just run and pass some tests. It should also be written with the available computing resources — and the energy bill — in mind.

Every inefficient loop, poorly chosen data structure, or redundant computation burns more electricity than necessary. Unlike C, for example, where you must reserve bits from your disk for each new variable you create, Python will consume resources as it sees fit. This makes it extremely beginner-friendly, but also rather energy-intensive when used wrong.

Sloppy algorithms are not just bad for the performance of a code. They are bad for the planet, too. Software companies like Microsoft are struggling to keep their carbon emissions low because of all of the energy they consume for AI and other tasks. At the same time, sustainability is a growing concern. Sustainability-minded programmers are therefore becoming a valuable resource for many companies.

In addition to making yourself more attractive on the job market, learning to code efficiently saves you money. Bye-bye bloated AWS bill! It also shortens your code runtimes — the only bad news with this is that you can’t quickly get a coffee while your code is running through then. And it frees up computing resources for other tasks.

Your Python code will rarely be as low-level efficient as C or Rust, but you can easily make some codes ten times more efficient with the right techniques. This involves loops, data structures, built-in functions, data types, parallelization, and benchmarking. We will cover each of these in the following piece.

1 — Optimize Loops and Iterations

Loops are part of any programming 101 course, in pretty much any programming language. They’re intuitive and take care of repetitive tasks. On the other hand, they can be a massive leakage of energy.

Consider how a beginner might write a simple loop:

# put squared numbers in a list
squared_numbers = []
for i in range(1, 10001):
    squared_numbers.append(i ** 2)

This gets the job done, but it is energy-consuming: Every time that Python appends an entry to this list, it must find a larger chunk of memory than the one the list is currently occupying. It must then copy over all the old entries (plus the new one), and then free up the old piece of memory.

As the list grows, the memory can become fragmented, which further slows down performance. Not great!

There are two solutions to this: The one is the (very Pythonic!) list comprehension. The second, worth using whenever you are dealing with numbers, is numpy.

The list comprehension looks like this:

squared_numbers = [i ** 2 for i in range(1, 10001)]

List comprehensions are much more performant because calculate the overall memory needed ahead of filling it, thus avoiding needless copying and pasting of new entries.

The numpy version looks like this:

import numpy as np

numbers = np.arange(1, 10001)
squared_numbers = numbers ** 2

The packages numpy is in fact written in C under the hood, making it far more efficient for numerical tasks. Operations are performed directly on arrays, avoiding the per-iteration overhead of Python’s native loops.

The speed gains depend a bit on the length of your list and the machine you are running it on. In general, though, you can easily get ten times more performance in loop situations like these.

2 — Choose the Right Data Structures

In Python, the default data structures — lists, dictionaries, sets, and tuples — offer a lot of flexibility. Each comes with its own trade-offs in terms of speed, memory usage, and functionality. As a result, one must choose wisely.

Generally speaking, lists are the most generalizable data structures. If you can use something more specific than them, you should. Using the data structure that is most specific to your needs tends to get you a far better performance. Below are some examples, but in a nutshell, this is how you should go about this:

• Lists are great for small, ordered data with infrequent lookups or changes.

• Sets are perfect for large datasets with frequent membership checks.

• Dictionaries are your go-to for key-value pairs and fast lookups.

• Tuples are best for immutable, fixed-size data.

• NumPy Arrays excel in high-volume numerical computations.

Sets for Membership Checks

Sets in Python are written in curly braces {} instead of the square brackets [] that lists come in. Sets are implemented as hash tables, making membership checks (x in my_set) average O(1), compared to O(n) for lists.

For example:

# Inefficient approach using a list
data = [1, 2, 3, 4, 5]
if 3 in data:
    print("Found")

# Efficient approach using a set
data_set = {1, 2, 3, 4, 5}
if 3 in data_set:
    print("Found")

Dictionaries for Key-Value Mapping

Dictionaries are also implemented as hash tables, making lookups, insertions, and deletions average O(1). What makes them different from sets is that they always map one key to one value.

For example:

# Inefficient approach using nested lists
data = [["id1", "Alice"], ["id2", "Bob"]]
for record in data:
    if record[0] == "id1":
        print(record[1])

# Efficient approach using a dictionary
data_dict = {"id1": "Alice", "id2": "Bob"}
print(data_dict["id1"])

Tuples for Fixed Data

Tuples are immutable and lighter than lists. If you do not need to modify the data, tuples are a more memory-efficient choice.

# Use a tuple instead of a list for fixed data
coordinates = (10, 20)

NumPy Arrays for Numerical Data

NumPy arrays are far more memory- and computationally-efficient than Python lists for numerical operations. They allow for vectorized operations, which avoids per-element overhead.

import numpy as np

# Inefficient approach using a list
numbers = [1, 2, 3, 4, 5]
squared_numbers = [x ** 2 for x in numbers]

# Efficient approach using a NumPy array
numbers = np.array([1, 2, 3, 4, 5])
squared_numbers = numbers ** 2