1784480935

Python Threads Don't Do What You Think: The Definitive Guide to the GIL


It's been a while since I posted anything here, so I decided to write something special for all of you. Today I want to talk about one of the most misunderstood and most debated topics in the entire Python community: the GIL, the Global Interpreter Lock. It's one of those subjects that shows up in almost every technical interview, sparks heated discussions on forums, and keeps confusing developers even after years of writing Python day to day. The good news is that, in 2026, the GIL's story is finally changing, which makes this the perfect moment to really understand what's going on. ## <br>The problem nobody expects to run into Imagine you write a Python program to process a folder full of images, and you decide to use threads because your computer has eight cores and you want to take advantage of all that processing power. You launch four threads working at the same time, expecting the total time to drop to a quarter of what it was. Instead, the program takes roughly the same time it would with a single thread, or sometimes even longer. This is probably the first time most Python developers hear about the GIL, usually through frustration rather than academic curiosity. And the explanation for this behavior sits at the very heart of how CPython, Python's reference implementation, was designed from the start. ## <br>What the GIL actually is The Global Interpreter Lock is, in simple terms, a mutex, a locking mechanism that ensures only one thread executes Python bytecode at a time, no matter how many cores the processor has available. Even if your program launches ten threads, only one of them is actually running Python code at any given moment; the others are waiting their turn. This is a good place to pause and highlight a point that causes a lot of confusion: the GIL is not a feature of the Python language itself. It's a particularity of CPython, which happens to be the most widely used implementation and the one you get by default when you download Python from the official site. Other implementations, like Jython or IronPython, never had this problem, and PyPy has been exploring alternatives for years. When someone says "Python has the GIL," what they really mean is "CPython has the GIL" — a small distinction on paper, but a huge one in practice. ## <br>Why this decision was made To understand why the GIL exists, we need to go back to how CPython manages memory. Python uses a system called reference counting to know when it can free the memory occupied by an object. Every Python object has an internal counter that goes up whenever a new reference is created and goes down whenever a reference stops existing. When that counter reaches zero, the object is destroyed and its memory is freed. The problem is that this counting needs to happen safely. If two threads try to modify the same object's counter at the same time, without any protection, the result can be an incorrect count, which could lead to memory leaks or, worse, freeing memory that's still in use — a particularly nasty bug to track down. The most obvious fix would be giving each object its own individual lock, but that would bring a huge performance cost for simple operations, and would make life much harder for anyone writing C extensions for Python, which are a fundamental part of the ecosystem. The alternative chosen by CPython's creators, back in the nineties, was more pragmatic: instead of protecting each object individually, protect the entire interpreter with a single global lock. This drastically simplified writing the interpreter and C extensions, and kept Python fast for single-threaded programs, which for decades were the overwhelming majority of use cases. The price to pay, as we've seen, was the impossibility of real parallelism within a single process. ## <br>What this means for everyday development In practice, the GIL's impact depends entirely on the kind of work the program is doing. For tasks that spend most of their time waiting, like network requests, file reads, or database queries, threads in Python remain quite useful. That's because the GIL is automatically released whenever a thread enters a waiting operation, letting another thread take advantage of that gap to do work. For tasks that demand heavy processor computation, though, like image processing, intensive math operations, or file compression, threads in traditional Python bring no advantage at all, because the GIL prevents the computation from truly happening in parallel. This is exactly the scenario from our opening example, and it's also the reason so many developers end up disappointed the first time they try threads in Python for this kind of work. ## <br>The solutions the community built over the years Since the GIL has always been there, the Python community developed several ways to work around its limitations, each suited to a different scenario. For heavy processor work, the most common solution has always been the `multiprocessing` module, which instead of creating threads within the same process, creates entirely separate processes, each with its own Python interpreter and, therefore, its own GIL. This allows real parallelism, at the cost of more memory usage and slower communication between processes compared to communication between threads. Libraries like NumPy, widely used in data science, work around the problem differently: much of the heavy lifting happens in C code under the hood, and that code releases the GIL during the most intensive calculations, allowing some parallelism even within a normal Python program. For programs dealing with many simultaneous waiting operations, like modern web servers, `asyncio` became the preferred approach, enabling concurrency without even needing real threads, through a cooperative execution model. ## <br>The change that was decades in the making For many years, the standard answer to "why does the GIL still exist" was simply "because removing it would break too much code and make single-threaded Python slower." That started changing seriously with PEP 703, proposed by Sam Gross and officially accepted by Python's Steering Council in October 2023. The plan laid out in PEP 703 defined three phases. In the first phase, make a GIL-free build of CPython available, but clearly marked as experimental. That phase happened in Python 3.13. In the second phase, make that GIL-free build officially supported, while still remaining optional — something that happened with the acceptance of PEP 779 and arrived in Python 3.14, released in October 2025. In the third phase, still without a confirmed date, the GIL-free build would become the default option. In Python 3.14, the technical implementation described in PEP 703 was essentially completed, including relevant changes to the C API, and many temporary workarounds used during the experimental phase were replaced with more permanent solutions. One of the most visible improvements was the reduced performance impact on single-threaded code, which dropped from roughly forty percent during the experimental phase in 3.13 to a range between five and ten percent in 3.14 — a number that's already much more acceptable for real-world use. The biggest challenge now is less about the interpreter's internals and more about the ecosystem: every C extension needs to explicitly declare that it's compatible with the GIL-free mode. If a program imports an extension that hasn't made that declaration yet, the interpreter automatically re-enables the GIL for the entire process, silently, without breaking the program but also without delivering the expected parallelism. ## <br>How to try this out right now One of the most interesting parts of this stage in Python's history is that there's no need to wait for anything anymore: anyone can download the free-threaded build of Python 3.14 and try it out in practice. This build is identified with a "t" in its name, for example `python3.14t`, clearly distinguishing it from the traditional build with the GIL. To confirm in real time whether the GIL is active or not in a running program, Python 3.13 introduced the function `sys._is_gil_enabled()`, which returns `False` when the GIL is actually disabled. It's worth noting that this state can change during the program's execution: if you import an incompatible C extension, the GIL turns back on automatically, and this function will reflect that change. There's still no official confirmed date for the third phase, the one where GIL-free Python would become the default option. The pace of that transition will depend largely on how quickly the most widely used libraries in the ecosystem, especially the ones relying on C extensions, declare support for the free-threaded mode. ## <br>Wrapping up The GIL was born from a pragmatic engineering decision, made at a time when multi-core processors weren't even the norm yet, and it made sense for decades given the alternatives available. The fact that it still exists today isn't a sign of stagnation, it's a sign of how deeply intertwined it is with how CPython manages memory and with an entire ecosystem of extensions built around that premise. The difference is that, for the first time in over thirty years, there's a real technical path, already partly walked, to leave that limitation behind, and that's something well worth following closely over the coming years.

(0) Comments

Welcome to Chat-to.dev, a space for both novice and experienced programmers to chat about programming and share code in their posts.

About | Privacy | Donate
[2026 © Chat-to.dev]