Folks, the whole point of this effort and what stands it apart from previous GIL-removal efforts is that it's carefully designed to:
1. Give the performance-benefits of GIL removal for multi-threaded code.
2. Not hurt the performance of single-threaded code, hopefully improving it too.
3. Not break any pure Python code that relies on the semantics of the GIL.
4. Minimize impact to extensions.
If you're commenting here that this is going to break things w/o having read the design doc, you're contributing FUD. It's disappointing to see the discussion go this way.
The project aims for a concurrency model that matches the threads + shared memory model implemented in common operating systems and in programming languages like Java, C++, and Swift. We also aim for similar safety guarantees to Java and Go -- the language doesn’t prevent data races, but data races in user code do not corrupt the VM state. (Notably, Swift and C++ do not have this property: data races in those languages may corrupt VM state leading to segfaults). This strikes a balance between safety and performance. Weaker guarantees, like in Swift and C++, can make debugging difficult, while much stronger guarantees can limit flexibility or hamper performance.
Compatibility:
The vast majority of Python programs should not require any changes to Python code to continue working. Most of the issues I have encountered so far are due to being based on 3.9.0a3 and libraries using features from the 3.9 final release. A few of the issues have been due to changes to code object internals or the bytecode format, but so far I have been able to address those by making changes to the interpreter to improve backwards compatibility.
All extensions that use the Python C API will require re-compilation, even the few that are limited to the stable ABI. (The reference counting changes do not preserve the stable ABI).
To compile successfully, some C API extensions will need minor modifications, typically replacement of direct access of PyObject reference count fields with the Py_REFCNT macro.
To run successfully, some C API extensions need modifications to protect global data structures in C code that were previously protected by the GIL. For an example, see these two patches to NumPy.
1. Give the performance-benefits of GIL removal for multi-threaded code.
2. Not hurt the performance of single-threaded code, hopefully improving it too.
3. Not break any pure Python code that relies on the semantics of the GIL.
4. Minimize impact to extensions.
If you're commenting here that this is going to break things w/o having read the design doc, you're contributing FUD. It's disappointing to see the discussion go this way.
Here's Sam's design doc:
https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsD...
Some highlights:
The project aims for a concurrency model that matches the threads + shared memory model implemented in common operating systems and in programming languages like Java, C++, and Swift. We also aim for similar safety guarantees to Java and Go -- the language doesn’t prevent data races, but data races in user code do not corrupt the VM state. (Notably, Swift and C++ do not have this property: data races in those languages may corrupt VM state leading to segfaults). This strikes a balance between safety and performance. Weaker guarantees, like in Swift and C++, can make debugging difficult, while much stronger guarantees can limit flexibility or hamper performance.
Compatibility:
The vast majority of Python programs should not require any changes to Python code to continue working. Most of the issues I have encountered so far are due to being based on 3.9.0a3 and libraries using features from the 3.9 final release. A few of the issues have been due to changes to code object internals or the bytecode format, but so far I have been able to address those by making changes to the interpreter to improve backwards compatibility.
All extensions that use the Python C API will require re-compilation, even the few that are limited to the stable ABI. (The reference counting changes do not preserve the stable ABI).
To compile successfully, some C API extensions will need minor modifications, typically replacement of direct access of PyObject reference count fields with the Py_REFCNT macro.
To run successfully, some C API extensions need modifications to protect global data structures in C code that were previously protected by the GIL. For an example, see these two patches to NumPy.