I understand the desire for multi-threaded CPython scripts, but why not do a PHP5 -> PHP7 and focus on single-thread performance more, particularly around implementing a JIT? We know from PyPy (and other JIT-enabled languages like NodeJS) that performance improvements can be significant. I feel like that should be the primary focus for speedups, and have always wondered why the idea has never been more popular. Has a CPython JIT been ruled out for some reason I'm not familiar with?
There have been several attempts at integrating a JIT into CPython. Google made a stab at it back around version 2.6 and there was a second attempt around the 2->3 change to build a JIT on top of LLVM. However none of these projects produced results that the core developers felt were good enough across the board and had problems with backwards compatibility and lot cross platform LLVM issues (as it was a pretty new project) so they were dropped after Google stopped funding the project.
That being said there is still work being done and people are hoping for at least some JITing in upcoming python. Pyston is project being worked on Guido Van Possum himself and both Microsoft Instagram have their own JITed versions of Python they use internally. Some initial parts of Pyston are even scheduled for being included in python 3.12 or 3.13
PyPy breaks compatibility with CPython in significant ways, as far as I understand the current no-gil proposal only got through because it has an automated fallback for modules that are not no-gil compatible.
The biggest one would be not using reference counting, which makes execution of cleanup code in __del__ somewhat non deterministic and tricks that rely on the exact reference count of an object outright impossible (for example reusing "immutable" objects that are only referenced once).
I thought that the C-API still had significant differences but going by its documentation it has a compatibility layer that is almost complete if somewhat slow.
You should look into the copy & patch efforts underway for Python[0]; an actual JIT will probably never exist but I think c&p has a shot of being mainlined in the next few years, such that Python could dynamically choose to either run the interpreter or a c&p option.
