Hacker Newsnew | past | comments | ask | show | jobs | submit | riyadparvez's commentslogin

> My understanding is that it's dangerous and few want to do it.

Yeah this is the part everyone is ignoring in the conversation. Many highly compensated physical labor jobs are highly compensated because of the associated risk. While being a developer (with exceptions) doesn't expose you to any physical risk.


I hurt my little pinky finger programming. RSI is a real thing.

but I don't think tech compensation is high because of risk of this type of injury.

vs underwater welding, where the types of possible injuries have greater impact on overall health.


I've heard it from someone who was in China after the pandemic emerged. Apparently some Chinese people believe exactly that. Some of the lab animals were sold to the wet market. And that's how the virus jumped from animal to host. Although evidence is scant. Then again it's China.


Now, IF this is true, it would also explain why it seems that the government is trying to cover up what happened at the institute. If this hypothesis were to be true, it might cause China/the CCP to lose face more than many other alternatives, as such corruption could be seen as a weakness of the current system.


> In my time at Amazon, I was actually trained and coached on removing myself from certain situations and not interfering or establishing certain relationships because it would help me ultimately do better whatever is right for the company.

Can you please elaborate on this with some concrete examples, if possible?


Yes it seems to me they have taken the same approach as AWS Aurora. Use Postgres frontend and query parsing and replace the storage layer and query execution with their own.


Why would it decrease single thread performance? How is python different than other languages that support native full-fledged multi-threading, eg Java, Go, C#?


A big part is that Python uses reference counting GC. Java, Go, C# all use tracing GC. Py_INCREF and Py_DECREF are responsible for inc/decreasing the reference count, and are not atomic. The GIL ensures refcount safety by allowing only one thread access to changing refcount. The naive approach to parallelization would require locking each ref inc/dec. There are some more sophisticated approaches (thanks to work by Sam Gross et al) that avoid a mutex hit for every inc/dec.

Tracing GC does not run into this problem. Why Python doesn't use tracing GC is not something I am qualified to answer.

Sam Gross' work: https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsD...

The GIL code: https://github.com/python/cpython/blob/main/Python/ceval_gil...

Py_INCREF: https://github.com/python/cpython/blob/a4460f2eb8b9db46a9bce...


I am by no means knowledgeable enough on the topic, but Swift has similar problem domain, and afaik only uses atomic ref counts for objects that “escape” from a given thread - is there a reason something like that wouldn’t work for python as well?


python made it's C api visible, so things like reference counting are widely observed by C libraries that interop with python. This makes it much harder to make changes since you can't change the implementation in ways that programs rely on.


I am curious what exactly is the reason GIL hasn't been removed in one of the most popular languages ever?


Until now attempts to remove the GIL have come at the cost of reduced single threaded performance.

This is the first suggestion that doesn’t degrade single threaded performance.


Well, all credit to Sam the author, but he "bought" that no degraded performance by adding performance optimizations! Unfortunately, Python 3.11+faster-cpython work has already superseded some of it, i.e negating the trade-in he's trying to accomplish! However, I still hope everyone helps out and that the nogil work can land as default in CPython.


> Unfortunately, Python 3.11+faster-cpython work has already superseded some of it,

Why is that unfortunate? If his patch being slower than python 3.11 isn't acceptably fast despite being faster than every other python version before that then python was never acceptably fast to begin with.

Linux got rid of the BKL ages ago, that the Python community still holds onto the GIL as if it was some multi threading holy grail isn't even remotely funny anymore.


It's unfortunate because I think it makes it slightly less likely that his work is accepted, now that part of the sweet single thread performance gains are already supplied from another source.


>Unfortunately, Python 3.11+faster-cpython work has already superseded some of it, i.e negating the trade-in he's trying to accomplish!

If "Python 3.11+faster-cpython" didn't also remove the GIL, then it didn't negate anything he's trying to acomplish. He wasn't going for a faster Cpython alone, but for a Cpython without GIL.


The first Gil-removal patch I'm aware of was against python 1.4, so it's not like people haven't been trying. The problem has never been removing the GIL per se, but removing the GIL without slowing down the language when running single thread


You could compile Python using the --without-threads configure option, before Python 3.7 [1] or thereabouts.

[1] 3.7.0 is the first major release following the removal of the config option in https://github.com/python/cpython/issues/75551 .. there's also a 2021 comment about "This has unfortunately turned out to be a blocker on getting WASI support as there's not direct threading support in WebAssembly."


Popular but historically un/underfunded.


Also: there were lots of independent projects to remove the GIL/speed up Python: from Google, Dropbox, and others.

But due to the leadership/community model/aversion to big changes (except for alienating people with 2 to 3 changes) none of this was in the context of CPython, they were independent versions, that never caught on, and didn't leave anything (or much) behind to the regular CPython when they folded.


Hi Simon, thank you for the great work. Do you have any plan to support DuckDB in future?


I am seriously considering adding a plugin hook to support alternative database backends - PostgreSQL and DuckDB are the two I'm most interested in for that.


The issue is there's a huge ecosystem developed on top of Pandas. So if it's different API, now you lose access to all the ecosystem.


Should not be a real blocker. There is good interop with arrow, numpy and pandas. Where arrow and numpy mostly is zero-copy.

So you are one `df.to_numpy()/df.to_pandas()` away to `X` libary you want to use.


Polars can export to arrow which you can then load into pandas.

Or you can convert it directly into a pandas Df


There's also bear-type for runtime type checking: https://github.com/beartype/beartype.


I've been perplexed as to why this hasnt been built in to the python runtime since day 1.


Python was first released in 1991.


Type inference has been around (as in "used in programming languages") since the early 1970s. In the beginning python hit a sweet spot: glue code that wasn't exactly write once, too difficult to do in bash, and too small to go all the way and use C++. However, it was not without its own design issues.


And day 1 of when type hints was introduced was in 2014.


The only valid use for NFT I can think of is a pure vehicle for financial speculation. It's better people do wild speculation on NFTs than housing market.


There is functional value to NFT’s. Think beyond JPEG NFT’s. If you’re a software engineer, consider building a decentralized system with unique instances of a model. Unique tokens make a lot of sense to me and certainly there is utility to them. I agree that JPEG NFT’s are ridiculous though.


NFTs aren't even the best option for pure financial speculation, stocks are much better. At least there the money being gambled can be put towards something productive. Plus it's much better for the gambler. Random stocks don't routinely lose 99.999% of their value in a matter of months.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: