I agree. This really seems like high impact too. A first party, supported solution would solve this mess. Getting the Python dependency build story under control I would include in this category. In this department, I think Conda shows the way. They leverage Azure Pipelines to build much of the c dependencies ahead of time, and just send binaries down the pipe (at least, that's the high level overview as I understand it).
I can't see a world where the Python Software Foundation can't get support for this, I think they have the contacts at the big companies to get it funded.
The lead Python has on machine learning, data science, and scientific computing won't last forever (though, I think its safe to say no matter what, it ain't going anywhere.
> I can't see a world where the Python Software Foundation can't get support for this, I think they have the contacts at the big companies to get it funded.
You might be surprised at how hard it is to get money donated for this sort of thing. Large companies often have their own build infrastructure and thus build things for themselves from scratch, to the point of building their own build toolchains (e.g. look at Google and bazel, Facebook and buck). So the way they handle packaging means how the overall packaging ecosystem functions does not impact them like it does individuals or smaller companies.
This is also why companies only started funding PyPI security work about two years ago; downloading source securely from PyPI is important to big companies as that's where they grab the source that they compile. But funding pip improvements only just happened in Decembe, starting with the new pip resolver work got money (https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-rol...). And all that money came from Mozilla and the Chan Zuckerberg Initiative; two huge non-profits, not companies.
Python overall very much runs on volunteers, but the packaging system especially. The core team of Python itself might generously have a total of 3 devs/week being paid to work on CPython spread across all contributors (IOW it is nowhere near as impactful as 3 actual full-time devs who aren't context-switching constantly between "work work" and "Python work"). But for Python packaging in general? If you don't count conda I think packaging has less than even 3.
First of all, as an aside, thank you for all the work you do on Python! I actually got started pretty early on in my professional career with Python (3.5+ too, which I am thankful for in hindsight). I was using VS Code and the Python Extension before MS even acquired its developer (is Don still there? I hope he is. Such a genuinely down to earth guy). Without the work you guys do I don't know if I would have taken the programming the same way, or at that the time I did in my life.
I will say, I didn't realize the funding situation was _that_ dire, especially since on Talk Python I remember specifically Michael and some others talking about the massive influx of money into Python and its ecosystem in the last year and half or so. Its not billions of dollars but it sounded like the situation had a positive uptick substantially, so, in short, I was wrong!
Has the PSF ever considered having an actual business? Like say, consulting and/or having developers work on features that businesses pay to have implemented? (A loose example of this would be Igalia[0], which makes surprisingly good amounts of money to implement features in web browsers on behalf of the major browser vendors)
I've always thought this would be a pretty natural way to have a steady source of revenue and the profits could all feed back into the PSF.
> First of all, as an aside, thank you for all the work you do on Python!
Welcome!
> is Don still there?
Yep, working on the data science features of the extension.
> on Talk Python I remember specifically Michael and some others talking about the massive influx of money into Python and its ecosystem in the last year and half or so.
As I said there have been donations for PyPI and pip just got its big donation. The CZI donation was also much bigger than pip and covered a lot of the science ecosystem.
> Has the PSF ever considered having an actual business? Like say, consulting and/or having developers work on features that businesses pay to have implemented?
There isn't a need for the PSF to have a business subsidiary to accomplish that as people can donate money for such things if they truly wanted to. It's also really hard to manage due to tax laws in the US around non-profits and such. Plus there would be start-up capital costs, etc. just like any other business. I just don't think the risk of trying to get it set up would lead to any benefit when people have always had the option to donate directly to see if something could be done.
If people want to help out financially they can go to https://donate.pypi.org for packaging donations and https://www.python.org/psf/donations/python-dev/ for the language and CPython. And if someone wanted to fund a specific thing I'm sure the PSF is happy to discuss such an idea to see how feasible it would be.
That makes sense, on its face, as to why they don't have a business arm, at least currently.
One thing I would suggest, on its face, is that any time someone has to reach you (as in, the foundation, not you specifically), that introduces friction, even the smallest amount of it, can leave lost donations on the table. If there was a simple page even, that said something like
Do you have a feature you need implemented in Python? Are there features you would like to fast track? For a simple donation, we can prioritize the work you need to grow your business
This with a nice little contact form, would eliminate that friction, rather than being in a situation where I might be reaching out to the wrong wing of the foundation, or not feeling comfortable sending a general inquiry etc.
you could also put a feature page that tracks how much money needed to implement x too, to solicit donations. I've seen this work in other communities. I think Django has gotten funding using this technique before.
Just a thought, I know you guys are doing everything you can and then some, and Python is unique in that its one of the few language communities of its size without any formal corporate benefactor.
> For a simple donation, we can prioritize the work you need...
I'm not an expert, but I'm pretty sure a non-profit can't do things like this. If the person giving the money expects a tangible benefit in return, it's not a donation.
The specific restriction from my understanding is that a US non-profit that accepts money for a specific thing must spend every penny of that money only on that thing. So if the PSF was given $1,000,000 to solve a specific problem and while trying to do it they go bankrupt they still couldn't touch that money to stay afloat. So non-profits avoid that situation since they typically have enough to survive a year or so, but not enough to be able to restrict what they might need to do with their cash.
It would probably go a long way if they advertised a roadmap of things they’d like to work on given donations. Maybe they have this out there and I just haven’t seen it, but I imagine it would be easy enough for me to persuade my small employer to throw a few thousand USD their way each year until some sane packaging solution is developed.
If there's a specific pain point your employer would like to see fixed and is willing to donate money to help resolve that pain point then please go to https://donate.pypi.org/.
I can't see a world where the Python Software Foundation can't get support for this, I think they have the contacts at the big companies to get it funded.
The lead Python has on machine learning, data science, and scientific computing won't last forever (though, I think its safe to say no matter what, it ain't going anywhere.