The point for me personally is the same as why I find it so powerful to self host SMTP, IMAP, HTTP. It’s in my hands, I know where it all begins and ends. I answer to no one.
For LLMs this means I am allowed their full potential. I can generate smut, filth, illegal content of any kind for any reason. It’s for me to decide. It’s empowering, it’s the hacker mindset.
Many would be users can’t send their data data to openAI. Think HIPPA and other laws restricting data sharing. Federation or distribution of the models for local training is the other solution to that problem.
I think it's mostly useful if you want to do your own fine tuning, or the data you are working with can't be sent to a third party for contractual, legal, or paranoid reasons.
OpenAIs paid GPT4 has few restrictions and is still cheap.
... Not to mention GPT4 with browsing feature is vastly superior to any home of the models you can run at home.