Curious: Do you have plans to bring in FOSS LLMs for summarization and Q&A style queries anytime in the coming months?
Btw, I was half expecting that you quit because of FUTO grants (saw your post on their forums), but I guess it wasn't that. Either ways, rooting for you!
Not in the short term, if for no other reason than not having the hardware for it. Maybe down the line. Would be neat if someone who was into that sort of stuff wanted to integrate with Marginalia though. I've got a free API ;-)
I do think LLMs has the potential to integrate well with search engines and I'd love to see a sort of open source search ecosystem emerge with different projects collaborating to exceed the sum of their parts.
Yeah I was in talks with FUTO and they agreed to help the project out a bit, I won't say more until I have money in hand. It's taking a while but it's not on their end, I just need to sort out some legal stuff first.
Btw, I was half expecting that you quit because of FUTO grants (saw your post on their forums), but I guess it wasn't that. Either ways, rooting for you!