Why are browsers so forgetful? Local text search is an easy problem at single-user scale.
I am working on a browser extension to automatically grab text (visible phrases) from the pages I am reading in the browser, save them to my index and implement search. I want to always be able to recall anything I see. In order to match texts I will use neural embeddings computed with a pretrained Huggingface model, fast enough even on CPU.
An alternative way to implement this would be to take screenshots every few seconds (only after recent user interaction). Then I can OCR the images and search anything that passed on the screen, in any app.
It's as if I become the Google Bot responsible with exploration and filtering of potentially useful data.
In the future I might be able to use my index for neural question answering, add a voice interface to my data.
Of course the local index could work for images too. You can grab images and embed them with CLIP to get Google Photos like semantic search.
I think web browsers are going to morph into personal internet agents, responsible with logging, searching, interfacing with external APIs and services, filtering ads, news and mail, making sure we don't accidentally leak personal information, etc. In other words AI that works for the user, locally.
"I think web browsers are going to morph into personal internet agents, responsible with logging, searching, interfacing with external APIs and services, filtering ads, news and mail, making sure we don't accidentally leak personal information"
And you think that will happen, while the browsers are still mostly add funded?
But I am with you, that this is a change, I want to see, too.
There is an extension for Firefox (Memex Go?) that attempts something very similar and succeeds to a large degree.
I think I sponsored it for a year or something by paying for it even though I didn't use it much but I must have used it to some degree because on one or more of my machines it injects interesting results from my earlier browsing when I search in DDG (or was it Google?) at least.
I'm really impressed how far they've taken this idea even as Mozilla has simultaneously been busy dismantling the extension API that made Firefox great.
I am working on a browser extension to automatically grab text (visible phrases) from the pages I am reading in the browser, save them to my index and implement search. I want to always be able to recall anything I see. In order to match texts I will use neural embeddings computed with a pretrained Huggingface model, fast enough even on CPU.
An alternative way to implement this would be to take screenshots every few seconds (only after recent user interaction). Then I can OCR the images and search anything that passed on the screen, in any app.
It's as if I become the Google Bot responsible with exploration and filtering of potentially useful data.
In the future I might be able to use my index for neural question answering, add a voice interface to my data.
Of course the local index could work for images too. You can grab images and embed them with CLIP to get Google Photos like semantic search.
I think web browsers are going to morph into personal internet agents, responsible with logging, searching, interfacing with external APIs and services, filtering ads, news and mail, making sure we don't accidentally leak personal information, etc. In other words AI that works for the user, locally.