Ohi, author here! Thanks for posting Hister. Feel free to A.M.A.
My first free software search project was Searx, a privacy respecting metasearch engine, but because of the limitations of the metasearch concept, I've decided to take a different approach.
Hister builds a personal search index from pages you visit, bookmarks, browser history, local files, and crawled websites. It stores extracted content with offline result previews, so information remains searchable even when the original page changes or disappears. It supports full text and semantic search, can run entirely on your own machine, and includes a web interface, command line tools, and an MCP endpoint for assistant integrations.
>Hister builds a personal search index from pages you visit, bookmarks, browser history, local files, and crawled websites.
Immediately interested and will check it out, thank you! I've wanted a "search stuff you've seen online" tool for a long time, but everything seems to be research-oriented or "archive but don't search" or some weird combination that means it's nigh useless to me. I've got decades of bookmarks and archives and I've kinda been stuck grepping them at best (it's rare but I do sometimes want a page I saw once three years ago and I love having that option), while hoping someone would build something better.
One question if ya don't mind, while I explore: any chance of singlefile support? Content-extraction is useful in lots of situations (e.g. wallabag) and it's a great default, but sometimes it fails and sometimes you really do want the page, relatively close to how it actually was. Singlefile does that much better than most, and it does so well enough (and manually-handle-able enough if needed) that I don't feel any desire to switch to WARCs or similar.
Though specifically I'm probably looking for something like "content-extract everything" + "key combo to save singlefile version too" + "upload singlefile archives to backfill / recover". Like 99% of the time content extraction is preferred, and I'm glad to see it... it's just not always enough, and having to go elsewhere for exceptions breaks a lot of the utility.
asciimoo 2 hours ago [-]
Exactly! I had the very same issues before Hister.
> One question if ya don't mind, while I explore: any chance of singlefile support?
Yes, partially. Hister can already import HTML files created by SingleFile, but there is no direct integration yet. In the longer term, I would like the SingleFile extension to be able to send snapshots directly to Hister.
Groxx 2 hours ago [-]
I assume it content-extracts that on upload? I'd really like to move storage into hister too, if possible. That way you could also switch from an extracted view to a "full" view in the UI. Though I assume that'd be fairly simple to build later.
Overall I really like what I'm seeing, it ticks a lot of important boxes for me and it's pleasantly straightforward. Hopefully I'll find time to contribute!
asciimoo 1 hours ago [-]
> I assume it content-extracts that on upload?
Hister always stores the original material.
> That way you could also switch from an extracted view to a "full" view in the UI.
It isn't even needed, we just need a SingleFile specific extractor (an interface in Hister to parse specific page content and provide custom previews) that provides the full original HTML for the preview panel.
> Hopefully I'll find time to contribute!
I'd appreciate it. <3
Groxx 1 hours ago [-]
Oooh, now I see the extractor-view setting in the UI. Yeah, that's essentially perfect \o/
Thank you again!
genxy 33 minutes ago [-]
I came here to mention https://github.com/gildas-lormeau/singlefile, I couldn't get it to do what I want so I built my own, for watched domains it pushes a copy of the serialized DOM to a local search database. For structured data, either extract and enrich in the browser or enrich on the server side.
Would Hister support this basic workflow? I'd love to retire my own software.
The next phase was going to move to a recording proxy.
iambateman 2 hours ago [-]
Very cool. Does it work to connect my phone and laptop to the same firehose?
I often find myself irritated because I read an article on my phone 6 months ago and the history is gone.
asciimoo 2 hours ago [-]
You can host Hister on a home server and access it from multiple devices.
Automatic page capture on mobile currently requires Firefox, since mobile Chrome does not support browser extensions.
dented42 2 hours ago [-]
What’s the safari story look like right now?
asciimoo 2 hours ago [-]
Safari support is currently stalled in this PR: https://github.com/asciimoo/hister/pull/46. I do not have access to macOS, so help with development and testing would be very welcome.
mfld 41 minutes ago [-]
What would be a typical size of the search index, let's say after 5 years of intense browsing?
asciimoo 29 minutes ago [-]
It depends on what you consider intense browsing. An indexed document uses about 100KB on average because Hister stores the full original HTML for offline previews. If storage is a concern, you can disable offline previews, reducing the average document size significantly.
oulipo 2 hours ago [-]
You should integrate with Karakeep.app, it's only natural that you'd want both a search engine, and a nice "archive" and "article pretty view" features :)
asciimoo 2 hours ago [-]
What do you mean by integration? Hister already supports importing data from Karakeep, Linkding, Linkwarden, Readeck, Shaarli, and wallabag: https://hister.org/docs/import
zuzululu 2 hours ago [-]
interesting you used AGPL 3 licensing, are you planning a hosted version?
would've been great with a more liberal license
asciimoo 2 hours ago [-]
> are you planning a hosted version
Not in the near term. Right now I am focused on developing Hister rather than operating a hosted service. There are already plenty of centralized hosted search engines, so my longer term interest is in federation and distributed search. I want to make the core system mature first.
> would've been great with a more liberal license
It depends on how do you define liberal. =] I chose AGPLv3+ because I want Hister to remain free software and available to their users.
zargon 1 hours ago [-]
AGPLv3 is awesome, great choice!
zuzululu 45 minutes ago [-]
it prevents certain innovations to be derived from it but with LLMs I do think these licensing are pretty much moot.
i still think it could've benefited by Apache 2.0 which more or less gets you to your goals
polymer8563 12 minutes ago [-]
it would not! asciimoo, thanks for using AGPL, I've been using hister for months now and it helped me a lot
pbronez 2 hours ago [-]
I've been using LinkDing + SingleFile for this. Nice to have another option!
asciimoo 2 hours ago [-]
That is a great combo. My main friction with it is having to manually capture pages I want to keep.
Jon_m 2 hours ago [-]
[dead]
quasigod 1 hours ago [-]
I just finished setting this up yesterday and I'm kind of obsessed with it. I was previously a heavy user of Karakeep, but I hated forgetting to save something and losing it. I also think Hister's semantic search is a better solution than AI generated summaries and tags. Support for local docs is super cool too, I have it set up to index my org notes directory.
wmchen 2 hours ago [-]
I set this up a few months ago based on asciimoo's comments on HN, and barely used it at first, but I realized not too long ago that it could be a pretty useful research tool for one of my hobbies (award travel), that revolves around being in the know around various concepts and quirks.
I scraped and imported posts from the blogs I regularly reference for award travel, then hooked it up to OpenCode/Codex as an MCP server and used that corpus for research on those topics. So I can ask things like "has anyone ever mentioned running into this problem before?" [1]
If you have a hobby or working situation that requires you to regularly reference a core set of websites or reference materials, Hister provides almost all the tools out of the box to start a search engine against it. The default datasets they promote include the Python Stlib, MDN and RFC corpus, as an example. [2]
Wow, this is a really inspiring use case and blog post. Thanks for sharing it.
What tools or features would Hister need to support your complete search workflow?
wmchen 1 hours ago [-]
[dead]
ebr4him 21 minutes ago [-]
Is the design vibe coded in Codex by any chance? Seeing a lot of similar designs out there and it’s really starting to annoy me.
asciimoo 13 minutes ago [-]
The whole design is created by a main contributor and IIRC the style is called neobrutalism.
hallole 11 minutes ago [-]
Really? The website, you mean? :( I thought to myself, "what a cute, well designed webpage!" when I saw it. Maybe with time this site will stand out like those Claudeslop ones.
lucaaa 15 minutes ago [-]
same
renegat0x0 2 hours ago [-]
I have created something similar, yet different. I maintain my own Internet domain index.
I believe hister you have to fill in with your data, right?
lnenad 2 hours ago [-]
I LOVE the concept. I will play around with the execution, if it works as described this is a great product.
Carrok 2 hours ago [-]
I tried this out this week and liked it but really wish this project had some form of auth. Opening the contents of every page you’ve ever visited, even to the local network, is not the best idea.
asciimoo 2 hours ago [-]
Hister supports token based, password based, and OIDC/OAuth authentications with optional multi-user handling. Details about user handling can be found here: https://hister.org/docs/user-handling
It also has a "public mode" where anyone can search the indexed content, but only authenticated users can add or modify it.
Carrok 2 hours ago [-]
It seemed like the public mode was the default when I set it up. If so, that’s a fairly dangerous default as keeping a “clean” history with no secrets leaked seems neigh impossible.
asciimoo 2 hours ago [-]
The default configuration binds only to localhost, and a fresh installation starts with an empty database/index. Could you clarify which specific attack surface you are concerned about in that scenario?
Carrok 56 minutes ago [-]
I’m not concerned about an attack scenario. I’m just saying that using the docker image, if someone (or their agent) isn’t careful, they could expose their browsing history publicly fairly easily. It might just be nice to default to at least a user and pass login rather than just wide open.
androidu 1 hours ago [-]
I've set it up today with OIDC AuthN (Keycloak), removed the username/password fields & kept only OIDC, it's working flawless.
Also indexed data is persisted on a per-user basis, so you got this isolation and certainty that your searches will not be polluted by your family's
robotmay 2 hours ago [-]
Ooh I’ve been thinking about this idea for years, I’m glad someone beat me to it. Guess I have something to play with over the rest of the weekend!
alexnewman 1 hours ago [-]
Time to shameless plug my own easy to host your own content search index
Webtm.io
All open source and small enough to deploy. I deploy to cf webworkers so it’s the only place it’s tested.
One cool thing is we work on iOS, chrome and friends, Firefox and pretty much everywhere. We do require you bring your own LLM though.
exe34 2 hours ago [-]
Can it import browser bookmarks? It says browser history and a bunch of other bookmark services but not specifically browser bookmarks.
asciimoo 2 hours ago [-]
Yes, it can import data from various sources, including browser history, and popular self-hosted bookmarking apps: https://hister.org/docs/import
exe34 1 hours ago [-]
Is this reply from Gemini? Because I'm used to gemini gaslighting me. The answer is the opposite of my question.
asciimoo 1 hours ago [-]
Doh, sorry, the answer was coming from me who did not read the question properly. It currently cannot import browser bookmarks, but it is a good idea. Added to my TODO.
exe34 58 minutes ago [-]
Ah sorry didn't mean to be mean, I just wondered if you had automated it haha!
The name coming from HISTory on STERoids. When I checked the name I found only a beetle and part of the Danube called "Hister".
wolfy1993 2 hours ago [-]
I'll be honest I wouldn't blame OP for not finding this (I found a species of hister beetle before anything to do with Hitler)
I feel this is a weird spin on the "Clicks to Hitler" game
thataccount 1 hours ago [-]
I was not using Wikipedia to get there. This is something I immediately recognized from research about World War II and possible prophetic fulfillment. I definitely wasn't playing a game when I came across this information either, but I get your drift.
Where it says that “Hister” is a variant of its Latin name.
I think the name is lovely with no reason to change. Surely there are few people who would connect this with silly medieval superstition. Perhaps you can take part in wiping out that association.
thataccount 1 hours ago [-]
User name checks out.
tim-projects 39 minutes ago [-]
I didn't know this link. But, I immediately read Hitler when I saw it. Coming back days later... Nope brain still reads Hitler.
It's a terrible choice of name
shevy-java 2 hours ago [-]
This is like Hipster. But missing a 'p'.
Rendered at 21:24:39 GMT+0000 (Coordinated Universal Time) with Vercel.
My first free software search project was Searx, a privacy respecting metasearch engine, but because of the limitations of the metasearch concept, I've decided to take a different approach.
Hister builds a personal search index from pages you visit, bookmarks, browser history, local files, and crawled websites. It stores extracted content with offline result previews, so information remains searchable even when the original page changes or disappears. It supports full text and semantic search, can run entirely on your own machine, and includes a web interface, command line tools, and an MCP endpoint for assistant integrations.
Project page: https://github.com/asciimoo/hister
Tiny read-only demo: https://demo.hister.org/
Immediately interested and will check it out, thank you! I've wanted a "search stuff you've seen online" tool for a long time, but everything seems to be research-oriented or "archive but don't search" or some weird combination that means it's nigh useless to me. I've got decades of bookmarks and archives and I've kinda been stuck grepping them at best (it's rare but I do sometimes want a page I saw once three years ago and I love having that option), while hoping someone would build something better.
One question if ya don't mind, while I explore: any chance of singlefile support? Content-extraction is useful in lots of situations (e.g. wallabag) and it's a great default, but sometimes it fails and sometimes you really do want the page, relatively close to how it actually was. Singlefile does that much better than most, and it does so well enough (and manually-handle-able enough if needed) that I don't feel any desire to switch to WARCs or similar.
Though specifically I'm probably looking for something like "content-extract everything" + "key combo to save singlefile version too" + "upload singlefile archives to backfill / recover". Like 99% of the time content extraction is preferred, and I'm glad to see it... it's just not always enough, and having to go elsewhere for exceptions breaks a lot of the utility.
> One question if ya don't mind, while I explore: any chance of singlefile support?
Yes, partially. Hister can already import HTML files created by SingleFile, but there is no direct integration yet. In the longer term, I would like the SingleFile extension to be able to send snapshots directly to Hister.
Overall I really like what I'm seeing, it ticks a lot of important boxes for me and it's pleasantly straightforward. Hopefully I'll find time to contribute!
Hister always stores the original material.
> That way you could also switch from an extracted view to a "full" view in the UI.
It isn't even needed, we just need a SingleFile specific extractor (an interface in Hister to parse specific page content and provide custom previews) that provides the full original HTML for the preview panel.
> Hopefully I'll find time to contribute!
I'd appreciate it. <3
Thank you again!
Would Hister support this basic workflow? I'd love to retire my own software.
The next phase was going to move to a recording proxy.
I often find myself irritated because I read an article on my phone 6 months ago and the history is gone.
Automatic page capture on mobile currently requires Firefox, since mobile Chrome does not support browser extensions.
would've been great with a more liberal license
Not in the near term. Right now I am focused on developing Hister rather than operating a hosted service. There are already plenty of centralized hosted search engines, so my longer term interest is in federation and distributed search. I want to make the core system mature first.
> would've been great with a more liberal license
It depends on how do you define liberal. =] I chose AGPLv3+ because I want Hister to remain free software and available to their users.
i still think it could've benefited by Apache 2.0 which more or less gets you to your goals
I scraped and imported posts from the blogs I regularly reference for award travel, then hooked it up to OpenCode/Codex as an MCP server and used that corpus for research on those topics. So I can ask things like "has anyone ever mentioned running into this problem before?" [1]
If you have a hobby or working situation that requires you to regularly reference a core set of websites or reference materials, Hister provides almost all the tools out of the box to start a search engine against it. The default datasets they promote include the Python Stlib, MDN and RFC corpus, as an example. [2]
[1]: https://wmchen.com/blog/revisiting-hister/
[2]: https://hister.org/datasets
What tools or features would Hister need to support your complete search workflow?
https://github.com/rumca-js/Internet-Places-Database
Also I maintain android app that can be used to search places.
https://f-droid.org/en/packages/io.github.rumcajs.offlineweb...
I believe hister you have to fill in with your data, right?
It also has a "public mode" where anyone can search the indexed content, but only authenticated users can add or modify it.
Also indexed data is persisted on a per-user basis, so you got this isolation and certainty that your searches will not be polluted by your family's
Webtm.io
All open source and small enough to deploy. I deploy to cf webworkers so it’s the only place it’s tested.
One cool thing is we work on iOS, chrome and friends, Firefox and pretty much everywhere. We do require you bring your own LLM though.
Got enough Hitler in my life with Google...
I feel this is a weird spin on the "Clicks to Hitler" game
Where it says that “Hister” is a variant of its Latin name.
I think the name is lovely with no reason to change. Surely there are few people who would connect this with silly medieval superstition. Perhaps you can take part in wiping out that association.
It's a terrible choice of name