yall really are making me want to massively overextend and start that federated search engine project based on human-driven indexing and whichever APIs each instance wants to query and cache. yes, like a fancy web directory
maybe this is a good idea for a FreeAssembly project once Philthy’s in a good state? it’s a better idea than starting a shitty Wikipedia clone at least
Personally: I’d love to, but I have a conflicting non-compete so I’d definitely have to quit my job first and I’m not ready for that level of adulting. The good news is that if I ever do quit I’ll have a lot of relevant skills
(feels quite fucky to type the following without coming across as a naysayer; not quite the intended meaning, but… I guess you’ll see)
it’d probably be cool if this could exist, but also there’s a couple of extremely hard problems in going for it, along with a couple of (to my current knowledge) entirely unsolved ones
one of the presently-unsolved things I know of is that we don’t yet have anything like scalable performant homomorphic encryption so there’s no way to do fully-private query operations on a dataset, which thus gives way to the operator snooping space combined with user privacy angles. there are some technical solutions to some aspects of this, and a number of social things that would apply too
Kagi basically admits they’re just piggybacking off of Google, Bing, et al, so getting into the space shouldn’t be a serious PITA.
yall really are making me want to massively overextend and start that federated search engine project based on human-driven indexing and whichever APIs each instance wants to query and cache. yes, like a fancy web directory
maybe this is a good idea for a FreeAssembly project once Philthy’s in a good state? it’s a better idea than starting a shitty Wikipedia clone at least
Personally: I’d love to, but I have a conflicting non-compete so I’d definitely have to quit my job first and I’m not ready for that level of adulting. The good news is that if I ever do quit I’ll have a lot of relevant skills
(feels quite fucky to type the following without coming across as a naysayer; not quite the intended meaning, but… I guess you’ll see)
it’d probably be cool if this could exist, but also there’s a couple of extremely hard problems in going for it, along with a couple of (to my current knowledge) entirely unsolved ones
one of the presently-unsolved things I know of is that we don’t yet have anything like scalable performant homomorphic encryption so there’s no way to do fully-private query operations on a dataset, which thus gives way to the operator snooping space combined with user privacy angles. there are some technical solutions to some aspects of this, and a number of social things that would apply too
might be interesting either way
definitely a hell of a big project.
maybe this is something you’re looking for idk https://github.com/StractOrg/stract they have their own index and their own crawler
covered by 404media https://www.404media.co/this-guy-is-building-an-open-source-search-engine-in-real-time/
Omg so much tech jargon in this comment
welcome to TechTakes!
no, welcome wasn’t the right word now was it