Decentralized Search in 2020

Parking ideas for a decentralized search engine from first principles in 2020

Goal - to build a new age search engine (fun project) that is semantic and decentralized.

Some initial thoughts

  • Electron UI
  • Modeling
    • Semantic modeling (schema.org)
    • Ability to add / edit models
    • Ability to add / edit objects
  • Data Extraction
    • Source Plug-in architecture (example Wikipedia plugin, StackOverflow plugin, Reddit plugin, Amazon plugin etc.).
    • The source plugins will be extract information and map it semantically in a local SQLite database.
  • View
    • Multiple object view. with plugins for rendering objects as list, graphs, images
    • Single object view. with templates for each object type and also showing modeled objects with internet links for further search
  • Sharing
    • Ability to publish / share models, objects, extractors, views in-app
  • Network
    • Offline first
    • Personal node first
    • Personal node can be setup as a server
    • Ability to create a peer network (node to node)
    • Ability to search a peer via a plugin

Reading

Pears: https://pearsearch.org/

Blog on personal search: https://beepb00p.xyz/pkm-search.html

3 Likes

Interesting idea. Seems more like a knowledge store/graph than a search engine. SQLite / or an RDBMS probably wouldn’t be the best way to model semantic relationships. A graph DB would be better suited.

Speaking of knowledge graphs, check out https://cayley.io and https://perkeep.org/ (previously Camlistore. Not a knowledge graph but a personal “life store”).

An interesting project is https://connectordb.io/ - intended to be a personal Google Analytics equivalent (known as “Quantified Self”). It has become dormant but they did make a number of data collection software for various platforms (browsers, OS).

A social search (powered by fossasia) https://loklak.org