Skip to content

Kennedy: Crawler and Search Engine for Gemini space. Leverages techniques and architecture from early WWW crawlers like Mercator, Archive.org, and GoogleBot

Notifications You must be signed in to change notification settings

acidus99/Kennedy

Folders and files

NameName
Last commit message
Last commit date

Latest commit

1bc232d · Jan 2, 2025
Nov 10, 2024
Jan 2, 2025
Jan 2, 2025
Jan 2, 2025
May 2, 2023
Jan 2, 2025
Jan 2, 2025
Nov 10, 2024
May 15, 2024
Jul 12, 2023
Jul 21, 2022
Jan 1, 2024
Nov 10, 2024
May 12, 2024
Aug 14, 2022
May 27, 2024
Jun 10, 2023
Oct 8, 2023
Jan 27, 2023

Repository files navigation

Kennedy

Kennedy is a search engine for Gemini space. It consists of a crawler, backend, and Gemini app server. Kennedy leverages many of the techniques and architecture from early WWW crawlers and search engines like Mercator, Archive.org, and GoogleBot.

Kennedy running in Lagrange client

Demo

Visit gemini://kennedy.gemi.dev with a Gemini client or via an HTTP-to-Gemini proxy

Features

  • Full Text search, with Porter Stemming
  • Suggested queries. Kennedy recommends other queries if you don't get many results
  • Complex search queries. For example: "cats AND dogs", "(cats OR dogs) NOT birds".
  • Image Search! Kennedy indexes link text and path info to enable searching for images.
  • Content language classification using ngrams instead of MIME types lang= parameters
  • PageRank-dervied algorithm to better determine results relevance
  • Clean Snippets: Search results include a snippet of content which matches your query. Your keywords are [surrounded] with brackets, and gemtext formatting is removed to make it cleaner to read.
  • Lines count: To help you sort quick articles from longer form content, search results tell you how many lines are in the content. If you want to be nerdy, I also include byte sizes.

Why?

Many years ago the great British explorer George Mallory, who was to die on Mount Everest, was asked why did he want to climb it. He said, "Because it is there."

John F. Kennedy Address at Rice University, Sept. 12, 1962

Projects

  • Kennedy.Crawler - Crawler logic (Url Frontiers, Queues, etc)
  • Kennedy.CrawlData - Models and storage systems for documents, meta data, and full text search
  • Kennedy.Server - Gemini Server to handle queries and search results. Built on top of RocketForce, a .NET Gemini server and application framework
  • Kennedy.SearchConsole - Console app for running FTS queries. Used for testing

About

Kennedy: Crawler and Search Engine for Gemini space. Leverages techniques and architecture from early WWW crawlers like Mercator, Archive.org, and GoogleBot

Topics

Resources

Stars

Watchers

Forks

Languages