Shikigami

Personal conjured assistant template for fellow onmyōji 🔖

Vision

A self-contained, learning, fully offline virtual assistant. This requires gigabytes of space for all the AI models (llama, wav2vec, etc.), probably a GPU if you expect semi-normal response times, and a decent CPU as well, plus peripherals (camera, mic, speakers, etc.).

This project doesn't use any APIs, and while in theory it could all run on one beast of a machine, in practice it is designed to allow each module to run independently across as many systems as needed, as long as they all share secure LAN access to the beanstalkd process.

Setup:

Start by running init.sh after you create your own config.yml file.

shikigami is meant to be run in parallel as a series of ruby scripts under pm2 (if you need another process, add it as another ruby script for pm2. Ruby files here respect the GIL).

This is currently the system I use to automate my life, sharing here in case anyone else finds it useful. Recommended to use with nvm and rvm under a dedicated user account.

Architecture:

                                            
 ┌────────────────────────────────────────┐ 
 │               PM2 Daemon               │ 
 └─────┬───────────────────────────┬──────┘ 
       │                           │        
       ▼                           ▼        
  ┌─────────┐                   ┌─────┐     
  │ core.rb ├───────────────────┤     │     
  └────┬────┘                   │     │     
       │                        │     │     
       ▼                        │     │     
     ┌───┐                      │  B  │     
     │   │     ┌───────────┐    │  e  │     
     │   ├────►│ module 00 ├────┤  a  │     
     │   │     └───────────┘    │  n  │     
     │ M │                      │  s  │     
     │ o │     ┌───────────┐    │  t  │     
     │ d ├────►│ module 01 ├────┤  a  │     
     │ u │     └───────────┘    │  l  │     
     │ l │                      │  k  │     
     │ e │     ┌───────────┐    │  d  │     
     │ s ├────►│ module 02 ├────┤     │     
     │   │     └───────────┘    │     │     
     │   │                      │     │     
     │   │     ┌───────────┐    │     │     
     │   ├────►│ module NN ├────┤     │     
     └───┘     └───────────┘    └─────┘     
                                            

Events from external resources (chat clients, databases, filesystems) are processed by the appropriate module, or queued into beanstalkd as raw lines of ruby code. Each module is responsible for routing its events, which can be sent to another module or core.rb, which will spawn a new thread and executes eval() on the message body. Every directory under modules with a valid wrapper.sh file will automatically be detected by core.rb and sent to PM2 for startup and persistence.

Resources:

• llama.cpp
• Llama Family
• Discordrb Ruby Gem
• Spriggan Ruby Gem
• RedFairyBook Ruby Gem

Training Data:

The Large Language Model (LLM) used in this project is currently Llama 3.1, which is trained on the following:

• 67.0% CommonCrawl
• 15.0% C4
• 4.5% GitHub
• 4.5% Wikipedia
• 4.5% Books
• 2.5% ArXiv
• 2.0% StackExchange

*** VERY MUCH A WORK IN PROGRESS ***