Code Audit

Here we audit our Laravel codebase and provide background info.

Known issues

In rough order of priority:

• Downtime for unknown reason: We deploy with Laravel Forge+Envoyer connected to Hetzner. On 5/26 our three domains openagents.com, staging.openagents.com, and streamtest.openagent.com were all down for all users (browser loading just hanging). Our analytics site showed 0 visits all day. As soon as I triggered a manual staging deploy via Envoyer, all three sites went back up. We had no notification of this and can't tell why this happened. Need some kind of logging/alerts and figure out how to avoid a repeat of this.
• Our Livewire Chat.php component has way too much logic in it. It is difficult to reason about and has little test coverage.
• We have complex & inconsistent handling of selectedModel/selectedAgent across Chat.php, ModelDropdown.php, ModelSelector.php, SelectedModelOrAgentTrait.php. Multiple components and classes across the app need to access the currently selected model or agent and it is difficult to identify the source of truth.
• We have a new Payments integration and want to see if our handling of agent<>agent, user<>agent, agent<>user, user<>user payments is well structured. (see main PR and our Payable trait)
• We load the FeaturedAgents component in the empty Chat screen, including multiple heavy database calls (via Agent.php attributes) that should be cached, lazy loaded, or otherwise refactored.
• Inconsistent/poor error handling: When a chat message fails like runs over the context window, we dd() the error which makes a modal. Good verbosity - the user can screenshot that and we know where to look - but bad user experience.
• Inconsistent logging: We use a mix of the Log facade, a LocalLogger that writes simple logs to database, and OpenObserve which is also used by our protocol nodes. All are instantiated separately each invocation and should be consolidated to use dependency injection.
• We use env() outside of config files a lot, giving IDE warnings "env() function is used outside of config files. It always returns the default value if config is cached." We should probably consolidate env usage to config files.
• Completed: We have a lot of database migrations that could be consolidated for easier inspection of a database's table structure, if we have a process to do this safely in production
• Our code connects to a gRPC service we call "Pool" that acts as a gateway to the Nostr network, but several classes and variables related to this where erroneously prefixed with Nostr- instead of Pool. This is inconsistent and an oversimplification, they should be renamed: eg. NostrService -> PoolService

Questions

• How can we maximize test coverage / readability / simplicity of Livewire components?
• Is Pusher best / most scalable for broadcasting? (vs. Reverb/Wave)
• What are best practices for supporting multiple authentication providers? (social/email/custom)

Background

Our main repo OpenAgentsInc/openagents is a Laravel web application served at openagents.com.

Core functionality

The core feature of OpenAgents is chat with top LLMs and user-created agents.

Interactions with third-party AI APIs are managed via provider-specific Gateways in app/AI.

Recently added functionality (re Agent Store) focuses on creating & discovering user-built agents.

Upcoming functionality:

• May
  • Pay-as-you-go revenue-sharing [Details]
• June
  • Profiles & comments
  • Agent plugins UI
• Q3
  • Agent nodes UI
  • More social network features

Data models

Italic = Adding soon

• Agent - A user-created chat agent
• AgentFile - A file uploaded to an agent
• Balance - An agent or user currency balance
• Log - A simple log used by LocalLogger
• Message - Chat messages belonging to threads
• NostrAccount - Belongs to a User with Nostr-specific metadata
• NostrJob - A job from Nostr associated with an Agent
• Payment - A payment to/from an agent or user
• Post - A piece of content from user or agent; could be a reply to another post or reference Agents, Threads, Messages
• Thread - A chat with an agent
• User - Standard user model

Protocol communication

Our app uses gRPC to communicate to one or more external services described here.

• App\Utils\PoolUtils - Provides utility functions for managing tasks and gRPC communications, including job handling and discovery services.
• App\Grpc\nostr - Generated gRPC code here

RAG handshake

We use our pool for retrieval augmented generation and soon advanced agent interactions.

Definitions

• Pool: This is a gRPC service that acts as a gateway for the nostr network and provides some utilities.
• Job Request: A request to the network to perform a specific task.
• Warm-up Job Request: A special job request that is fired just to get the network to generate or refresh cache.

Flow

1. The user asks something to an agent.
2. We take the user question and use an LLM to generate a better question.
3. We call PoolUtils::sendRAGJob() with this question.
   • From there a gRPC request is made to the pool.
   • This request will post a Job Request to the network and return it to the Laravel app.
   • We save a reference for the Job Request in the database. The flow is suspended at this point, since the rest happens asynchronously.
4. At some point in time, the pool submits a JSON payload to the webhook.
   • The payload is an array with two values [type, data]. Type can be Job or Event; data is a JSON object.
   • Currently, we handle only payloads of type Job. The data of every payload of type Job contains the full Job object as tracked by the pool.
   • We parse this job object to find a result with status==3 (success).
   • We propagate this result by dispatching a JobResultReceiverJob.