LLMeter

LLMeter is a tool designed to help you test and evaluate prompts for various language models. It provides insights into performance and cost, allowing you to optimize your usage effectively.

Features

• Multi-Model Support: Test prompts across GPT-3.5-turbo, Gemini-2.0-flash, and more
• Cost Intelligence: Real-time token cost estimation with currency conversion
• Performance Metrics: Response time tracking and quality comparisons
• Prompt Versioning: Save and compare different prompt variations
• API Agnostic: Works with multiple LLM providers simultaneously
• Security First: API keys stored locally in browser storage

Quick Start

Live Demo

Access the production deployment directly:
👉 llmeter.vercel.app

Local Development

1. Clone the repository:

   git clone https://github.com/IremOztimur/llmeter.git

2. Install dependencies:

   cd llmeter && npm install

3. Start the development server:

   npm run dev

4. Access at http://localhost:3000

Configuration

Model Setup

1. Click Config in the navigation bar
2. Select your target model from the dropdown
3. Add your API keys:
   • Get OpenAI API Key
   • Get Gemini API Key
4. Save configuration

Configuration Screenshot

Contributing 🤝

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit changes (git commit -m 'Add amazing feature')
4. Push to branch (git push origin feature/amazing-feature)
5. Open a Pull Request