🧰 LLMKit

Your complete LLM toolkit for API testing and code generation

Compare Models

Compare 2-5 different models (GPT-4o, Claude 3.5, Gemini...) with the same input to see which model performs best in terms of quality, speed, and cost.

Comparing...: Claude 3.5 Haiku, GPT-4o Mini

Quick Start

Get started quickly with popular model combinations for common use cases

Popular Comparisons

See how top models stack up against each other

O
A
GPT-4o vs Claude 3.5 Sonnet
Coding tasks
Compare the latest flagship models for programming tasks
O
A
G
Budget Models Battle
Cost-effective options
Compare mini/haiku/flash models for cost-conscious projects
o1
O
Reasoning Champions
Complex problem solving
Test reasoning capabilities with o1 and Opus
O
A
G
Creative Writing Test
Content generation
Compare creativity and writing quality across top models

Real-time Streaming

  • • Live response streaming
  • • TTFT (Time To First Token) tracking
  • • Side-by-side comparison
  • • Performance metrics

Cost Analysis

  • • Accurate token counting
  • • Input/output cost breakdown
  • • Budget tracking & alerts
  • • Cost per comparison

Smart Scoring

  • • Custom scoring weights
  • • Speed vs cost vs quality
  • • Winner determination
  • • Export results to CSV

Use Quick Start above or enable Advanced Mode

Selected Models (0/5)

Enter API keys and test message to start comparison

No models selected yet

Select models from left column to start