🧰 LLMKit

Your complete LLM toolkit for API testing and code generation

Compare Models

Compare 2-5 different models (GPT-4o, Claude 3.5, Gemini...) with the same input to see which model performs best in terms of quality, speed, and cost.

Comparing...: Claude 3.5 Sonnet (Latest), Gemini 2.0 Flash (Experimental)

Quick Start

Get started quickly with popular model combinations for common use cases

Popular Comparisons

See how top models stack up against each other

O
A
GPT-4o vs Claude 3.5 Sonnet
Coding tasks
Compare the latest flagship models for programming tasks
O
A
G
Budget Models Battle
Cost-effective options
Compare mini/haiku/flash models for cost-conscious projects
o1
O
Reasoning Champions
Complex problem solving
Test reasoning capabilities with o1 and Opus
O
A
G
Creative Writing Test
Content generation
Compare creativity and writing quality across top models

Real-time Streaming

  • • Live response streaming
  • • TTFT (Time To First Token) tracking
  • • Side-by-side comparison
  • • Performance metrics

Cost Analysis

  • • Accurate token counting
  • • Input/output cost breakdown
  • • Budget tracking & alerts
  • • Cost per comparison

Smart Scoring

  • • Custom scoring weights
  • • Speed vs cost vs quality
  • • Winner determination
  • • Export results to CSV

Use Quick Start above or enable Advanced Mode

Selected Models (0/5)

Enter API keys and test message to start comparison

No models selected yet

Select models from left column to start