Getting Started

Quickstart

Get your first comparison running in under 5 minutes.

1. Get your API key

2. Make your first comparison

quickstart.sh

curl -X POST https://promptdiff.bizmarq.com/api/v1/compare \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer pd_your_api_key" \
  -d '{
    "prompt": "What is the capital of France?",
    "models": ["gpt-4o-mini", "claude-3-haiku"]
  }'

3. Parse the results

Each result includes the model output, latency in milliseconds, cost in USD, and token counts.

response.json

{
  "eval_id": "eval_01j8abc123",
  "results": [
    {
      "model": "gpt-4o-mini",
      "provider": "openai",
      "output": "The capital of France is Paris.",
      "latency_ms": 612,
      "cost_usd": 0.000012,
      "input_tokens": 10,
      "output_tokens": 8,
      "total_tokens": 18
    }
  ],
  "meta": {
    "total_cost_usd": 0.000031,
    "fastest_model": "gpt-4o-mini",
    "cheapest_model": "gpt-4o-mini",
    "created_at": "2025-03-20T10:00:00Z"
  }
}

Using the Python SDK

example.py

import promptdiff

client = promptdiff.Client(api_key="pd_your_api_key")

result = client.compare(
    prompt="Explain async/await in JavaScript.",
    models=["gpt-4o-mini", "claude-3-haiku", "gemini-1.5-flash"],
    options={"temperature": 0.7, "max_tokens": 300}
)

for r in result.results:
    print(r.model, r.latency_ms, r.cost_usd)
    print(r.output[:100])
    print()