Getting Started

Quickstart

Name: PromptDiff
Author: Bizmarq

Get your first comparison running in under 5 minutes.

1. Get your API key

2. Make your first comparison

quickstart.sh

curl -X POST https://promptdiff.bizmarq.com/api/v1/compare \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer pd_your_api_key_here" \
  -d '{
    "prompt": "What is the capital of France?",
    "models": ["gpt-4o-mini", "claude-haiku-4-5"]
  }'

3. Parse the results

Each result includes the model output, latency in milliseconds, cost in USD, and token counts.

response.json

{
  "eval_id": "eval_01j8abc123",
  "results": [
    {
      "model": "gpt-4o-mini",
      "provider": "openai",
      "output": "The capital of France is Paris.",
      "latency_ms": 612,
      "cost_usd": 0.000012,
      "input_tokens": 10,
      "output_tokens": 8,
      "total_tokens": 18
    }
  ],
  "meta": {
    "total_cost_usd": 0.000031,
    "fastest_model": "gpt-4o-mini",
    "cheapest_model": "gpt-4o-mini",
    "created_at": "2025-03-20T10:00:00Z"
  }
}

Using the Python SDK

example.py

from promptdiff import PromptDiff

pd = PromptDiff(api_key="pd_your_api_key")

result = pd.compare(
    prompt="Explain async/await in JavaScript.",
    models=["gpt-4o-mini", "claude-haiku-4-5", "gemini-2.5-flash"],
)

for r in result.results:
    print(r.model, r.latency_ms, r.cost_usd)
    print(r.output[:100])
    print()

Using the TypeScript SDK

example.ts

import { PromptDiff } from 'promptdiff';

const pd = new PromptDiff({ apiKey: 'pd_your_api_key' });

const result = await pd.compare({
  prompt: 'Explain async/await in JavaScript.',
  models: ['gpt-4o-mini', 'claude-haiku-4-5', 'gemini-2.5-flash'],
  temperature: 0.7,
  maxTokens: 300,
});

for (const r of result.results) {
  console.log(r.model, r.latencyMs, r.costUsd);
  console.log(r.output.slice(0, 100));
}