| Model | Provider | Context Window |
|---|---|---|
| Claude Sonnet 4.6 | Anthropic (via Bedrock) | 1M tokens |
| Claude Opus 4.6 | Anthropic (via Bedrock) | 1M tokens |
| GPT-OSS 20B | OpenAI (via Bedrock) | 128K tokens |
| GLM 5 | Zhipu AI (via Bedrock) | 200K tokens |
| Model | Avg TTFT | Avg Total | Notes |
|---|---|---|---|
| Claude Sonnet 4.6 | ~1248ms | ~4.59s | Consistent, balanced output |
| Claude Opus 4.6 | ~1575ms | ~4.30s | Slower TTFT, similar total |
| GPT-OSS 20B | ~452ms | ~1.16s | 3x faster TTFT |
| GLM 5 | ~1415ms | ~2.54s | Variable TTFT, shorter responses |
| Model | Input / 1K tokens | Output / 1K tokens | Est. per request |
|---|---|---|---|
| Claude Sonnet 4.6 | $0.003 | $0.015 | $0.0060 |
| Claude Opus 4.6 | $0.015 | $0.075 | $0.0300 |
| GPT-OSS 20B | $0.00015 | $0.0002 | $0.000135 |
| GLM 5 | $0.0005 | $0.002 | $0.00085 |
Opus is 5x the cost of Sonnet and ~220x the cost of GPT-OSS 20B per request.
8 test cases across reasoning, factual recall, code generation, summarization, instruction following, creative writing, and safety. Claude Opus 4.6 served as the judge, scoring accuracy, relevance, and coherence (1-5).
→ Run the interactive evaluation dashboard
| Use Case | Model | Rationale |
|---|---|---|
| Quick Q&A / chat | Claude Sonnet 4.6 | Best balance of quality and cost at $0.006/req |
| Complex reasoning | Claude Opus 4.6 | Highest quality, similar total latency despite higher TTFT |
| High-volume / cost-sensitive | GPT-OSS 20B | ~220x cheaper, 3x faster TTFT |
| Multilingual / CJK | GLM 5 | Purpose-built for CJK, mid-range cost |
GPT-OSS 20B was the fastest model (~452ms TTFT vs ~1200-1600ms for others) but produced more verbose and occasionally less focused responses. Opus and Sonnet had similar total response times despite Opus having a higher TTFT, suggesting Opus generates tokens faster once started.
The quality gap between Sonnet and Opus was smaller than the 5x cost difference suggests. For most conversational use cases, Sonnet delivers comparable quality at a fraction of the cost. Opus justifies its premium only for complex multi-step reasoning or analysis tasks.
Sonnet and Opus were highly consistent across repeated runs (TTFT variance <300ms). GPT-OSS 20B showed more variance in output length (191-467 tokens for the same prompt). GLM 5 had the widest TTFT variance (758ms-1999ms), suggesting less predictable cold-start behavior.
For a production GenAI chat application, Claude Sonnet 4.6 offers the best overall value — strong quality at reasonable cost ($0.006/request) and acceptable latency. Opus should be reserved for high-stakes reasoning tasks where accuracy is critical and the 5x cost premium is justified. GPT-OSS 20B is ideal for cost-sensitive, high-volume workloads where sub-second TTFT matters more than output refinement.