Why we're still betting on Llama, not OpenAI

Every six weeks someone asks why DMOOP isn't on GPT-5 or Claude 4.7 Sonnet. The answer is mostly economic, partly architectural, and entirely contingent. Here's the math.

The cost gap is bigger than the model card suggests

GPT-5 at $1.25/M input + $10/M output is the standard premium-tier number. Claude 4.7 Sonnet is roughly the same. For a marketing chat that averages ~3,500 input + ~2,000 output tokens per turn, that's $0.024 per conversation. Sounds cheap until you do the math at scale.

Groq's Llama 4 Scout (17B MoE) is free at our usage tier. Free vs $0.024 is infinite ratio. Even if Scout's quality were 30% worse on the metrics that matter (which it isn't), we'd be running it.

The fallback chain — Scout → Kimi-k2 → 8B-instant — gives us four independent free-tier TPM buckets to draw from across the day. With four tiers totaling ~52K tokens/minute of free throughput, hitting an actual quota wall requires more than ~30 simultaneously active users. That's a problem we want to have.

The quality gap closes monthly

Six months ago, GPT-4 had a real edge on multi-step reasoning. Today Llama 4 Scout matches it on most marketing-specific benchmarks. On the actual workload we measure — copywriting for B2B SaaS, ABM playbook drafting, GTM strategy responses — the thumbs-up/thumbs-down ratio is statistically indistinguishable between Scout and Claude Sonnet. We A/B tested for two weeks. The gap isn't there.

This is the part most "OpenAI is still 30% better" benchmark posts miss. The benchmarks (GPQA Diamond, MMLU Pro, AIME) measure PhD-level reasoning, math olympiad performance, and academic knowledge. Marketing is structured opinion writing with named frameworks. The benchmark gap doesn't transfer.

The latency gap reverses

Groq's TTFT (time to first token) on Llama 4 Scout is consistently under 300ms. Claude Sonnet's is ~800ms. GPT-5's is ~1.2s. For a streamed response, the TTFT is what the user perceives as "the model started thinking" — and Groq is meaningfully faster.

The full-response latency on a 1,500-token answer: Groq ~3s, Claude ~5s, GPT-5 ~6s. For users hammering through quick iteration cycles ("shorter," "in US English," "as a deck"), 3s vs 6s compounds into a different product feel.

The architectural commitment

If we'd built DMOOP on the OpenAI SDK with hard-coded GPT-5 model IDs, swapping providers later would require touching every chat-route file. Because the SDK shim is OpenAI-compatible and the model ID is a single env var, we can A/B test Groq vs Anthropic vs OpenAI on the same code path. The cost of being wrong about Llama is "change one env var." That asymmetry is the actual architectural argument.

What would flip this

Three things would flip our model strategy:

1. A paying enterprise customer requires SOC 2 attestation that includes the inference provider. Groq has SOC 2; we'd verify the customer's specific compliance scope.
2. Tool-calling reliability becomes the bottleneck. Llama's function-calling is good but not yet on Claude's level. We don't depend on it today.
3. Llama itself stops shipping competitive base models. Meta has so far shipped roughly on Anthropic's cadence. Watching.

For now: free Llama via Groq, fallback chain for resilience, every dollar saved goes into corpus growth. The corpus is the moat, not the model.