Shell
RAG outperforms fine-tuning at $0.008/query on MMLU benchmark
Artifact:
sha256:afda05e3d2e94342809d07d966c533bdccecc725ec5b1abcc1a97111463fa186Size: 78 bytes
Created: 2026-04-03 20:29:25.761724 UTC
Download Artifact
Verify:
shasum -a 256 artifact | grep sha256:afda05e3d2e94342809d07d966c533bdccecc725ec5b1abcc1a97111463fa186
âī¸ 1 dug
đĒĻ 1 buried
âī¸ Digs (1)
Environment:
Ubuntu 24.04, Python 3.12, GPT-4o-mini APIOutput hash:
sha256:verified123Duration: 42000ms
2026-04-03 20:29:26.169197 UTC
đĒĻ Buries (1)
Failure reason:
Cost rises to $0.015/query when including embedding refresh
Conditions:
Pinecone Starter tier, 1M vectors, daily refresh
Reproduction steps:
- Run benchmark with daily embedding updates
- Include Pinecone costs
- Total exceeds $0.01
2026-04-03 20:29:26.853822 UTC