← All hardware
A deterministic inference engine built for the lowest latency per token.
Pros
- Industry-leading low latency and TTFT
- Predictable, deterministic performance
- Easy OpenAI-compatible cloud access
Cons
- Tiny per-chip SRAM (230 MB) needs many chips for big models
- Inference-only; no training
- Throughput trails wafer-scale rivals on some models
✓ Where it shines / best for
- Real-time, latency-critical LLM applications
- Agentic and conversational AI needing fast TTFT
- Developers wanting drop-in OpenAI-compatible inference
✕ Not the best fit for
- Model training or fine-tuning
- Single-device deployment of very large models
- Maximum aggregate throughput over latency
Features
- ✓ Deterministic, software-scheduled dataflow architecture
- ✓ 230 MB SRAM per chip (no HBM bottleneck)
- ✓ Sub-100ms time-to-first-token
- ✓ Hundreds to 1,000+ tokens/sec depending on model
- ✓ GroqCloud inference API and on-prem GroqRack
- ✓ Predictable, low-latency performance
- ✓ OpenAI-compatible API endpoints
- ✓ Designed purely for inference
Pricing
| Plan | Price | Billing | Notes |
|---|---|---|---|
| GroqCloud Free | $0 | monthly | Free tier, no credit card; ~14,400 requests/day, 30 req/min rate limit, all models |
| GroqCloud Developer (pay-as-you-go) | $0.05-$0.79 per 1M tokens | per million tokens | Per-model input/output rates: e.g. Llama 3.1 8B $0.05 in / $0.08 out; GPT-OSS 120B $0.15 in / $0.60 out; Llama 3.3 70B $0.59 in / $0.79 out. Batch API and prompt caching each cut rates ~50%. |
| Enterprise / on-prem GroqRack | Contact sales | custom | Dedicated capacity and on-prem deployments; higher rate limits |
Pricing verified from the official source. Prices change often — confirm on the vendor's site before buying.
Specifications
| use | inference-only |
| latency | sub-100ms time-to-first-token |
| throughput | hundreds to 1,000+ tokens/sec (model-dependent) |
| architecture | LPU (deterministic dataflow), SRAM-based |
| on_chip_sram | 230 MB per chip |
Sponsored
A full review is being generated for this product and will appear here shortly.