Speech-to-text (STT) has quietly become good enough to build real products on. Meeting notes, call-center analytics, voice assistants, podcast search, medical scribing — they all start with the same step: turn audio into accurate text. The hard part isn't finding a model that works. It's picking the one that fits your accuracy needs, your latency budget, and your cost ceiling without painting you into a corner.
This post walks through how we think about that choice: the two big families of options, the trade-offs that actually matter, and the specific situations where self-hosting an open model beats paying for an API.
Two families: open models and hosted APIs
Almost every STT decision comes down to a choice between two camps.
Open models you run yourself. OpenAI's Whisper is the best-known example, and the wider ecosystem has grown around it — faster reimplementations, quantized versions that run on modest GPUs, and streaming forks. You download the weights, run inference on your own hardware, and own the whole pipeline.
Hosted APIs. Deepgram, AssemblyAI, Google, and others expose transcription as a paid endpoint. You send audio, you get text back. Someone else owns the GPUs, the scaling, and the model updates.
Neither is universally "better." The right answer depends on a handful of concrete requirements, so let's go through them.
The trade-offs that actually matter
Accuracy — but on your audio
Every vendor publishes accuracy numbers, and they're all measured on clean, read-aloud speech that looks nothing like your data. What matters is accuracy on your audio: your accents, your background noise, your domain jargon, your phone-line compression.
The only reliable way to know is to test. Collect 20–50 representative clips — real calls, real meetings, real field recordings — transcribe them by hand once, and use that as a reference set. Run each candidate model against it and compare. This takes a day and saves you from choosing a model on marketing claims.
A few things reliably hurt accuracy regardless of model:
- Domain vocabulary. Drug names, product SKUs, legal terms, and people's names are where general models stumble. Hosted APIs often let you supply a custom vocabulary or "keyword boost" list; open models can be fine-tuned but that's a bigger project.
- Overlapping speakers and cross-talk. Two people talking at once degrades every model. If you need to know who said what, that's speaker diarization — a separate capability, and quality varies a lot between vendors.
- Audio quality. An 8kHz phone recording carries less information than a 48kHz studio mic. No model recovers what the signal never captured.
Latency and streaming
There's a hard split between two modes, and it drives the whole architecture.
Batch (asynchronous): you have a finished audio file and want a transcript. A minute of processing delay is fine. This is the easy case — most models and APIs handle it well.
Streaming (real-time): you need words to appear as someone speaks, for live captions or a voice assistant. This is much harder. The model has to emit partial results with only a fraction of a second of audio, then revise them as more context arrives. Not every model streams well, and a model that's excellent in batch mode can be mediocre in streaming mode.
If your product is real-time, make streaming quality a first-class evaluation criterion from day one. Retrofitting it later often means changing models.
Cost — and how it's shaped
Cost structure differs fundamentally between the two camps, and that shape matters more than the headline rate.
| Hosted API | Self-hosted open model | |
|---|---|---|
| Pricing model | Per minute (or per second) of audio | Cost of GPU time you run |
| Scales with | Volume — you pay for every minute | Utilization — idle GPUs still cost money |
| Cheapest when | Volume is low or spiky | Volume is high and steady |
| Hidden costs | Egress, add-ons (diarization, etc.) | Ops, on-call, model updates |
The crossover is about utilization, not just volume. A hosted API charges you only for audio you actually process — perfect for low or unpredictable traffic. A self-hosted GPU charges you whether it's busy or idle, so it only wins when you keep it reasonably full. Processing a steady, high volume of audio around the clock is where self-hosting tends to pull ahead; bursty, business-hours-only traffic often favors the API even at higher per-minute rates.
Run the arithmetic with your own expected volume before deciding. And include the operational cost of self-hosting — someone has to keep those servers running.
Privacy, compliance, and data residency
For some workloads this decides everything. If you're transcribing medical conversations, legal calls, or anything under strict data-handling rules, sending audio to a third-party API may be off the table — or may require a specific agreement and region. Self-hosting keeps the audio inside your own environment, which is sometimes the entire reason to do it.
Don't assume, though. Most serious STT vendors offer compliance options, regional processing, and contractual data protections. The question isn't "API or private" — it's "does this specific vendor meet our specific obligations."
Languages and specialization
If you work in one or two major languages, almost everything supports you well. The picture changes for less-common languages, code-switching (speakers mixing languages mid-sentence), or heavy domain specialization. Here, test candidates head-to-head on real samples — support for a language on a feature list doesn't tell you whether it's good.
When self-hosting actually pays off
Self-hosting an open model is appealing — no per-minute bill, full control — but it comes with real operational weight: GPUs to provision, autoscaling to tune, models to update, and an on-call rotation when the pipeline stalls at 2 a.m. It pays off in a few clear situations:
- High, steady volume. When you can keep GPUs well-utilized, the per-minute economics beat an API.
- Hard data-residency requirements that make sending audio out impractical.
- A need to fine-tune the model deeply on your domain — accents, jargon, or audio conditions a general model handles poorly.
- You already run GPU infrastructure for other workloads, so the marginal ops cost is low.
If none of those apply, a hosted API is usually the faster, cheaper-in-practice path — especially early, when your volume is uncertain and your team is small. The worst outcome is standing up a GPU cluster to save on a bill you don't have yet.
A decision path that works
Here's the sequence we'd follow on a new project:
- Define the mode. Batch or streaming? This narrows the field immediately.
- Build a reference set. 20–50 real clips, hand-transcribed once. This is your ground truth for every comparison that follows.
- Start with a hosted API. It's the fastest way to a working prototype and a real accuracy read on your data. You'll learn what "good enough" means for your users before committing to infrastructure.
- Measure accuracy, latency, and features against your reference set — including diarization and custom vocabulary if you need them.
- Project the cost curve. Estimate your steady-state volume and utilization. Only if the numbers clearly favor self-hosting — and you can carry the ops load — should you plan that move.
- Keep the interface swappable. Put transcription behind a thin internal interface so switching providers later is a small change, not a rewrite.
That last point is the cheapest insurance you can buy. The STT landscape moves fast; a model that's best today may be second-best in a year. If your application talks to a small transcription abstraction rather than a specific vendor's SDK sprinkled throughout the code, you keep the freedom to change your mind.
The short version
Pick based on your real requirements, not benchmark tables. Test candidates on your own audio before committing. Start hosted, measure honestly, and self-host only when steady volume, compliance, or deep customization clearly justify the operational cost. And whatever you choose, wrap it so you can change it — because you probably will.
Working on a product that needs speech-to-text? Get in touch — we design and build audio and AI pipelines end to end.