Compare AI Transcription Tools With Word Error Rate
Word error rate counts substitutions, deletions and insertions relative to the number of words in the reference. Use the same normalization for each tool and keep a separate record of high-impact errors.
Worked example
Synthetic word error rate calculation
Reference: we packed six blue boxes Output: we packed seven boxes One substitution (six → seven), one deletion (blue), zero insertions. Word error rate = 2 ÷ 5 = 40%.
Steps
- Prepare a short approved recording and verify a human reference transcript against it.
- Choose one normalization rule for punctuation, case and number spelling. Apply it equally to the reference and each result.
- Align the words, count substitutions, deletions and insertions, then divide by the reference word count. Flag changed dates, names and quantities separately.
What to check
- The denominator is the five reference words.
- Decide beforehand whether a numeral and its spelled-out equivalent should count as the same word.