Multilingual Meeting Transcription: How AI Handles Code-Switching

Multilingual meeting transcription works well when each speaker or part of the call stays in one language and makes more errors when people switch languages mid-sentence, which linguists call code-switching. A meeting transcript follows the usual rules of how to format a transcript, but one line may hold two languages. Below: what AI can do, code-switching types, why recognition fails, language detection, our test, scoring, meeting rules and translation.

Can AI transcribe a meeting in two languages?

Yes, AI can transcribe a meeting in two languages, and it does best when each speaker or each stretch of the meeting stays in one language; errors rise when people switch languages mid-sentence or talk over each other.

A transcription service's list of supported languages tells you which languages it can transcribe one at a time, not whether it survives code-switching on a bilingual call. To transcribe audio in multiple languages, find out how your transcription app picks the language, keep one language per speaker or per agenda item where you can, and check the second-language parts against the recording.

What is code-switching in a meeting?

Code-switching is moving between two languages in one conversation — between speakers, between sentences, or inside a single sentence. The classic typology in linguistics comes from Shana Poplack's 1980 paper "Sometimes I'll start a sentence in Spanish y termino en español", based on the speech of 20 Puerto Rican residents of East Harlem, New York. She sorted code-switches into tag-like, sentential (now usually called inter-sentential) and intra-sentential; speech recognition guides add turn-level switching, where each speaker keeps one language.

Intra-sentential switching is often called code-mixing, or mixing languages mid-sentence: Spanglish for Spanish and English, Hinglish for Hindi and English. Poplack found that fluent bilinguals favored intra-sentential switches and used tag-like ones least, so the pattern hardest for AI is the most natural one for fluent bilingual speakers.

A second language at home is common in the US. In the Census Bureau's 2018–2022 American Community Survey, 78.3% of people age 5 and older spoke only English at home, so more than 1 in 5 spoke another language, most often Spanish (61.1% of them). In business, international teams and customer calls add more language pairs.

Why does code-switching break speech recognition?

Most speech recognition decides which language it is hearing before it transcribes; when that guess is wrong, the second language comes out as nonsense in the first language's spelling, gets dropped, or is translated instead of transcribed. OpenAI's open-source Whisper model shows the mechanism: unless you name a language, its transcribe function detects one from up to the first 30 seconds of audio and decodes the whole file in it. That's a language lock: a call that opens in English stays English to the model.

Peng, Yan, Watanabe and Harwath (Interspeech 2023) measured the cost with Whisper Large on two Mandarin-English code-switched corpora: ASCEND, recorded in Hong Kong, and SEAME, from Singaporean and Malaysian speakers.

• Wrong-language output. Told to expect English, the model's error rate on Mandarin sentences in ASCEND hit 90.4%; told to expect Mandarin, its error rate on English sentences hit 93.1%.
• Mixed sentences stay hard. With automatic detection, the error rate on code-switched sentences was 26.6% on ASCEND and 43.2% on SEAME.
• Accents mislead detection. On SEAME, automatic detection pushed the English error rate to 85.5%, against 33.8% when the model was told the audio was English; the authors suspect accents.
• Translated output. Checking errors by hand, the authors "found a common scenario where the model outputs monolingual translation for code-switched utterances."

The authors' fix, prompting the model with both language tokens at once, cut the overall error rate on SEAME by 19% relative to the default.

Training data is the deeper cause. The team behind ASCEND (Holy Lovenia and colleagues, LREC 2022) calls code-switching recognition "several times inferior" to monolingual recognition and attributes the gap to limited code-switching corpora. Code-switches are also short: around 70% of English segments inside mixed ASCEND sentences were one or two words long, which leaves little context. That's why a client's name dropped into a Spanish sentence is easy to miss.

How do transcription tools handle more than one language?

Tools pick the language at one of four levels — per file, per speaker, per segment or per word — and only the last one is built for mid-sentence switching. That choice, called language identification (automatic language detection on speech audio), decides which code-switching patterns a transcription service can follow.

Per file is the simplest setup in transcription software. The Whisper function above works this way. Meetlens also detects the dominant language of the recording, and its feature page warns that frequent code-switching can lower transcription accuracy, though in our test below it kept second-language words in their own language.

Per speaker is how multilingual speech recognition in Microsoft Teams works: "each participant can select their own preferred spoken language" for live captions and transcripts. It's turned on in the meeting's Options, under Recording and transcription; it needs Teams Premium or a Microsoft Copilot license and supports nine languages, including English, Spanish and French. In a recorded file, where the software doesn't know who is speaking, voices first have to be told apart by speaker diarization, which adds speaker labels, so a per-speaker setting is only as good as those labels.

To choose between one language and automatic detection, select the language yourself when the whole call is in one language, use automatic detection when you don't know it, and never force one language on a bilingual call: the forced-language results above show what happens to the other half.

Our test: Meetlens on real English, Spanish and Spanish-English conversations

On single-language stretches of real conversation, Meetlens had an 11.8% word error rate (WER); it handled one language per speaker well and never locked into one language: Spanish stayed Spanish and English stayed English.

On speech without crosstalk, the bilingual conversations stayed close to the single-language controls: 13.5% WER when each speaker kept mostly to one language and 10.6% when whole sentences switched language, against 8.7% (English) and 9.1% (Spanish). A Mandarin-English conversation scored an 11.0% mixed error rate, with Chinese characters and English words counted together. Harder conditions cost more: when languages switched inside a sentence and people talked over each other, the error rate over all words reached 22.3%, about 22 words in 100, and short second-language phrases were sometimes translated or dropped, mostly where two people spoke at once. For contrast, open-source Whisper (small model, automatic language detection) took that same Spanish-English excerpt for English and turned most of the Spanish into English: 95% of the Spanish words came out wrong.

Disclosure: we build Meetlens, so this is a test of our own product, and weak results stay in.

How we tested. Meetlens is a Chrome extension that records browser meetings without a bot and turns them into transcripts and AI reports; it also imports recorded audio files, which is how we ran this test. Instead of staging meetings, we used real, unscripted conversations from two open research corpora, in which trained transcribers had already written down every word and tagged its language: the Bangor Miami corpus of Spanish-English conversations recorded in Miami in 2008–2011 (GNU GPL v3 or later), and ASCEND, Mandarin-English conversations recorded in Hong Kong (CC BY-SA 4.0). Participants in both gave written consent to the recording and to its use in research. These are casual talks among family, friends, neighbors and students, not work meetings. We cut six excerpts of about 6.5 minutes each and uploaded them as audio files through Import in the Meetlens web app on October 16, 2026, rather than recording live calls. Two controls ran in one language, an English and a Spanish stretch of the Miami conversations; the others varied the language mix and the type of code-switching. We planned a Russian-English call too, but found no openly licensed recording of real Russian-English conversation with a checked transcript, so it is not in the test.

Second-language share is the share of second-language words in the reference transcript (for M6, English words among Chinese characters and English words). The controls are the most single-language stretches we found in the corpus: M1 still has 2% Spanish words and M2 7% English words.

Each excerpt's reference transcript is the corpus's own: every word in its original language and script, tagged by language, written by trained transcribers (the Miami transcripts were proofread by a second member of the team; ASCEND was transcribed by a professional annotation company). Before looking at any Meetlens output, we wrote an answer key of 5 key points per excerpt (plans, tasks and facts a report should keep), some spoken in the second language; in casual conversation these are mostly plans and facts rather than formal decisions. The answer key measures report coverage: how many items reached the English report.

"WER, all words" and the main- and second-language columns count every word; "no crosstalk" counts only words spoken while no one else was talking (in M4 and M5, 42% and 46% of words overlapped another speaker, against 19% and 25% in the controls; M6 is one microphone track, so it has no such split). For M6 the scores are character error rate (CER) on Mandarin and mixed error rate (MER) over Chinese characters and English words. "In the report" includes points the report kept with a detail missing or wrong (one in M3, two in M5, one in M6); in M4 the report left out one whole topic, in both languages.

In the bilingual conversations (M3–M5), 27% of all errors fell within two words of a code-switch point, where 20% of the words are, so the switch itself adds only a little; most second-language phrases that were dropped (13 of 15) were spoken while someone else was talking.

What this means:

• Use the meeting transcript as is for single-language calls, one language per speaker and whole sentences in one language: in M3 and M4, second-language words had fewer errors than main-language words (16.7% vs 18.9% and 16.3% vs 20.3% over all words), and on speech without crosstalk both conversations stayed within 5 points of the single-language controls.
• Follow the meeting rules below for mixing languages mid-sentence, in Spanish-English and Mandarin-English: on speech without crosstalk the Spanish-English conversation (M5) scored 8 points above the Spanish control, and a few short second-language phrases came out translated.

Limits. Six excerpts of about 6.5 minutes from two research corpora make a small sample. They are casual conversations among family, friends, neighbors and students, not work meetings, so the answer keys hold plans and facts rather than formal decisions, and the report results say more about what a report picks from a chat than about language. We uploaded audio files instead of recording live calls, ran one product version on one date, and scored against the corpus transcripts as published; the corpus replaced people's names with pseudonyms, so names of people were not scored. There is no Russian-English result: we found no openly licensed recording of real Russian-English conversation with a checked transcript. Read the scores as one team's check of its own product, not a benchmark.

How to measure multilingual transcription accuracy

Score each language separately — word error rate for languages with spaces, character error rate for Chinese or Japanese, mixed error rate for both — and count translated or dropped phrases on their own, because WER hides them. One score for multilingual meeting transcription averages a clean main language with a broken second one. Word error rate itself is the standard speech-to-text measure; here is what changes when a meeting has two languages:

1. Build a reference transcript (ground truth) in the original languages and scripts: Spanish in Latin letters, Russian in Cyrillic, Mandarin in Chinese characters.
2. Tag every word with its language and mark each code-switch point.
3. Compute per-language WER by splitting the errors by the language tag of each reference word.
4. Use CER or MER where words have no spaces. Mixed error rate counts Mandarin characters and English words as tokens in one score, as in the TALCS corpus paper. In the Python library jiwer, "MER" means match error rate, a different metric.
5. Count errors within two words of each code-switch point to see whether the code-switch itself causes them.
6. Count translated and dropped phrases by hand. WER shows a translated phrase as substitutions and a dropped one as deletions, so it can't tell meaning kept in the wrong language from meaning lost.
7. Check critical terms: names, numbers, dates, product names, and negations or conditions ("no", "unless", "only if"), where one wrong word flips a decision.

Public corpora make a fair benchmark for comparing models, but a 10-minute recording of your own team, scored this way, shows how transcription accuracy holds up for your language mix.

How to run a bilingual meeting so the transcript works

Agree on one main language, keep each agenda item in one language where you can, and restate decisions in the main language before moving on. These rules don't stop code-switching; they make sure that what matters lands in the meeting transcript in a form you can check:

1. Pick the main language before the call and put it in the invite: the language for decisions, numbers and the report.
2. Assign a language to agenda items. The Madrid team's update can run in Spanish while the budget item runs in English.
3. Restate decisions in the main language. Illustrative: "Entonces, vamos a contratar a dos personas más para soporte. In English: we're hiring two more people for support." Confirm with a full sentence, not a one-word "sí": a very short reply can end up in the neighboring speaker's line.
4. Say names and numbers clearly, then repeat them. Spell a surname or product name that comes up in the other language.
5. Let a speaker finish the sentence where they change language. Overlapping speech at a code-switch gives the model two problems at once.
6. Record long other-language parts as a separate recording. If your transcription app locks onto one dominant language per recording, as Whisper did in our test, stop before a long Spanish segment and start a new recording, so each part gets its own dominant language.
7. Check decisions by timestamps after the call. Jump to each decision in the recording and confirm the words, especially those said in the second language.

Transcribing two languages is not the same as translating between them.

Multilingual transcription vs. translation: what you get

Transcription keeps every speaker's original words; translation turns them into another language — Meetlens does the first and writes the AI report in English, Spanish, German or Russian. The meeting transcript stays in the detected spoken language, with speaker labels and timestamps, so the original-language transcript keeps a Spanish quote in Spanish. The report language is shown above the report; switching it to another language (Report translation) is part of the Business plan.

Meetlens doesn't translate the meeting transcript line by line and doesn't provide interpretation or translated captions during the call: the transcript and report are prepared after you stop the recording, not in real time. If participants need to understand each other live, that takes an interpreter or a live-caption feature in the meeting platform. Report translation is listed as a Business plan feature on the pricing page.

For the language list and report options, see transcription in 99+ languages. To check multilingual meeting transcription on your own language mix, record a short bilingual call on the Free plan and score it with the steps above.