by Hazel
Converting audio files to text can reduce the time spent listening back to meetings or customer service calls. Real-world recordings, however, often contain sounds that interfere with speech recognition, such as vacuum cleaners, road noise, or people talking nearby. In these conditions, short words may be omitted or mistaken for other words with similar pronunciations.
A transcript still takes work to review: you need to check for errors, find key details, and identify next steps.
In this tutorial, we will build a Python CLI that uses the Return Zero (RTZR) STT OpenAPI to run base transcription, transcript refinement, and insight generation. We will then use Korean audio with background noise and structured conversation data to examine the following questions:
- Which transcription errors does refinement correct?
- How do the refined results compare with other STT models on the same audio?
- Does insight generation correctly extract information from the transcript based on the prompt?
1. Goals
What We Will Build
This example uses three features.
| Feature | What It Does | Output to Review |
|---|---|---|
| Batch STT | Transcribes audio and returns text for each utterance. | Base transcript |
| Transcript refinement | Checks the transcript against the audio to correct text errors | Refined transcript |
| Insight generation | Extracts information from the transcript based on a prompt | Title, category, and summary |
Some utterances may remain unchanged after refinement.
Insight generation extracts information from the transcript based on a prompt. The result includes a title, summary, and one of the following categories: 일상, 문의, 업무, or 스팸. Adding a prompt lets you focus on the information you need, such as requests made during a customer service call or decisions made in a meeting.
💡 Enterprise Features
Transcript refinement and insight generation require an Enterprise license. Check that they are enabled for your account. If they are unavailable, contact RTZR about Enterprise access.
Audio file
↓
Submit a transcription request to the RTZR STT API
↓
The RTZR STT API returns results
├─ Base transcription results
├─ Title, category, and summary included if insight generation was requested
└─ Refined results available separately if refinement was requested
↓
The CLI formats and saves the results
Output Preview
By default, running the CLI creates an output directory with the following structure.
outputs/<audio-name>-<UTC-run-timestamp>/
├── transcript.txt # Final transcript organized by speaker
├── response.json # Base transcription API response
├── insight.txt # Created when insight generation is enabled
└── refined.json # Created when refinement is enabled
The RTZR API returns transcription and insight results as JSON. The example CLI collects the speaker ID and transcribed text for each utterance into transcript.txt, and writes the insight title, category, and summary to insight.txt. The files response.json and refined.json preserve the original JSON responses for further analysis.
2. Project Setup
Project Structure
c1-test/
├── src/rtzr_transcribe/
│ ├── cli.py # CLI options and output saving
│ ├── client.py # Authentication, transcription requests, and polling
│ ├── config.py # API request settings and environment variables
│ ├── formatters.py # Conversion of JSON responses to TXT
│ └── io.py # Input validation and file saving
├── tests/
├── .env.example
├── pyproject.toml
└── uv.lock
The example runs on Python 3.10 or later and uses uv to manage packages and virtual environments. After downloading the repository, install the dependencies with the following commands.
uv sync --locked
cp -n .env.example .env
Enter the Client ID and Client Secret issued through the RTZR developer site in .env.
RTZR_CLIENT_ID=YOUR_CLIENT_ID
RTZR_CLIENT_SECRET=YOUR_CLIENT_SECRET
RTZR_BASE_URL=https://openapi.vito.ai
This example accepts files in the mp4, m4a, mp3, amr, flac, and wav formats supported by the RTZR Batch STT API.
3. Calling the RTZR STT API
API Request Flow
First, obtain an access token from the authentication API and upload the audio file. The API returns a transcribe_id. The client uses this ID to poll the processing status and retrieves the final JSON response once the status becomes completed.
RTZR_CLIENT_ID / RTZR_CLIENT_SECRET
↓
Obtain an access token
↓
Submit the audio file for transcription
↓
Receive transcribe_id
↓
Poll for completion every 5 seconds
↓
Save the base and refined results as JSON
Retrieve the base transcription results with GET /v1/transcribe/{TRANSCRIBE_ID}. If insight generation was requested, its results are included in results.insight in the same response. Retrieve refined results separately with GET /v1/transcribe/{TRANSCRIBE_ID}?result=refined.
Running Batch STT
Pass only the audio file path to run base transcription.
The examples below use sample.wav. Replace it with your audio file’s name or path.
uv run --locked rtzr-transcribe sample.wav
Default Transcription Settings in This Example
When you pass only an audio file, the CLI requests transcription with the settings below. The model, language, and domain are fixed in the example code, while optional features can be enabled through command-line options.
| Setting | Default | Available Option |
|---|---|---|
| Speech recognition model | sommers |
Fixed in the example code |
| Language | Korean (ko) |
Fixed in the example code |
| Domain | General (GENERAL) |
Fixed in the example code |
| Speaker diarization | Enabled; speaker count estimated automatically | If you know the count, specify it with an option such as --speaker-count 2 |
| Transcript refinement | Disabled | Enable with --refinement |
| Insight generation | Disabled | Enable with --insight |
💡 Processing Applied by Default
This example does not explicitly include the following settings in the request; it uses the defaults of the RTZR Batch STT API.
- Inverse text normalization for English words, numbers, and units: enabled
- Disfluency filter: enabled
- Profanity filter: disabled
As a result, hesitation sounds such as 음 and 아, as well as unnecessary repetitions, may be removed from the transcript.
Using Transcript Refinement
Adding --refinement makes the CLI wait for the refined results after base transcription completes.
uv run --locked rtzr-transcribe sample.wav --refinement
The base response is saved to response.json, and the refined response to refined.json. The human-readable transcript.txt contains the refined transcript.
Using Insight Generation
Adding --insight includes a title, category, and summary in the base transcription response. This prompt asks for key points, requests, and follow-up actions.
uv run --locked rtzr-transcribe sample.wav \
--insight \
--insight-prompt "대화의 핵심 내용과 중요한 요청, 후속 조치를 정리해 주세요."
The CLI reads results.insight from response.json and also creates insight.txt. If you omit the prompt, it uses the default instruction defined in the project. If you explicitly pass an empty string, insight generation remains enabled, but no additional instruction is sent to the API.
If you need both refinement and insight generation, you can use the two options together.
uv run --locked rtzr-transcribe sample.wav --refinement --insight
4. Refining Transcripts of Noisy Audio
Data
For the refinement experiment, we use AI Hub’s Speech Recognition Data in Noisy Environments. This dataset provides audio mixed with background noise, along with reference transcripts.
This comparison uses a single file, 01_01_000606_210809_SN.wav. We transcribed the full recording and selected passages where the outputs differed.
Comparing RTZR Results Before and After Refinement
At the beginning of the recording, the speakers discuss seeing water deer on the road. Below are verbatim excerpts of four consecutive utterances from the same segment, with the main changes shown in bold.
Base Transcript
도로 위를 진짜 무겁자처럼 막 뛰어다녀요. 아니, 근데 걔네는 법을 모르니까 무법자일 수밖에 없죠.
근데 새끼 고라이가 진짜 귀엽게 생기긴 했어요, 되게 귀엽더라고요.
걔네가 막 깡총깡총 뛰어다니는데 집으로 데려가고 싶기도 했어요.
근데 걔네가 밤에는 정말 차에 진짜 졸진을 한다고 그러더라고요. 걔네가 진짜 위험하다고 그런 소리를 많이 들었어요.
Refined Transcript
도로 위를 진짜 무법자처럼 막 뛰어다녀요. 아니, 근데 걔네는 법을 모르니까 무법자일 수밖에 없죠.
근데 새끼 고라니가 진짜 귀엽게 생기긴 했어요. 되게 귀엽더라고요.
걔네가 막 깡총깡총 뛰어다니는데 집으로 데려가고 싶기도 했어요.
근데 걔네가 밤에는 정말 차에 진짜 돌진을 한다고 그러더라고요. 걔네가 진짜 위험하다고 그런 소리를 많이 들었어요.
In the refined transcript, the following misrecognized words were corrected:
무겁자처럼→무법자처럼고라이가→고라니가졸진을→돌진을
Comparing Results with Other Publicly Available Models
For inference code and run settings for each model, see the model inference examples.
For comparison, we used OpenAI’s Whisper large-v3-turbo, the Alibaba Qwen team’s Qwen3-ASR-1.7B, NVIDIA’s Nemotron 3.5 ASR 0.6B, and Microsoft’s VibeVoice-ASR. We examined their transcriptions of the same audio alongside RTZR’s base and refined results.
The table compares selected passages from the recording. Where a word-by-word alignment was difficult, we included a slightly longer excerpt from the corresponding passage.
| Reference Text | Whisper large-v3-turbo | Qwen3-ASR-1.7B | Nemotron 3.5 ASR 0.6B | VibeVoice-ASR | RTZR Base Transcript | RTZR Refined Transcript |
|---|---|---|---|---|---|---|
| 무법자처럼 | 무겁자처럼 | 무섭더라고 | 무엇자처럼 | 엄마 언니를 진짜 못 잡혀 | 무겁자처럼 | 무법자처럼 |
| 다 큰 고라니 | 다크는 고라니 | 다 큰 보라니고요 | 다크는 고란이고 | 닭도 고라니가 | 다크롱 고라니 | 다 큰 고라니 |
| 이목이 집중이 될 것 같은데 | 이목이 집중이 될 것 같은데 | 무기 집중해 그러더라고요 | 대목이 집중이 될 것 같은데 | 의무기 집중이 될 것 같은데 | 이 집중이 될 것 같은데 | 이목이 집중이 될 것 같은데 |
| 손을 파닥 거린 거고요 | 손을 파닥버린 거고요 | 소를 파닥거리고요 | 손을 끄닥거리거고요 | 손을 파닥거린 거고요 | 손을 파닥버린 거고요 | 손을 파닥거린 거고요 |
| 킥라니 | 킹남 | 킹라니 | 킹나니 | 킹란이라는 | 킹라니 | 킥라니 |
| 킥보드 타는 사람들이 | 슈퍼드 타는 사람들이 | 킴보드 타는 사람들이 | 축구듯하는 사람들이 | 킥보드 타는 사람들이 | 킥보드 찾는 사람들이 | 킥보드 타는 사람들이 |
RTZR’s base transcription of 이 집중이 changed to 이목이 집중이 after refinement. Whisper recognized this passage correctly, while Qwen, Nemotron, and VibeVoice transcribed it as 무기, 대목이, and 의무기, respectively. VibeVoice did, however, correctly recognize 손을 파닥거린 거고요 and 킥보드 타는 사람들이. In the examples shown in the table, RTZR’s refined results matched the meaning of the reference expressions.
These are observations from a specific recording and do not represent the models’ overall performance.
5. Extracting Insights from Conversations
Data
For the insight experiment, we use Wi-Fi’s Korean Full-Duplex Synthetic Dataset Preview, which consists of synthetic Korean conversations.
Each conversation includes the original utterance text, speakers, and start and end times, along with the conversation’s topic (topic) and the user’s goal (user_goal). We use this information to check whether the insight captures the key points of the conversation.
For this example, we use prod_0031838. The following excerpt from its metadata shows the topic and user goal:
{
"topic": "국제배송 통관 문의",
"user_goal": "관세나 부가세를 어디서 결제하는지도 확인하고 싶다"
}
Running Insight Generation
The recording contains a conversation between a customer and a service representative, so we set the speaker count to 2. We use the default instruction defined in the CLI without providing a custom insight prompt.
uv run --locked rtzr-transcribe sample.wav \
--speaker-count 2 \
--insight
The key excerpts from the generated transcript.txt are shown below.
화자 1: 국제 배송 통관 처리 방법과 관세, 부가세, 결제 위치를 알려주세요.
화자 2: 통관 절차는 배송사와 세관에서 진행되며, 관세와 부가세는 결제 시 자동 계산됩니다.
화자 1: 결제는 어디서 하나요? 카드로 가능한가요?
화자 2: 결제는 배송 완료 후 안내되는 링크에서 카드로 가능합니다.
화자 1: 통관 지연 시 추가 비용이 발생하나요?
화자 2: 지연 시 보관료가 부과될 수 있으나, 관세나 부가세는 변동 없습니다.
[중략]
화자 1: 부가세 포함 가격이 맞나요?
화자 2: 네, 상품 가격에 부가세가 포함되어 있습니다.
화자 1: 결제 후 환불은 어떻게 되나요?
화자 2: 환불은 카드사 정책에 따라 처리되며 관세는 환급되지 않습니다.
화자 1: 감사합니다. 대략적인 범위만 알고 싶어요.
화자 2: 관세와 부가세는 상품 가격의 10 에서 20% 내외로 예상됩니다.
The contents of the generated insight.txt are shown below.
제목: 국제 배송 통관 절차 및 관세·부가세 안내 문의
분류: 문의
내용: 고객이 국제 배송의 통관 처리 방법, 관세·부가세, 결제 방식 등에 대해 문의하였으며, 주요 내용은 다음과 같다. 통관 절차는 배송사와 세관이 진행하며, 관세와 부가세는 결제 시 자동 계산된다. 결제는 배송 완료 후 안내되는 링크를 통해 카드로 가능하다. 관세 납부 기한은 통관 완료 후 일반적으로 3일 이내이며, 관세·부가세는 상품 가격의 10~20% 내외로 예상된다. 통관 지연 시 보관료가 부과될 수 있으나 관세·부가세는 변동 없다. 배송 추적은 배송사 앱 또는 웹사이트에서 실시간 조회 가능하다. 환불 시 관세는 환급되지 않으며, 부가세는 상품 가격에 포함되어 있다.
In the conversation, the customer asks about customs clearance for international shipping and how to pay customs duties and VAT, then checks additional charges for delays, shipment tracking, payment deadlines, refunds, and the expected cost range. The summary brings these details together in one paragraph. Its title and 문의 category also align with the dataset’s topic and user goal.
6. Wrapping Up
In this tutorial, we used a single Python CLI to transcribe audio, apply refinement and insight generation, and save the results as JSON and TXT files.
Transcript refinement can help correct misrecognized words and phrases while preserving the base transcript’s utterance timing and speaker information. Insight generation organizes a conversation into a title, category, and summary, helping you identify key details and follow-up actions.
See the example code below for the full implementation and instructions for running it.
rtzr
Data Sources
- Refinement experiment: AI Hub, Speech Recognition Data in Noisy Environments. This article uses data developed and made publicly available through a project of the National Information Society Agency (NIA).
- Insight experiment: Wi-Fi, Korean Full-Duplex Synthetic Dataset Preview, CC BY 4.0. Excerpts from the conversations and the transcription and insight results generated from them are used as examples.
