KRZN.live
Features

Live captions

KRZN Live writes captions of the speech while it happens, with Whisper running on the central computer (no cloud service, no cost). The operator switches them on when the speech starts and off for the praise and music. The captions go into a vMix title and/or are drawn on the picture sent to Go Live, and each speech is saved as subtitles (.srt) and text (.txt).

How it works

Two models work together, so the words appear quickly and are still written well:

Stage Model Runs on Delay Quality
Draft Whisper small CPU about 1 s after the words (up to 3 s when the computer is busy) fair: some words wrong
Correction Whisper large-v3-turbo graphics card (Vulkan) the sentence is replaced about 6 s after the pause (7-12 s while the burn-in also uses the card) very good, Taglish included

While a sentence is spoken, the draft is written again every 0.4 s of new sound. At a pause, the sentence goes to the accurate model and its text replaces the draft. Without a graphics card, the drafts are not corrected.

Setting up

  1. Settings > Live captions: turn on Live captions, choose the path with the sound of the speech (empty: the Go Live program), add the words to know (names of people, places and products), and Apply.
  2. Captions page: click Download (the engine, 51 MB, and the models, 764 MB: once).
  3. Choose where the captions go: - vMix title (recommended): add a title input in vMix (a lower third), turn on vMix Settings > Web Controller, and enter the title's name in Settings > Live captions. The title is put in the overlay channel while speech and taken out during the music. vMix's program then has the captions: nothing else to do. - In the picture sent to Go Live (without vMix titles): KRZN Live draws the captions on the program and encodes it again (path program-cc, graphics card encoder). Go Live sends it once it has worked for 10 s; if it stops, Go Live sends the plain program again by itself. Don't use both, or the captions show twice.

During the service

On the Captions page: press Speech when the speech starts, Music when the songs start. Music makes Whisper write nonsense: keep captions off during the songs. The page shows the captions as they are sent, the sound received, and the state of both models. ON is shown next to Captions in the menu while speech.

Files

What Where
Engine and models captions\ next to krznlive.yml
Captions of each speech recordings\captions\<date time>.srt and .txt

Settings

Setting Default Meaning
captions false The whole feature.
captionsSource "" Path whose sound is captioned (empty: goLiveSource).
captionsModel / captionsDraftModel turbo / small Accurate and fast models (empty: not used). Each computer of a cluster has its own.
captionsGPU true Run the accurate model on the graphics card.
captionsLanguage tl tl (Tagalog and Taglish), en, auto.
captionsWords [] Words to know.
captionsLines, captionsLineLength, captionsClearAfter 2, 42, 4s Lines shown, characters per line, cleared after this much silence.
captionsVMix, captionsVMixAddress, captionsVMixInput, captionsVMixField, captionsVMixOverlay off, http://127.0.0.1:8088, "", first field, 1 vMix title. Overlay 0: KRZN Live doesn't show or hide the title.
captionsBurnIn, captionsBurnInEncoder off, auto Draw the captions on the program sent to Go Live.
captionsTranscript true Save .srt and .txt.
Message us