Sound and music
Transcripts timed to the word, the beats in a track, music that ducks under the voice, narration spoken on your Mac, and music and sound effects made on it.
Transcription, beats and the mix
Transcribe with Whisper, with word-level timestamps on by default
Write caption cues for one processed video. To burn them in, pass the returned ffmpeg_argv to render_ffmpeg
Pull the audio out of a clip
Line up separately recorded audio with its video automatically, by listening to both
Save a separately recorded audio file as a clip's sound
Remove a clip's separately recorded audio
Mix narration and music in one call, with the music ducked automatically under the voice
Tempo and beat map for a clip's audio: BPM, beats, downbeats, energy and suggested cut points, computed locally
Loudness over time and the loudest moments. path can also be a clip id, filename or stem
An edit map for an indexed song: structure, drops, cut points that respect phrases, loudness and key
Isolate or remove one sound by describing it: prompt="music", "dog barking", "the speaker's voice"
Whether SAM-Audio is installed, which separate_sound needs
Voices and narration
Speak a script locally in one of this profile's voices. Returns a job_id; job_wait(job_id) gives back the audio path and word timings. Every take is saved in the library. Only speed 1.0 is accepted
List every voice in this profile
Find voices by text
One voice's full record: what you need to narrate with it and whether you may
Hear a few seconds of a voice before using it for a whole script. Cached, so a repeat costs nothing
Create a voice from a written description or a stock speaker, with no real person in it. Returns a job_id because it runs in the background
Link a voice to the person or avatar who speaks in it, so "narrate as Mara" just works
Unlink a voice from a person or avatar
Whether local speech is ready: the engine, the chosen model size, and whether its files are downloaded
Everything your Mac's engines made in this profile, newest first, as shown in AI Voice's Generations tab and AI Video's Recent column
The voice effects: rooms and halls, devices such as a telephone or radio, characters such as a monster or robot, and neural effects that re-speak a take in a different voice
Put an effect on one recording, saved as a new take in the library, or on a voice so every take it speaks from then on wears it
Cloning a voice from a real person's recording is done by you, on the app's AI Voice page
Music and effects, made on your Mac
Make an instrumental music bed from a description, locally, with Stable Audio 3 through MLX
Make a sound effect from a description, locally, with Stable Audio 3 through MLX