analyze_audio

io.clueso/video · Verify this server

Analyze an audio file. Modes: - transcript: Speech-to-text with word-level timestamps. Works on BOTH speech and sung lyrics — routes to a music-aware provider when content is detected as music. Use to get `words[{w,start_ms,end_ms}]` for caption timing, script editing, or word-level visual sync. Note: features.has_speech may report false on pure music while transcript still successfully extracts lyrics. - silences: list of silent regions with start/end/peak. Use for auto-trim, smart-split, or pause-aware editing. - beats: tempo (bpm) + beat positions for music tracks. Returns empty for non-music. Use to align animations/zooms to a beat. - features: duration, peak/rms/lufs loudness, speech-vs-music heuristic. Cheap dispatcher — call once to decide which other mode to use, or to get duration_ms for source_duration when calling add_audio. Source: provide exactly one of mcp_upload_id (from upload tool) or file_url (already-public URL, e.g. a find(type='music') result or Clueso CDN file). Optional time range: range_start_ms / range_end_ms crops the analysis window.