Talk mode

• Native macOS/iOS/Android Talk: native speech recognition, Gateway chat, and talk.speak TTS. Apple Speech recognition on macOS/iOS may use network services; Android behavior depends on the installed speech service. Nodes advertise the talk capability and declare which talk.* commands they support.
• iOS Talk (realtime): client-owned WebRTC for OpenAI realtime configs that select webrtc transport or omit transport, including framed and frameless transcript/audio events. Explicit gateway-relay, provider-websocket, and non-OpenAI realtime configs stay on the Gateway-owned relay; non-realtime configs use the native speech loop.
• Browser Talk: talk.client.create for client-owned webrtc/provider-websocket sessions, or talk.session.create for Gateway-owned gateway-relay sessions. managed-room is reserved for Gateway handoff and walkie-talkie rooms.
• Android Talk (realtime): Android uses Gateway-owned relay realtime when talk.catalog reports the realtime group ready and the configured model passes the Android client gate; it never opens a client-owned WebRTC session. The Gateway now supports gpt-live-* relay sessions, but Android intentionally keeps those models on native speech recognition, Gateway chat, and talk.speak until the relay path is proven live from an Android device.
• Transcription-only clients: talk.session.create({ mode: "transcription", transport: "gateway-relay", brain: "none" }), then talk.session.appendAudio, talk.session.cancelTurn, and talk.session.close for captions/dictation without an assistant voice response. One-shot uploaded voice notes still use the media understanding audio path.

​

Behavior (macOS)

• Always-on overlay while Talk mode is enabled.
• Listening → Thinking → Speaking phase transitions.
• On a short pause (silence window), the current transcript is sent.
• Replies are written to WebChat (same as typing).
• Interrupt on speech (default on): if the user talks while the assistant is speaking, playback stops and the interruption timestamp is noted for the next prompt.

​

Voice directives in replies

{ "voice": "<voice-id>", "once": true }

• First non-empty line only; the JSON line is stripped before TTS playback.
• Unknown keys are ignored.
• once: true applies to the current reply only; without it, the voice becomes the new Talk mode default.

​

Config (~/.openclaw/openclaw.json)

{
  talk: {
    provider: "elevenlabs",
    providers: {
      elevenlabs: {
        voiceId: "elevenlabs_voice_id",
        modelId: "eleven_v3",
        outputFormat: "mp3_44100_128",
        apiKey: "elevenlabs_api_key",
      },
      mlx: {
        modelId: "mlx-community/Soprano-80M-bf16",
        // Fish S2 Pro can also use a local reference voice:
        // referenceAudioPath: "/Users/example/Voices/reference.wav",
        // referenceText: "Exact transcript of the reference clip.",
      },
      system: {},
    },
    speechLocale: "ru-RU",
    silenceTimeoutMs: 1500,
    interruptOnSpeech: true,
    realtime: {
      provider: "openai",
      providers: {
        openai: {
          apiKey: "openai_api_key",
          model: "gpt-realtime-2.1",
          speakerVoice: "cedar",
        },
      },
      instructions: "Speak warmly and keep answers brief.",
      mode: "realtime",
      transport: "webrtc",
      brain: "agent-consult",
    },
  },
}

​

macOS UI

• Menu bar toggle: Talk
• Config tab: Talk Mode group (voice id + interrupt toggle)
• Overlay: the orb renders the universal talk waveform (shared with iOS, watchOS, and Android). Listening follows the live mic level, Speaking follows the actual TTS playback envelope, Thinking breathes softly. Click the orb to pause/resume, double-click to stop speaking, click X to exit Talk mode.

​

Android UI

• Android’s main navigation is Home, Chat, and Settings. Voice input lives in the Chat composer rather than a separate Voice tab.
• Tap the composer microphone for on-device dictation. Long-press it to record a voice-note attachment. Start continuous Talk from the Talk waveform.
• Dictation, voice-note recording, and Talk are mutually exclusive microphone paths; starting one stops or blocks the others.
• Realtime Talk prefers a connected Bluetooth Classic or BLE headset microphone; if it disconnects, the app requests another headset input or falls back to the default microphone, restoring the default preference once capture stops.
• Dictation and voice-note recording stop when the app leaves the foreground or the user leaves Chat.
• Talk Mode keeps running until toggled off or the node disconnects, using Android’s microphone foreground-service type while active.
• Android supports pcm_16000, pcm_22050, pcm_24000, and pcm_44100 output formats for low-latency AudioTrack streaming.

​

Notes

• Requires Speech + Microphone permissions.
• Native Talk uses the active Gateway session and only falls back to history polling when response events are unavailable.
• The gateway resolves Talk playback through talk.speak using the active Talk provider. Android falls back to local system TTS only when that RPC is unavailable.
• macOS local MLX playback uses the bundled openclaw-mlx-tts helper when present, or an executable on PATH. Set OPENCLAW_MLX_TTS_BIN to point at a custom helper binary during development. The helper streams PCM, keeps one selected model resident, and supports Fish S2 Pro reference audio through providers.mlx.referenceAudioPath plus referenceText.
• Voice directive value ranges (ElevenLabs): stability, similarity, and style accept 0..1; speed accepts 0.5..2; latency_tier accepts 0..4.

​

Related

• Voice wake
• Audio and voice notes
• Media understanding