samayly.com is live. Hosted plans open soon. Join the waitlist →
Phone optionreference14 min read

Option 5: Siri Voice-First Integration

Turn your tracker into a voice-controlled assistant - say "Hey Siri, tracker check" and get spoken responses. Works on iPhone, Apple Watch, AirPods, HomePod, and CarPlay.

⏱️ Setup time: 40 min 💰 Cost: ~$1-3/month (Claude API only) 👥 Users: iOS/Apple ecosystem per-user setup


What you're building

transcribed text

structured action

POST

read/write

result

🗣️ 'Hey Siri, tracker check'

🤖 Siri

📱 iOS Shortcut
(voice recognition)

🧠 Claude API
(parse intent)

☁️ Apps Script
or Cloud Run

📊 Your Sheet

🔊 Siri speaks reply
(via AirPods/HomePod/Watch)

Sample interactions: - 🎤 "Hey Siri, morning brief" → 🔊 "You have 3 items due this week. Ethics assignment in 4 days..." - 🎤 "Hey Siri, tracker add" → 🔊 "What to add?" → 🎤 "GPU quiz Oct 15" → 🔊 "Added row 8" - 🎤 (in car, hands-free) "Hey Siri, what's due tomorrow" → 🔊 spoken response


Prerequisite

Complete Option 3 (Apps Script + Shortcut) first - Siri voice sits on top of that infrastructure. If you've done Option 3, you're 80% of the way here.


Setup Overview

Step 1
Complete Option 3 setup
(Apps Script webhook working)

Step 2
Build voice-first Shortcut
(Dictate + Speak actions)

Step 3
Add trigger phrase to Siri

Step 4
Test on iPhone

Step 5
Enable Apple Watch complication

Step 6
Test CarPlay + AirPods

Step 7 (optional)
Multi-shortcut for different actions


1Verify Option 3 works

Before adding voice, confirm the underlying flow works:

  • Your Apps Script Web App URL responds to POST requests
  • Your Claude API key is valid
  • You have a basic Shortcut that can add rows

If not, complete OPTION_3_APPS_SCRIPT_SHORTCUT.md first.


2Build the voice-first Shortcut

This time, redesign the shortcut around voice input and voice output.

2.1 - Create new shortcut

Shortcuts app → + → name it "Tracker Voice"

2.2 - Add actions in order

Action 1: Dictate Text (voice input)

  • Search: Dictate Text
  • Language: English (India) or your locale
  • Stop Listening: On Tap or After Pause (short pause = 1.5s)

💡 "After Pause" is better for natural speech - you can pause to think without cutting off.

Action 2: Get Contents of URL (call Claude)

Same as Option 3 Step 4, but modify system prompt for voice:

  • URL: https://api.anthropic.com/v1/messages
  • Method: POST
  • Headers: (same as Option 3)
  • Body: json { "model": "claude-sonnet-4-5", "max_tokens": 512, "system": "You parse spoken tracker requests. Voice input may have transcription errors — be forgiving. Valid subjects: FoVL-Language, GPU Programming, Ethics and Professional Life, Innovation and IP Management, Advanced AI. Return ONLY JSON: {action, subject, type, title, due_date (DD-Mmm-YYYY), time (HH:MM IST), notes}. Include 'voice_reply' field with a natural-sounding sentence I should speak back to the user.", "messages": [ {"role": "user", "content": [DICTATED_TEXT_VARIABLE]} ] }

Action 3: Extract voice_reply

  • Get Dictionary Value → get content[0].text
  • Then Get Dictionary Value → get voice_reply

Action 4: Speak Text

  • Text: the voice_reply from Action 3
  • Language: same as Dictate
  • Rate: 0.5 (natural pace)
  • Pitch: 1.0
  • Wait Until Finished: On

Action 5: POST to Apps Script (for writes)

Only run if action != "read": - Wrap Actions 5-6 in an If block: If action is not "read"

Inside the If: - Get Contents of URL → POST parsed JSON to your Apps Script URL

Action 6: Optional final Speak (confirmation)

  • Speak: "Done." or the response from Apps Script

3Add to Siri

  1. Open the Tracker Voice shortcut
  2. Tap the settings icon (top right, ⓘ)
  3. Add to Siri → tap the red record button
  4. Say your trigger phrase clearly: "tracker" or "log tracker" or "morning brief"
  5. Tap Done

Trigger phrases that work well:

  • "Hey Siri, tracker check" (read)
  • "Hey Siri, tracker add" (write)
  • "Hey Siri, morning brief" (daily summary)
  • "Hey Siri, what's due" (upcoming)

Trigger phrases to avoid:

  • Anything with "sheet" (Siri may launch Google Sheets app instead)
  • Anything one word (may conflict with other shortcuts)
  • Anything with numbers or dates in the phrase itself

4Test on iPhone

  1. Lock your phone
  2. Say "Hey Siri, tracker check"
  3. Speak your query when Siri activates the shortcut: "What's due this week"
  4. Listen for the spoken reply

Common test cases: - "What's due tomorrow?" → should list items - "Add GPU quiz October 15 at 6 PM" → should ask confirmation, then add - "Mark row 5 as done" → status update


5Enable Apple Watch

  1. Shortcuts app → tap Tracker Voice
  2. Settings icon → toggle Show on Apple Watch ON
  3. On Apple Watch → open Shortcuts app → your shortcut appears

Or set it as a Watch complication: 1. Long-press watch face → Edit → Complications 2. Choose slot → select Shortcuts → pick Tracker Voice 3. Now it's one-tap from your wrist

Voice on Watch works via Raise to Speak - no button press needed if configured.


6Test CarPlay + AirPods

CarPlay

  1. Connect to your car
  2. Long-press Siri button on steering wheel
  3. Say: "Hey Siri, morning brief"
  4. Reply plays through car speakers

AirPods

  • Squeeze/tap AirPod → "Tracker check"
  • Reply plays in your ear

HomePod / HomePod mini

  • "Hey Siri, tracker check" from anywhere in room
  • Reply plays through HomePod

Step 7 (Optional): Multiple specialized shortcuts

Instead of one big shortcut, create several focused ones:

Shortcut name Trigger Purpose
Tracker Check "tracker check" Read this week's items
Morning Brief "morning brief" Read today + tomorrow
Log Tracker "log tracker" Add new item (with dictation)
Mark Done "mark done" Update status to completed

Each does one thing well = better voice recognition + faster response.


Advanced ideas

Automations (no voice, triggered by events)

Shortcuts app → Automation tab → +

Examples: - When I arrive at work → speak "Today's tracker items:" - 9 AM every weekday → read new items overnight - When I open Google Sheets app → auto-run "Show updated items" - When I connect AirPods → speak morning brief

Focus Mode integration

Only enable tracker voice during work hours: - Settings → Focus → Work → Add Shortcut trigger

Multiple language support

  • Set Dictate Text language to Hindi, Tamil, etc.
  • Claude Sonnet 4.5 handles most languages natively
  • Voice reply also speaks in same language

Pros / Cons

✅ ❌
Fully hands-free (driving, cooking, walking) iOS/Apple ecosystem only
Works on 6+ device types (iPhone/Watch/AirPods/HomePod/CarPlay/iPad) Voice recognition sometimes misses technical terms
No screen needed Requires per-user setup
Feels magical in demos Each shortcut needs its own SSH key if using SSH
Layer on top of Options 3/4 easily Battery drain if used heavily

Sample voice_reply examples

Good system prompts produce natural spoken responses:

User says: "What's due this week?" Voice reply: "You have 3 items this week. Ethics assignment in 4 days, GPU quiz on Saturday, and Innovation assignment next Monday."

User says: "Add GPU quiz October 15 at 6 PM" Voice reply: "Adding GPU Programming quiz on October 15th at 6 PM IST. Say yes to confirm."

User says: "Nothing due" Voice reply: "You're all caught up. Nothing due in the next 7 days."


Troubleshooting

Siri doesn't hear "tracker" correctly

  • Re-record the phrase more slowly
  • Try a longer phrase: "check my tracker" instead of just "tracker"
  • Ensure Siri language matches your accent (Settings → Siri & Search → Language)

Voice reply cut off mid-sentence

  • Speak action: enable Wait Until Finished
  • Increase Claude's max_tokens if replies are truncated

Works on iPhone but not Apple Watch

  • Watch needs the shortcut synced: iPhone Shortcuts app → toggle "Show on Apple Watch"
  • If it uses SSH, add the SSH key to Watch keychain separately

CarPlay says "I can't help with that"

  • Enable Use with CarPlay in shortcut settings
  • CarPlay's Siri is stricter - use simpler trigger phrases

Dictation not stopping automatically

  • Change Stop Listening to After Pause (2 seconds)
  • Or use After Tap if pauses interrupt your thinking

Cost note

Voice adds no extra cost - same Claude API calls as text. Voice recognition is on-device (Siri) or included in iOS.


Summary

You now have hands-free tracker access from any Apple device. Best combined with Option 3 (the infrastructure) or Option 4 (for batch text access).

The full stack: - Mac at desk: Claude Code direct - iPhone away from Mac: SSH (Option 1) or Shortcut (Option 3) - Voice / hands-free: Siri (Option 5) - Batch access: WhatsApp bot (Option 4) or Telegram bot (Option 2)

All layered on the same Google Sheet as source of truth. 🚀

Questions or corrections: [email protected]