agentOS

# AgentOS

**[English](README.md)** | **[繁體中文](README.zh-TW.md)** | **[日本語](README.ja.md)**

[![iOS 16+](https://img.shields.io/badge/iOS-16%2B-blue)](https://developer.apple.com/ios/)
[![Swift 5.9+](https://img.shields.io/badge/Swift-5.9%2B-orange)](https://swift.org)
[![Claude Opus 4](https://img.shields.io/badge/AI-Claude%20Opus%204-blueviolet)](https://anthropic.com)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![GitHub Issues](https://img.shields.io/github/issues/claude-world/agentOS)](https://github.com/claude-world/agentOS/issues)

**Turn your iPhone into an AI agent host.** AgentOS is a native iOS app that runs Claude AI with full tool-use capabilities — your phone can see, hear, speak, browse, code, trade, and communicate autonomously.

## What is AgentOS?

AgentOS transforms an iPhone into a powerful AI agent that can:

- **See** — Camera, photo library, computer vision, AR scene analysis
- **Hear & Speak** — On-device STT (Whisper), TTS (ElevenLabs), voice channel
- **Browse** — Web search, fetch pages, cloud browser automation (Browserbase)
- **Code** — GitHub repos, branches, PRs, issues, file editing (10 tools)
- **Trade** — Taiwan stocks, US equities, and cryptocurrency markets (11 tools)
- **Communicate** — 10 channels: Telegram, Discord, LINE, WhatsApp, Slack, Teams, Matrix, Signal, Voice, built-in chat
- **Remember** — Basic + semantic vector memory with cross-session persistence
- **Automate** — Scheduled tasks (cron), background execution, multi-agent discussions
- **Create** — DALL-E image generation, HeyGen video, Canva design
- **Manage** — Google Drive, Gmail, Calendar, Contacts, Photos, Reminders, HomeKit, Health

Built with **pure Swift** calling the Claude Messages API directly — no subprocess spawning, no bridging frameworks.

## Key Numbers

| Category | Count | Details |
|----------|-------|---------|
| **Native Tools** | 165+ | 59 tool files across hardware, cloud, social, coding, market data |
| **AI Agents** | 30 | Specialized personas with dedicated dashboard UIs |
| **Smart Skills** | 34 | Lightweight composable capabilities |
| **Channels** | 10 | Built-in, Telegram, Discord, LINE, WhatsApp, Slack, Teams, Matrix, Signal, Voice |
| **AI Providers** | 5 | Anthropic, z.ai, OpenRouter, Gemini, MLX Local (on-device) |
| **MCP Servers** | Dynamic | Streamable HTTP transport (iOS-native) |
| **Languages** | 3 | English, 繁體中文, 日本語 (live-switch) |
| **Siri Shortcuts** | 11 | 6 voice-activated + 5 Shortcuts-only intents |
| **Swift Files** | 232+ | ~76K LOC |

## AI Agents

30 pre-built agents, each with a dedicated dashboard UI:

| Agent | What It Does |
|-------|-------------|
| **Personal Assistant** | General assistant — web search, notes, journal, memories |
| **GitHub Agent** | Full coding workflow: planner → coding → reviewer pipeline |
| **Market Agent** | Taiwan/US stocks + crypto with real-time data |
| **Threads Publisher** | Meta Threads: search → write → analyze (14 tools) |
| **Schedule Manager** | Calendar, reminders, and scheduled task automation |
| **Photo Manager** | Photo library management with ML-powered search |
| **Automation Agent** | Cron jobs, webhooks, multi-step automation |
| **Cloud Dev Agent** | Browser + code execution + image gen in the cloud |
| **Channel Hub** | Multi-channel monitoring and response |
| **+ 21 more** | Including hardware agents (BLE, NFC, AR, HomeKit, Health, Music, Navigation) |

Create your own agents with the **5-step Agent Creation Wizard** or import community agents via `.agentpkg` packages.

## Smart Skills

34 composable skills that enhance any agent — no dedicated UI needed:

| Skill | Description |
|-------|------------|
| Market Analyst | Taiwan stock market analysis |
| US Stock Analyst | US equity market data (Polygon) |
| Crypto Analyst | Cryptocurrency market analysis |
| Threads Publisher | Meta Threads social media (14 tools) |
| Coding | GitHub coding workflow (10 tools) |
| Brainstormer | Creative ideation and brainstorming |
| Planner | Project and task planning |
| Reviewer | Code and content review |
| Translator | Multi-language translation |
| Document Processor | Document analysis and processing |
| **+ 24 more** | Covering SEO, email, cloud infra, security scanning, and more |

## 10 Communication Channels

| Channel | Foreground | Background (Always-On) |
|---------|-----------|----------------------|
| Built-in Chat | Native SwiftUI | Always available |
| Telegram | Long polling | APNs relay |
| Discord | WebSocket Gateway | APNs relay |
| LINE | Webhook | APNs relay |
| WhatsApp | Webhook | APNs relay |
| Slack | WebSocket | APNs relay |
| Teams | Webhook | APNs relay |
| Matrix | Sync API | APNs relay |
| Signal | Webhook | APNs relay |
| Voice | On-device STT/TTS | — |

**Always-On Architecture**: Cloudflare Worker relay enables background message processing via APNs push notifications, even when the app is suspended.

## 5 AI Providers

| Provider | Model | Use Case |
|----------|-------|----------|
| **Anthropic** | Claude Opus 4 | Primary (adaptive thinking, tool_use) |
| **z.ai** | Claude via z.ai | Alternative endpoint |
| **OpenRouter** | Multiple models | Model diversity |
| **Gemini** | Gemini Pro | Google integration |
| **MLX Local** | Qwen3, Gemma, Llama | On-device inference (no API key) |

Auto-failover between providers when one is unavailable.

## MCP Integration

Extend AgentOS with any [Model Context Protocol](https://modelcontextprotocol.io) server using **Streamable HTTP** transport — the only MCP transport that works on iOS (no subprocess spawning).

Pre-configured servers included. Add your own via the in-app MCP settings.

## Architecture

```
┌──────────────────────────────────────────────────┐
│               AgentOS (iOS App)                  │
├───────────┬───────────┬────────────┬─────────────┤
│ Channel   │  Agent    │   Tool     │    MCP      │
│ Manager   │  Engine   │ Dispatch   │  Client     │
│ (10 ch)   │  (core)   │ (165+)     │ (HTTP)      │
├───────────┴───────────┴────────────┴─────────────┤
│    Claude Messages API (agentic tool_use loop)   │
├──────────────────────────────────────────────────┤
│  5 AI Providers  │  Always-On Relay (CF Worker)  │
├──────────────────┴───────────────────────────────┤
│  iOS Hardware: Camera, GPS, Audio, BLE, NFC, AR  │
└──────────────────────────────────────────────────┘
```

**Agentic Loop**: Message → Claude API → tool_use? → execute tools → append results → loop until end_turn. Safety limit: 50 tool calls per turn.

## Tool Hooks (Safety Layer)

| Hook | Purpose |
|------|---------|
| **ActivityLogHook** | Logs all tool usage for audit trail |
| **PrivacyHook** | Requires explicit permission for sensitive operations (Drive, Gmail, GitHub writes, contacts, BLE, HomeKit) |
| **ConstitutionHook** | Constitutional AI guardrails |

## Privacy & Security

- All API keys stored in **iOS Keychain** (never in plain text, never in UserDefaults)
- Channel **allow-lists**: only explicitly approved chats can send commands (fail-closed)
- MCP connections over **HTTPS only**
- **No data collection** — everything stays on your device
- Sensitive operations gated by **PrivacyHook** (requires user approval)
- Rate limiting via token-bucket algorithm
- Telegram/Discord webhook verification (Ed25519, HMAC-SHA256)

## Requirements

- iPhone running **iOS 16.0** or later
- At least one AI provider API key:
  - [Anthropic](https://console.anthropic.com) (recommended)
  - [z.ai](https://z.ai)
  - [OpenRouter](https://openrouter.ai)
  - Or use **MLX Local** for fully on-device inference (no API key needed)
- Optional: Telegram Bot Token, Discord Bot Token, LINE Channel Token, etc.

## Reporting Issues

Found a bug or have a feature request? Please [open an issue](https://github.com/claude-world/agentOS/issues/new/choose).

- **Bug reports**: Use the bug report template — include iOS version, steps to reproduce, and expected vs actual behavior
- **Feature requ

[truncated…]