🧠 Iris — A Second Brain That Maintains Itself

🇬🇧 English  |  🇩🇪 Deutsch

---

⭐ Star this repo  |  🐛 Report a bug  |  💬 Discussions

---

"Every knowledge system I tried had the same problem: it became useless the moment I stopped maintaining it. I wanted a system that maintains itself — one that creates notes automatically, explains itself, and gives intelligent answers based on my personal context. All on local hardware." — Alf Lewerken, builder

---

🌟 What is Iris?

Every personal knowledge system has the same fundamental problem: it decays the moment you stop maintaining it. Maintaining it takes effort that always gets dropped first in daily life.

Iris solves this at the architectural level. There is no maintenance step. You work — and Iris documents, organises, and cross-references the result automatically. Notes appear as a byproduct of real work, not as an extra step afterwards. The vault navigates itself, explains itself, and keeps itself current — like Wikipedia for your personal knowledge.

When you ask questions, Iris doesn't return raw note content. It returns a thoughtful answer that knows your personal context and addresses your specific question.

Everything runs on your own hardware. No service, no subscription, no cloud upload. Your knowledge stays yours.

Not a single note in this vault was created manually. Every entry was written by Iris during conversations or agent jobs.

---

Iris adapts to the hardware you have:

Your setup	How Iris runs
Apple Silicon + Ollama / LM Studio	✅ Fully offline — no API key, no internet
Any Mac + API key	✅ Cloud mode — Anthropic, OpenAI, OpenRouter
In between	✅ Mix & match — local STT + cloud LLM, or any other combination

---

🎯 Features

🎙️ Voice & Screen Awareness

• Push-to-talk — ctrl+option system-wide, works in any app
• Screen-aware — every query includes a screenshot of all monitors; ask about anything visible
• Editable dictation — floating panel at your cursor with live transcript; edit before sending
• Drag & drop — attach images and code files to any message
• Markdown rendering — responses render as formatted GFM: tables, headings, code blocks, lists
• TTS replay — speaker button on every response to hear it again

🤖 Three Coding Agents

Launch full AI agent sessions with the same ctrl+option shortcut — no Terminal needed:

Agent	Best for	License
Claude Code	Code tasks, refactoring, file analysis	Anthropic
Hermes	Research, analysis, long-running tasks	MIT
opencode	Claude Code alternative, no Anthropic account needed	MIT

All agents persist across restarts, support follow-up messages, drag-drop file attachments, and fire a macOS notification when done. Iris detects which agents are installed and hides unavailable buttons automatically.

🧩 Skill System (System Prompts)

The feature that sets Iris apart from other AI tools.

Attach reusable system prompts to any agent job — individually, in combination, or let the AI decide:

• Single select — pick one prompt from the list
• Multi-select — combine multiple prompts; their contents are merged automatically
• Auto-Select — Iris asks the LLM which prompts are relevant for your current request, based on each prompt's usage description. Fully automatic routing.

The active prompt(s) are shown as a badge in every job chat. When multiple prompts are active, a chevron reveals all individual names.

📓 Obsidian Integration — Vault as Memory

Every agent job chat is automatically saved to your Obsidian vault as a structured Markdown file — with YAML frontmatter (job ID, model, status, system prompt names), full conversation history, and wikilinks for any attached files. Files attached via iPhone are stored in the vault and linked automatically.

This creates an emergent learning loop that requires zero extra code:

1. Hermes solves a task (e.g. saving a recipe) → the chat is logged to the Vault
2. Next time a similar request comes in, RAG search finds the old conversation
3. Hermes sees exactly how it handled the task last time — folder structure, frontmatter format, tags — and gets it right immediately

The more tasks Iris completes, the better it understands your specific workflow. Not because the model is retrained, but because the context grows. Few-shot learning from your own Vault. The log is the memory.

🌙 Iris Dreamer — Self-Improving Memory

The Dreamer is Iris's overnight learning loop. A scheduled Hermes agent runs at 3:00 AM, reads every chat you upvoted (👍), and distills reusable templates into Iris-Dreamer/ in your Vault.

Before each new agent job starts, TemplateMatcherService asks the configured local LLM which template best fits the current request — and prepends it to the prompt.

The result: Iris gets better at your specific workflows over time, automatically, without retraining any model.

Daily use → Chat logged → You upvote good results
    → Dreamer distills templates overnight
        → Templates prepended to future prompts
            → Better results → more upvotes → better templates

This is not RAG. There are no embeddings, no vector databases, no scoring functions. Just curated Markdown that a small local model can use as few-shot examples. Fully transparent, fully debuggable.

📱 iOS Companion App

Iris Remote (App Store) turns your iPhone into a full remote for Iris on your Mac:

• Chat with the same history as the Mac
• Start, monitor, and follow up on agent jobs
• Attach photos and documents to jobs
• Configure LLM backend and model remotely
• Push-to-talk with Apple Speech (de-DE)
• Share Extension: send anything from any iOS app directly to Iris

Pair once by scanning a QR code. All traffic is TLS-encrypted with certificate pinning and bearer-token authentication.

🔌 MCP Server Support

All three agents are pre-configured with the macos-automator MCP server — run AppleScript and JXA automations from any agent job. Open apps, read windows, interact with system UI.

🌍 Multi-Language Support

Iris ships with full English and German localisation out of the box:

• UI language — switch between English, Deutsch, and System default in Other Settings; takes effect after restart
• Speech recognition — pin Apple Speech to en-US, de-DE, or follow the system locale
• TTS — Apple TTS reads responses in the language chosen by the system; ElevenLabs and Kokoro are language-independent

All 250+ UI strings are managed via Xcode String Catalogs (.xcstrings). Adding a third language requires only a new translation file — no code changes.

🔒 Privacy by Design

• ✅ No analytics, no crash reporting, no telemetry of any kind
• ✅ All API keys stored in macOS Keychain — never in config files or app bundle
• ✅ iOS connection: TLS + certificate pinning + bearer token
• ✅ Fully offline operation with local models — no data leaves your machine

---

📊 Comparison

Feature	Iris	Raycast AI	PopClip AI	ChatGPT for Mac
Local models (Ollama/LM Studio)	✅	❌	❌	❌
Screen-aware (every query)	✅	⚠️	❌	⚠️
Coding agents (Claude Code etc.)	✅	❌	❌	❌
iOS companion app	✅	❌	❌	✅
Obsidian integration	✅	❌	❌	❌
Self-building memory (Dreamer)	✅	❌	❌	❌
Auto-Select skill routing	✅	❌	❌	❌
No subscription	✅	❌	✅	❌
No telemetry	✅	❌	✅	❌
Open source	✅ MIT	❌	❌	❌

---

📸 Screenshots

🖥️ Menu Bar & Chat Panel

Screenshots coming soon

🤖 Agent Job with Skill System Badge

Screenshots coming soon

📱 iOS Companion App

Screenshots coming soon

---

🎬 Iris in Action

Real examples from daily use — all running locally on an M3 Ultra with minimax-m2.7, no cloud API required.

---

🕹️ "Find the Arkanoid prompt and build the game"

1. Voice command: "Search the Obsidian board for an AI game prompt for Arkanoid"
2. Hermes searches the Obsidian Vault, finds KI-Spieleprompts/Arkanoid/Breakout.md
3. Follow-up: "Build it" → Claude Code generates a single-file HTML game
4. Result: a fully playable Breakout clone with particle effects, power-ups, and Web Audio API — opened directly from the Vault in the browser

No API key. No cloud. One voice command.

---

🍽️ "Save this as a recipe"

1. Scan a physical recipe card with iPhone → attach the PDF to the Iris prompt
2. Voice command: "Save this as a recipe"
3. Hermes reads the Obsidian Skill to determine the correct format, finds the existing 04 Ressourcen/Kochrezepte/ folder, transcribes the full recipe including ingredients, nutritional values, allergens, and step-by-step instructions
4. Result: a clean Obsidian note with frontmatter, tags, and a wiki-link back to the original PDF scan — filed in the right place automatically

The only instruction was "Save this as a recipe."

---

💡 What else is possible

Because Hermes reads, writes, and searches your Vault — and Claude Code can generate anything — the same workflow applies to:

Prompt	What happens
"Scrape this documentation page and summarize it"	Web content → structured Obsidian note, cross-referenced with existing docs
"What does this error mean?" + screenshot	Screen-aware answer with context from your own knowledge base
"Scan this board game manual and build a digital version"	Photo → rules extracted → Claude Code implements playable HTML game
"Find everything I know about X and write a summary"	RAG search across entire Vault → synthesis
"Log this meeting" + voice memo	Transcription → structured note with action items → filed by project

The key insight: Iris doesn't need to be taught these tasks. The Skill System (system prompts) and Vault structure are enough context for the agent to figure out what "the right thing to do" is.

---

🚀 Quick Start

Option A — Build from source (free)

# Prerequisites: Xcode 16+, macOS 14.2+
git clone https://github.com/alflewerken/iris.git
cd iris/frontend/macos-iris
open Iris.xcodeproj
# Set your signing team in project settings → Cmd+R

⚠️ Always launch via Xcode (Cmd+R), never xcodebuild — it resets TCC permissions.

Option B — Download release

Download the notarized .dmg from Releases and drag to /Applications.

First launch checklist

• Grant Microphone access (for push-to-talk)
• Grant Accessibility access (for the global ctrl+option shortcut)
• Grant Screen Recording access (for screenshots)
• Configure an LLM backend (local or cloud) in the Settings tab
• Optionally install Claude Code, Hermes, or opencode via the onboarding wizard

---

🛠️ AI Backends

🤖 LLM Backends

Backend	Type	Notes
Ollama	🟢 Local	Any vision-capable model (gemma4, llama4, …)
LM Studio	🟢 Local	Any OpenAI-compatible local model
OpenAI-compatible	🟢 Local	vLLM, llama.cpp, text-generation-webui, mlx-lm, …
Anthropic	🔴 Cloud	API key · models discovered dynamically
OpenAI	🔴 Cloud	API key · models discovered dynamically
OpenRouter	🔴 Cloud	API key · 300+ models discovered dynamically

Iris LLM (companion chat) and Agent LLM (Claude Code / Hermes / opencode) are configured independently. Use a fast local model for chat and a powerful cloud model for agents, or run everything locally.

🎙️ Speech-to-Text

Provider	Notes
Apple Speech (default)	Local, no key, prefers de-DE
AssemblyAI	Real-time streaming, lower latency, API key needed
OpenAI Whisper	Upload-based, API key needed

🔊 Text-to-Speech

Provider	Notes
Apple TTS (default)	Local, includes Markdown stripper + pause markers
Kokoro	Local neural TTS via kokoro-fastapi (port 8880)
ElevenLabs	Cloud, highest quality, API key + Voice ID needed

---

🏗️ Architecture

┌─────────────────────────────────────────────────────────────┐
│                      Iris (macOS)                           │
│                                                             │
│  ctrl+option                                                │
│      │                                                      │
│      ▼                                                      │
│  DictationInputOverlay  ←→  SystemPromptStore (Auto-Select) │
│      │                                                      │
│      ├── ⌘↩  ──►  CompanionManager ──► LLM Backend         │
│      │              (chat + TTS)        Ollama / Anthropic  │
│      │                                  OpenAI / OpenRouter │
│      └── Agent Button                                       │
│              │                                              │
│              ▼                                              │
│         AgentJobManager                                     │
│              │                                              │
│              ├──► claude --print (stream-json)              │
│              ├──► hermes chat --yolo [--resume]             │
│              └──► opencode run                              │
│                        │                                    │
│              ObsidianChatLogger ──► ~/Obsidian/Vault/       │
│              IrisRemoteServer   ──► iOS App (TLS + Token)   │
└─────────────────────────────────────────────────────────────┘

Key source files:

File	Purpose
CompanionManager.swift	Central state machine — LLM routing, TTS, screen capture
DictationInputOverlay.swift	Floating chat panel — push-to-talk, drag-drop, agent buttons
AgentJob.swift	Agent job system — subprocess lifecycle, streaming, persistence
AgentJobChatPanel.swift	Per-job chat window — streaming UI, follow-ups, token count
IrisRemoteServer.swift	Embedded HTTPS server — REST + SSE for all remote clients (iOS, Linux, Mac)
RegisteredDeviceStore.swift	Device registry — tracks, blocks and removes connected clients
RemoteConnectionProfile.swift	Mac→Mac connection profiles (RemoteConnectionStore singleton)
RemoteServerClient.swift	HTTP client for Mac→Mac connections — TLS pinning, device headers
AnthropicProxyManager.swift	Anthropic↔OpenAI format translation for local agent backends
SystemPromptStore.swift	Skill system — prompts, usage descriptions, Auto-Select
ObsidianChatLogger.swift	Obsidian vault writer — Markdown + YAML frontmatter
VaultIndexUpdater.swift	Auto-generates _Index.md per vault folder (bottom-up)

Full architecture reference for AI coding assistants: CLAUDE.md

---

📋 Requirements

• macOS 14.2 (Sonoma) or later
• Xcode 16+ to build from source
• At least one LLM backend (local or cloud API key)

Agent prerequisites (only for the agents you want to use):

Agent	Requires
Claude Code	Node.js → npm install -g @anthropic-ai/claude-code
Hermes	Homebrew + Python 3.10+
opencode	Node.js → npm install -g opencode-ai

The onboarding wizard installs agents for you — but Node.js / Homebrew / Python must already be present.

---

🆕 Recent Changes

June 2026

• ✅ Device management — all connected clients (iOS, Linux, Mac) appear in Settings → Connected Devices; block or remove individual devices without rotating the global token
• ✅ IP fallback tracking — clients without X-Device-ID header are tracked by IP address; no client update required
• ✅ Mac remote client device headers — RemoteServerClient now sends X-Device-ID (persistent UUID) and X-Device-Name on every request
• ✅ Remote Server renamed — "iOS Remote Server" → "Remote Server" (all clients connect, not just iOS); "Mac Remote Clients" → "Mac Remote Server"
• ✅ Vault Maintenance: Wikilink repair — LLM-based broken wikilink detection and repair (REMOVE / REPLACE / SKIP per link)
• ✅ Vault Maintenance: LLM description enrichment — auto-generates description frontmatter for notes that lack one

May 2026

• ✅ Full EN/DE localisation — 250+ strings in Xcode String Catalogs; UI language + STT language selectable in settings
• ✅ Auto-Select Skill System — LLM-based system prompt routing with usage descriptions
• ✅ Multi-select system prompts — combine multiple prompts, badge shows all names
• ✅ Obsidian chat logger — all agent chats auto-saved as structured Markdown
• ✅ iOS: Share Extension — send anything from any app directly to Iris
• ✅ iOS: File attachments — photos and documents in jobs and follow-ups
• ✅ Stale path detection — auto-detects moved files in saved chats, Mac or iOS dialog
• ✅ Bearer-token auth — QR pairing now includes a 64-char security token
• ✅ OpenAI + OpenRouter — fully supported cloud providers with dynamic model discovery
• ✅ AnthropicProxy in Swift — pure Swift replacement for the Python subprocess proxy
• ✅ Hermes --resume — real session continuation for follow-up messages

April–May 2026 (initial build)

• ✅ macOS menu bar app with push-to-talk + screen capture
• ✅ Claude Code, Hermes, opencode agent integration
• ✅ iOS companion app (all 28 phases — chat, agent jobs, PTT, TTS, settings sync)
• ✅ TLS-pinned HTTPS server + QR pairing
• ✅ System prompt manager
• ✅ Markdown rendering in all chat views
• ✅ MCP server integration (macos-automator)
• ✅ Onboarding wizard with agent installation
• ✅ Job persistence, rename, notifications
• ✅ Rich chat media (file icons, folder browser, zip sharing)
• ✅ Removed PostHog analytics, Sparkle updater, mlx-vlm (privacy + simplicity)

---

💡 The Vision — A Knowledge System That Maintains Itself

I've tried every personal knowledge system out there. They all hit the same wall: the moment you stop actively maintaining them, they become useless. You can't find things anymore because the structure drifted, the index is stale, or you simply forgot to file something.

The real problem isn't discipline — it's that documentation and filing are always extra steps on top of the actual work. And extra steps get dropped first.

Iris removes the extra step entirely.

Notes appear because Iris writes them. Structure stays current because VaultIndexUpdater rebuilds the index automatically after every change. Cross-references exist because Iris knows the whole vault when it writes a new entry. The vault becomes navigable like Wikipedia — not because someone maintained it, but because the system maintains itself.

When you ask questions later, Iris doesn't return a raw note. It returns a contextual answer drawn from everything it knows about your specific situation, workflow, and history.

Not a single note in this vault was created manually — every entry was written by Iris during conversations or agent jobs.

---

How this looks in practice

You develop a feature with Iris. Iris writes the code, documents the architectural decision, creates a note in the right folder, and updates the index. Three weeks later you ask: "Why did we solve it that way?" — and get a precise answer that references exactly that note.

You scan a recipe card with your iPhone, attach it to Iris, and say "Save this as a recipe." Hermes reads the Obsidian skill to determine the correct format, finds the existing Kochrezepte/ folder, transcribes the full recipe including ingredients, allergens, and instructions, and files it correctly. The only instruction was those five words.

"Give me an overview of my unread emails" Running locally on a Qwen3.6-35B MoE model. Hermes read two months of accumulated context from the Vault and filtered 22 newsletters to junk — automatically, no rules written. That's not a large model being clever. That's a small model with the right context.

This is fundamentally different from a chat interface. A chat interface gives you a smart stranger. Iris gives you something closer to a colleague who has read all your notes, remembers every conversation, and gets better the more you work together.

Closest alternatives: mem.ai (cloud, no action capability), Khoj (open source, no distillation loop), Rewind (passive capture only). None combine local-first operation, a curated feedback loop, and real action capability in one tool.

---

Why menu bar?

The menu bar is not the product — it's the solution to a specific UX problem: how do you give a permanently running assistant a presence that doesn't interrupt, but is instantly reachable from any context? One shortcut (ctrl+option) from any app, any window, any space. No context switch. No app to bring to the front.

The data you accumulate is also a future fine-tuning dataset. A model trained on your upvoted conversations would carry your context in its weights — no prompt overhead, faster responses, even more personal.

---

🤝 Contributing

Issues and pull requests are welcome. For larger changes, open a discussion first.

The codebase has a detailed CLAUDE.md that serves as architecture reference for both human developers and AI coding assistants. DOKUMENTATION.md has deeper dives into non-obvious implementation details.

---

❤️ Support

If Iris is useful to you:

• ⭐ Star the repo — the simplest way to help
• 🐛 Open an issue — bugs and feature requests welcome
• 📱 Download Iris Remote from the App Store — small one-time fee, supports continued development

---

📜 License

MIT — see LICENSE.

Third-party agents (Claude Code, Hermes, opencode) are separate projects with their own licenses. Iris detects and invokes them as external subprocesses and does not redistribute them.

---

_{Built with ❤️ on an M3 Ultra · Local by default, Cloud by choice · No subscription, no telemetry}