OverShoulder User Guide

Installation

1. Download — Get the DMG from overshoulder.ai. The file is named OverShoulder-x.x.x-arm64.dmg.
2. Install — Open the DMG and drag the OverShoulder icon into your Applications folder.
3. First launch — macOS will warn about an unidentified developer. Go to System Settings → Privacy & Security and click "Open Anyway".
4. Grant permissions — When prompted, allow Screen Recording access (required for screen sharing). You can enable this later in System Settings if you skip it.

First Launch & Login

1. Launch OverShoulder from your Applications folder. A transparent panel appears on the right side of your screen.
2. Click Sign Up to create a new account with your email and password, or Log In if you already have one.
3. After logging in, you'll see Andy — your AI assistant — greet you. A 7-day free trial begins automatically with limited model access.
4. The panel floats above all apps. Click through it to interact with your desktop apps underneath — the panel only captures clicks on its own UI elements.

Panel Overview

The main panel is divided into these areas from top to bottom:

• Top bar — Settings gear icon (left), tab bar (center), plus (+) button to add new tabs (right).
• Model selector — Shows the active AI model name. Click to open the model picker and switch between different AI models.
• Chat area — Scrollable conversation history. Each message shows the AI model used, and action buttons appear on hover.
• Input bar — Text input at the bottom. Type your message and press Enter to send. The microphone icon activates voice input.
• Bottom controls — Screen share toggle (play/stop), Memory Box icon, and credit display.

Chat & Tabs

Sending a message

1. Click the input bar at the bottom of the panel, or just start typing.
2. Press Enter to send. Press Shift+Enter for a new line.
3. The AI processes your message and streams the response in real-time.

Using multiple tabs

Each tab runs independently with its own conversation and AI model. This lets you work on different topics simultaneously.

1. Click the + button in the tab bar to create a new tab.
2. Switch between tabs by clicking their names. Each tab remembers its own conversation history.
3. Right-click a tab to rename or close it. Double-click the tab name to edit it inline.
4. Each tab can use a different AI model — change the model in one tab without affecting others.

Clearing a conversation

To start fresh in the current tab, open Settings → General and click Clear Chat. This clears the chat history for the active tab only.

AI Models

OverShoulder provides access to multiple AI models from three providers. Each model has different strengths and credit costs.

How to switch models

1. Click the model name displayed below the tab bar (e.g., "Claude Sonnet 4.5").
2. A dropdown appears showing all available models with their tags and credit cost per message.
3. Click the model you want. It applies to the current tab only — other tabs keep their models.
4. Your selection is saved and persists across sessions.

Available models

AI Model Tips & Strategy

Each model has different strengths. Choosing the right model for the right task saves credits and gets better results.

Coding & Technical Work

Research & Information Gathering

Creative Writing & Content

Quick Questions & Simple Tasks

Screen Sharing & App Guidance

Screen Sharing

Screen sharing lets the AI see your screen and provide context-aware guidance. It's completely optional — OverShoulder works without it too.

How to start screen sharing

1. Click the play button (▶) at the bottom of the panel.
2. If this is your first time, macOS will ask for Screen Recording permission. Grant it in System Settings → Privacy & Security → Screen Recording.
3. Once active, the button changes to a stop icon (■) and a green indicator appears.
4. The AI can now see what's on your screen and will automatically detect which app you're using.

How it works

• OverShoulder takes periodic screenshots of your screen (every few seconds while active).
• Screenshots are processed locally on your device — text is extracted using OCR, sensitive data is masked, and the screenshot is deleted immediately after processing.
• Only the extracted text context (not the image) is sent to the AI model for analysis.
• The AI uses this context to understand what you're working on and provide relevant guidance.

Proactive guidance (Observe mode)

When screen sharing is active and a Knowledge Pack matches your current app, the AI may proactively offer guidance based on what it sees on your screen. For example, if you're in Photoshop and have the Layers panel open, it might offer tips about layer management.

To stop screen sharing

Click the stop button (■) at the bottom of the panel. Screen sharing stops immediately. All captured data is discarded.

Active Guide

Active Guide is a hands-free, step-by-step coaching mode. When enabled, the AI watches your screen and automatically advances to the next instruction as you complete each step — no need to type "next" or "and then?" between steps.

How to start Active Guide

1. Turn on screen sharing.
2. Ask a how-to question (e.g., "How do I file a tax invoice on HomeTax?" or "Show me how to create a pivot table").
3. The AI responds with the first step and offers an Active Guide button.
4. Click the button. A green indicator appears above the input bar, confirming Active Guide is running.

How it works

• After each instruction, the AI watches for screen changes (you clicking a button, a page loading, a dialog appearing).
• When it detects you've completed the step, it automatically praises your progress and gives the next step — all without you typing anything.
• If it detects an error or you went to the wrong page, it corrects course immediately.

How Active Guide ends

• Manual: Type "종료" (or "stop" / "done" / "end") in the input bar, or click the ✕ button on the green indicator.
• Auto-complete: When the AI determines all steps are done, it ends the session automatically.
• Timeout: If no screen activity is detected for 3 minutes, Active Guide deactivates to save credits.

Voice Input & TTS

Voice input (speech-to-text)

1. Press Cmd+Shift+; or click the microphone icon in the input bar.
2. Speak your message. The panel shows a pulsing indicator while recording.
3. Press the shortcut again or click the mic icon to stop recording.
4. Your speech is transcribed and inserted into the input bar. Press Enter to send, or edit it first.

Text-to-speech (TTS)

When enabled, the AI reads its response aloud. This is useful when you want to keep your eyes on your work.

1. Open Settings → General.
2. Toggle Voice Response on.
3. Choose a voice avatar from the available options.
4. The AI will automatically read new responses aloud. You can stop playback at any time.

Knowledge Packs

Knowledge Packs are curated libraries of expert-written, step-by-step guides for specific apps and topics. When you ask a question, OverShoulder automatically searches through 3,700+ guides using vector search and injects the most relevant guide into the AI's context.

How Knowledge Packs work

OverShoulder uses a 5-stage pipeline to find the best answer for your question:

1. Detect — OverShoulder identifies which app you're using from your shared screen (Photoshop, Excel, HomeTax, etc.).
2. Enrich — Your question is combined with screen context (window title, URL) for smarter matching. Even a vague question like "what do I do next?" becomes actionable when the AI knows you're on a specific page.
3. Search — The enriched query is converted to a vector embedding and semantically matched against 22,000+ guide chunks using pgvector cosine similarity (threshold: 60%).
4. Fallback — If no Knowledge Pack matches, OverShoulder automatically searches the web using the enriched query — so you still get relevant, real-time information.
5. Inject — The best matching guide (or web result) is injected into the AI's context. The AI responds with verified, step-by-step instructions instead of generic guesses.

What packs are available?

23 packs covering software tools, finance, and more:

• Software: Adobe Photoshop (360 guides), Figma (219), Microsoft Excel, Word, PowerPoint, Google Docs, Google Sheets, Canva, Notion, Slack, Zoom
• Finance: US Tax Filing, Korean Tax Filing, Stock Investing, ETF/Index Funds, Investment Analysis, Crypto/Digital Assets
• Life: Health Insurance, Home Buying, Car Buying, Student Loans, Legal Self-Help
• OverShoulder: In-app help guide for OverShoulder itself

See the full catalog: Explore All Knowledge Packs →

Memory Box

Memory Box is OverShoulder's long-term memory system. It automatically extracts and remembers important facts from your conversations so the AI can personalize its responses over time.

How to access Memory Box

1. Click the folder icon at the bottom of the panel (next to the screen share button).
2. A panel slides out showing all stored memories, organized by category.

What gets remembered

• Facts about you — Your preferences, interests, skill level, tools you use
• Session summaries — Key takeaways from past conversations
• Preferences — How you like answers formatted, your communication style preferences

Managing memories

• Edit — Click a memory to edit its content. Fix anything the AI got wrong.
• Delete — Click the trash icon on any memory to permanently remove it.
• Pin — Pin important memories so they're always included in the AI's context.

Message Actions

Hover over any message to reveal action buttons:

• Copy — Copies the message text to your clipboard.
• Reply — Quote a specific message when responding (like Telegram). Click the reply arrow on any message, add your text, and send. The chat bubble shows both the quoted message and your reply for clear context.
• Send to Tab — Sends the message content to another tab. Click the share icon to see a dropdown of your other tabs, then select the destination. You can also create a new tab directly from this menu.
• Feedback (thumbs up/down) — Rate AI responses to help improve quality over time.

Settings: Account

Open Settings by clicking the gear icon in the top-left corner of the panel.

• Email — Your registered email address (read-only).
• Display Name — Set how you'd like the AI to address you.
• Use Purpose — Tell OverShoulder what you primarily use it for (work, study, personal). This helps personalize AI responses.
• Log Out — Signs you out and returns to the login screen.

Settings: API Keys (BYOK)

BYOK (Bring Your Own Key) lets you use your own API keys from AI providers. When you use BYOK, messages go through your own API key and don't consume OverShoulder credits.

How to set up BYOK

1. Open Settings → API Keys.
2. You'll see three provider sections: Anthropic (Claude models), OpenAI (GPT models), and Google (Gemini models).
3. Paste your API key into the text field for the provider you want to use.
4. Click Save. The key is encrypted and stored locally on your device.
5. When you select a model from that provider, OverShoulder will use your key instead of consuming credits.

Where to get API keys

• Anthropic: console.anthropic.com → API Keys
• OpenAI: platform.openai.com → API Keys
• Google: aistudio.google.com → API Keys

Settings: Billing & Credits

• Current Plan — Shows your subscription tier (Trial, Basic, Pro, or Enterprise).
• Credits Remaining — Your current credit balance. Credits are consumed per message based on the model used (1, 3, or 12 credits per message).
• Upgrade — Opens the subscription management page in your browser.
• Top-up Credits — Purchase additional credits without changing your plan (available for Basic and Pro tiers).

Settings: Usage

Track how you're using OverShoulder over time.

• Time period — Switch between Today, This Week, and This Month views.
• Token Usage — Total tokens consumed in the selected period.
• By Model — Breakdown of usage per AI model, showing which models you use most frequently.

Settings: Privacy

Control how OverShoulder handles your data. See Privacy Modes below for detailed explanations.

• Privacy Mode — Choose between Strict, Balanced, or Full Assist. Controls what data is sent to AI providers.
• Allow Screen Capture — Toggle screen sharing capability on/off globally.
• Auto-mask Sensitive Info — When enabled, automatically detects and masks sensitive data (API keys, emails, IP addresses) before sending to the AI.

Settings: General

• Language — Switch the interface between English, Korean (한국어), and Japanese (日本語). The AI also adjusts its response language to match.
• Color Theme — Choose your panel theme (Default, Midnight, Aurora, etc.).
• Panel Transparency — Adjust how transparent the floating panel is. Higher transparency lets you see more of the app behind it.
• Edge Glow — Toggle the decorative glow effect around the panel border.
• Launch at Login — Automatically start OverShoulder when you log into macOS.
• Clear Chat — Deletes the conversation history for the current tab.

Settings: Shortcuts

View all keyboard shortcuts in one place. See the Keyboard Shortcuts reference below.

Privacy Modes

OverShoulder offers three privacy levels. Choose based on how much data you're comfortable sharing with AI providers.

How to change your privacy mode

1. Open Settings → Privacy.
2. Click on the mode card you want to use (Strict, Balanced, or Full Assist).
3. The change takes effect immediately for all future messages.

Data Masking

When running in Balanced mode, OverShoulder's privacy engine automatically detects and replaces sensitive information before it reaches the AI model.

What gets masked

• API keys — Keys from Anthropic, OpenAI, Google, AWS, and other providers are detected and replaced with [API_KEY]
• Email addresses — Replaced with [EMAIL]
• IP addresses — Both IPv4 and IPv6 addresses are replaced with [IP_ADDR]
• Bearer tokens — Authentication tokens are replaced with [TOKEN]

Masking happens entirely on your local device before any data is transmitted to AI providers.

Where Your Data Lives

OverShoulder stores data in multiple locations. Here's exactly what goes where:

On your device (local only)

• API keys — Encrypted using macOS Keychain (safeStorage). Never leave your device.
• Memory Box data — Stored in data/memory/ as JSON files, organized by user email.
• Chat history — Stored in data/chat-history/ as JSON files per session.
• Settings & preferences — Stored in localStorage and synced to the server for cross-device access.
• Screenshots — Captured in-memory only. Never written to disk. Deleted immediately after OCR processing.

On OverShoulder servers

• Account info — Email, hashed password, subscription status.
• Settings — Your settings preferences for cross-device sync.
• Billing — Managed through Stripe. OverShoulder never stores card numbers.

Sent to AI providers (per-message)

• Your message — The text you typed or spoke.
• Screen context — Extracted text from your screen (masked in Balanced mode, omitted in Strict mode).
• Memory context — Relevant memories retrieved from your Memory Box.
• Knowledge guide — If a relevant Knowledge Pack guide is found.

Keyboard Shortcuts

These shortcuts work globally — even when OverShoulder isn't focused.

Frequently Asked Questions