Introducing Mecum: let agents work in your apps

Agents can write your code. Now they can use your apps.

We're launching Mecum, a free, open source macOS app that lets agents work in desktop software while you keep using your Mac. Give a worker a task in Mecum, or connect Claude Code or Codex through MCP.

A lot of work happens inside an interface. You edit a clip, move through an export dialog, create an archive and send the result to someone. Mecum gives agents a way to do that work in the apps you already use, including software with no API.

Watch Mecum in action on GitHub.

A screen for your agent, on your Mac

We built a background driver that moves app windows to a virtual display and routes the agent's input there. The agent gets a place to work on the same Mac while your cursor stays available for your own work.

You use your existing apps and files. The driver works in the current Mac session without a virtual machine.

Some apps can still grab focus when they launch or open a dialog. Mecum tries to restore your window, and a run can pause if recovery fails. You can follow the activity and stop a worker from the app. Stopping a run does not undo actions already completed.

From an interface to an action

Mecum's Perception pipeline turns the current window into structured text on your Mac. It reads the pixels and adds accessibility information when available. This gives the model context even when an app has no usable accessibility tree.

The engine uses the current interface to find the target, act and check what changed. You can inspect the result in the app and see where the agent could confirm an effect or needs another look.

Perception can miss controls. Support depends on the application, its version and the task. The current scope includes AppKit, Chromium, Electron and Qt. UXP support for Adobe apps is in development. The compatibility guide describes the limits.

Two workflows from the launch demo

In one run, an agent edits a clip in DaVinci Resolve, makes it black and white, exports it, creates a ZIP in Finder and sends the archive on Slack. The task moves through several apps to reach a deliverable.

In another run, an agent uses Chrome to research posts about computer use agents on X and prepares a draft based on what it finds. The recording ends at the draft.

These are two separate recordings, sped up and trimmed for the launch video. During the DaVinci export, the user switched windows once after Mecum paused its background controls. The full run took about 25 minutes. The short video shows selected steps from that run.

App knowledge that grows with use

As agents work, Mecum's Brain builds a map of the controls, relationships and transitions they encounter in each app. That knowledge gives later tasks more context about how the app works.

Memory stores past interactions and their outcomes for reuse. Recall is still in development and can be inconsistent. The agent still needs to check the current interface before acting.

You can explore the Brain in Mecum and report controls or relationships it missed. Those examples help us improve the system around real workflows.

Less context for the model

Our first benchmark compared Mecum's structured text with Cua's screenshot and accessibility-tree responses. Across paired full reads, Mecum returned 85–97% fewer estimated tokens.

This measures the context in individual tool responses on one Mac. It does not measure model bills, complete-task success or end-to-end speed. The full report includes the method, application results and limitations.

Start in Mecum, or bring your agent

Create a worker in Mecum and connect a supported model. If you already use Claude Code or Codex, connect it through the local MCP bridge with the configuration Mecum generates.

Mecum is free. Model access has its own requirements: your connection may use a subscription, an API key or a model server. The desktop window-to-text pipeline runs on your Mac; a remote model can receive window text, task context and relevant app knowledge. Other tools can return screenshots, so the data sent depends on the tools and connection you use.

You can create multiple workers. They take turns controlling desktop apps while the others wait for access.

Open source, including the driver

Mecum is open source under Apache 2.0, including the app, the engine and the background driver. You can inspect the code, build from source and use the Driver, Perception and Engine libraries in your own macOS projects.

We're looking for contributions that expand app support, improve Perception and make Memory more reliable. A reproducible report of a task that failed is useful too.

Download Mecum and follow the setup guide. Start with a small task and a result you can check. Tell us what works and what breaks on GitHub or Discord.