desktop-operator

What This Skill Does

The desktop-operator skill is a specialized automation tool designed for macOS environments to interact with Electron-based desktop applications. By leveraging the Puppeteer Chrome DevTools Protocol (CDP), this agent can bridge the gap between AI commands and graphical user interfaces. It enables users to programmatically launch applications, navigate to specific visual elements defined by text, interact with them via click events, and capture high-fidelity screenshots for confirmation. This effectively turns standard desktop software into programmable entities that the OpenClaw AI can manipulate.

Installation

To add this skill to your workspace, utilize the OpenClaw CLI tool. Run the following command in your terminal:

clawhub install openclaw/skills/skills/bondli/desktop-operator

Ensure that your environment meets the prerequisites: the application you intend to control must be located in your /Applications/ directory, and your Terminal must have granted Accessibility permissions under "System Settings → Privacy & Security" to allow the execution of automated UI interactions.

Use Cases

This skill is ideal for repetitive GUI-based tasks that lack a formal API. It is frequently used for automating workflows in Electron management dashboards, local data analysis tools, and proprietary utility software. For instance, if you regularly need to log into a local dashboard, export reports, or toggle settings across multiple sessions, the desktop-operator automates the manual mouse-and-keyboard effort, reducing human error and increasing productivity.

Example Prompts

1. "Open the DaveBella app, find the 'Order Center' button, click it, and send me a screenshot of the results."
2. "Can you launch the Obsidian app and click on the 'Settings' icon, then take a screenshot for me?"
3. "Please open the AnalyticsTool application and navigate to the data analysis dashboard by clicking the relevant text, then capture the current view."

Tips & Limitations

• Targeting: This skill relies strictly on visible text within the Electron UI. Ensure the text passed as the --target argument exactly matches the button label or link text in the UI.
• Compatibility: This tool is strictly limited to Electron applications. It will not work for native macOS applications (e.g., Pages, Mail) or web browsers not built with Electron.
• Environment: Always ensure the target application is not already crashing or stuck in a "not responding" state, as the CDP connection requires a stable, running process to inject commands successfully.
• Security: Since this tool interacts with your local system's UI and processes, ensure you only use it on trusted applications to prevent unauthorized interface manipulation.