Macpilot

What This Skill Does

MacPilot is a powerful automation suite that bridges the gap between AI agents and the macOS operating system. By leveraging native Accessibility APIs, this skill set allows your AI agent to control your computer as if it were a human user. It provides comprehensive capabilities for interacting with desktop applications, managing system windows, manipulating UI elements, and handling native OS dialogs. Whether it is performing repetitive clicks, OCR-based interactions, or complex window management, MacPilot transforms your AI agent into a full-fledged macOS operator.

Installation

To integrate this functionality into your environment, use the OpenClaw installation command: clawhub install openclaw/skills/skills/adhikjoshi/macpilot

Alternatively, you can add it via npx if using the standard skills registry: npx skills add adhikjoshi/macpilot-skills

Ensure you have granted MacPilot.app the necessary 'Accessibility' and 'Screen Recording' permissions within your macOS System Settings under 'Privacy & Security'.

Use Cases

MacPilot is designed for users who want to automate macOS workflows that lack traditional APIs. Common use cases include:

• Automated UI testing: Interacting with buttons and menus in native apps that are otherwise inaccessible via command line.
• Window management: Automatically organizing your workspace by snapping windows to sides or managing Spaces.
• File operations: Automating the 'Save As' or 'Open' dialog processes when dealing with legacy software.
• Workflow orchestration: Combining browser-based tasks with native desktop apps (e.g., pulling data from a browser and pasting it into an Excel spreadsheet).

Example Prompts

1. "Open the Calculator app, perform a complex calculation, and copy the result to my clipboard."
2. "Find the 'Settings' button in the active window, click it, and then resize the window to take up the left half of my screen."
3. "Save the current document to my Desktop by interacting with the native file dialog, ensuring I select the PDF format."

Tips & Limitations

Because MacPilot relies on Accessibility APIs, it is sensitive to UI changes. If an app updates its interface, your AI agent might need a moment to re-inspect the UI elements. For the best reliability, ensure your screen resolution is stable and try to avoid moving windows while the agent is executing tasks. The OCR capability is highly effective for visual cues, so if direct element interaction fails, ask the agent to 'use OCR to find and click the text' instead. Always maintain human oversight when the agent is performing file deletions or system-level configuration changes.