linux-desktop
Control and automate the Linux desktop GUI on X11. Use this skill to take screenshots, find and click UI elements, type text, send keyboard shortcuts, scroll, manage windows (focus, minimize, maximize, close, move, resize), and use your own vision to locate elements on screen. Requires X11 (not Wayland). Use for desktop automation, GUI testing, remote desktop control, and any task requiring interaction with graphical applications.
Install via CLI (Recommended)
clawhub install openclaw/skills/skills/gedigi/peekabooxLinux Desktop GUI Automation
Automate any X11 Linux desktop: capture screens, find and click elements, type, use hotkeys, manage windows.
Preferred screenshot interpretation path: capture with capture.sh and interpret the image directly in your OpenClaw chat (existing image-capable model connection).
Prerequisites
- X11 session running (XFCE, GNOME on X11, KDE on X11, i3, openbox, etc.)
DISPLAYenvironment variable set (usually:0)- Run
bash install.shonce to install dependencies - No extra key needed for screenshot interpretation when using OpenClaw's image-capable chat path
Quick Reference
| Task | Command |
|---|---|
| Take screenshot | bash capture.sh |
| Screenshot of window | bash capture.sh --window "Firefox" |
| List windows | bash inspect.sh |
| Active window info | bash inspect.sh --active |
| Find window by name | bash inspect.sh --window "Firefox" |
| Click at coordinates | bash click.sh --x 500 --y 300 |
| Right-click | bash click.sh --x 500 --y 300 --button right |
| Double-click | bash click.sh --x 500 --y 300 --double |
| Click relative to window | bash click.sh --window "Firefox" --x 200 --y 150 |
| Type text | bash type.sh "hello world" |
| Type into window | bash type.sh --window "Terminal" "ls -la" |
| Send hotkey | bash hotkey.sh "ctrl+c" |
| Send Enter | bash hotkey.sh "Return" |
| Scroll down | bash scroll.sh --direction down --amount 3 |
| Scroll up at position | bash scroll.sh --x 500 --y 300 --direction up --amount 3 |
| Focus window | bash window.sh --action focus --window "Firefox" |
| Minimize window | bash window.sh --action minimize --window "Firefox" |
| Maximize window | bash window.sh --action maximize --window "Firefox" |
| Close window | bash window.sh --action close --window "Firefox" |
| Move window | bash window.sh --action move --window "Firefox" --x 100 --y 50 |
| Resize window | bash window.sh --action resize --window "Firefox" --width 1280 --height 800 |
Typical Automation Workflow
For most GUI automation tasks, follow this pattern:
- Capture a screenshot with
capture.sh— note the file path printed - Look at the screenshot yourself to understand what's on screen
- Find the target element by examining the screenshot and estimating its pixel coordinates
- Act using the coordinates:
click.sh --x X --y Y - Verify by capturing another screenshot and checking the result
Example: Click the Save button in a dialog
# Step 1: Capture the screen
SCREENSHOT=$(bash capture.sh | tail -1)
# Step 2: Look at the screenshot (read the image file with your vision)
# Examine the image and identify the Save button's position
# Step 3: Click at the coordinates you identified
bash click.sh --x 450 --y 320
Example: Type into a specific application
# Focus the terminal window and type a command
bash type.sh --window "Terminal" "ls -la"
bash hotkey.sh "Return"
Metadata
Not sure this is the right skill?
Describe what you want to build — we'll match you to the best skill from 16,000+ options.
Find the right skillPaste this into your clawhub.json to enable this plugin.
{
"plugins": {
"official-gedigi-peekaboox": {
"enabled": true,
"auto_update": true
}
}
}