Computer Use
Computer use lets the agent operate your desktop: screenshot, read UI elements, click, type, press keys, and scroll.
Enabling
Settings > Computer Use:
- Enable the master toggle
- macOS requires Accessibility and Screen Recording permissions
- Restart LanMate after granting permissions
Tools
| Tool | Capability | Confirmation |
|---|---|---|
| computer_screenshot | Capture screen (with UI tree) | No |
| computer_get_active_app | Get foreground app name | No |
| computer_get_ui_tree | Get UI element tree | No |
| computer_click | Click at logical coordinates | Yes |
| computer_type | Type text at focus | Yes |
| computer_press_key | Press key combos | Yes |
| computer_scroll | Scroll at coordinates | Yes |
Security
- Master toggle: Off by default, checked per-call
- App allowlist: Non-allowlisted apps prompt on write operations
- Unattended sessions: Channel/scheduler sessions only allow allowlisted apps
Standard agent loop: screenshot → locate element → click/type → verify.
Platform Support
| Platform | Status |
|---|---|
| macOS | Supported |
| Windows | Supported |
| Linux | Implemented (xdotool + pyatspi2), pending E2E validation |