Computer Vision Actions

Computer vision actions let an AI Employee work the way a person does: by looking at the screen. Instead of clicking a fixed pixel position, the AI Employee finds what it needs by matching text or images on screen, so the job keeps working even when windows move or layouts shift. Most of these actions share the same targeting settings - an Action Target Region (currently focused window, most recently defined region, a specific region, or the entire screen) and a locator that finds text, a stored image, an image variable, or coordinates within that region. Text locators support regular expressions; see Computer Vision: Find Text Using Regex for the matching rules.

Click On Visible Text

The simplest way to click something: give the AI Employee the exact text as it appears on screen (a button label, menu item, or link) and it finds that text in the currently focused window and clicks its center. Use this whenever the click target has stable visible text - it is faster to set up and more reliable than an AI prompt, and consumes no AI usage. Key settings: Target Text (required; supports {{Field}} tokens, so you can click a runtime value like an invoice number) and Hold Key While Clicking (for example, hold CONTROL to open a link in a new browser tab).

Computer Vision - Define Region

Locates a region of the screen and remembers it as the “most recently defined region” so later computer vision actions can search inside it. Use it to narrow the search area first - for example, find a panel or table once, then run several reads or clicks within it. Key settings: Action Target Region and What to Find Within the Above Target Region? (find single text or an image, a label’s value, a table row value, region borders, or a region directly above/below/left/right of something), plus optional Expand Located Region and Adjust Located Region fine-tuning.

Computer Vision - Find on Screen

Checks whether something is currently visible on screen: the action succeeds if the target is found and fails if it is not. Use it to verify a screen state before continuing, or to branch the job using the action’s success and failure handling. Key settings: the shared targeting options (Action Target Region, What to Find Within the Above Target Region?, and the locator), an optional Search Everywhere Key the AI Employee presses to move around while searching (down arrow or tab are most common), and an optional Abort Action If Seen locator that stops the action if unwanted content appears.

Computer Vision - Find by Iterating Over a Table

Works through the records of a process table, taking one field value at a time and searching the screen for it until a match is found. Use it when the screen shows one of many possible values - for example, matching an on-screen order number against your orders table. When a match is found, the matched text and the matching record are made available to later actions; the action fails if no record’s value appears on screen. Key settings: Action Target Region, Process Table, Table Query Builder (Optional) to reduce the records searched, Table Field Value to Find on Screen, Text Search Direction, and the optional Search Everywhere Key and Search Everywhere Keystroke Count.

Computer Vision - Mouse Click

Locates a region on screen using the shared targeting settings and clicks it. Use this instead of Click On Visible Text when you need a different click type, an image locator, or precise control over where the click lands. Key settings: Mouse Click Type (left, right, double, triple, or hover with no click), Mouse Click Location within the found region, Leave Mouse at Click Location, Hold Key While Clicking, and an optional Abort Action If Seen locator.

Computer Vision - Mouse Scroll

Locates a region on screen and scrolls the mouse wheel over it. Use it to bring off-screen content into view - inside a scrollable list or panel - before reading or clicking. Key setting: Mouse Scroll Direction Type (scroll up or scroll down), plus the shared targeting settings that decide where the scroll happens.

Computer Vision - Read Text

Reads the text in a located region of the screen and stores it so later actions can use it (for example, to save into a process field). Use it to capture values from applications that offer no other way to extract data. Key settings: the shared targeting options, Detect and Ignore Partial Text (skip text that is cut off at the region edge), Validation Regex to confirm the text was read correctly, and Apply Regex Adjustments to automatically clean up the result until it passes validation.

Computer Vision - Type Field Value

Finds an input field on screen, clicks it, and types into it in one step. Use it to fill forms in desktop applications where fields are located by a nearby label or image rather than a fixed position. Key settings: the shared targeting options plus Text to Type (supports {{Field}} tokens, so you can type a value from the process), Clear Text Before Typing, Verify Text After Typing, and Press Key After Typing (for example, TAB to move to the next field).

Computer Vision - Date Picker

Selects a date from an on-screen calendar control. The AI Employee reads the calendar, clicks the backward or forward navigation until the target month and year are displayed, then clicks the day. Key settings: Date To Select (supports {{Field}} tokens), Current Selected Date (defaults to today’s date if left blank), Month Name List (All 12) for calendars that display non-standard month names, and locators for the calendar’s Navigate Backward and Navigate Forward buttons.

Computer Vision - Reset Find Next

When a locator uses “Find Next”, the AI Employee remembers which matches it has already visited so each search moves to the next one - useful for looping through a list of results. This action clears that memory so the next “Find Next” search starts from the beginning again. It has no settings; place it wherever a loop should restart from the top.