Can you teach an AI agent a task by showing it once?

What is the difference between recording a task and describing one?

When you describe a task in plain language — 'fetch the latest invoice, extract the total, and post it to the accounting sheet' — the agent has to infer every intermediate step. That inference draws on general reasoning, which is flexible but occasionally wrong in ways that are hard to predict.

Recording a task means the agent observes a real execution: the exact field you clicked, the exact value you typed, the exact endpoint that responded. The resulting instruction set is far more specific. Specificity helps when the environment is stable. It becomes a liability the moment the environment changes, because the agent has no general model to fall back on — it only knows what it saw.

Why does a recorded step have to be matched by name rather than position?

Position-based matching means the agent finds a field by where it sits on the screen or in a data structure: 'the third column', 'the second input in the form'. This works until a developer adds a column, reorders a form, or a responsive layout shifts elements on a smaller viewport. The agent then acts on the wrong field with complete confidence.

Name-based matching anchors each step to a stable identifier: a field label, an HTML attribute, an API key name. Names survive layout changes. They also survive pagination and infinite scroll, which position-based logic handles poorly. The practical rule is simple: if you cannot give a step a meaningful name, the task probably should not be recorded — it should be described instead, so the agent can reason about what it is actually trying to achieve.

Position-based matching

Finding an element by where it appears in a layout or data structure. Fast to record, fragile when the interface changes.

Name-based matching

Finding an element by a stable identifier such as a label, attribute or API key. Slower to set up, resilient to layout changes.

Silent failure

When an automation acts on the wrong target without raising an error. Common with position-based steps after an interface update.

Graceful degradation

When an agent detects it cannot match a named element and pauses for human review rather than proceeding on a wrong assumption.

What should never be captured in a recorded task?

Credentials are the obvious answer. Any recording that captures a password, an API key, or a session token creates a secret embedded in an instruction file. Instruction files get shared, version-controlled, and sometimes exported. The correct pattern is to record the shape of the authentication step — 'enter the API key from the secure vault' — and resolve the actual value at runtime from a secrets store the agent has permission to read.

Personal data about third parties is a less obvious but equally important exclusion. If you record a task that processes customer records, the recording itself should reference the data source, not a specific row you happened to use during the demonstration. A recording containing a real customer's name, address or payment detail is a data handling problem waiting to surface during an audit. Use synthetic or anonymised examples when teaching a task that will later run against live personal data.

• Passwords, API keys and session tokens — resolve these at runtime from a secrets store.
• Personal data used during the demonstration — substitute synthetic or anonymised records.
• Hardcoded file paths that only exist on your machine — use relative paths or environment variables.
• Timestamps recorded as literal values — capture the logic ('yesterday's date') not the value ('2026-08-15').
• UI coordinates — replace with named selectors before saving the recording.

When is a simpler tool the better choice?

A spreadsheet macro, a shell script, or a no-code automation tool is often the right answer for a task that is purely linear, touches only one application, and never needs to reason about context. These tools are auditable, version-controllable, and understood by more people than an agent instruction file.

An agent adds value when the task crosses multiple tools, requires judgement at decision points, or needs to handle exceptions that a rigid script would simply crash on. If you find yourself recording a task that has no branches and no ambiguity, ask whether a simpler automation would be easier to maintain. GROX Circuits are designed for exactly the cases where the task is genuinely multi-step and context-dependent — not as a replacement for a three-line script.