Automation Reliability: Success Criteria and Error Modes
Reliable automation is not about writing perfect code the first time. It is about building systems that surface failures clearly, recover gracefully when possible, and give you the information you need to fix problems quickly. Chase Agents provides several interlocking mechanisms to achieve this.
Success Criteria
When you create or edit an automation, you can define a success criteria: a plain-language description of what a successful run looks like. This criteria is evaluated by an AI model after each run completes. If the run's outputs and logs satisfy the criteria, the run is marked successful. If they do not, the run is flagged for review even if no exceptions were thrown.
Success criteria close a significant reliability gap: many automations can complete all their steps without error but still produce wrong outputs. For example, a reporting automation might run without exceptions but generate a report with zero rows because a filter condition was set incorrectly. With success criteria, you can catch this by specifying that a successful run should produce a report with at least one row.
Error Handling Mode
Every automation has an error handling mode that controls how it responds when a step fails. The two modes are ai and any.
- In ai mode, when a step fails the platform engages the AI auto-fix engine: it analyzes the error, rewrites the failing step, and re-runs it. This mode is best for automations that call external APIs or execute dynamic code where transient errors and minor logic bugs are common.
- In any mode, any step failure immediately halts the automation and logs the error without attempting a fix. This mode is appropriate when you want strict control and need to be notified immediately of any deviation from expected behavior.
Pre-Execution Auto-Fix
The automation_pre_execution_auto_fix setting enables the platform to analyze your automation's steps before each run and check for obvious problems: missing required fields, malformed references to prior step data, invalid API calls. If it detects issues, it attempts to correct them before execution begins. This prevents entire runs from failing due to configuration errors that could have been caught earlier.
Post-Execution Auto-Fix
The automation_post_execution_auto_fix setting enables the platform to analyze a failed run after the fact, identify which step caused the failure and why, rewrite that step, save the corrected version, and schedule a re-run. This is particularly effective for automations that interact with third-party APIs where endpoint formats or authentication requirements change over time.
Retry Behavior
Chase Agents performs one internal retry attempt per automation run (MAX_WORKFLOW_RETRY_ATTEMPTS is set to 1). For scheduled automations and webhook-triggered runs, external retry logic is also managed by the platform: if a run fails and auto-fix produces a corrected version, the corrected automation is re-executed automatically. This design ensures that retries always run against the most up-to-date version of the automation rather than blindly repeating a broken configuration.
Rate Limiting
To prevent runaway automations from consuming excessive resources, each automation is limited to five executions per five-minute window. If a trigger fires more frequently than this, additional executions are queued and processed once the rate window resets. You can monitor execution frequency in the Workflow History page.
Step-Level Logging
Every step in every automation run is logged individually. You can expand any run in Workflow History to see the inputs, outputs, and any errors for each step in sequence. This step-level visibility makes debugging fast: instead of guessing which part of a multi-step workflow failed, you can see exactly which step threw an error and what data it was working with at the time.