Skip to main content

Troubleshooting

Storage backend: SQLite as of v0.2.6. File paths under agent_io/target/.../data.json in older snippets are stale; this page uses the current SQLite layout (agent_io/store/<workflow>.db, table target_data).

What happens when something goes wrong in your agentic workflow? This guide helps you debug and fix common errors. Let's explore the error types, their causes, and how to resolve them.

Error Types​

SchemaValidationError​

The most common error. Occurs when data fails JSON Schema validation.

text
TXT
SchemaValidationError: Input schema validation failed for tool 'add_answer_text'
at target_word_counts -> correct_answer_words: 18 is not of type 'string'
[Context: function=add_answer_text, validation_type=input,
 error_path=target_word_counts -> correct_answer_words,
 failed_value=18, schema_constraint={'type': 'string'}]

Context Fields:

FieldDescriptionExample
error_pathDot-path to failing fieldtarget_word_counts -> correct_answer_words
failed_valueThe actual value received18, 'some text', ['a', 'b']
schema_constraintExpected schema rule{'type': 'string'}
functionTool that failedadd_answer_text
validation_typeInput or output validationinput, output

ProcessingError​

Wraps lower-level errors with record context. Think of this as a breadcrumb trail—it tells you which specific record caused the problem.

text
TXT
ProcessingError: Failed to process item
[Context: source_guid=37812c37-80a2-596b-8747-8f93e7a34e7f,
         agent_name=add_answer_text]

Context Fields:

FieldDescription
source_guidUUID of the record being processed
agent_nameAction that failed

RecordContextError (Record Namespace)​

Occurs when a downstream action tries to read fields from an upstream action's output, but the expected fields are missing. This typically means the upstream action stored corrupted or empty data.

text
TXT
[RECORD NAMESPACE] 'generate_optimal_code': declared fields ['optimal_code'] not found.
Available: ['title', 'type', 'properties', 'required', 'additionalProperties']

Common causes:

CauseWhat happenedFix
Schema-echoLLM returned the JSON Schema definition instead of dataEnable on_schema_mismatch: reprompt — the validator now detects schema-echo payloads and triggers reprompt
Empty objectLLM returned {}Same fix — empty objects are now rejected when the schema declares fields
Existing poisonBad data already in target_data from a previous runRe-run the upstream action with reprompt enabled, or manually clean the SQLite row

Behavior: When a single record has corrupted namespace data, that record is skipped and processing continues for the remaining records in the file. The skipped record is logged at WARNING level. This prevents one bad record from failing an entire file of valid records.

AgentActionsError​

Top-level agentic workflow failure. This wraps other errors to provide the full context of what went wrong.

text
TXT
AgentActionsError: Error generating target: Failed to process content
[Context: file_path=/path/to/data.json,
         agent_name=my_workflow]

Common Errors & Fixes​

Let's walk through the most common errors and how to fix them.

Template Variable Error​

Error:

text
TXT
Template rendering failed for agent 'write_question'

  Reference: classify.question_type
  Namespace 'classify' exists: YES
  Field 'question_type' in namespace: NO
  Available in 'classify': question_category, difficulty_level, tags

  Did you mean 'classify.question_category'?

Fix: Check that 'classify' produces the referenced field.

Cause: Template references a field that doesn't exist in the namespace. The error message shows:

  • Whether the namespace (e.g., classify) exists
  • Whether the field exists within that namespace
  • Available fields you can use instead
  • A suggestion if there's a similar field name (typo detection)

Fix options:

  1. Check the field name - Use the suggested alternative if it's a typo
  2. Verify the dependency - Ensure the action producing this field is in your dependencies list
  3. Check the upstream action's output - Verify the action produces the expected field

Source Data Structure Mismatch​

Error:

text
TXT
Template rendering failed for agent 'my_action'

  Reference: source.content
  Namespace 'source' exists: NO
  Available namespaces: items, metadata

Cause: Source data is a wrapper object, not a flat array of records. This happens when your JSON file has metadata alongside the actual records.

Wrong format:

json
JSON
{
  "exam_name": "My Exam",
  "items": [
    {"id": "1", "content": "..."},
    {"id": "2", "content": "..."}
  ]
}

Prompt expects {{ source.content }} but source is the wrapper, not individual items.

Correct format:

json
JSON
[
  {"id": "1", "content": "..."},
  {"id": "2", "content": "..."}
]

Fix options:

  1. Restructure input - Extract array to staging file:

    python
    PY
    data = json.load(open("wrapper.json"))
    records = data["items"]
    json.dump(records, open("staging/data.json", "w"))

  2. Add preprocessing action - First action extracts items:

    yaml
    YAML
    - name: extract_items
     kind: tool
     impl: extract_items_from_wrapper
     granularity: file

  3. Update prompts - If wrapper is intentional:

    jinja2
    JINJA2
    {% for item in source.items %}
      {{ item.content }}
    {% endfor %}

Type Mismatch​

Error:

text
TXT
18 is not of type 'string'

Cause: Integer value where string expected.

Fix: Convert in your tool:

python
PY
data['field'] = str(data['field'])

Or fix the TypedDict:

python
PY
# If the field can be int or string
field: Union[int, str]

Array vs String Mismatch​

Error:

text
TXT
'Audit client code...' is not of type 'array'

Cause: String value where array expected.

Fix: Normalize in your tool:

python
PY
answer_text = data["upstream_action"]["answer_text"]
if isinstance(answer_text, str):
    answer_text = [answer_text]

Unexpected Field​

Error:

text
TXT
'new_field' was unexpected

Cause: Field exists in data but not declared in TypedDict.

Fix: Add field to TypedDict:

python
PY
class MyInput(TypedDict, total=False):
    existing_field: str
    new_field: str  # Add the missing field

Or use total=False to allow any fields.

Mixed-Type Dict Values​

Error:

text
TXT
'greater_than' is not of type 'integer'

Cause: Using Dict[str, int] but values include strings.

Fix: Use plain dict for mixed-type dictionaries:

python
PY
# BAD - Fails if values include strings
target_word_counts: Dict[str, int]

# GOOD - Allows any structure
target_word_counts: dict

Missing Required Field​

Error:

text
TXT
'question' is a required property

Cause: Schema requires field but tool didn't return it.

Fix: Ensure tool returns all required fields:

python
PY
@udf_tool()
def my_function(data: dict) -> dict:
    return {
        'question': data["upstream_action"]["question"],
        'processed': True
    }

Or enable reprompting for LLM actions:

yaml
YAML
reprompt:
  max_attempts: 4
  on_exhausted: return_last

Debugging with Prompt Traces​

When an LLM action produces unexpected output, the fastest path to understanding "why" is inspecting the compiled prompt and raw response. Agent Actions captures both automatically.

warning

Prompt traces only exist for LLM actions. Tool (UDF) actions do not generate prompt traces. If a tool action produces empty or wrong output, debug the tool function directly — check the input data it receives, not prompt_trace.

Using the Data Explorer​

  1. Run agac docs to generate the documentation catalog
  2. Open the Data Explorer in your browser
  3. Navigate to the action's output in the Data tab
  4. Find the record with unexpected output
  5. Click the Prompt Trace accordion below the record — it shows:
    • Compiled Prompt: The exact text the LLM received (with all template variables resolved)
    • LLM Response: The raw text the LLM returned (before parsing)

What to Look For​

  • Missing context: Template variables resolved to empty strings or wrong values — check your context_scope and dependency chain
  • Ambiguous instructions: The prompt doesn't clearly constrain the output format — tighten the prompt template
  • Schema mismatch: The LLM response doesn't match the expected JSON structure — consider enabling reprompting
  • Model badge: Check if the model name matches what you expected — a misconfigured provider can route to the wrong model

Querying Traces Directly​

For bulk analysis across many records, you can query the storage backend directly. With the default SQLite backend:

bash
BASH
sqlite3 my_workflow/agent_io/store/my_workflow.db \
  "SELECT record_id, response_text FROM prompt_trace WHERE action_name = 'classify_issue' LIMIT 10"

See Prompt Traces reference for the full table schema and query examples.

Inspecting Failed Record Input Snapshots​

When records fail or exhaust retries, the framework captures the input record at the moment of failure. Query the disposition table to see what data was being processed:

sql
SQL
-- View failed records with their input snapshots
SELECT action_name, record_id, reason, input_snapshot
FROM record_disposition
WHERE disposition = 'failed' AND input_snapshot IS NOT NULL;

-- View exhausted records (retries exceeded)
SELECT action_name, record_id, reason, input_snapshot
FROM record_disposition
WHERE disposition = 'exhausted' AND input_snapshot IS NOT NULL;

Debugging Agentic Workflows​

When an error occurs, resist the urge to start changing code immediately. Follow this systematic approach to understand what went wrong before fixing it.

Step 1: Parse the Error​

Extract key information from the error message. Agent Actions provides structured error context—use it:

  1. What type? SchemaValidationError, ProcessingError, etc.
  2. Which field? Look at error_path
  3. What value? Look at failed_value
  4. What expected? Look at schema_constraint
  5. Which action? Look at function or agent_name

Step 2: Find the Source Record​

Use source_guid to trace the record. Each action's output is a JSON array stored in the data column of the target_data table, keyed by action_name. Use json_each(data) to scan records and json_extract to filter by source_guid:

bash
BASH
# Find every action that produced a record with this source_guid
sqlite3 agent_io/store/<workflow>.db "
  SELECT DISTINCT t.action_name
  FROM target_data t, json_each(t.data) r
  WHERE json_extract(r.value, '\$.source_guid') = '37812c37-80a2-596b-8747-8f93e7a34e7f'
"

Step 3: Check Node Outputs​

Compare data at each stage. Filter the JSON blob in the data column by source_guid to inspect the same record at each step:

bash
BASH
# Input to failing action — fetch the matching record
sqlite3 agent_io/store/<workflow>.db "
  SELECT json_extract(r.value, '\$')
  FROM target_data t, json_each(t.data) r
  WHERE t.action_name = '<failing_action>'
    AND json_extract(r.value, '\$.source_guid') = '37812c37-80a2-596b-8747-8f93e7a34e7f'
"

# Previous action's output for the same record
sqlite3 agent_io/store/<workflow>.db "
  SELECT json_extract(r.value, '\$')
  FROM target_data t, json_each(t.data) r
  WHERE t.action_name = '<previous_action>'
    AND json_extract(r.value, '\$.source_guid') = '37812c37-80a2-596b-8747-8f93e7a34e7f'
"

Step 4: Enable Debug Mode​

Get detailed tracebacks:

bash
BASH
AGENT_ACTIONS_LOG_LEVEL=DEBUG agac run -a my_workflow

Step 5: Enable Prompt Debug​

See rendered prompts for LLM actions:

yaml
YAML
- name: my_action
  prompt: $workflow.My_Prompt
  prompt_debug: true  # Logs full prompt

Data Lineage​

Here's where it gets interesting: every record in Agent Actions maintains tracking fields that let you trace its journey through the agentic workflow. This lineage data is invaluable for debugging.

source_guid​

Original content UUID. Never changes through the agentic workflow.

json
JSON
"source_guid": "37812c37-80a2-596b-8747-8f93e7a34e7f"

Use this to trace any record back to its source, regardless of how many actions it has passed through.

lineage​

Array of all node_ids visited. Grows at each node.

json
JSON
"lineage": [
  "node_0_693094fb-53d1-48d6-bdc9-781a4989d35c",
  "node_1_361c54c6-7080-4527-9a00-aaeccfd0e6ba_0",
  "node_2_e546c260-de10-4f20-8950-e09f01ea468f"
]

The _0 suffix indicates this was the first record from a flattening operation.

node_id​

Identifies the specific processing node:

text
TXT
node_0_693094fb-...          # Single output
node_1_361c54c6-..._0        # Flattened, index 0
node_1_361c54c6-..._1        # Flattened, index 1

Action Output Storage​

Action outputs are stored as JSON arrays in the target_data table of agent_io/store/<workflow>.db, keyed by (action_name, relative_path). List what each action produced:

bash
BASH
sqlite3 agent_io/store/<workflow>.db "
  SELECT action_name, relative_path, record_count
  FROM target_data
  ORDER BY action_name, relative_path
"

Typical layout for a multi-action workflow:

text
TXT
target_data
├── action_name='node_0_extract_raw_qa'      relative_path='input.json'  record_count=42
├── action_name='node_1_flatten_questions'   relative_path='input.json'  record_count=128
├── action_name='node_2_classify_type'       relative_path='input.json'  record_count=128
└── action_name='final_workflow_output'      relative_path='input.json'  record_count=128

TypedDict Best Practices​

Use total=False for Optional Fields​

python
PY
class MyInput(TypedDict, total=False):
    required_field: str  # Still works, just not enforced
    optional_field: str

Use Plain dict for Mixed Types​

python
PY
# When dict values can be int, string, or other types
metadata: dict  # Not Dict[str, int]

Use Union for Polymorphic Fields​

python
PY
# When a field can be different types
answer_text: Union[str, List[str]]

Access Namespaced Fields​

python
PY
@udf_tool()
def safe_function(data: dict) -> dict:
    value = data["upstream_action"]["field"]
    items = data["upstream_action"]["items"]
    return {'result': value}

Reprompting​

What happens when an LLM returns invalid JSON? Rather than failing immediately, Agent Actions can automatically retry with feedback about what went wrong. This is reprompting.

Configuration​

Reprompt requires explicit configuration:

yaml
YAML
reprompt:
  max_attempts: 3          # Number of retry attempts
  on_exhausted: return_last   # return_last | raise

To disable: reprompt: false

Configuration Options​

OptionDescription
max_attemptsMaximum retry attempts (default: 2)
on_exhaustedBehavior when exhausted: return_last, raise

When to Use​

Consider what your agentic workflow needs:

  • Simple schemas — Low max_attempts (2-3)
  • Complex schemas — Higher max_attempts (4-5)
  • Critical outputs — Maximum attempts, on_exhausted: raise
warning

Reprompting adds latency and token cost. For high-volume agentic workflows, consider fixing schema issues at the source rather than relying on retries.

Log Analysis​

Log Location​

text
TXT
agent_io/logs/events.json

Search Patterns​

Events are JSON Lines. Structured fields live under the .data object (e.g. .data.action_name); message and level are top-level.

bash
BASH
# Find all schema validation errors
jq -c 'select(.message | test("SchemaValidationError"))' agent_io/logs/events.json

# Find events for a specific action
jq -c 'select(.data.action_name == "add_answer_text")' agent_io/logs/events.json

# Find events that mention a specific field
jq -c 'select(.message | test("answer_text"))' agent_io/logs/events.json

Execution History​

Check artefact/runs.json for execution metrics:

bash
BASH
# Count failed runs
grep '"status": "FAILED"' artefact/runs.json | wc -l

# Find recent failures
grep -A5 '"status": "FAILED"' artefact/runs.json | tail -20

Quick Reference​

These commands and patterns are your debugging toolkit.

Debug Commands​

bash
BASH
# Debug mode (full tracebacks)
AGENT_ACTIONS_LOG_LEVEL=DEBUG agac run -a workflow

Common Fixes Cheatsheet​

Error PatternQuick Fix
X is not of type 'string'str(value) or Union[int, str]
X is not of type 'array'[value] if isinstance(value, str) else value
X was unexpectedAdd field to TypedDict
X is a required propertyReturn field from tool or use reprompt
Dict[str, int] failsUse plain dict

Debug Checklist​

When debugging agentic workflow errors, work through this checklist:

  1. Read full error message, note error_path and failed_value
  2. Find record by source_guid in node outputs
  3. Compare data before/after failing action
  4. Check TypedDict matches actual data shape
  5. Enable prompt_debug: true for LLM actions
  6. Run with AGENT_ACTIONS_LOG_LEVEL=DEBUG for tracebacks
  7. Consider enabling reprompt with max_attempts: 3 for LLM schema failures

Most errors fall into one of two categories: schema mismatches (the data structure doesn't match expectations) or missing fields (required data wasn't provided). The checklist above helps you identify which category you're dealing with.

Dependency Patterns​

Understanding Fan-in vs Parallel​

When using multiple dependencies, understanding the pattern detection is crucial:

yaml
YAML
# Pattern 1: Parallel Branches (MERGE)
dependencies: [classify_1, classify_2, classify_3]
# Same base name "classify" → outputs are merged
# Execution count: N (from merged outputs)

# Pattern 2: Fan-in (PRIMARY + CONTEXT)
dependencies: [extract, enrich, validate]
# Different actions → first is primary, others via context
# Execution count: N (from extract only)

# Pattern 3: Aggregation (MERGE with reduce_key)
dependencies: [validator_A, validator_B, validator_C]
reduce_key: parent_id
# reduce_key set → all outputs merged and grouped by key

Missing Context Data in Fan-in​

Symptom: Action only sees data from first dependency, not others.

Cause: Fan-in pattern requires context sources to be in context_scope:

yaml
YAML
# WRONG - enrich and validate data not accessible
- name: generate_report
  dependencies: [extract, enrich, validate]

# CORRECT - all dependencies in context_scope
- name: generate_report
  dependencies: [extract, enrich, validate]
  context_scope:
    observe:
      - extract.*
      - enrich.*     # Loaded from record namespaces
      - validate.*   # Loaded from record namespaces

Unexpected Execution Count​

Symptom: Action executes more/fewer times than expected.

Debug:

  1. Check if dependencies are parallel branches (same base name) or different actions
  2. For fan-in: first dependency determines execution count
  3. For parallel: merged outputs determine execution count
  4. For aggregation: reduce_key groups determine execution count