Guardrails

Runtime quality gates in workflows that enforce evaluation thresholds and route execution based on quality scores.

What are Guardrails?

Guardrails are workflow nodes that evaluate outputs and route execution along different paths based on quality:

This pattern enables:

Quality enforcement - Reject low-quality outputs automatically
Retry logic - Regenerate outputs that fail quality checks
Conditional routing - Different downstream processing based on quality
Graceful degradation - Fallback to human review after max retries

Configuration

Basic Guardrail


{
  "type": "guardrail",
  "id": "quality_check",
  "config": {
    "evaluators": [
      {
        "type": "faithfulness",
        "context": "$.nodes.rag_retrieval.output.chunks",
        "answer": "$.nodes.llm_generation.output.text",
        "threshold": 0.8
      }
    ],
    "paths": [
      {
        "path_id": "pass",
        "target_node_ids": ["format_output"]
      },
      {
        "path_id": "fail",
        "target_node_ids": ["human_review"]
      }
    ]
  }
}

Fields:

evaluators - Array of evaluation metrics to apply
paths - Routing rules for pass/fail outcomes
aggregation - How to combine multiple evaluator scores (optional)
pass_threshold - Minimum aggregate score to pass (optional)

Multiple Evaluators

Combine multiple quality dimensions:


{
  "type": "guardrail",
  "config": {
    "evaluators": [
      {
        "type": "faithfulness",
        "context": "$.nodes.rag_retrieval.output.chunks",
        "answer": "$.nodes.llm_generation.output.text",
        "threshold": 0.8,
        "weight": 0.5
      },
      {
        "type": "json_schema",
        "schema": {
          "type": "object",
          "properties": {
            "answer": {"type": "string"},
            "confidence": {"type": "number"}
          },
          "required": ["answer", "confidence"]
        },
        "output": "$.nodes.llm_generation.output.data",
        "weight": 0.3
      },
      {
        "type": "length_check",
        "text": "$.nodes.llm_generation.output.text",
        "min_length": 50,
        "max_length": 500,
        "unit": "characters",
        "weight": 0.2
      }
    ],
    "aggregation": "weighted_average",
    "pass_threshold": 0.75,
    "paths": [
      {"path_id": "pass", "target_node_ids": ["next_step"]},
      {"path_id": "fail", "target_node_ids": ["retry_node"]}
    ]
  }
}

Score calculation:

Faithfulness: 0.9 * 0.5 = 0.45
JSON Schema: 1.0 * 0.3 = 0.30
Length Check: 1.0 * 0.2 = 0.20
Aggregate: 0.95 (passes threshold 0.75)

Aggregation Methods

Weighted Average

Combine scores with configurable weights:


{
  "aggregation": "weighted_average",
  "evaluators": [
    {"type": "faithfulness", "threshold": 0.8, "weight": 0.6},
    {"type": "relevance", "threshold": 0.7, "weight": 0.4}
  ],
  "pass_threshold": 0.75
}

Formula: sum(score_i * weight_i) / sum(weight_i)

Use when: Different quality dimensions have different importance

Minimum

Pass only if all evaluators pass:


{
  "aggregation": "min",
  "evaluators": [
    {"type": "faithfulness", "threshold": 0.8},
    {"type": "json_schema", "schema": {...}}
  ]
}

Formula: min(score_1, score_2, ...)

Use when: All criteria are mandatory (hard requirements)

Maximum

Pass if any evaluator passes:


{
  "aggregation": "max",
  "evaluators": [
    {"type": "exact_match", "reference": "$.expected"},
    {"type": "llm_judge", "criteria": "Semantically equivalent?"}
  ]
}

Formula: max(score_1, score_2, ...)

Use when: Multiple acceptable quality definitions (fallback logic)

All

Binary pass if every evaluator passes threshold:


{
  "aggregation": "all",
  "evaluators": [
    {"type": "faithfulness", "threshold": 0.8},
    {"type": "relevance", "threshold": 0.7},
    {"type": "json_schema", "schema": {...}}
  ]
}

Use when: Strict quality requirements (no partial credit)

Any

Binary pass if at least one evaluator passes:


{
  "aggregation": "any",
  "evaluators": [
    {"type": "regex_match", "pattern": "^[A-Z]{2}[0-9]+$"},
    {"type": "llm_judge", "criteria": "Is format correct?"}
  ]
}

Use when: Flexible acceptance criteria (multiple valid outputs)

Routing Paths

Pass/Fail Routing

Basic binary routing:


{
  "paths": [
    {
      "path_id": "pass",
      "target_node_ids": ["format_output", "log_success"]
    },
    {
      "path_id": "fail",
      "target_node_ids": ["retry_generation", "log_failure"]
    }
  ]
}

Execution:

Guardrail evaluates output
If aggregate score >= pass_threshold, execute “pass” path nodes
If aggregate score < pass_threshold, execute “fail” path nodes

Multi-Path Routing

Route based on score ranges:


{
  "paths": [
    {
      "path_id": "high_quality",
      "condition": "score >= 0.9",
      "target_node_ids": ["publish_immediately"]
    },
    {
      "path_id": "medium_quality",
      "condition": "score >= 0.7 and score < 0.9",
      "target_node_ids": ["queue_for_review"]
    },
    {
      "path_id": "low_quality",
      "condition": "score < 0.7",
      "target_node_ids": ["regenerate"]
    }
  ]
}

Use cases:

Progressive quality thresholds (auto-publish high quality, review medium)
Different processing based on confidence
Tiered retry strategies

Evaluator-Specific Routing

Route based on which evaluator failed:


{
  "evaluators": [
    {"type": "faithfulness", "id": "faith", "threshold": 0.8},
    {"type": "json_schema", "id": "schema", "schema": {...}}
  ],
  "paths": [
    {
      "path_id": "all_pass",
      "condition": "faith.pass and schema.pass",
      "target_node_ids": ["success"]
    },
    {
      "path_id": "faith_fail",
      "condition": "not faith.pass",
      "target_node_ids": ["retry_with_more_context"]
    },
    {
      "path_id": "schema_fail",
      "condition": "not schema.pass",
      "target_node_ids": ["retry_with_schema_prompt"]
    }
  ]
}

Enables targeted retry strategies based on failure mode.

Retry Patterns

Simple Retry

Retry generation with same prompt:


{
  "type": "guardrail",
  "config": {
    "evaluators": [...],
    "max_retries": 3,
    "paths": [
      {"path_id": "pass", "target_node_ids": ["success"]},
      {"path_id": "fail", "target_node_ids": ["llm_generation"]}
    ]
  }
}

Behavior:

LLM generates output
Guardrail evaluates
If fail and retries < max_retries, loop back to LLM
If fail and retries >= max_retries, execute fallback path

Use when: LLM non-determinism may produce better output on retry

Retry with Better Prompt

Enhance prompt based on failure:


{
  "paths": [
    {"path_id": "pass", "target_node_ids": ["success"]},
    {
      "path_id": "fail",
      "target_node_ids": [
        "enhance_prompt",  // Add "Ensure output is faithful to context"
        "llm_generation_retry"
      ]
    }
  ]
}

Prompt enhancement node:


# Add guidance based on failed evaluator
if failure_reason == "faithfulness":
    enhanced_prompt = original_prompt + "\n\nIMPORTANT: Only use information from the provided context. Do not add external knowledge."
elif failure_reason == "json_schema":
    enhanced_prompt = original_prompt + f"\n\nOutput must match this schema: {schema}"

Use when: Specific failure modes have known prompt fixes

Retry with Different Model

Fallback to more capable (or different) model:


{
  "paths": [
    {"path_id": "pass", "target_node_ids": ["success"]},
    {
      "path_id": "fail",
      "target_node_ids": ["llm_gpt4_fallback"]
    }
  ]
}

Node configuration:

Initial LLM node: model: gpt-3.5-turbo
Fallback LLM node: model: gpt-4

Use when: Quality issues stem from model capability limits

Exponential Backoff

Increase retry delay to avoid rate limits:


{
  "retry_strategy": {
    "type": "exponential_backoff",
    "initial_delay_ms": 1000,
    "max_delay_ms": 10000,
    "multiplier": 2,
    "max_retries": 5
  }
}

Delays:

Retry 1: 1000ms
Retry 2: 2000ms
Retry 3: 4000ms
Retry 4: 8000ms
Retry 5: 10000ms (capped at max_delay_ms)

Use when: API rate limiting may cause transient failures

Human-in-the-Loop Integration

Route low-quality outputs to human review:


{
  "type": "guardrail",
  "config": {
    "evaluators": [...],
    "max_retries": 2,
    "paths": [
      {"path_id": "pass", "target_node_ids": ["publish"]},
      {
        "path_id": "fail_retry",
        "condition": "retries < max_retries",
        "target_node_ids": ["retry_generation"]
      },
      {
        "path_id": "fail_final",
        "condition": "retries >= max_retries",
        "target_node_ids": ["human_review_node"]
      }
    ]
  }
}

Human Review Node:


{
  "type": "hitl",
  "config": {
    "review_type": "correction",
    "instructions": "The AI output failed quality checks. Please correct or regenerate.",
    "fields": [
      {
        "name": "corrected_output",
        "type": "text",
        "default": "$.nodes.llm_generation.output.text"
      }
    ]
  }
}

See Human-in-the-Loop for detailed HITL configuration.

Creating Guardrail Nodes

Add Guardrail Node

In the workflow canvas:

Open workflow editor
Drag “Guardrail” node from palette
Position between source node (LLM) and downstream nodes

Configure Evaluators

In the node properties panel:

Click “Add Evaluator”
Select evaluator type (faithfulness, relevance, etc.)
Configure JSONPath references to workflow data
Set threshold and weight (if applicable)
Repeat for additional evaluators

Define Routing Paths

In the “Paths” section:

Add “Pass” path
- Click downstream nodes that should execute on pass
Add “Fail” path
- Click downstream nodes that should execute on fail
Optionally add conditional paths for multi-tier routing

Set Aggregation

If using multiple evaluators:

Select aggregation method (weighted_average, min, max, all, any)
Set pass_threshold (0.0-1.0)
Adjust evaluator weights if using weighted_average

Test Guardrail

Save workflow
Click “Test Run” with sample input
View guardrail evaluation results in execution logs
Check which path was taken
Iterate configuration based on results

Monitoring Guardrails

Execution Metrics

Track guardrail performance:

Metric	Definition	Target
Pass Rate	Percentage of evaluations passing	> 80%
Fail Rate	Percentage failing (requiring retry/review)	< 20%
Retry Rate	Percentage requiring retries	< 10%
HITL Rate	Percentage escalated to human review	< 5%
Evaluation Latency	Time spent in guardrail evaluation	< 500ms

Failure Analysis

Group failures by:

Evaluator - Which evaluator failed most often?
Score Range - Distribution of failing scores (just below threshold vs far below)
Input Type - Certain inputs fail more than others?
Timestamp - Quality degrading over time?

Example dashboard:


Guardrail: RAG Faithfulness Check (Last 7 Days)

Pass Rate:     ████████████░░░░░░░░ 72%  (↓ 5% from last week)
Retry Rate:    ████░░░░░░░░░░░░░░░░ 18%
HITL Rate:     ██░░░░░░░░░░░░░░░░░░ 10%

Failed Evaluators:
- Faithfulness:  65%  (score: 0.65 avg)
- JSON Schema:   25%
- Length Check:  10%

Common Failure Patterns:
1. Long context (> 5 chunks) → hallucination (40% of failures)
2. Multi-part questions → incomplete answers (30%)
3. Technical jargon → low relevance scores (15%)

This analysis informs:

Threshold adjustment - Lower threshold if failure rate too high
Prompt improvements - Address common failure patterns
Model changes - Upgrade if capability insufficient

Alerts

Set up alerts for:

Pass rate drops below 70%:

Indicates quality degradation
Check if upstream data changed or model updated

HITL rate exceeds 10%:

Too many manual reviews needed
Review guardrail configuration or improve prompts

Evaluation latency > 1000ms:

Slow evaluations bottleneck workflow
Cache results or use faster evaluators

Best Practices

Evaluator Selection

Start with fast evaluators (schema, regex) before slow ones (LLM-based)
Use domain-specific evaluators when available
Combine reference-free metrics (faithfulness, relevance) with structural checks (schema)
Avoid redundant evaluators (high correlation = wasted computation)

Threshold Tuning

Collect baseline data - Run guardrails without enforcement, log scores
Analyze distribution - Plot score histogram
Set threshold at target percentile (e.g., p10 to reject bottom 10%)
Monitor false positives - Review outputs that fail but shouldn’t
Iterate - Adjust threshold based on production metrics

Example: Faithfulness threshold tuning


Score Distribution (1000 samples):
0.9-1.0: ████████████████████  45%
0.8-0.9: ████████████████      32%
0.7-0.8: ██████                12%
0.6-0.7: ████                   8%
0.5-0.6: ██                     3%

Recommended threshold: 0.75 (rejects bottom 11%, passes 89%)

Retry Strategy

Limit retries - Max 2-3 retries (diminishing returns)
Fail fast - Don’t retry if score is extremely low (< 0.3)
Vary approach - Don’t retry same prompt/model repeatedly
Fallback to HITL - Escalate after max retries

Performance

Evaluate selectively - Not every workflow run needs guardrails
Sample in production - Evaluate 10-20% of traffic, not 100%
Cache results - Identical inputs = identical scores
Parallel evaluation - Run independent evaluators concurrently

Documentation

Document guardrail purpose in workflow:


{
  "type": "guardrail",
  "description": "Ensures LLM output is faithful to retrieved context. Retry with better prompt if failed. Escalate to human review after 3 retries.",
  "config": {...}
}

This helps team understand why guardrails fail and how to improve.

Common Patterns

RAG Faithfulness Gate


{
  "evaluators": [
    {
      "type": "faithfulness",
      "context": "$.nodes.rag_retrieval.output.chunks",
      "answer": "$.nodes.llm_generation.output.text",
      "threshold": 0.8
    }
  ],
  "max_retries": 2,
  "paths": [
    {"path_id": "pass", "target_node_ids": ["return_answer"]},
    {
      "path_id": "fail",
      "target_node_ids": ["retry_with_instruction"],
      "instruction": "Only use information from provided context. Do not add external knowledge."
    }
  ]
}

Structured Extraction Validation


{
  "evaluators": [
    {
      "type": "json_schema",
      "schema": {
        "type": "object",
        "properties": {
          "invoice_number": {"type": "string"},
          "date": {"type": "string", "format": "date"},
          "total": {"type": "number"}
        },
        "required": ["invoice_number", "date", "total"]
      },
      "output": "$.nodes.extraction.output.data"
    },
    {
      "type": "regex_match",
      "pattern": "^INV-[0-9]{6}$",
      "text": "$.nodes.extraction.output.data.invoice_number"
    }
  ],
  "aggregation": "all",
  "paths": [
    {"path_id": "pass", "target_node_ids": ["save_to_database"]},
    {"path_id": "fail", "target_node_ids": ["manual_extraction"]}
  ]
}

Multi-Tier Quality Routing


{
  "evaluators": [
    {"type": "faithfulness", "threshold": 0.8, "weight": 0.5},
    {"type": "relevance", "threshold": 0.7, "weight": 0.5}
  ],
  "aggregation": "weighted_average",
  "paths": [
    {
      "path_id": "high_quality",
      "condition": "score >= 0.9",
      "target_node_ids": ["auto_publish"]
    },
    {
      "path_id": "medium_quality",
      "condition": "score >= 0.7",
      "target_node_ids": ["queue_for_review"]
    },
    {
      "path_id": "low_quality",
      "condition": "score < 0.7",
      "target_node_ids": ["regenerate"]
    }
  ]
}

Troubleshooting

Guardrails Always Fail

Possible causes:

Threshold too high (reduce to 0.6-0.7 and tune up)
Evaluator misconfigured (wrong JSONPath references)
Output format doesn’t match evaluator expectations
Insufficient context for evaluation

Debugging:

Check evaluation logs for actual scores
Review failed outputs manually - should they pass?
Lower threshold temporarily to collect score distribution
Test evaluator in isolation with known good/bad examples

Guardrails Always Pass

Possible causes:

Threshold too low (increase to 0.8-0.9)
Evaluator not sensitive enough to quality differences
Bug in evaluator logic (always returns high score)

Debugging:

Test with known bad outputs
Check if evaluator is even running (view logs)
Try different evaluator types
Manually review “passing” outputs for quality

High Retry Rate

Solutions:

Improve upstream prompts (reduce quality variance)
Relax thresholds (accept more outputs)
Switch to better model (fewer failures)
Add HITL earlier (don’t retry forever)

Slow Evaluation

Optimizations:

Use faster evaluators (regex before LLM judge)
Cache evaluation results
Run evaluators in parallel
Sample evaluation (not every run)

Next Steps

Configure evaluation metrics for your use case
Build workflows with quality gates
Monitor guardrail performance in the dashboard
Integrate human review for fallback handling