Human-in-the-Loop Guide

Overview

Human-in-the-Loop (HITL) allows workflows to pause and wait for human interaction before continuing. Perfect for approval workflows, content moderation, complex decisions, and compliance requirements. HITL uses Durable Objects to maintain state across requests, allowing workflows to pause for minutes, hours, or days while waiting for human input.

Quick Example

name: content-approval-workflow
description: Generate content and wait for human approval

flow:
  # Generate content with AI
  - member: generate-blog-post
    type: Think
    config:
      provider: anthropic
      model: claude-3-5-sonnet-20241022
    input:
      topic: ${input.topic}

  # Pause for human review
  - member: request-approval
    type: HITL
    config:
      prompt: "Review the generated blog post"
      fields:
        - name: approved
          type: boolean
          label: "Approve this content?"
        - name: feedback
          type: text
          label: "Feedback (optional)"
      notifyUrl: "${env.APPROVAL_WEBHOOK_URL}"
      timeout: 86400000  # 24 hours

  # Continue only if approved
  - member: publish-post
    condition: ${request-approval.output.approved}
    input:
      content: ${generate-blog-post.output.text}

  # Revise if not approved
  - member: revise-post
    condition: ${!request-approval.output.approved}
    input:
      feedback: ${request-approval.output.feedback}

output:
  status: ${request-approval.output.approved ? 'published' : 'needs-revision'}
  feedback: ${request-approval.output.feedback}

See HITL Concept for detailed explanation.

HITL Member Configuration

Basic HITL Member

name: simple-approval
type: HITL
description: Basic approval request

config:
  prompt: "Please review and approve"
  fields:
    - name: approved
      type: boolean
      label: "Approve?"

With Multiple Fields

config:
  prompt: "Review this expense report"
  fields:
    - name: approved
      type: boolean
      label: "Approve expense?"
      required: true

    - name: amount
      type: number
      label: "Approved amount"
      required: true

    - name: notes
      type: text
      label: "Approval notes"
      required: false

With Timeout

config:
  prompt: "Urgent: Review security alert"
  timeout: 3600000  # 1 hour
  fields:
    - name: action
      type: select
      label: "Action to take"
      options:
        - "Block"
        - "Allow"
        - "Investigate"

With Notification

config:
  prompt: "Review user registration"
  notifyUrl: "${env.SLACK_WEBHOOK_URL}"
  notifyPayload:
    text: "New user registration requires approval"
    user: ${input.userName}
    reviewUrl: "${env.HITL_REVIEW_URL}/${execution.id}"
  fields:
    - name: approved
      type: boolean
      label: "Approve registration?"

Field Types

Boolean (Checkbox)

fields:
  - name: approved
    type: boolean
    label: "Approve this request?"
    required: true

Text (Single Line)

fields:
  - name: comment
    type: text
    label: "Add a comment"
    required: false
    placeholder: "Enter your feedback here"

Textarea (Multi-Line)

fields:
  - name: feedback
    type: textarea
    label: "Detailed feedback"
    required: false
    placeholder: "Provide detailed comments"
    rows: 5

Number

fields:
  - name: score
    type: number
    label: "Quality score (1-10)"
    required: true
    min: 1
    max: 10

fields:
  - name: category
    type: select
    label: "Select category"
    required: true
    options:
      - "Urgent"
      - "Normal"
      - "Low Priority"

Email

fields:
  - name: assignee
    type: email
    label: "Assign to"
    required: true

Date

fields:
  - name: deadline
    type: date
    label: "Set deadline"
    required: true

Common Patterns

Content Approval

name: content-moderation
description: AI generates, human approves

flow:
  - member: generate-content
    type: Think

  - member: human-review
    type: HITL
    config:
      prompt: "Review generated content for quality and appropriateness"
      fields:
        - name: approved
          type: boolean
          label: "Approve content?"
        - name: rating
          type: number
          label: "Quality rating (1-5)"
          min: 1
          max: 5
        - name: concerns
          type: textarea
          label: "Any concerns?"

  - member: publish-content
    condition: ${human-review.output.approved && human-review.output.rating >= 3}

output:
  published: ${publish-content.success}
  rating: ${human-review.output.rating}

Expense Approval

name: expense-approval-workflow
description: Multi-level approval for expenses

flow:
  - member: check-amount
    type: Function

  # Manager approval for $500+
  - member: manager-approval
    condition: ${check-amount.output.total >= 500}
    type: HITL
    config:
      prompt: "Manager approval required for $${check-amount.output.total}"
      notifyUrl: "${env.MANAGER_WEBHOOK}"
      timeout: 172800000  # 48 hours
      fields:
        - name: approved
          type: boolean
          label: "Approve expense?"
        - name: notes
          type: text
          label: "Comments"

  # Director approval for $5000+
  - member: director-approval
    condition: ${check-amount.output.total >= 5000 && manager-approval.output.approved}
    type: HITL
    config:
      prompt: "Director approval required for $${check-amount.output.total}"
      notifyUrl: "${env.DIRECTOR_WEBHOOK}"
      timeout: 172800000  # 48 hours
      fields:
        - name: approved
          type: boolean
          label: "Approve expense?"

output:
  status: ${director-approval.output.approved || (manager-approval.output.approved && check-amount.output.total < 5000) ? 'approved' : 'rejected'}

Data Validation

name: data-entry-validation
description: AI extracts, human validates

flow:
  - member: extract-data
    type: Think
    input:
      document: ${input.document}

  - member: validate-extraction
    type: HITL
    config:
      prompt: "Verify extracted data is correct"
      context:
        extracted: ${extract-data.output}
        original: ${input.document}
      fields:
        - name: correct
          type: boolean
          label: "Is extraction correct?"
        - name: corrections
          type: textarea
          label: "Corrections needed"

  - member: apply-corrections
    condition: ${!validate-extraction.output.correct}
    input:
      corrections: ${validate-extraction.output.corrections}

output:
  data: ${validate-extraction.output.correct ? extract-data.output : apply-corrections.output}
  validated: true

Quality Control

name: quality-control-workflow
description: Automated checks with human oversight

flow:
  - member: automated-tests
    type: Function

  # Human review only if tests fail
  - member: manual-inspection
    condition: ${!automated-tests.output.passed}
    type: HITL
    config:
      prompt: "Automated tests failed. Manual inspection required."
      context:
        failures: ${automated-tests.output.failures}
      fields:
        - name: override
          type: boolean
          label: "Override test failures?"
        - name: justification
          type: textarea
          label: "Justification"
          required: true

output:
  passed: ${automated-tests.output.passed || manual-inspection.output.override}

Security Review

name: security-review-workflow
description: Automated scan with manual approval

flow:
  - member: security-scan
    type: Function

  # Require review for high/critical findings
  - member: security-approval
    condition: ${security-scan.output.severity >= 7}
    type: HITL
    config:
      prompt: "Security vulnerabilities detected"
      timeout: 7200000  # 2 hours
      context:
        findings: ${security-scan.output.findings}
      fields:
        - name: action
          type: select
          label: "Action"
          options:
            - "Block deployment"
            - "Deploy with risk"
            - "Request remediation"
        - name: notes
          type: textarea
          label: "Security notes"

output:
  canDeploy: ${security-scan.output.severity < 7 || security-approval.output.action === 'Deploy with risk'}

Timeout Handling

Default Timeout

config:
  timeout: 86400000  # 24 hours
  fields:
    - name: approved
      type: boolean

Workflow fails if no response within 24 hours.

Custom Timeout Behavior

flow:
  - member: approval-request
    type: HITL
    config:
      timeout: 3600000  # 1 hour
    continue_on_error: true

  # Auto-reject if timeout
  - member: handle-timeout
    condition: ${!approval-request.success}

No Timeout

config:
  timeout: 0  # Wait indefinitely

Use with caution - workflow will wait forever until response.

Notification Patterns

Slack Notification

config:
  prompt: "Approve deployment"
  notifyUrl: "${env.SLACK_WEBHOOK_URL}"
  notifyPayload:
    text: "Deployment approval required"
    blocks:
      - type: "section"
        text:
          type: "mrkdwn"
          text: "*Deployment Approval Required*\n\nEnvironment: ${input.environment}\nVersion: ${input.version}"
      - type: "actions"
        elements:
          - type: "button"
            text:
              type: "plain_text"
              text: "Review"
            url: "${env.HITL_REVIEW_URL}/${execution.id}"

Email Notification

config:
  prompt: "Review invoice"
  notifyUrl: "${env.EMAIL_WEBHOOK_URL}"
  notifyPayload:
    to: ${input.approverEmail}
    subject: "Invoice Approval Required"
    body: |
      An invoice requires your approval.

      Amount: ${input.amount}
      Vendor: ${input.vendor}

      Review: ${env.HITL_REVIEW_URL}/${execution.id}

Custom Webhook

config:
  prompt: "Approve request"
  notifyUrl: "${env.CUSTOM_WEBHOOK_URL}"
  notifyPayload:
    event: "approval_required"
    executionId: ${execution.id}
    data: ${input}

Building HITL UI

Fetching Pending Requests

// Get pending HITL requests
const response = await fetch(`https://your-worker.workers.dev/hitl/${executionId}`, {
  headers: {
    'Authorization': `Bearer ${apiKey}`
  }
});

const hitl = await response.json();

console.log(hitl.prompt);     // Display prompt
console.log(hitl.fields);     // Render form fields
console.log(hitl.context);    // Show additional context

Submitting Response

// Submit HITL response
await fetch(`https://your-worker.workers.dev/hitl/${executionId}/respond`, {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${apiKey}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    approved: true,
    comments: "Looks good!"
  })
});

React Component Example

import { useState, useEffect } from 'react';

function HITLReview({ executionId, apiKey }) {
  const [hitl, setHitl] = useState(null);
  const [values, setValues] = useState({});

  useEffect(() => {
    fetch(`https://api.example.com/hitl/${executionId}`, {
      headers: { 'Authorization': `Bearer ${apiKey}` }
    })
      .then(res => res.json())
      .then(setHitl);
  }, [executionId]);

  const handleSubmit = async () => {
    await fetch(`https://api.example.com/hitl/${executionId}/respond`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${apiKey}`,
        'Content-Type': 'application/json'
      },
      body: JSON.stringify(values)
    });
  };

  if (!hitl) return <div>Loading...</div>;

  return (
    <div>
      <h2>{hitl.prompt}</h2>

      {hitl.context && (
        <pre>{JSON.stringify(hitl.context, null, 2)}</pre>
      )}

      {hitl.fields.map(field => (
        <div key={field.name}>
          <label>{field.label}</label>
          {field.type === 'boolean' && (
            <input
              type="checkbox"
              checked={values[field.name] || false}
              onChange={e => setValues({
                ...values,
                [field.name]: e.target.checked
              })}
            />
          )}
          {field.type === 'text' && (
            <input
              type="text"
              value={values[field.name] || ''}
              onChange={e => setValues({
                ...values,
                [field.name]: e.target.value
              })}
            />
          )}
        </div>
      ))}

      <button onClick={handleSubmit}>Submit</button>
    </div>
  );
}

Testing HITL Workflows

import { describe, it, expect } from 'vitest';
import { TestConductor } from '@ensemble-edge/conductor/testing';

describe('content-approval', () => {
  it('should wait for human approval', async () => {
    const conductor = await TestConductor.create();

    // Start workflow (will pause at HITL)
    const execution = await conductor.executeEnsemble('content-approval', {
      topic: 'AI Safety'
    });

    // Verify workflow is paused
    expect(execution.status).toBe('waiting_for_input');
    expect(execution.currentMember).toBe('request-approval');

    // Simulate human approval
    await conductor.respondToHITL(execution.id, {
      approved: true,
      feedback: 'Looks great!'
    });

    // Workflow continues and completes
    const result = await conductor.waitForCompletion(execution.id);

    expect(result).toBeSuccessful();
    expect(result.output.status).toBe('published');
  });

  it('should handle rejection', async () => {
    const conductor = await TestConductor.create();

    const execution = await conductor.executeEnsemble('content-approval', {
      topic: 'AI Safety'
    });

    // Simulate rejection
    await conductor.respondToHITL(execution.id, {
      approved: false,
      feedback: 'Needs more detail'
    });

    const result = await conductor.waitForCompletion(execution.id);

    expect(result).toBeSuccessful();
    expect(result.output.status).toBe('needs-revision');
    expect(result.output.feedback).toBe('Needs more detail');
  });

  it('should timeout after deadline', async () => {
    const conductor = await TestConductor.create();

    const execution = await conductor.executeEnsemble('urgent-approval', {
      data: { ... }
    });

    // Fast-forward time past timeout
    await conductor.advanceTime(3600000 + 1000);  // 1 hour + 1 second

    const result = await conductor.getExecution(execution.id);

    expect(result.status).toBe('failed');
    expect(result.error).toContain('timeout');
  });
});

Best Practices

Set appropriate timeouts - Balance urgency vs. availability
Provide clear prompts - Explain what needs review
Include context - Show relevant data for decision-making
Send notifications - Alert approvers via Slack/email
Handle timeouts gracefully - Default action or retry
Track approval history - Log who approved what when
Test thoroughly - Verify both approval and rejection paths
Secure endpoints - Require authentication for responses
Monitor pending requests - Dashboard of awaiting approvals
Set SLAs - Define expected response times

Security Considerations

Authentication

// Verify request signature
export default {
  async fetch(request, env) {
    const signature = request.headers.get('X-Signature');

    if (!verifySignature(signature, env.WEBHOOK_SECRET)) {
      return new Response('Unauthorized', { status: 401 });
    }

    // Process HITL response
  }
};

Authorization

// Check user has permission to approve
const user = await authenticateUser(request);

if (!user.permissions.includes('approve_content')) {
  return new Response('Forbidden', { status: 403 });
}

await respondToHITL(executionId, response);

Audit Trail

flow:
  - member: log-approval
    type: Data
    config:
      storage: d1
      operation: query
      query: |
        INSERT INTO approval_log (execution_id, approver, decision, timestamp)
        VALUES (?, ?, ?, CURRENT_TIMESTAMP)
    input:
      params:
        - ${execution.id}
        - ${request-approval.output.approver}
        - ${request-approval.output.approved}

HITL Concept

Deep dive into Human-in-the-Loop

HITL Member

Complete HITL member reference

Durable Objects

State persistence with Durable Objects

Testing Guide

Test HITL workflows

Conductor

Core Concepts

Guides

Member Types

Built-In Members

Examples

API Reference

Deployment

​Overview

​Quick Example

​HITL Member Configuration

​Basic HITL Member

​With Multiple Fields

​With Timeout

​With Notification

​Field Types

​Boolean (Checkbox)

​Text (Single Line)

​Textarea (Multi-Line)

​Number

​Select (Dropdown)

​Email

​Date

​Common Patterns

​Content Approval

​Expense Approval

​Data Validation

​Quality Control

​Security Review

​Timeout Handling

​Default Timeout

​Custom Timeout Behavior

​No Timeout

​Notification Patterns

​Slack Notification

​Email Notification

​Custom Webhook

​Building HITL UI

​Fetching Pending Requests

​Submitting Response

​React Component Example

​Testing HITL Workflows

​Best Practices

​Security Considerations

​Authentication

​Authorization

​Audit Trail

​Related Documentation