TestConductor - Ensemble

Overview

TestConductor is a powerful testing utility that provides a complete test environment for Conductor workflows. It includes mocking capabilities, execution tracking, and comprehensive assertions.

import { TestConductor } from '@ensemble-edge/conductor/testing';

const conductor = await TestConductor.create({
  projectPath: './conductor'
});

const result = await conductor.executeEnsemble('user-onboarding', {
  email: 'test@example.com'
});

expect(result.success).toBe(true);
expect(result.output).toMatchObject({ userId: expect.any(String) });

Installation

npm install --save-dev @ensemble-edge/conductor

Constructor

create()

Create a new TestConductor instance:

static async create(options?: TestConductorOptions): Promise<TestConductor>

options

TestConductorOptions

Test conductor configuration

options.projectPath

string

Path to Conductor project directory

options.env

object

Mock environment variables and bindings

options.mocks

object

Mock configuration for AI, database, HTTP, Vectorize

Example:

const conductor = await TestConductor.create({
  projectPath: './conductor',
  env: {
    OPENAI_API_KEY: 'test-key',
    DATABASE_URL: 'mock-db'
  },
  mocks: {
    ai: {
      'generate-greeting': { message: 'Hello, Test!' }
    },
    database: {
      users: [
        { id: 1, email: 'test@example.com' }
      ]
    }
  }
});

Execution Methods

executeEnsemble()

Execute an ensemble in test mode:

async executeEnsemble(
  name: string,
  input: Record<string, unknown>
): Promise<TestExecutionResult>

name

string

required

Name of the ensemble to execute

input

object

required

Input data for the ensemble

Returns: Promise<TestExecutionResult>

interface TestExecutionResult {
  success: boolean;
  output?: unknown;
  error?: Error;
  executionTime: number;
  stepsExecuted: ExecutedStep[];
  stateHistory: StateSnapshot[];
  aiCalls: AICall[];
  databaseQueries: DatabaseQuery[];
  httpRequests: HTTPRequest[];
}

Example:

const result = await conductor.executeEnsemble('order-processing', {
  orderId: 'ORD-123',
  customerId: 'CUST-456'
});

console.log(result.success); // true
console.log(result.output); // { status: 'processed', total: 99.99 }
console.log(result.executionTime); // 234 (ms)
console.log(result.stepsExecuted.length); // 5

executeMember()

Execute a single member directly:

async executeMember(
  name: string,
  input: unknown
): Promise<TestMemberResult>

name

string

required

Name of the member to execute

input

unknown

required

Input data for the member

Example:

const result = await conductor.executeMember('validate-email', {
  email: 'test@example.com'
});

expect(result.output.valid).toBe(true);

Mocking Methods

mockAI()

Mock AI provider responses:

mockAI(memberName: string, response: unknown | Error): void

memberName

string

required

Name of the AI member to mock

response

unknown | Error

required

Mock response or error to return

Example:

// Mock successful AI response
conductor.mockAI('generate-summary', {
  summary: 'This is a test summary'
});

// Mock AI error
conductor.mockAI('generate-summary', new Error('AI service unavailable'));

mockDatabase()

Mock database data:

mockDatabase(table: string, data: unknown[]): void

table

string

required

Database table name

data

array

required

Array of records to populate the table

Example:

conductor.mockDatabase('users', [
  { id: 1, email: 'alice@example.com', name: 'Alice' },
  { id: 2, email: 'bob@example.com', name: 'Bob' }
]);

conductor.mockDatabase('orders', [
  { id: 'ORD-1', userId: 1, total: 99.99, status: 'pending' }
]);

mockAPI()

Mock external API responses:

mockAPI(url: string, response: unknown): void

url

string

required

API URL to mock

response

unknown

required

Mock response data

Example:

conductor.mockAPI('https://api.stripe.com/v1/charges', {
  id: 'ch_123',
  status: 'succeeded',
  amount: 9999
});

conductor.mockAPI('https://api.weather.com/forecast', {
  temperature: 72,
  condition: 'sunny'
});

Catalog Methods

addEnsemble()

Add an ensemble to the test catalog:

addEnsemble(name: string, config: EnsembleConfig): void

Example:

conductor.addEnsemble('test-ensemble', {
  name: 'test-ensemble',
  description: 'Test ensemble',
  flow: [
    {
      member: 'greet',
      type: 'Think',
      config: {
        provider: 'cloudflare',
        model: '@cf/meta/llama-3.1-8b-instruct'
      },
      input: {
        prompt: 'Say hello to ${input.name}'
      }
    }
  ],
  output: {
    greeting: '${greet.output.text}'
  }
});

addMember()

Add a member to the test catalog:

addMember(name: string, config: MemberConfig): void

Example:

conductor.addMember('validate-email', {
  name: 'validate-email',
  type: 'Function',
  config: {
    handler: (input) => ({
      valid: /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(input.email)
    })
  }
});

getEnsemble()

Get an ensemble from the catalog:

getEnsemble(name: string): EnsembleConfig | undefined

getMember()

Get a member from the catalog:

getMember(name: string): MemberConfig | undefined

History Methods

getExecutionHistory()

Get all execution records:

getExecutionHistory(): ExecutionRecord[]

Example:

const history = conductor.getExecutionHistory();

console.log(`Total executions: ${history.length}`);
console.log(`Success rate: ${history.filter(e => e.success).length / history.length * 100}%`);

getAICalls()

Get all AI calls from execution history:

getAICalls(): AICall[]

Example:

const aiCalls = conductor.getAICalls();

const totalTokens = aiCalls.reduce(
  (sum, call) => sum + (call.usage?.totalTokens || 0),
  0
);
console.log(`Total tokens used: ${totalTokens}`);

getDatabaseQueries()

Get all database queries from execution history:

getDatabaseQueries(): DatabaseQuery[]

Example:

const queries = conductor.getDatabaseQueries();

console.log(`Total queries: ${queries.length}`);
console.log(`Slowest query: ${Math.max(...queries.map(q => q.duration))}ms`);

Utility Methods

snapshot()

Create a snapshot of the current project state:

async snapshot(): Promise<ProjectSnapshot>

Example:

const snapshot = await conductor.snapshot();

console.log(`Ensembles: ${snapshot.catalog.ensembles.size}`);
console.log(`Members: ${snapshot.catalog.members.size}`);

cleanup()

Cleanup test resources:

async cleanup(): Promise<void>

Example:

afterEach(async () => {
  await conductor.cleanup();
});

Testing Patterns

Basic Test

import { TestConductor } from '@ensemble-edge/conductor/testing';
import { describe, it, expect, beforeAll, afterEach } from 'vitest';

describe('User Onboarding', () => {
  let conductor: TestConductor;

  beforeAll(async () => {
    conductor = await TestConductor.create({
      projectPath: './conductor'
    });
  });

  afterEach(async () => {
    await conductor.cleanup();
  });

  it('creates new user', async () => {
    const result = await conductor.executeEnsemble('user-onboarding', {
      email: 'newuser@example.com',
      name: 'New User'
    });

    expect(result.success).toBe(true);
    expect(result.output).toMatchObject({
      userId: expect.any(String),
      email: 'newuser@example.com'
    });
  });
});

With Mocks

it('handles AI failures gracefully', async () => {
  conductor.mockAI('generate-welcome', new Error('AI service down'));

  const result = await conductor.executeEnsemble('user-onboarding', {
    email: 'test@example.com'
  });

  expect(result.success).toBe(false);
  expect(result.error?.message).toContain('AI service down');
});

Testing Performance

it('completes within 500ms', async () => {
  const result = await conductor.executeEnsemble('quick-check', {
    data: 'test'
  });

  expect(result.success).toBe(true);
  expect(result.executionTime).toBeLessThan(500);
});

Testing AI Costs

it('uses less than 1000 tokens', async () => {
  await conductor.executeEnsemble('content-generation', {
    topic: 'testing'
  });

  const aiCalls = conductor.getAICalls();
  const totalTokens = aiCalls.reduce(
    (sum, call) => sum + (call.usage?.totalTokens || 0),
    0
  );

  expect(totalTokens).toBeLessThan(1000);
});

Testing Data Operations

it('creates database records', async () => {
  conductor.mockDatabase('users', []);

  await conductor.executeEnsemble('create-user', {
    email: 'test@example.com'
  });

  const queries = conductor.getDatabaseQueries();
  const insertQueries = queries.filter(q => q.operation === 'INSERT');

  expect(insertQueries).toHaveLength(1);
  expect(insertQueries[0].table).toBe('users');
});

Testing External APIs

it('calls payment API', async () => {
  conductor.mockAPI('https://api.stripe.com/v1/charges', {
    id: 'ch_test',
    status: 'succeeded'
  });

  const result = await conductor.executeEnsemble('process-payment', {
    amount: 9999,
    currency: 'usd'
  });

  expect(result.success).toBe(true);
  expect(result.httpRequests).toHaveLength(1);
  expect(result.httpRequests[0].url).toContain('stripe.com');
});

Best Practices

Use projectPath - Load ensembles and members automatically
Mock external services - Avoid real API calls in tests
Clean up after tests - Use cleanup() in afterEach
Test error paths - Mock failures to test error handling
Check execution time - Ensure workflows are performant
Monitor AI usage - Track tokens and costs
Verify state - Check state history for correctness
Test with real data - Use realistic test data
Isolate tests - Each test should be independent
Use snapshots - Compare outputs with snapshots

Custom Matchers

Vitest matchers for assertions

Mocks

Mock utilities and helpers

Testing Guide

Testing workflows guide

CI/CD

Continuous integration setup

Conductor API

Core Classes

Member Types API

Built-In Members API

SDK API

Testing API

Durable Objects API

AI Providers API

HTTP API

​Overview

​Installation

​Constructor

​create()

​Execution Methods

​executeEnsemble()

​executeMember()

​Mocking Methods

​mockAI()

​mockDatabase()

​mockAPI()

​Catalog Methods

​addEnsemble()

​addMember()

​getEnsemble()

​getMember()

​History Methods

​getExecutionHistory()

​getAICalls()

​getDatabaseQueries()

​Utility Methods

​snapshot()

​cleanup()

​Testing Patterns

​Basic Test

​With Mocks

​Testing Performance

​Testing AI Costs

​Testing Data Operations

​Testing External APIs

​Best Practices

​Related Documentation

Custom Matchers

Mocks

Testing Guide

CI/CD

Overview

Installation

Constructor

create()

Execution Methods

executeEnsemble()

executeMember()

Mocking Methods

mockAI()

mockDatabase()

mockAPI()

Catalog Methods

addEnsemble()

addMember()

getEnsemble()

getMember()

History Methods

getExecutionHistory()

getAICalls()

getDatabaseQueries()

Utility Methods

snapshot()

cleanup()

Testing Patterns

Basic Test

With Mocks

Testing Performance

Testing AI Costs

Testing Data Operations

Testing External APIs

Best Practices

Related Documentation