Built-In Members Overview

Overview

Conductor includes powerful built-in members for common operations like web scraping, validation, RAG, human-in-the-loop approvals, API calls, and database queries. These members are production-ready and optimized for Cloudflare Workers.

Available Members

Scrape

Web scraping with three-tier fallback strategy

Validate

Multi-evaluator quality validation system

RAG

Retrieval-augmented generation with Vectorize

HITL

Human-in-the-loop approval workflows

Fetch

HTTP requests with retry and timeout

Queries

SQL queries with Hyperdrive pooling

Transform

Data transformation utilities

Schedule

Cron-based workflow scheduling

Registry System

Built-in members use a lazy-loading registry for optimal performance:

import { getBuiltInMember } from '@ensemble-edge/conductor';

// Lazy loads only when first used
const scrapeMember = await getBuiltInMember('scrape');

Registry API

interface BuiltInRegistry {
  get(name: string): Promise<BaseMember>;
  has(name: string): boolean;
  list(): string[];
  getMetadata(name: string): BuiltInMemberMetadata;
}

Usage in Ensembles

Basic Usage

flow:
  - member: scrape-page
    type: Function
    config:
      builtin: scrape
      url: https://example.com
      format: markdown

With Custom Configuration

flow:
  - member: validate-content
    type: Function
    config:
      builtin: validate
      strategy: judge
      model: '@cf/meta/llama-3.1-8b-instruct'
      criteria: "Is this content appropriate for all audiences?"

Member Metadata

Each built-in member provides metadata:

interface BuiltInMemberMetadata {
  name: string;
  type: 'Think' | 'Function' | 'Data' | 'API';
  description: string;
  version: string;
  configSchema: JSONSchema;
  examples: Example[];
}

Access metadata:

import { getBuiltInMetadata } from '@ensemble-edge/conductor';

const metadata = getBuiltInMetadata('scrape');
console.log(metadata.description);
console.log(metadata.configSchema);

Configuration Schemas

All built-in members have JSON schemas for validation:

import { validateBuiltInConfig } from '@ensemble-edge/conductor';

const config = {
  url: 'https://example.com',
  format: 'markdown'
};

const result = validateBuiltInConfig('scrape', config);

if (!result.valid) {
  console.error('Invalid config:', result.errors);
}

Performance

Lazy Loading

Members are loaded only when first used:

// No members loaded yet
const conductor = new Conductor({ env });

// Scrape member loaded on first use
await conductor.execute('scrape-workflow', {});

// Subsequent uses are instant (already loaded)
await conductor.execute('scrape-workflow-2', {});

Tree Shaking

Unused members are eliminated from bundle:

// Only includes 'scrape' in bundle
import { scrape } from '@ensemble-edge/conductor/built-in';

// Entire registry (all members)
import { registry } from '@ensemble-edge/conductor/built-in';

Caching

Built-in members support caching:

- member: cached-scrape
  type: Function
  config:
    builtin: scrape
    url: ${input.url}
  cache:
    enabled: true
    ttl: 3600000  # 1 hour
    key: ${input.url}

Common Patterns

Scrape → Validate → Store

flow:
  - member: scrape-content
    type: Function
    config:
      builtin: scrape
      url: ${input.url}

  - member: validate-quality
    type: Function
    config:
      builtin: validate
      strategy: judge
      content: ${scrape-content.output.content}

  - member: store-if-valid
    condition: ${validate-quality.output.passed}
    type: Data
    config:
      type: d1
      operation: execute
      query: INSERT INTO content (url, text) VALUES (?, ?)
      params: [${input.url}, ${scrape-content.output.content}]

RAG Pipeline

flow:
  - member: index-documents
    type: Function
    config:
      builtin: rag
      operation: index
      documents: ${input.documents}
      namespace: kb

  - member: search-relevant
    type: Function
    config:
      builtin: rag
      operation: search
      query: ${input.question}
      namespace: kb
      topK: 5

  - member: generate-answer
    type: Think
    config:
      model: '@cf/meta/llama-3.1-8b-instruct'
      prompt: |
        Context: ${search-relevant.output.results}

        Question: ${input.question}

        Answer:

HITL Approval

flow:
  - member: request-approval
    type: Function
    config:
      builtin: hitl
      operation: suspend
      prompt: "Review this content"
      fields:
        - name: approved
          type: boolean
          label: Approve?

  - member: process-if-approved
    condition: ${request-approval.output.approved}
    type: Function
    config:
      handler: () => ({ status: 'processed' })

Error Handling

Built-in members use consistent error codes:

flow:
  - member: safe-scrape
    type: Function
    config:
      builtin: scrape
      url: ${input.url}
    retry:
      maxAttempts: 3
      backoff: exponential

Common error codes:

SCRAPE_FAILED - Web scraping error
VALIDATION_FAILED - Content validation error
RAG_INDEX_ERROR - Vectorize indexing error
HITL_TIMEOUT - Approval timeout
FETCH_ERROR - HTTP request error
QUERY_ERROR - Database query error

Testing

Built-in members can be mocked for testing:

import { mockBuiltInMember } from '@ensemble-edge/conductor/testing';

// Mock scrape member
mockBuiltInMember('scrape', {
  execute: async (input) => ({
    content: 'Mocked content',
    strategy: 'fast'
  })
});

// Run test
const result = await conductor.execute('test-ensemble', {});
expect(result.output).toBeDefined();

Versioning

Built-in members are versioned:

- member: specific-version
  type: Function
  config:
    builtin: scrape@1.2.0  # Pin to version
    url: ${input.url}

Check available versions:

import { getBuiltInVersions } from '@ensemble-edge/conductor';

const versions = getBuiltInVersions('scrape');
console.log(versions); // ['1.0.0', '1.1.0', '1.2.0']

Custom Built-In Members

import { registerBuiltInMember } from '@ensemble-edge/conductor';

registerBuiltInMember({
  name: 'my-custom-member',
  type: 'Function',
  version: '1.0.0',
  description: 'My custom reusable member',
  factory: (config) => new MyCustomMember(config),
  configSchema: {
    type: 'object',
    properties: {
      option1: { type: 'string' },
      option2: { type: 'number' }
    }
  }
});

Best Practices

Use built-in members - Battle-tested, optimized code
Cache when possible - Reduce redundant operations
Handle errors - Built-in members can fail
Pin versions - For production stability
Check schemas - Validate configuration
Test with mocks - Fast, predictable tests
Monitor usage - Track performance
Contribute back - Share improvements
Read documentation - Understand capabilities
Stay updated - New members added regularly

Creating Members

Build custom members

Member Types

Base member classes

Testing Guide

Test workflows

Contributing

Contribute members

Conductor API

Core Classes

Member Types API

Built-In Members API

SDK API

Testing API

Durable Objects API

AI Providers API

HTTP API

​Overview

​Available Members

Scrape

Validate

RAG

HITL

Fetch

Queries

Transform

Schedule

​Registry System

​Registry API

​Usage in Ensembles

​Basic Usage

​With Custom Configuration

​Member Metadata

​Configuration Schemas

​Performance

​Lazy Loading

​Tree Shaking

​Caching

​Common Patterns

​Scrape → Validate → Store

​RAG Pipeline

​HITL Approval

​Error Handling

​Testing

​Versioning

​Custom Built-In Members

​Best Practices

​Related Documentation