CI/CD Guide - Ensemble

Overview

Automate your Conductor workflow deployment with CI/CD pipelines. This guide covers GitHub Actions, GitLab CI, testing automation, and deployment strategies.

GitHub Actions

Complete Pipeline

# .github/workflows/deploy.yml
name: Test and Deploy

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-node@v3
        with:
          node-version: '18'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run linter
        run: npm run lint

      - name: Run tests
        run: npm test

      - name: Build
        run: npm run build

  deploy-staging:
    needs: test
    if: github.ref == 'refs/heads/develop'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-node@v3
        with:
          node-version: '18'

      - name: Install dependencies
        run: npm ci

      - name: Deploy to staging
        env:
          CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
          CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
        run: |
          npx wrangler deploy --env staging

  deploy-production:
    needs: test
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-node@v3
        with:
          node-version: '18'

      - name: Install dependencies
        run: npm ci

      - name: Deploy to production
        env:
          CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
          CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
        run: |
          npx wrangler deploy --env production

      - name: Create release
        uses: actions/create-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          tag_name: v${{ github.run_number }}
          release_name: Release v${{ github.run_number }}

Test-Only Workflow

# .github/workflows/test.yml
name: Test

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-node@v3
        with:
          node-version: '18'

      - run: npm ci
      - run: npm run lint
      - run: npm run type-check
      - run: npm test
      - run: npm run build

      - name: Upload coverage
        uses: codecov/codecov-action@v3
        with:
          file: ./coverage/coverage-final.json

Manual Deployment

# .github/workflows/manual-deploy.yml
name: Manual Deploy

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to deploy to'
        required: true
        type: choice
        options:
          - staging
          - production

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-node@v3
        with:
          node-version: '18'

      - run: npm ci

      - name: Deploy
        env:
          CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
        run: |
          npx wrangler deploy --env ${{ github.event.inputs.environment }}

GitLab CI

# .gitlab-ci.yml
stages:
  - test
  - deploy

variables:
  NODE_VERSION: "18"

.node_template:
  image: node:$NODE_VERSION
  cache:
    paths:
      - node_modules/
  before_script:
    - npm ci

test:
  extends: .node_template
  stage: test
  script:
    - npm run lint
    - npm run type-check
    - npm test
    - npm run build

deploy:staging:
  extends: .node_template
  stage: deploy
  only:
    - develop
  script:
    - npx wrangler deploy --env staging
  environment:
    name: staging

deploy:production:
  extends: .node_template
  stage: deploy
  only:
    - main
  script:
    - npx wrangler deploy --env production
  environment:
    name: production
  when: manual

Secrets Management

GitHub Secrets

# Add secrets via GitHub CLI
gh secret set CLOUDFLARE_API_TOKEN
gh secret set CLOUDFLARE_ACCOUNT_ID
gh secret set OPENAI_API_KEY
gh secret set ANTHROPIC_API_KEY

Wrangler Secrets

# Add secrets to Cloudflare Workers
echo "sk-..." | npx wrangler secret put OPENAI_API_KEY --env production
echo "sk-ant-..." | npx wrangler secret put ANTHROPIC_API_KEY --env production

Environment Variables

# wrangler.toml
[env.staging]
name = "my-worker-staging"
vars = { ENVIRONMENT = "staging", API_URL = "https://staging-api.example.com" }

[env.production]
name = "my-worker"
vars = { ENVIRONMENT = "production", API_URL = "https://api.example.com" }

Testing in CI

Unit Tests

- name: Run unit tests
  run: npm run test:unit

Integration Tests

- name: Run integration tests
  run: npm run test:integration
  env:
    TEST_API_KEY: ${{ secrets.TEST_API_KEY }}

E2E Tests

- name: Deploy to test environment
  run: npx wrangler deploy --env test

- name: Run E2E tests
  run: npm run test:e2e
  env:
    TEST_URL: https://test.example.workers.dev

Deployment Strategies

Blue-Green Deployment

deploy:
  steps:
    # Deploy to blue
    - name: Deploy blue
      run: npx wrangler deploy --env blue

    # Test blue
    - name: Test blue
      run: npm run test:smoke -- --url https://blue.example.workers.dev

    # Switch traffic to blue
    - name: Switch traffic
      run: npx wrangler route update blue.example.com blue-worker

    # Keep green as rollback option

Canary Deployment

deploy:
  steps:
    # Deploy canary (10% traffic)
    - name: Deploy canary
      run: npx wrangler deploy --env canary

    # Monitor canary
    - name: Monitor metrics
      run: npm run monitor:canary

    # Promote to production if successful
    - name: Promote canary
      if: success()
      run: npx wrangler deploy --env production

Rolling Deployment

Cloudflare Workers automatically do rolling deployments across edge locations.

Pre-Deployment Checks

Validate Configuration

- name: Validate wrangler.toml
  run: npx wrangler deploy --dry-run

Check Bundle Size

- name: Check bundle size
  run: |
    npm run build
    SIZE=$(stat -f%z dist/index.js)
    if [ $SIZE -gt 1000000 ]; then
      echo "Bundle too large: $SIZE bytes"
      exit 1
    fi

Lint Ensembles

- name: Lint ensemble definitions
  run: npm run lint:ensembles

Post-Deployment

Health Check

- name: Health check
  run: |
    sleep 10
    curl -f https://my-worker.workers.dev/health || exit 1

Smoke Tests

- name: Run smoke tests
  run: npm run test:smoke
  env:
    API_URL: https://my-worker.workers.dev

Notify Team

- name: Notify Slack
  if: success()
  uses: slackapi/slack-github-action@v1
  with:
    payload: |
      {
        "text": "✅ Deployed to production successfully"
      }
  env:
    SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

Rollback Strategy

Automatic Rollback

- name: Deploy
  id: deploy
  run: npx wrangler deploy --env production

- name: Test deployment
  id: test
  run: npm run test:smoke

- name: Rollback on failure
  if: failure() && steps.deploy.outcome == 'success'
  run: npx wrangler rollback --env production

Manual Rollback

# Rollback to previous version
npx wrangler rollback --env production

Monitoring in CI

Performance Monitoring

- name: Run performance tests
  run: npm run test:performance

- name: Compare with baseline
  run: npm run compare:performance

Cost Monitoring

- name: Estimate costs
  run: npm run estimate:costs

Multi-Environment Setup

# wrangler.toml
[env.dev]
name = "my-worker-dev"
vars = { ENVIRONMENT = "dev" }

[env.staging]
name = "my-worker-staging"
vars = { ENVIRONMENT = "staging" }

[env.production]
name = "my-worker"
vars = { ENVIRONMENT = "production" }

# Deploy to specific environment
- name: Deploy to ${{ matrix.env }}
  strategy:
    matrix:
      env: [dev, staging, production]
  run: npx wrangler deploy --env ${{ matrix.env }}

Best Practices

Test before deploy - Always run tests in CI
Use secrets management - Never commit secrets
Automate everything - Manual steps are error-prone
Deploy frequently - Small, incremental changes
Monitor deployments - Health checks and smoke tests
Have rollback plan - Quick rollback on failure
Use environments - Dev, staging, production
Version releases - Tag deployments
Notify team - Slack/email on deploy
Document pipeline - README with CI/CD info

Troubleshooting

Deploy Fails

- name: Debug deployment
  if: failure()
  run: |
    npx wrangler whoami
    npx wrangler deploy --dry-run --verbose

Tests Fail in CI but Pass Locally

- name: Debug environment
  run: |
    node --version
    npm --version
    env

Secrets Not Available

# Verify secret is set
npx wrangler secret list --env production

# Re-add secret
echo "value" | npx wrangler secret put SECRET_NAME --env production

Deployment Guide

Deploy to Cloudflare Workers

Testing Guide

Test workflows

Edgit Integration

Component versioning

GitHub Actions

Official GitHub Actions docs

Conductor

Core Concepts

Guides

Member Types

Built-In Members

Examples

API Reference

Deployment

​Overview

​GitHub Actions

​Complete Pipeline

​Test-Only Workflow

​Manual Deployment

​GitLab CI

​Secrets Management

​GitHub Secrets

​Wrangler Secrets

​Environment Variables

​Testing in CI

​Unit Tests

​Integration Tests

​E2E Tests

​Deployment Strategies

​Blue-Green Deployment

​Canary Deployment

​Rolling Deployment

​Pre-Deployment Checks

​Validate Configuration

​Check Bundle Size

​Lint Ensembles

​Post-Deployment

​Health Check

​Smoke Tests

​Notify Team

​Rollback Strategy

​Automatic Rollback

​Manual Rollback

​Monitoring in CI

​Performance Monitoring

​Cost Monitoring

​Multi-Environment Setup

​Best Practices

​Troubleshooting

​Deploy Fails

​Tests Fail in CI but Pass Locally

​Secrets Not Available

​Related Documentation

Deployment Guide

Testing Guide

Edgit Integration

GitHub Actions

Overview

GitHub Actions

Complete Pipeline

Test-Only Workflow

Manual Deployment

GitLab CI

Secrets Management

GitHub Secrets

Wrangler Secrets

Environment Variables

Testing in CI

Unit Tests

Integration Tests

E2E Tests

Deployment Strategies

Blue-Green Deployment

Canary Deployment

Rolling Deployment

Pre-Deployment Checks

Validate Configuration

Check Bundle Size

Lint Ensembles

Post-Deployment

Health Check

Smoke Tests

Notify Team

Rollback Strategy

Automatic Rollback

Manual Rollback

Monitoring in CI

Performance Monitoring

Cost Monitoring

Multi-Environment Setup

Best Practices

Troubleshooting

Deploy Fails

Tests Fail in CI but Pass Locally

Secrets Not Available

Related Documentation