E2E Playbooks

The Playbook pattern is the primary way to write E2E tests for the workflow engine. A playbook defines a complete execution scenario declaratively: the CMS state, the workflow, the function blocks, the expected jobs, and the expected terminal state.

Architecture

graph TD
    PB["Playbook<br/>(test scenario definition)"] --> PE["PlaybookExecutor"]
    PE --> EI["E2eInteractor<br/>(HTTP helpers)"]
    EI --> App["NestJS App<br/>(test instance)"]

Playbook -- A TypeScript object that declares the complete test scenario. PlaybookExecutor -- Runs the playbook step by step, simulating the worker side. E2eInteractor -- Provides HTTP helper methods for API interaction.

Playbook Structure

interface Playbook {
  cmsDbBeforeExecution: CmsDB           // CMS mock state
  createWorkflowDefinitionDto: CreateWorkflowDefinitionDto  // Workflow to deploy
  registerFunctionBlockDtos: RegisterFunctionBlockPlaybookDto[]  // FBs to register
  executeWorkflowDto?: ExecuteWorkflowDto   // Execution trigger
  expectedWorkflowExecutionStateAfterSubmit?: WorkflowExecutionStatus
  expectedAcquireJobs?: PlaybookJobAndResponse[]  // Expected ACQUIRE jobs + responses
  expectedExecuteJobs?: PlaybookJobAndResponse[]  // Expected EXECUTE jobs + responses
  expectedRollbackJobs?: PlaybookJobAndResponse[]  // Expected ROLLBACK jobs + responses
  expectedTerminalState?: WorkflowExecutionStatusTerminalStates
  cmsDbAfterExecution?: CmsDB            // Expected CMS state after execution
  expectedStateOverrides?: ExpectedStateOverrides
}

Execution Flow

The PlaybookExecutor runs the playbook in this order:

Initialize CMS -- Set up mock CMS data (devices, interfaces, groups)
Register workflow -- Deploy the workflow definition
Register function blocks -- Register all FBs the workflow uses
Execute workflow -- Trigger the execution
Wait for RESOURCE_DISCOVERY -- Or terminal state if validation fails
Register worker -- Create a test worker
Acquire phase -- Poll for ACQUIRE jobs, post expected responses, wait for RUNNING
Execute phase -- Poll for EXECUTE jobs, post expected responses in order
Rollback phase -- If the workflow enters ROLLBACK, poll and respond to rollback jobs
Assert terminal state -- Verify the execution ended in the expected state
Assert CMS state -- Optionally verify CMS data after execution

Writing a Playbook

Directory Structure

test/workflow-lifecycle/
├── playbooks/
│   ├── base/                     # E2eInteractor base class
│   ├── playbook.ts               # Playbook interface
│   ├── playbook-executor.ts      # PlaybookExecutor
│   └── tests/                    # Playbook definitions
│       ├── simple-workflow-2.ts
│       ├── design-bit/
│       └── design-technopark/
└── workflow-lifecycle.e2e-spec.ts # Test spec

Minimal Example

const myPlaybook: Playbook = {
  cmsDbBeforeExecution: {
    devices: [
      { id: 1, hostname: 'router-1', ip: '10.0.0.1' },
      { id: 2, hostname: 'router-2', ip: '10.0.0.2' },
    ],
  },

  createWorkflowDefinitionDto: {
    workflow: {
      label: 'test_workflow',
      name: 'test_workflow',
      package: 'wf.test.neops.io',
      majorVersion: 1,
      minorVersion: 0,
      patchVersion: 0,
      seedEntity: 'device',
      type: 'workflow',
      steps: [{
        type: 'functionBlock',
        label: 'ping',
        functionBlock: 'fb.test.neops.io/ping:1.0.0',
      }],
    },
  },

  registerFunctionBlockDtos: [{
    package: 'fb.test.neops.io',
    name: 'ping',
    majorVersion: 1,
    minorVersion: 0,
    patchVersion: 0,
    fbType: 'check',
    isPure: true,
    isIdempotent: true,
    parameterJsonSchema: {},
    resultDataJsonSchema: {},
  }],

  executeWorkflowDto: {
    workflow: 'wf.test.neops.io/test_workflow:1.0.0',
    executeOnParameters: { ids: [1, 2] },
    parameters: {},
  },

  expectedExecuteJobs: [
    {
      functionBlock: 'fb.test.neops.io/ping:1.0.0',
      response: { status: 'success', result: { reachable: true } },
    },
    {
      functionBlock: 'fb.test.neops.io/ping:1.0.0',
      response: { status: 'success', result: { reachable: true } },
    },
  ],

  expectedTerminalState: WorkflowExecutionStatus.COMPLETED_ACK,
}

Running the Test

describe('Workflow Lifecycle', () => {
  let app: INestApplication

  beforeAll(async () => {
    app = await initNestApplication()
  })

  afterAll(async () => {
    await initNestApplication.teardown(app)
  })

  it('should complete a simple workflow', async () => {
    const executor = new PlaybookExecutor(myPlaybook)
    await executor.execute(app)
  })
})

E2eInteractor Helpers

The E2eInteractor base class provides HTTP helpers:

Method	Description
`registerWorker()`	Register a test worker
`registerWorkflow(dto)`	Deploy a workflow definition
`registerFunctionBlocks(dtos)`	Register function blocks
`executeWorkflow(dto)`	Trigger a workflow execution
`pollJobs(workerId, fbIds)`	Poll the blackboard for jobs
`postJobResult(jobId, result)`	Push a job result
`waitForState(executionId, state)`	Wait until execution reaches a state
`waitForJobs(executionId, type)`	Wait until jobs of a type are available
`initCms(cmsDb)`	Set up CMS mock data