Neops Workflow Engine

Description

Neops workflow enigne

Schema

classDiagram
direction BT
class function_block {
   timestamp with time zone created_at
   timestamp with time zone updated_at
   jsonb json_schema
   varchar(255) description
   varchar(255) package
   varchar(255) name
   varchar(255) version
}
class function_block_registration {
   timestamp with time zone created_at
   timestamp with time zone updated_at
   varchar(255) function_block_package
   varchar(255) function_block_name
   varchar(255) function_block_version
}
class job {
   timestamp with time zone created_at
   timestamp with time zone updated_at
   varchar(255) function_block_package
   varchar(255) function_block_name
   varchar(255) function_block_version
   jsonb parameters
   jsonb context
   uuid workflow_execution_uuid
   uuid uuid
}
class job_result {
   timestamp with time zone created_at
   timestamp with time zone updated_at
   text status
   jsonb result
   uuid job_uuid
}
class worker_registration {
   timestamp with time zone created_at
   timestamp with time zone updated_at
   uuid uuid
}
class workflow {
   timestamp with time zone created_at
   timestamp with time zone updated_at
   jsonb workflow_definition
   varchar(255) id
}
class workflow_execution {
   timestamp with time zone created_at
   timestamp with time zone updated_at
   varchar(255) workflow_id
   uuid uuid
}

function_block_registration  -->  function_block : function_block_package, function_block_name, function_block_version-package, name, version
job  -->  function_block : function_block_package, function_block_name, function_block_version-package, name, version
job_result  -->  job : job_uuid-uuid
job  -->  workflow_execution : workflow_execution_uuid-uuid
workflow_execution  -->  workflow : workflow_id-id

Project setup

$ npm install
$ npm link # To make neops-mermaid available globally

Compile and run the project

# development
$ npm run start

# watch mode
$ npm run start:dev

# production mode
$ npm run start:prod

Run tests

# unit tests
$ npm run test

# e2e tests
$ npm run test:e2e

# test coverage
$ npm run test:cov

Docker

Build the application image

# 1) Provide your GitHub Packages token for private dependencies
export NPM_TOKEN=ghp_xxx

# 2) Build the full production image (multi-stage)
DOCKER_BUILDKIT=1 docker build \
  --secret id=npm_token,env=NPM_TOKEN \
  -t zebbra/neops-workflow-engine:latest .

# 3) Run it
docker run --rm -p 3030:3030 zebbra/neops-workflow-engine:latest

Notes: - The build uses BuildKit secrets so the token is not written to layers. - If you only want to build without running e2e tests, you can skip the --target and just build the final image as above.

Run the CI stages inside Docker

The Dockerfile contains CI stages you can run locally in a single image build.

# Run lint, unit tests, and e2e tests in the multi-stage "run-ci" target
export NPM_TOKEN=ghp_xxx
DOCKER_BUILDKIT=1 docker build \
  --secret id=npm_token,env=NPM_TOKEN \
  --target run-ci \
  -t zebbra/neops-workflow-engine:ci .

# The resulting image contains marker files proving stage success:
# lint.txt, test.txt, test-e2e.txt
docker run --rm zebbra/neops-workflow-engine:ci sh -c 'ls -1'

Individual stages can also be built directly if needed:

# Lint only
DOCKER_BUILDKIT=1 docker build --secret id=npm_token,env=NPM_TOKEN --target linter .

# Unit tests only
DOCKER_BUILDKIT=1 docker build --secret id=npm_token,env=NPM_TOKEN --target tester .

# E2E tests only
DOCKER_BUILDKIT=1 docker build --secret id=npm_token,env=NPM_TOKEN --target tester-e2e .