Skip to content

Network Zones

Network zones need Workflow Engine 0.46.0, neops-core 2.4 and Worker SDK 0.5, or later.

A network zone is a network that only its own workers reach; zones may reuse each other’s addresses. 10.0.0.1 in zone-a and 10.0.0.1 in zone-b are different devices. Every job runs in exactly one zone, and only a worker that serves that zone takes it.

Zones are created in the CMS (neops-core), in Django admin or with networkZoneUpsert. default is reserved: it is the zone of every device that has no zone stored, and no CMS zone can take the name. Workflows name it like any other zone.

Declaring a zone

networkZone is a key on every step kind (function blocks, embedded workflows, workflow references) and on the root workflow. Nested steps inherit it.

networkZone: zone-b                              # a zone in the CMS
networkZone: default                             # the zone of devices with no zone
networkZone: "{{ parameters.zone || 'default' }}" # an expression over parameters
networkZone: { name: zone-b, override: true }    # beat the zone of the device the job runs on

A string is short for { name: <string> }; override defaults to false.

Where a job runs

  1. Take the nearest networkZone: the step’s own, else its enclosing workflow’s, and so on up to the root. Its override flag comes with it: an outer override: true does not reach a step that declares its own zone.
  2. If that declaration has override: true, the job runs in the declared zone.
  3. Otherwise a job on one device runs in the device’s zone; a job on one interface runs in the zone of the interface’s device.
  4. Any other job (a runOn: global job, a group job, and every step’s ACQUIRE job) runs in the declared zone, else in default.
Declarations Runs on Job zone Why
none r1 in zone-a zone-a device
none sw9, no zone stored default device
workflow: zone-b r1 in zone-a zone-a the device beats a zone without override; the step’s ACQUIRE job, with no device, runs in zone-b
step: { name: zone-b, override: true } r1 in zone-a zone-b override
workflow: { name: zone-a, override: true }, step: zone-b sw9 in default default the nearest declaration (zone-b) has no override, so the device wins
step: "{{ parameters.zone }}", run with zone: zone-b global zone-b step
none global default default
none a group default default; a group job names its members by id only

A declared zone places jobs; it does not select devices. A selector by IP or hostname matches every zone’s device with that address unless it also filters on network_zone.name. Because every function-block step gets an ACQUIRE job when the run starts, a device step that inherits zone-b needs a zone-b worker with its function block even when all its devices are elsewhere.

Retries and rollback jobs run in the zone of the job they belong to.

Override reaches whatever answers there

With override: true a worker in the declared zone connects to the device’s address in its own network, with the device’s credentials, and writes what it finds onto the device’s record. Where addresses overlap, that may be a different device. Check the device’s identity before moving it, as in the onboarding example below.

Zone expressions

A zone expression reads only parameters. Every step’s ACQUIRE job takes its zone when the run starts, before any step has run, so a zone cannot read a step’s result, its meta, or the context.

  • A workflow’s own networkZone is evaluated where its other keys are: in the parent’s scope for an embedded workflow, in the run’s parameters for the root.
  • Inside an embedded workflow, parameters are what the parent passes. A parameter the parent fills from a step result or the context is only known after a step runs, so a zone that reads it fails the run.
  • An embedded workflow’s parameters get no defaults; a default in the root workflow’s parameterSchema does fill a missing run parameter.

An expression must resolve to a zone

When a run starts, the engine resolves every networkZone in the tree, including one that a nearer declaration shadows and one on a step whose condition may skip it. The run fails with invalid_parameters, before anything is locked, when a declaration:

  • does not resolve to a zone name: null, a missing parameter, an empty or a blank string, or a value that is not a string, such as a number;
  • reads a value only known after a step runs.

The failure names the step or workflow and its expression, as in The networkZone of 'discover' ({{ parameters.zone }}) does not resolve to a zone name. Put the fallback into the expression; JMESPath’s || covers a missing, null or empty value, but not a blank one:

networkZone: "{{ parameters.zone || 'default' }}"

The fallback replaces the parent’s zone, so a sub-workflow that should follow its parent takes the zone as a parameter.

What publishing checks

  • A networkZone expression that reads anything but parameters, or does not compile, is refused with 400 InvalidDocument.
  • A plain zone name the CMS does not hold is reported as the advisory finding NETWORK_ZONE_UNKNOWN: the zone may be created before the first run. See Publishing.

Workers and their zones

A worker names the zones it serves when it registers, from NEOPS_WORKER_NETWORK_ZONES (comma-separated) in the Worker SDK. The engine sends it networkZones on POST /workers/register and echoes the list, default first:

{ "name": "worker-b", "version": "0.5.0", "networkZones": ["zone-b", "default"] }
  • A worker that lists no zones serves default. Once the CMS holds a zone besides default, a worker must list its zones, even if only default: registration without a list (or with an empty one) is refused with 400 This installation has network zones. Set NEOPS_WORKER_NETWORK_ZONES, for example to default.
  • A zone the CMS does not hold is refused with 400 Unknown network zone: <name>. Create it in the CMS first, in Django admin or with networkZoneUpsert.
  • A worker that lists several zones registers, with a warning in the engine log: the addresses of those zones must not overlap as seen from the worker’s host.
  • Registration answers 503 only while the engine has never read the zones from the CMS (see below).

A worker claims only jobs of the zones it serves. The zone is part of the claim query, so a job in a zone no worker serves waits, while later jobs of other zones are claimed around it. GET /version answers "networkZones": {"supported": true}, so a worker can tell that the engine routes by zone.

Each job in a poll response carries meta with where it runs and why:

{
  "meta": {
    "networkZone": "zone-b",
    "networkZoneSource": "override",
    "step": { "networkZone": "zone-b", "override": true }
  }
}

networkZoneSource is device, step, override or default, as in Where a job runs; step is the step’s nearest declaration (networkZone is null when none encloses the step).

GET /network-zones (worker:read) lists the zones, default first, each with its online workers and its waiting jobs:

[
  { "name": "default", "description": "", "workers": { "online": 2 }, "jobs": { "pending": 0 } },
  { "name": "zone-b", "description": "Branch office B", "workers": { "online": 0 }, "jobs": { "pending": 3 } }
]

The engine does not check whether a zone has an online worker before it creates a job there: a job in a zone nobody serves waits until a worker of that zone comes online or BLACKBOARD_PENDING_JOB_TIMEOUT (one hour by default) fails it with No worker serving network zone ‘’ polled the job within …. A waiting EXECUTE job holds the engine’s drain phase, so new runs in every zone wait with it. The Monitor App’s Workers page flags such zones.

Run at least two workers for every zone, default included, so a zone’s jobs keep moving while one of its workers restarts. The developer-preview engine admits at most 2 workers per installation, so two workers in each of several zones need the licensed (enterprise) engine.

Where the engine reads zones

The engine keeps its own table of zones, read from the CMS when it starts and every 60 seconds after: the CMS’s zones plus default. A zone the CMS deletes is marked deleted in the table and comes back if the CMS creates it again. Once read, the table serves while the CMS is down.

Before refusing a name its table lacks, the engine reads the CMS once more, so a zone created a moment ago is found. It does so at worker registration, when a run starts, and when a job is created:

  • When a run starts, a declared zone the CMS does not hold fails the run FAILED_SAFE with reason unknown_network_zone, before anything is locked. The reason names each declaration and its zone. While the CMS is down, the engine decides from its last read.
  • When a job is created, a declared zone the CMS has deleted since the run started fails the step with unknown_network_zone. If the CMS cannot be read, the job is created and waits like any other. A zone read from the job’s device is not checked again: the CMS holds it.

Reading where a step ran

<step_label>.meta holds the zone a step’s job ran in and why, the step’s own declaration, the worker that ran it and the job’s identity. See Where a step ran: meta.

- type: functionBlock
  label: notify
  functionBlock: "fb.examples.neops.io/post_message:1.0.0"
  runOn: global
  networkZone: default                    # the chat bridge is only reachable from the central site
  parameters:
    text: "Discovery finished in {{ discover.meta.networkZone }}"

Examples

Discover one zone, then report from the central site:

label: zone_discovery
name: zone_discovery
package: wf.lab.neops.io
majorVersion: 1
minorVersion: 0
patchVersion: 0
seedEntity: global
type: workflow
networkZone: "{{ parameters.zone }}"        # every step below runs here unless it says otherwise
parameterSchema:
  type: object
  properties:
    zone: { type: string, description: "Network zone to discover" }
    subnets: { type: array, items: { type: object } }
  required: [zone]
steps:
  - type: functionBlock
    label: discover
    functionBlock: "fb.base.neops.io/global_discover_network:0.2.0"
    runOn: global
    parameters:
      subnets: "{{ parameters.subnets }}"
  - type: functionBlock
    label: notify
    functionBlock: "fb.examples.neops.io/post_message:1.0.0"
    runOn: global
    networkZone: default
    parameters:
      text: "Discovery finished in {{ discover.meta.networkZone }}"

Onboard devices that were imported into default but live in network B. Find the ones still in default with the CMS’s devices filter, where networkZone_Name: "default" matches the devices that have no zone stored, and run the workflow on their ids ("executeOnParameters": { "ids": [...] }):

query { devices(networkZone_Name: "default") { results { id hostname ip } } }

fb.examples.neops.io/get_facts and fb.examples.neops.io/check_identity are illustrative names; no repository ships them.

steps:
  - type: functionBlock
    label: first_contact
    functionBlock: "fb.examples.neops.io/get_facts:1.0.0"
    runOn: device
    networkZone: { name: zone-b, override: true }   # reach the device from zone-b, wherever it is recorded
  - type: functionBlock
    label: same_device
    functionBlock: "fb.examples.neops.io/check_identity:1.0.0"
    runOn: device
    parameters:                                     # fails unless the device that answered matches the CMS record
      hostname: "{{ first_contact.result.data.hostname }}"
      serial: "{{ first_contact.result.data.serial }}"
  - type: functionBlock
    label: rehome
    functionBlock: "fb.base.neops.io/device_update_device:0.2.0"
    runOn: device
    parameters:
      network_zone: zone-b                          # record the move; later steps and runs follow it

The first step sends the record’s credentials to whatever answers at that address in network B before any identity check runs, so onboard with credentials meant for that network.