# Manage workflows as code

> Export workflows to version control, make them portable between a sandbox and production, deploy them with their secrets, alert on failed runs, and switch them off in an emergency.

Source: https://developer.kustomer.com/docs/guides/manage-workflows-via-api

Workflows are JSON documents underneath the builder, so you can treat them like any other code: review changes in pull requests, test in a sandbox, and promote the same files to production. This guide builds that pipeline. By the end you'll have:

- an **export** that writes every workflow to a file, with org-specific IDs and secrets replaced by readable placeholders;
- a **deploy** script that creates or updates workflows by name, gives them their variables, and enables them in the right order;
- an **alert** on new failed runs;
- a **kill switch** that pauses workflows on one trigger, for incidents and data migrations.

```mermaid
flowchart LR
  SB[Sandbox org] -- "export" --> G[(Git repo<br/>workflows/*.json)]
  G -- "pull request review" --> G
  G -- "deploy: POST/PUT, variables, enable" --> P[Production org]
  P -- "GET …/processes" --> AL[Failure alerts]
```

This guide is about moving and operating workflows. To write one from scratch, start with [Write workflow JSON](https://developer.kustomer.com/docs/workflows/write-workflow-json).

**You'll need** an API key in each org with `org.permission.workflow.read`, `org.permission.workflow.create` and `org.permission.workflow.update`, plus `org.permission.workflow` to read, change and delete variables. The admin roles `org.admin.workflow.read`, `org.admin.workflow.write` and `org.admin.workflow` also work. The code uses the `kfetch` helper from [Making requests](https://developer.kustomer.com/docs/get-started/making-requests#idempotency-and-retries), with a base URL and key per org.

## What you can send back

`GET` returns more than `POST` and `PUT` accept. Only these root keys can be written: `name`, `description`, `meta`, `trigger`, `steps`, `enabled`, `replaces`, `logging`, `response` (a callable workflow's outputs) and `objectMapping`. Read-only attributes such as `lastError`, `editable`, `containsVariables` and timestamps, and the `_id` Kustomer adds to each step, are rejected with `400` if you send them back.

`name` is the workflow's identity in this pipeline. It's unique in each org, stored lowercased and trimmed, and the same in every org you deploy to, unlike the `id`.

## 1. Export workflows to files

Page through `GET /v1/workflows`; each record already holds the whole document. Keep the writable keys and drop step `_id`s:

`GET /v1/workflows` ([reference](https://developer.kustomer.com/api/workflows/get-workflows))

```ts title="export.ts"
const WRITABLE = ['name', 'description', 'meta', 'trigger', 'steps', 'replaces', 'logging', 'response', 'objectMapping'] as const

async function* allWorkflows(org: Org) {
  for (let page = 1; ; page++) {
    const { data, links } = await kfetch(org, `/v1/workflows?page=${page}&pageSize=100`).then((r) => r.json())
    yield* data
    if (!links?.next) return
  }
}

function toFile(wf: { attributes: Record<string, unknown> }) {
  const doc = Object.fromEntries(WRITABLE.filter((k) => wf.attributes[k] !== undefined).map((k) => [k, wf.attributes[k]]))
  doc.steps = (doc.steps as { _id?: string }[]).map(({ _id, ...step }) => step)
  return doc
}
```

Leave `enabled` out of the files. Whether a workflow is on is a per-org decision, made at deploy time.

## 2. Make the files portable

Workflows reference records by ID: tags, teams, users, queues, klasses, workflow variables, and in `replaces`, other workflows. Those IDs differ between orgs, so the export replaces them with placeholders and the deploy puts the target org's IDs back.

For records, keep one small map per org, and write placeholders such as `@@tag.vip@@` in place of the IDs:

```json title="ids/production.json"
{
  "tag.vip": "6512f0c2a1b2c3d4e5f60718",
  "team.billing": "6512f0c2a1b2c3d4e5f60719",
  "queue.billing": "6512f0c2a1b2c3d4e5f6071a",
  "workflow.app-order-sync": "6513a0d1a4b1e1001a6b3e77"
}
```

A `workflow.` entry is for a workflow you override with `replaces`. An app installs its workflows with a different ID in each org, so look the workflow up by name in each org and record its ID here.

For variables, write the variable's **name** where its ID goes. Steps reference a variable as `/#variables.<variableId>` or `{{{variables.<variableId>}}}`, so the export turns `variables.6512…` into `variables.crmToken`, and the deploy turns it back once the variable exists. Keep variable names unique within each workflow for this to work.

```ts title="placeholders.ts"
type IdMap = Record<string, string> // placeholder key → ID in this org

export function toPlaceholders(json: string, ids: IdMap, vars: Record<string, string>): string {
  for (const [key, id] of Object.entries(ids)) json = json.replaceAll(id, `@@${key}@@`)
  for (const [name, id] of Object.entries(vars)) json = json.replaceAll(`variables.${id}`, `variables.${name}`)
  return json
}

export function fromPlaceholders(json: string, ids: IdMap): string {
  return json.replace(/@@([\w.-]+)@@/g, (_, key) => {
    if (!ids[key]) throw new Error(`no ID for ${key} in this org`)
    return ids[key]
  })
}
```

Run both steps in the export, so no file is written with raw IDs. `variableNames` maps each of the workflow's variable IDs to its name, from [`GET /v1/workflows/{id}/variables`](https://developer.kustomer.com/api/workflows/get-all-variables-for-a-workflow):

```ts title="export.ts"
import { writeFile } from 'node:fs/promises'

for await (const wf of allWorkflows(org)) {
  const json = toPlaceholders(JSON.stringify(toFile(wf), null, 2), ids, await variableNames(org, wf.id))
  await writeFile(`workflows/${wf.attributes.name}.json`, `${json}\n`)
}
```

After the export, search the files for any remaining 24-character hex strings. Each one is an ID you haven't mapped yet, and deploying it to another org would point at nothing.

## 3. Deploy by name

For each file, look the workflow up by name in the target org. New workflows are created **disabled**, given their variables, and only then enabled, so no run starts without its secrets. Existing workflows get every key in the file sent again, with `trigger` and `steps` together. A `PUT` keeps the root keys you leave out, so the deploy sends `replaces` as `null` when the file has none, which clears an override you've removed from Git.

`POST /v1/workflows` ([reference](https://developer.kustomer.com/api/workflows/create-a-workflow))

`PUT /v1/workflows/{id}` ([reference](https://developer.kustomer.com/api/workflows/update-a-workflow-based-on-the-id))

`POST /v1/workflows/{id}/variables` ([reference](https://developer.kustomer.com/api/workflows/create-a-workflow-variable-for-a-workflow))

```ts title="deploy.ts"
async function deploy(org: Org, file: string, ids: IdMap, secrets: Record<string, string>, enable: boolean) {
  const doc = JSON.parse(fromPlaceholders(file, ids))
  const existing = await findByName(org, doc.name.trim().toLowerCase())

  const id = existing?.id ?? (await post(org, '/v1/workflows', { ...doc, enabled: false })).data.id
  const varIds = await ensureVariables(org, id, secrets)

  // Point variable references at this org's variable IDs, then send trigger and steps together.
  const withVars = JSON.parse(
    JSON.stringify({ trigger: doc.trigger, steps: doc.steps }).replace(/variables\.([\w-]+)/g, (_, name) => {
      if (!varIds[name]) throw new Error(`${doc.name}: no value for variable ${name}`)
      return `variables.${varIds[name]}`
    }),
  )
  const { name: _name, ...rest } = doc
  await put(org, `/v1/workflows/${id}`, { ...rest, replaces: doc.replaces ?? null, ...withVars })

  if (enable) await put(org, `/v1/workflows/${id}`, { enabled: true })
}

async function ensureVariables(org: Org, workflowId: string, secrets: Record<string, string>) {
  const { data } = await kfetch(org, `/v1/workflows/${workflowId}/variables?pageSize=100`).then((r) => r.json())
  const byName: Record<string, string> = Object.fromEntries(data.map((v: { id: string; attributes: { name: string } }) => [v.attributes.name, v.id]))
  for (const [name, value] of Object.entries(secrets)) {
    if (byName[name]) await put(org, `/v1/workflow-variables/${byName[name]}`, { value })
    else byName[name] = (await post(org, `/v1/workflows/${workflowId}/variables`, { name, value, type: 'string' })).data.id
  }
  return byName
}
```

`findByName` pages through `GET /v1/workflows` and compares `attributes.name`, and `post` and `put` wrap `kfetch` and throw on any non-2xx response. Take secret values from your own secret store: variable values are write-only, so Kustomer can't give them back to you.

The errors you'll see while deploying:

| Status | Cause                                                                                                             |
| ------ | ----------------------------------------------------------------------------------------------------------------- |
| `400`  | An unknown root key, a leftover step `_id`, or a transition that loops back to a step already on its path         |
| `409`  | On create, the name is taken. On update, the workflow has `editable: false`, usually because an app installed it. |

To change an app-installed workflow, create your own with `replaces` set to the app workflow's **ID**, through a `workflow.` entry in each org's ID map. When both match an event, only yours runs. See [Override an app workflow with replaces](https://developer.kustomer.com/docs/workflows/workflow-json#override-an-app-workflow-with-replaces).

> **Changes take about a minute:** Creating, enabling or changing a workflow, and changing a variable, takes about a minute to reach new runs. Deploy, wait, then run your checks.

## 4. Alert on failed runs

Kustomer records a process for each run, with its `status`: `queued`, `running`, `sleeping`, `finished`, `cancelled` or `failed`. Poll each enabled workflow for processes since your last check, keep the failed ones, and send them to your alerting tool:

`GET /v1/workflows/{id}/processes` ([reference](https://developer.kustomer.com/api/workflows/get-processes-errors-by-workflow))

```ts title="alerts.ts"
async function newFailures(org: Org, since: string) {
  const out = []
  for await (const wf of enabledWorkflows(org)) {
    let next: string | null = `/v1/workflows/${wf.id}/processes?createdAtFrom=${since}&pageSize=100`
    while (next) {
      const { data, links } = await kfetch(org, next).then((r) => r.json())
      for (const p of data) {
        if (p.attributes.status !== 'failed') continue
        const { message, step } = JSON.parse(p.attributes.error)
        out.push({ workflow: wf.attributes.name, step, message, at: p.attributes.createdAt })
      }
      next = links?.next ?? null
    }
  }
  return out
}
```

`createdAtFrom` is inclusive, so store the time you started each check and use it as the next `since`, but only after every page has been read; a check that fails partway should run again from the old `since`. `error` and `meta.context` are JSON strings; `meta.context` holds each step's output, with variable values removed, which is usually enough to reproduce the problem. Process records are kept for about a week. A REST step that gets an error status from your service doesn't fail the run, so it won't show up here; handle those in the workflow's transitions. The [debugging guide](https://developer.kustomer.com/docs/workflows/debugging) explains the common messages.

## 5. Keep a kill switch

To stop workflows quickly, during an incident or while you bulk-update records, send only `enabled`. It works even on workflows with `editable: false`, and nothing else changes:

```ts title="pause.ts"
async function pauseTrigger(org: Org, eventName: string): Promise<string[]> {
  const paused: string[] = []
  const q = new URLSearchParams({ enabled: 'true', eventName, pageSize: '100' })
  for (;;) {
    // Always read the first page: each workflow you disable drops out of this list.
    const { data } = await kfetch(org, `/v1/workflows?${q}`).then((r) => r.json())
    if (!data.length) return paused // save these, and re-enable only them afterwards
    for (const wf of data) {
      await put(org, `/v1/workflows/${wf.id}`, { enabled: false })
      paused.push(wf.id)
    }
  }
}
```

Allow about a minute before starting a migration. Events that happen while a workflow is off don't start runs later, so plan a catch-up if the workflow keeps something in sync.

## 6. Run it in production

#### Review diffs, not screenshots

Export after every builder change and commit the files. Pull requests then show exactly which condition or param changed. Keep step IDs stable: other steps reference them as `/#steps.<id>`.

#### Deploy to a sandbox first

Run the same deploy against a sandbox with its own ID map, send a test event, and check `lastError` and the processes endpoint before promoting.

#### Guard update triggers

A workflow that updates the record that triggered it fires again. Add a condition on `changes` so it only acts on the fields you care about. See [How workflows run](https://developer.kustomer.com/docs/workflows/how-workflows-run).

#### Don't delete, disable

The API has no delete for workflows. Remove a file from the repo by disabling the workflow in each org, and keep the file in history in case you need it back.

## Related

- [Workflows](https://developer.kustomer.com/api/workflows#workflows), [workflow variables](https://developer.kustomer.com/api/workflows#workflow-variables) and [failed runs](https://developer.kustomer.com/api/workflows#workflow-processes-errors) reference
- [Workflow JSON](https://developer.kustomer.com/docs/workflows/workflow-json), for every field and limit
- [Patterns](https://developer.kustomer.com/docs/workflows/patterns), for common workflows to start from