Durable Flows

Durable flows let you build multi-step workflows that survive server restarts, crashes, and deployments. Every step's input and output is persisted to a SQLite database using Write-Ahead Logging (WAL), so bext can resume interrupted flows exactly where they left off. This is a Pro feature.

Use durable flows for operations that span multiple services or take longer than a single request: order processing, payment workflows, onboarding sequences, data pipelines, and scheduled batch jobs.

How It Works

A flow is a named sequence of steps. When you start a flow run, bext:

  1. Creates a run record in SQLite with status pending
  2. Executes steps in order, persisting each step's result before moving to the next
  3. On crash/restart, queries for runs with status running and resumes from the last completed step
  4. On failure, applies the retry policy; if exhausted, marks the run as failed and executes compensation steps

The SQLite WAL mode ensures that writes are durable even if the process is killed mid-transaction.

Flow Definition

Define flows using the JavaScript/TypeScript SDK:

import { defineFlow, step } from "bext/flows";

export const orderFlow = defineFlow("process-order", {
  maxAttempts: 3,
  timeout: "30m",
  version: "1.0",

  steps: [
    step("validate", async (input, ctx) => {
      const order = await validateOrder(input.orderId);
      if (!order.valid) throw new Error("Invalid order");
      return { order };
    }),

    step("charge-payment", {
      timeout: "60s",
      retries: 2,
      compensate: async (result, ctx) => {
        // Rollback: refund if a later step fails
        await refundPayment(result.chargeId);
      },
      run: async (input, ctx) => {
        const charge = await chargeCard(input.order.paymentMethod, input.order.total);
        return { chargeId: charge.id };
      }
    }),

    step("fulfill", async (input, ctx) => {
      await createShipment(input.order.items, input.order.address);
      return { shipped: true };
    }),

    step("notify", async (input, ctx) => {
      await sendConfirmationEmail(input.order.email, input.order.id);
      return { notified: true };
    })
  ]
});

Starting a Flow Run

Trigger a flow run from an API handler, a scheduled task, or another flow:

import { startFlow } from "bext/flows";

// Start a run and get the run ID
const runId = await startFlow("process-order", {
  tenantId: "tenant_abc",
  input: { orderId: "ord_12345" },
  priority: 1,                    // Higher priority runs first
  idempotencyKey: "ord_12345",    // Prevents duplicate runs
});

The idempotencyKey ensures that retried API calls do not create duplicate flow runs. If a run with the same key already exists, the existing run ID is returned.

Retry Policies

Each step and the overall flow can define retry behavior:

step("external-api-call", {
  retries: 5,           // Max retry attempts for this step
  timeout: "30s",       // Per-attempt timeout
  backoff: "exponential", // "fixed", "exponential", or "linear"
  backoffBase: "1s",    // Starting delay (doubles each retry with exponential)
  run: async (input) => {
    return await callExternalService(input);
  }
})

The flow-level maxAttempts controls how many times the entire flow restarts from the beginning after an unrecoverable step failure.

Compensation (Rollback)

When a step fails after earlier steps have completed, bext runs compensation handlers in reverse order. This is the saga pattern:

Step 1: validate      -> OK
Step 2: charge-payment -> OK (chargeId: ch_123)
Step 3: fulfill       -> FAILED

Compensation runs:
  Step 2 compensate: refundPayment(ch_123)

Compensation handlers receive the step's original result, so they have the data needed to undo the operation. If a compensation handler itself fails, the failure is logged and bext continues compensating earlier steps.

Timeout Handling

Flows support timeouts at two levels:

  • Flow timeout -- the total time a run is allowed to take across all steps and retries (e.g. "30m")
  • Step timeout -- the time a single step execution is allowed (e.g. "60s")

Duration strings support ms, s, m, h, and d suffixes. When a timeout fires, the step is marked as failed and the retry/compensation logic activates.

Run Status

Each run progresses through these statuses:

Status Description
pending Created, waiting to execute
running Currently executing steps
completed All steps finished successfully
failed Exhausted retries or unrecoverable error
cancelled Manually cancelled via API

Steps have a similar lifecycle: pending, running, completed, failed, skipped.

Inspecting a Run

Fetch the full state of a single run -- including every step's status, attempts, and result -- from the admin API. Poll it for live dashboards and monitoring:

GET /__bext/admin/api/flows/{id}

The response reflects each status change as the run progresses:

{
  "id": "01abc...",
  "flow_id": "process-order",
  "status": "completed",
  "attempts": 1,
  "steps": [
    { "step_name": "charge-payment", "step_index": 0, "status": "completed",
      "attempt": 1, "result_json": "{\"chargeId\":\"ch_123\"}" }
  ]
}

Querying Runs

List and filter flow runs via the admin API:

GET /__bext/admin/api/flows?flow_id=process-order&status=failed&limit=50

{
  "flows": [
    {
      "id": "018f3a2b...",
      "flow_id": "process-order",
      "tenant_id": "tenant_abc",
      "status": "failed",
      "error": "Payment declined",
      "created_at": 1712150400000,
      "attempts": 3
    }
  ]
}

Timers and Delays

Insert a delay between steps or schedule a step to run at a future time:

step("wait-for-confirmation", {
  delay: "24h",  // Wait 24 hours before executing
  run: async (input, ctx) => {
    const confirmed = await checkConfirmation(input.order.id);
    if (!confirmed) throw new Error("Not confirmed in time");
    return { confirmed: true };
  }
})

Timers are persisted to SQLite with a wake_at timestamp. On restart, bext queries for fired timers and resumes the corresponding flows.

Flow Visualization

The bext companion app provides a visual timeline of flow runs, showing each step's status, duration, retries, and errors. This is invaluable for debugging production workflows.

Access it at /__bext/admin/api/flows in the admin dashboard, or connect the standalone companion app to your bext instance.

Warning

Durable flows are a Pro feature backed by a per-app SQLite WAL database. On Community edition, startFlow and defineFlow are present in the SDK but calls will throw at runtime. Check your license tier before building production workflows on this API.

Tip

Use an idempotencyKey whenever a flow is triggered from a retriable context (an API handler, a webhook, or a scheduler tick). Without one, a retried trigger creates a duplicate run that will execute all steps again — including payment charges and emails.

Related