Lane-Based Execution Queueing

Traditional agent systems serialize all operations through a single execution queue, creating bottlenecks that limit throughput. Concurrent execution is desirable but risky:

• Interleaving hazards: Multiple commands writing to stdin/stdout simultaneously corrupt user-facing output
• Race conditions: Shared state access without proper synchronization causes data corruption
• Deadlock risks: Naive concurrent queuing can create circular dependencies between operations

Agentic systems need parallelism to remain responsive (e.g., background tasks shouldn't block user interactions), but must preserve isolation guarantees.

Isolated execution lanes with independent queues and configurable concurrency per lane. Each lane is a named queue with its own concurrency limit, drained independently without interference.

Core concepts:

• Session lanes: Per-conversation queues prevent message interleaving. Each user session gets an isolated lane (e.g., session:telegram:user123).
• Global lanes: Background tasks (cron jobs, health checks) execute in dedicated lanes (e.g., cron, subagent) without blocking session lanes.
• Hierarchical composition: Operations can nest lanes (session → global) via queue chaining. The outer lane waits for the inner lane, preventing deadlocks through structured queuing.
• Configurable concurrency: Each lane supports a maxConcurrent cap. Unconfigured lanes default to 1 (serial); the main lane defaults to max(8, available CPU parallelism × 4) and is bounded by agents.defaults.maxConcurrent.
• Wait-time warnings: Tasks that sit queued too long trigger warnings and callbacks, surfacing performance issues.

Implementation sketch:

type LaneState = {
  queue: QueueEntry[];
  active: number;           // Currently running tasks
  maxConcurrent: number;    // Concurrency limit
  draining: boolean;
};

const lanes = new Map<string, LaneState>();

function drainLane(lane: string) {
  const state = getLaneState(lane);
  // Pump tasks until concurrency limit reached
  while (state.active < state.maxConcurrent && state.queue.length > 0) {
    const entry = state.queue.shift();
    state.active += 1;
    entry.task().finally(() => {
      state.active -= 1;
      pump();  // Drain next entry
    });
  }
}

function enqueueCommandInLane<T>(
  lane: string,
  task: () => Promise<T>,
): Promise<T> {
  return new Promise((resolve, reject) => {
    getLaneState(lane).queue.push({ task, resolve, reject });
    drainLane(lane);
  });
}

Deadlock prevention via hierarchical composition:

// Session lane queues a task that itself queues to global lane
await enqueueCommandInLane(sessionLane, () =>
  enqueueCommandInLane(globalLane, () =>
    doBackgroundWork()
  )
);
// Outer promise resolves when inner completes; no circular wait

Lane examples from OpenClaw:

• main: Inbound interactive turns; concurrency defaults to max(8, available CPU parallelism × 4)
• system-agent and system-agent-inference: Internal agent work, isolated from user traffic
• cron and cron-nested: Scheduled turns; an isolated cron turn holds a cron slot while its inner agent execution uses cron-nested
• hook-dispatch: External hook agent-run dispatch, in its own lane rather than the cron budget
• background, active-memory, nested: Housekeeping work that must not block user lanes
• subagent:<session>: Per-session spawned-agent queues (default 8 per spawning session); subagent:swarm:<group> collectors default to 32
• session:<key>: Per-session serial queues shared by CLI, embedded, and CLI-backend runs

1. Identify isolation boundaries: Group operations that must not interleave (e.g., per-user, per-channel).
2. Define lane names: Use a hierarchy (e.g., session:telegram:user123) for scoping.
3. Set concurrency limits: Serial lanes use maxConcurrent=1; parallel lanes use higher values.
4. Queue tasks: Call enqueueCommandInLane(lane, task) to schedule work.
5. Compose hierarchically: When a queued task must spawn work in another lane, await the inner enqueue from the outer task.

Observability: Track per-lane metrics (queue depth, active count, wait times) to detect backpressure and starvation. Key signals include queue_size_per_lane, active_tasks_per_lane, and wait_time_p95.

Pitfalls to avoid:

• Over-parallelization: Too many concurrent workers can exhaust resources (file handles, memory). Monitor active count.
• Starvation: Low-priority lanes may wait indefinitely if high-priority lanes are always full. Use wait-time warnings to detect.
• Missing hierarchy: Direct cross-lane dependencies without nested queuing risk deadlocks. Always compose via enqueueCommandInLane(() => enqueueCommandInLane(...)).
• Dynamic lane proliferation: Creating lanes with unstable identifiers (e.g., timestamps) causes unbounded memory growth. Use stable names and implement lane garbage collection for session-scoped lanes.

Pros:

• Isolation guarantees: No interleaving between lanes; each lane preserves ordering.
• Flexible parallelism: Concurrency per lane allows mixed workloads (serial UI, parallel background).
• Simple mental model: Hierarchical composition maps to structured programming patterns.
• Observability: Lane-level metrics (queue depth, active count, wait times) aid debugging.

Cons/Considerations:

• Memory overhead: Each lane maintains a queue; thousands of idle lanes may waste memory.
• Tuning required: Concurrency limits need calibration based on workload characteristics.
• Not a general scheduler: No priority queues, deadlines, or work stealing. Use a proper scheduler for complex needs.

• OpenClaw command-queue.ts - Core queue implementation
• OpenClaw lanes.ts - Lane enum: main, system-agent, cron, cron-nested, hook-dispatch, background, subagent, active-memory, nested
• OpenClaw queue documentation - Lane concurrency defaults and session lanes
• Related: Parallel Tool Execution for tool-level parallelism
• Academic foundations: Actor Model (IJCAI '73), Work-Stealing (SOSP '95), CALM Theorem (PODS '11)
• Industry analogs: Sidekiq queues, BullMQ isolation, Airflow pools