Background Jobs in Node.js with BullMQ and Redis

Sending an email, crawling a site, generating a report — anything that takes more than a moment shouldn’t block an HTTP response. The fix is a queue: the API enqueues a job and returns immediately, and a separate worker processes it. I reach for BullMQ on Redis.

A queue and a worker

Producers add jobs to a named queue; a worker (often a separate process) pulls and runs them. Keeping the worker separate means you can scale and restart it independently of the API.

import { Queue, Worker } from 'bullmq'

const connection = { host: '127.0.0.1', port: 6379 }

// Producer (in your API)
export const emailQueue = new Queue('email', { connection })
await emailQueue.add(
  'welcome',
  { userId },
  { attempts: 3, backoff: { type: 'exponential', delay: 2000 } },
)

// Consumer (in your worker process)
new Worker(
  'email',
  async (job) => {
    await sendWelcomeEmail(job.data.userId)
  },
  { connection, concurrency: 5 },
)

Make jobs safe to retry

Jobs fail — networks blip, third parties rate-limit. BullMQ retries with backoff, which means your handler must be idempotent: running it twice should not send two emails or double-charge anyone. Guard with a key (the job id, or a dedupe record) before doing the side effect.

• Idempotency: check "already done?" before the side effect.
• Backoff: exponential, so retries don’t hammer a struggling dependency.
• Dead-letter: after max attempts, keep the failed job for inspection.

Watch what’s happening

A queue you can’t see is a liability. I expose a dashboard (Bull Board) behind auth to watch waiting/active/failed counts, and add a scheduled "cron" worker for recurring jobs. When the failed count climbs, that’s the alert.

The payoff: a fast API, work that survives restarts and retries cleanly, and a system you can scale by simply adding more workers.