Troubleshooting

Common errors and solutions when using BullMQ

This section provides hints and solutions for common errors you might encounter when using BullMQ.

Missing Locks

Error Message

Missing lock for job 1234. moveToFinished.

What It Means

This error occurs when a job being processed by a worker unexpectedly loses its “lock.”

When a worker processes a job, it requires a special lock key to ensure the job is currently “owned” by that worker. This prevents other workers from picking up the same job.

However, this lock can be deleted, and the deletion may not be detected until the worker tries to move the job to completed or failed status.

Common Causes

High CPU Usage

Problem: The worker is consuming too much CPU and has no time to renew the lock every 30 seconds (default expiration time).

Solution:

• Reduce worker concurrency
• Optimize job processing code
• Move CPU-intensive work to separate processes
• Monitor worker CPU usage

const worker = new Worker(
  'myqueue',
  processor,
  {
    concurrency: 5, // Reduce if CPU usage is high
  }
);

Redis Connection Lost

Problem: The worker lost communication with Redis and cannot renew the lock in time.

Solution:

• Check network stability between worker and Redis
• Implement proper connection retry logic
• Monitor Redis connection health
• Consider increasing lock renewal interval

const worker = new Worker(
  'myqueue',
  processor,
  {
    lockDuration: 60000, // Increase to 60 seconds
    connection: {
      retryStrategy: (times) => {
        return Math.min(times * 1000, 20000);
      },
    },
  }
);

Job Forcefully Removed

Problem: The job was forcefully removed using BullMQ’s APIs or by removing the entire queue.

Solution:

• Avoid removing jobs that are currently being processed
• Use proper job state checks before removal
• Implement graceful job cancellation

// Check job state before removing
const job = await queue.getJob(jobId);
const state = await job.getState();

if (state !== 'active') {
  await job.remove();
}

Wrong maxmemory-policy

Problem: Redis instance has wrong maxmemory-policy setting. It should be noeviction to avoid Redis removing lock keys before expiration.

Solution:

• Set maxmemory-policy to noeviction in Redis config
• See Going to Production for details

# In redis.conf
maxmemory-policy noeviction

This is a critical configuration. Without noeviction, Redis may remove lock keys, causing jobs to lose their locks.

Invalid or Undefined Environment Variables

Error Message

ERR Error running script ... Lua redis() command arguments must be strings or integers

What It Means

This error typically happens when a parameter passed into a Redis command ends up being something other than a valid string or number.

Common Causes

If you rely on environment variables (e.g., for queue names or job data), this error can occur when those variables are:

• Undefined (not set at all)
• Empty strings (i.e., "")
• Non-string values (e.g., objects or arrays)

Solutions

1. Validate Environment Variables Early

   Check all required environment variables during initialization:

   const queueName = process.env.QUEUE_NAME;
   
   if (!queueName) {
     throw new Error('QUEUE_NAME is not defined or is empty.');
   }
   
   const queue = new Queue(queueName, { connection });
   

   This ensures you fail fast if a variable isn’t set, instead of causing hidden Lua script errors.

2. Use TypeScript Strictness

   Enable strictNullChecks and explicitly type environment variables:

   // types/env.d.ts
   declare global {
     namespace NodeJS {
       interface ProcessEnv {
         QUEUE_NAME: string;
         REDIS_HOST: string;
         REDIS_PORT: string;
       }
     }
   }
   
   export {};
   

   // Compile-time error if not defined
   const queueName: string = process.env.QUEUE_NAME;
   
3. Provide Defaults Where Appropriate

   Use fallback values when appropriate:

   const queueName = process.env.QUEUE_NAME ?? 'defaultQueue';
   const redisPort = parseInt(process.env.REDIS_PORT ?? '6379', 10);
   

   Be sure fallback values are actually valid in your production workflow.

Example: Robust Configuration

import { Queue } from 'bullmq';

// Validation function
function requireEnv(name: string): string {
  const value = process.env[name];
  
  if (!value || value.trim() === '') {
    throw new Error(`Environment variable ${name} is required but not set`);
  }
  
  return value;
}

// Safe configuration
const queueName = requireEnv('QUEUE_NAME');
const redisHost = requireEnv('REDIS_HOST');
const redisPort = parseInt(requireEnv('REDIS_PORT'), 10);

const queue = new Queue(queueName, {
  connection: {
    host: redisHost,
    port: redisPort,
  },
});

Job Stuck in Active State

Symptoms

• Jobs remain in “active” state indefinitely
• Jobs are not being processed
• No error messages in logs

Solutions

Worker Not Running

Ensure workers are running and connected:

worker.on('ready', () => {
  console.log('Worker is ready and connected');
});

worker.on('error', (err) => {
  console.error('Worker error:', err);
});

Job Processor Hanging

Make sure job processors complete or throw errors:

// Bad - hanging promise
const worker = new Worker('myqueue', async (job) => {
  someAsyncFunction(); // Missing await!
});

// Good - proper async handling
const worker = new Worker('myqueue', async (job) => {
  await someAsyncFunction();
});

Stalled Job Check

Jobs should automatically become stalled after 30 seconds. Check stalled jobs:

const stalledJobs = await queue.getJobs(['stalled']);
console.log('Stalled jobs:', stalledJobs.length);

Memory Issues

Symptoms

• Redis running out of memory
• Jobs disappearing
• “OOM command not allowed” errors

Solutions

1. Configure maxmemory-policy

   # In redis.conf
   maxmemory-policy noeviction
   
2. Enable job auto-removal

   const queue = new Queue('myqueue', {
     defaultJobOptions: {
       removeOnComplete: 100,
       removeOnFail: 1000,
     },
   });
   
3. Increase Redis memory

   # In redis.conf
   maxmemory 2gb
   
4. Monitor job data size

   Keep job data small - store large data elsewhere:

   // Bad - large data in job
   await queue.add('process', {
     largeFile: Buffer.from(...), // Don't do this!
   });
   
   // Good - reference to external storage
   await queue.add('process', {
     fileUrl: 's3://bucket/file.txt',
   });
   

Connection Issues

Symptoms

• “ECONNREFUSED” errors
• Workers not picking up jobs
• Intermittent failures

Solutions

Check Redis Connection

import { createClient } from 'redis';

const client = createClient({
  url: 'redis://localhost:6379',
});

client.on('error', (err) => {
  console.error('Redis Client Error', err);
});

await client.connect();
await client.ping();
console.log('Redis connected successfully');

Configure Retry Strategy

const worker = new Worker('myqueue', processor, {
  connection: {
    host: 'localhost',
    port: 6379,
    retryStrategy: (times) => {
      if (times > 10) {
        return null; // Stop retrying
      }
      return Math.min(times * 1000, 20000);
    },
  },
});

Check Network Connectivity

# Test Redis connection
redis-cli -h localhost -p 6379 ping

# Check if Redis is listening
netstat -an | grep 6379

# Test from application server
telnet redis-host 6379

Performance Issues

Jobs Processing Slowly

Increase Concurrency

const worker = new Worker(
  'myqueue',
  processor,
  { concurrency: 10 }
);

Add More Workers

Scale horizontally by running multiple worker instances

Optimize Job Processing

Profile and optimize slow job processors

Use Redis Cluster

Distribute load across multiple Redis nodes

Debugging Tips

1. Enable debug logging

   import { Queue, Worker } from 'bullmq';
   
   const queue = new Queue('myqueue', {
     connection: {
       // Add logging
       lazyConnect: false,
     },
   });
   
   queue.on('error', (err) => console.error('Queue error:', err));
   queue.on('waiting', (job) => console.log('Job waiting:', job.id));
   
   const worker = new Worker('myqueue', processor);
   
   worker.on('active', (job) => console.log('Job active:', job.id));
   worker.on('completed', (job) => console.log('Job completed:', job.id));
   worker.on('failed', (job, err) => console.log('Job failed:', job.id, err));
   
2. Check job state

   const job = await queue.getJob(jobId);
   
   if (job) {
     console.log('Job state:', await job.getState());
     console.log('Job data:', job.data);
     console.log('Job stacktrace:', job.stacktrace);
     console.log('Job attempts:', job.attemptsMade);
   }
   
3. Inspect Redis directly

   # Connect to Redis
   redis-cli
   
   # List all BullMQ keys
   KEYS bull:myqueue:*
   
   # Check queue length
   LLEN bull:myqueue:wait
   
   # Get job data
   HGETALL bull:myqueue:1234
   

Related Resources

Going to Production

Production deployment best practices

Redis Configuration

Redis compatibility and configuration