Migrating to Newer Versions

Strategies for upgrading BullMQ to newer versions in production

BullMQ’s team regularly releases new versions with bug fixes, new features, and occasionally breaking changes. This guide helps you upgrade safely while maintaining service continuity in production.

Always consult the Changelog before upgrading to understand the extent of changes and any special considerations.

Upgrade Strategy by Type

The approach you take depends on the type of upgrade:

Bugfix Upgrades

Version Pattern: Micro version increase (e.g., 3.14.4 → 3.14.7)

Strategy: Simple update, no special considerations

1. Update your package

   npm install bullmq@latest
   
2. Deploy to all instances

   While not critical that all instances run the same version, we recommend it for consistency.

Bugfix releases follow Semantic Versioning and contain no breaking changes or new features.

New Feature Upgrades

Version Pattern: Minor version increase (e.g., 3.14.7 → 3.20.5)

Strategy: Update all instances, then deploy code using new features

1. Update BullMQ on all instances

   npm install bullmq@3.20.5
   
2. Deploy updated version

   Ensure all Queue and Worker instances run the new version before using new features.

3. Verify all instances are updated

   Check that all services are running the new version.

4. Deploy code using new features

   Only after confirming all instances are updated, deploy code that depends on the new functionality.

If you deploy code using new features before all instances are updated, older Workers might fail when encountering jobs that use features they don’t understand.

Breaking Changes

Version Pattern: Major version increase (e.g., 3.x.x → 4.0.0)

Strategy: Depends on the type of breaking change

Breaking changes fall into two categories:

API Breaking Changes

These involve:

• Altered method parameters
• Removed methods
• Different operational patterns

How to handle:

1. Review the changelog

   Read the changelog for all API changes.

2. Update your code

   Modify your code to match the new API.

   // Example: API change
   // Before v4
   await queue.add('job', data, { removeOnComplete: true });
   
   // After v4 (hypothetical)
   await queue.add('job', data, { 
     remove: { onComplete: true } 
   });
   
3. Run tests

   Run your BullMQ-dependent unit tests to catch issues.

4. Fix TypeScript errors

   If using TypeScript, compilation errors will surface.

5. Deploy

   Deploy the updated code and BullMQ version together.

Data Structure Breaking Changes

These alter the underlying queue structure in Redis and are more challenging.

Types:

Additive Changes

Introduce new data structures that older versions don’t understand.

Strategy: Similar to new feature upgrades

1. Upgrade all instances to the new version
2. The new structures will be created automatically
3. Older versions ignore structures they don’t understand

Destructive Changes

Change or remove existing data structures.

Strategy: Requires careful planning

Destructive changes can make rollback impossible if the upgrade fails. Always have a backup strategy.

See Advanced Strategies below for handling destructive changes.

General Upgrade Advice

Avoid large version jumps. Upgrade incrementally whenever possible.

Recommended Upgrade Path

If you’re on version 1.3.7 and want to reach 4.2.6:

1. Apply bugfix releases

   npm install bullmq@1.3.12
   

   Upgrade through all bugfix releases first (e.g., 1.3.7 → 1.3.12).

2. Move to latest minor version

   npm install bullmq@1.20.5
   

   Proceed to new feature releases (e.g., 1.3.12 → 1.20.5).

3. Upgrade major versions incrementally

   npm install bullmq@2.0.0
   npm install bullmq@3.0.0
   npm install bullmq@4.2.6
   

   Only then tackle major releases that include breaking changes.

Advanced Strategies

For challenging upgrades with destructive breaking changes:

Strategy 1: Pause/Upgrade/Unpause

Suitable when your business can tolerate a brief pause:

1. Pause all queues

   import { Queue } from 'bullmq';
   
   const queue = new Queue('myqueue');
   await queue.pause();
   
2. Wait for active jobs to complete

   let activeCount = await queue.getActiveCount();
   
   while (activeCount > 0) {
     console.log(`Waiting for ${activeCount} active jobs...`);
     await new Promise(resolve => setTimeout(resolve, 5000));
     activeCount = await queue.getActiveCount();
   }
   
   console.log('All jobs completed');
   
3. Perform the upgrade

   Deploy the new BullMQ version to all instances.

4. Unpause queues

   await queue.resume();
   

This strategy is less useful if breaking changes affect Queue instances. Always consult the changelog.

Strategy 2: Use New Queues

The most drastic but safest solution:

1. Create new queues with different names

   // Option 1: Different queue name
   const oldQueue = new Queue('myqueue');
   const newQueue = new Queue('myqueue-v2');
   
   // Option 2: Different prefix
   const newQueue = new Queue('myqueue', {
     prefix: 'bull-v2',
   });
   
   // Option 3: Different Redis host
   const newQueue = new Queue('myqueue', {
     connection: {
       host: 'new-redis.example.com',
       port: 6379,
     },
   });
   
2. Run both versions in parallel

   Maintain two versions of your service:

   • Old service: Older BullMQ version with old queues
   • New service: Latest BullMQ version with new queues

   // Old service
   import { Worker as OldWorker } from 'bullmq-v3';
   
   const oldWorker = new OldWorker('myqueue', processor, {
     connection: oldRedisConfig,
   });
   
   // New service
   import { Worker } from 'bullmq';
   
   const newWorker = new Worker('myqueue-v2', processor, {
     connection: newRedisConfig,
   });
   
3. Direct new jobs to new queues

   Update your producers to add jobs to the new queues:

   // Instead of:
   await oldQueue.add('job', data);
   
   // Use:
   await newQueue.add('job', data);
   
4. Monitor old queues until drained

   const checkOldQueues = async () => {
     const waiting = await oldQueue.getWaitingCount();
     const active = await oldQueue.getActiveCount();
     const delayed = await oldQueue.getDelayedCount();
     
     console.log('Old queue status:', { waiting, active, delayed });
     
     return waiting === 0 && active === 0 && delayed === 0;
   };
   
   // Check periodically
   const interval = setInterval(async () => {
     if (await checkOldQueues()) {
       console.log('Old queues drained - safe to remove old service');
       clearInterval(interval);
     }
   }, 60000);
   
5. Retire old version

   When the old queues have no more jobs to process, retire the old version.

Testing Upgrades

1. Test in staging first

   Always test upgrades in a non-production environment before deploying to production.

2. Create a test suite

   import { Queue, Worker } from 'bullmq';
   import { describe, it, expect } from 'vitest';
   
   describe('BullMQ Upgrade Tests', () => {
     it('should process jobs after upgrade', async () => {
       const queue = new Queue('test-queue');
       const results: any[] = [];
       
       const worker = new Worker(
         'test-queue',
         async (job) => {
           results.push(job.data);
         }
       );
       
       await queue.add('test', { value: 1 });
       await queue.add('test', { value: 2 });
       
       // Wait for processing
       await new Promise(resolve => setTimeout(resolve, 1000));
       
       expect(results).toHaveLength(2);
       expect(results[0].value).toBe(1);
       expect(results[1].value).toBe(2);
       
       await worker.close();
       await queue.close();
     });
   });
   
3. Monitor key metrics

   Watch these metrics during upgrade:

   • Job completion rate
   • Error rate
   • Queue length
   • Stalled jobs
   • Memory usage

Rollback Plan

Always have a rollback plan:

1. Take a Redis snapshot

   redis-cli BGSAVE
   
2. Document the current version

   npm list bullmq
   
3. Prepare rollback commands

   # Rollback package version
   npm install bullmq@3.14.7
   
   # Redeploy previous version
   git revert HEAD
   # Deploy...
   
4. Test rollback procedure

   Test the rollback in staging before production upgrade.

Upgrade Checklist

Pre-Upgrade

• Read the changelog
• Identify breaking changes
• Test upgrade in staging environment
• Take Redis snapshot
• Document current version
• Prepare rollback plan
• Schedule maintenance window if needed

During Upgrade

• Update BullMQ package
• Update code for API changes
• Run tests
• Deploy to staging
• Monitor staging metrics
• Deploy to production
• Monitor production metrics closely

Post-Upgrade

• Verify job processing is normal
• Check error rates
• Monitor queue lengths
• Review logs for warnings
• Document any issues
• Update documentation

Related Resources

Bull to BullMQ

Migrating from Bull to BullMQ

Migration Overview

General migration guidance

Going to Production

Production deployment best practices

Troubleshooting

Common issues and solutions

External Resources

BullMQ Changelog

Official changelog with all version changes