Job Schedulers Overview

Learn how to create and manage repeatable jobs using Job Schedulers in BullMQ

A Job Scheduler acts as a factory, producing jobs based on specified “repeat” settings. The Job Scheduler is highly flexible, accommodating various scenarios, including jobs produced at fixed intervals, according to cron expressions, or based on custom requirements.

Job Schedulers replace “repeatable jobs” and are available in v5.16.0 and onwards. They are the recommended approach for creating recurring jobs in BullMQ.

Creating a Job Scheduler

To create a scheduler, use the upsertJobScheduler method:

import { Queue } from 'bullmq';

const queue = new Queue('myQueue', {
  connection: { host: 'localhost', port: 6379 },
});

// Creates a new Job Scheduler that generates a job every 1000 milliseconds (1 second)
const firstJob = await queue.upsertJobScheduler('my-scheduler-id', {
  every: 1000,
});

This example creates a new Job Scheduler that produces a new job every second. It also returns the first job created for this Job Scheduler, which will be in “delayed” status waiting to be processed after 1 second.

Key Concepts

Upsert vs. Add

The upsert operation (update or insert) is used instead of add to simplify management of recurring jobs, especially in production deployments. It ensures the scheduler is updated or created without duplications.

// First call creates the scheduler
await queue.upsertJobScheduler('my-scheduler-id', {
  every: 1000,
});

// Subsequent calls update the existing scheduler
await queue.upsertJobScheduler('my-scheduler-id', {
  every: 2000, // Updated interval
});

Job Production Rate

The scheduler will only generate new jobs when the last job begins processing. Therefore:

• If your queue is very busy
• If you don’t have enough workers or concurrency
• Jobs may be produced less frequently than the specified repetition interval

The actual job execution frequency depends on worker availability and processing speed. The scheduler guarantees jobs won’t be created more often than specified, but they may be created less frequently if the queue is congested.

Job Status

As long as a Job Scheduler is producing jobs, there will always be one job associated with the scheduler in the “Delayed” status, waiting to be processed.

Using Job Templates

You can define a template with standard names, data, and options for jobs added to a queue. This ensures that all jobs produced by the Job Scheduler inherit these settings:

// Create jobs every day at 3:15 AM
const firstJob = await queue.upsertJobScheduler(
  'my-scheduler-id',
  { pattern: '0 15 3 * * *' },
  {
    name: 'my-job-name',
    data: { foo: 'bar' },
    opts: {
      backoff: 3,
      attempts: 5,
      removeOnFail: 1000,
    },
  },
);

All jobs produced by this scheduler will use the given settings. You can call upsertJobScheduler again with the same scheduler ID to update any settings of this particular job scheduler, such as the repeat options or job template settings.

Template Options

name · string

Job name for all produced jobs

data · object

Default data object passed to every job

opts · JobOptions

Default job options (attempts, backoff, priority, etc.)

Job IDs

Since jobs produced by the Job Scheduler get a special job ID to guarantee that jobs will never be created more often than the given repeat settings, you cannot choose a custom job ID. However, you can use the job’s name to discriminate these jobs from other jobs.

The job ID format is: repeat:{jobSchedulerId}:{nextMillis}

Complete Example

import { Queue, Worker } from 'bullmq';

const connection = {
  host: 'localhost',
  port: 6379,
};

const queue = new Queue('notifications', { connection });

// Create a scheduler that sends daily reports at 9 AM
await queue.upsertJobScheduler(
  'daily-report-scheduler',
  {
    pattern: '0 0 9 * * *', // 9 AM every day
    tz: 'America/New_York',
  },
  {
    name: 'daily-report',
    data: {
      reportType: 'summary',
      recipients: ['[e-mail]'],
    },
    opts: {
      attempts: 3,
      backoff: {
        type: 'exponential',
        delay: 5000,
      },
      removeOnComplete: 100,
    },
  },
);

// Worker to process the scheduled jobs
const worker = new Worker(
  'notifications',
  async (job) => {
    console.log(`Processing ${job.name} with data:`, job.data);
    // Send report logic here
    return { sent: true, timestamp: Date.now() };
  },
  { connection },
);

Best Practices

1. Use descriptive scheduler IDs

   Choose meaningful IDs that describe the scheduler’s purpose:

   await queue.upsertJobScheduler('daily-backup', { ... });
   
2. Set appropriate job options

   Configure attempts, backoff, and auto-removal for scheduled jobs:

   await queue.upsertJobScheduler(
     'scheduler-id',
     { every: 60000 },
     {
       opts: {
         attempts: 3,
         backoff: { type: 'exponential', delay: 5000 },
         removeOnComplete: { count: 1000 },
       },
     },
   );
   
3. Consider timezone for cron patterns

   Always specify a timezone when using cron patterns:

   await queue.upsertJobScheduler(
     'scheduler-id',
     {
       pattern: '0 9 * * *',
       tz: 'America/New_York',
     },
   );
   
4. Clean up unused schedulers

   Remove schedulers that are no longer needed:

   await queue.removeJobScheduler('old-scheduler-id');
   

Related Topics

Repeat Strategies

Learn about every, cron, and custom repeat strategies

Repeat Options

Configure start dates, end dates, limits, and more

Management

Manage, retrieve, and remove job schedulers

RepeatableOptions

API reference for repeatable options

API Reference

• Queue.upsertJobScheduler API Reference
• JobScheduler Class