Duplicate “Enqueued” log entries for recurring jobs with multiple Solid Queue processes

[gdelbo-safe]

We’re seeing duplicate “Enqueued” log entries for recurring jobs, where two separate job records appear to be created. The logs show different Job IDs, but only one of them actually executes (we see only one “Performing” and “Performed” entry).

It looks like ActiveJob logs both enqueue attempts, but Solid Queue enforces uniqueness at the DB level so that only one job runs. I’m wondering if this is the expected behavior, or if it indicates a race condition when multiple Solid Queue processes are scheduling recurring jobs.

Environment

• Solid Queue version: 1.2.1
• Rails version: 8.0.2.1
• Ruby version: 3.3.8
• Deployment: Kubernetes with 2 Solid Queue containers
• Solid Queue is running as separate processes (not as Puma plugin)

Recurring Job Config (config/recurring.yml)

collect_metrics: &collect_metrics
  collect_solid_queue_metrics:
    class: QueueMetricsJob
    schedule: every minute

production:
  <<: *collect_metrics

Sample logs

[ActiveJob] Enqueued QueueMetricsJob (Job ID: aaa111) to SolidQueue(default)
[ActiveJob] Enqueued QueueMetricsJob (Job ID: bbb222) to SolidQueue(default)
[ActiveJob] [QueueMetricsJob] [aaa111] Performing QueueMetricsJob ...
[ActiveJob] [QueueMetricsJob] [aaa111] Performed QueueMetricsJob ...
// No execution logs for job bbb222

Database Schema

We have the standard Solid Queue tables included in the app's database, including:

• solid_queue_recurring_executions with a unique constraint on (task_key, run_at)

Suspected Behavior

From what we can tell, when multiple Solid Queue processes are running in separate containers, both may detect the same job as ready to be scheduled at the same time. As a result, they each attempt to enqueue it, which leads to two different job records being created. In the database we don’t actually see duplicate jobs, but we do notice that job IDs skip a number. Our assumption is that one of the jobs gets created and then removed due to the uniqueness constraint, which explains why only one job record remains while the logs show both enqueue attempts. Since the logging appears to come from ActiveJob, both enqueue attempts are still written to the logs even though only one job ends up being executed.

• Is this the expected behavior when running multiple Solid Queue processes with a scheduler?
• Is there a known race condition in recurring job scheduling where multiple schedulers may attempt to enqueue the same job?
• What’s the recommended deployment pattern to avoid duplicate enqueue logs in this scenario?

Thanks a lot for your time!