Updated October 2026: Updated the setup for Sidekiq 8 (Sidekiq::Job, JSON-only arguments, a protected Web UI), corrected the reliability claims, replaced the archived sidekiq-pool section and added a real tuning section.

In today’s web development landscape, handling background job processing efficiently has become crucial for building robust and scalable applications. Sidekiq stands out as a popular choice among developers due to its simplicity, performance, and powerful features.

I. What is Sidekiq?

Sidekiq is a highly efficient background job processing library for Ruby. It works in any Ruby application, though it is most commonly used with Ruby on Rails. It stores jobs in Redis and runs them on a pool of threads inside a separate process, so developers can offload resource-intensive tasks, such as sending emails, processing large datasets, or performing periodic maintenance, and keep web requests fast.

Sidekiq 8 (the current major version) requires Ruby 3.2+ and Redis 7.0+ (Valkey 7.2+ and Dragonfly 1.27+ also work), and supports Rails and Active Job 7.0+.

II. Why Use Sidekiq?

  1. Asynchronous Task Execution: Sidekiq enables developers to execute time-consuming tasks asynchronously, improving the responsiveness and scalability of web applications. By processing background jobs outside the main request-response cycle, Sidekiq ensures a smoother user experience.

  2. Scalability and Performance: One Sidekiq process runs many jobs concurrently on threads, which uses far less memory than one process per job. To handle more load, you add processes (and machines), all pulling from the same Redis queues.

  3. Retries and Error Handling: A job that raises an error is retried automatically with exponential backoff, and jobs that exhaust their retries are kept in the Dead set where you can inspect and re-run them. This is not the same as “no job is ever lost”, though:

    • Sidekiq executes a job at least once, not exactly once — a job can run again even after it finished (for example, if Redis goes away before Sidekiq acknowledges it). Jobs must be idempotent.
    • The open-source version removes a job from Redis when it fetches it. If the process crashes or is killed (e.g. by the OOM killer) while the job is running, that job is lost. On a normal restart Sidekiq tries to push unfinished jobs back to Redis, but a network problem can still prevent it. Sidekiq Pro’s super_fetch keeps in-progress jobs in Redis and recovers orphaned ones after a crash.

  4. Real-time Monitoring: Sidekiq comes with a web interface that shows queues, busy workers, scheduled jobs, retries and dead jobs, and lets you retry or delete them.

III. Installation and Setup in Rails

To integrate Sidekiq into a Ruby on Rails application, follow these steps:

  1. Add Sidekiq to your Gemfile:

    bundle add sidekiq

  2. Configure Redis: Ensure that Redis is installed and running. By default Sidekiq connects to localhost:6379; the easiest way to point it elsewhere is the REDIS_URL environment variable, which Sidekiq picks up automatically. If you configure it in code instead (usually config/initializers/sidekiq.rb), you must define both blocks:

    Sidekiq.configure_server do |config|
  config.redis = { url: ENV.fetch("REDIS_URL", "redis://localhost:6379/0") }
end

Sidekiq.configure_client do |config|
  config.redis = { url: ENV.fetch("REDIS_URL", "redis://localhost:6379/0") }
end

    Set maxmemory-policy noeviction on the Redis server Sidekiq uses, so Redis never silently drops job data under memory pressure.

  3. Start Sidekiq: Run the Sidekiq server in your Rails application directory:

    bundle exec sidekiq

  4. Create Job Classes: Sidekiq’s own docs put job classes in app/sidekiq (the generator bin/rails generate sidekiq:job hard creates the file and its test). Include Sidekiq::Job — it was added in Sidekiq 6.3 as the new name for Sidekiq::Worker:

    # app/sidekiq/hard_job.rb
class HardJob
  include Sidekiq::Job

  def perform(user_id, count)
    user = User.find(user_id)
    # Background job logic goes here
  end
end

  5. Enqueue Jobs: perform is an instance method, while perform_async, perform_in and perform_at are called on the class:

    HardJob.perform_async(user.id, 5)
HardJob.perform_in(5.minutes, user.id, 5)

    Arguments are serialized to JSON, so pass simple, JSON-native values (strings, integers, floats, booleans, nil, arrays and hashes with string keys) — pass user.id, not user. Since Sidekiq 7.0, passing anything else (a model, a Symbol, a Time) raises an error by default instead of only logging a warning.

  6. Mount and Protect the Web Interface: Mount Sidekiq::Web inside the routes.draw block so it reuses the Rails session, and never expose it without authentication. With Devise:

    # config/routes.rb
require "sidekiq/web"

Rails.application.routes.draw do
  authenticate :user, ->(user) { user.admin? } do
    mount Sidekiq::Web => "/sidekiq"
  end
end

    Without Devise, HTTP Basic auth also works:

    # config/routes.rb
require "sidekiq/web"

Sidekiq::Web.use Rack::Auth::Basic do |username, password|
  # Compare digests so the check doesn't leak length; use & (not &&) so it doesn't short circuit.
  ActiveSupport::SecurityUtils.secure_compare(
    ::Digest::SHA256.hexdigest(username), ::Digest::SHA256.hexdigest(ENV["SIDEKIQ_USERNAME"])
  ) & ActiveSupport::SecurityUtils.secure_compare(
    ::Digest::SHA256.hexdigest(password), ::Digest::SHA256.hexdigest(ENV["SIDEKIQ_PASSWORD"])
  )
end

Rails.application.routes.draw do
  mount Sidekiq::Web => "/sidekiq"
end

Note: Sidekiq can also be used as the backend of Active Job, Rails’ built-in job framework, by setting config.active_job.queue_adapter = :sidekiq in config/application.rb. Active Job adds some overhead compared with the native Sidekiq::Job API, and Active Job’s retry_on runs before Sidekiq’s own retries. For details, refer to the Sidekiq Active Job documentation.

IV. Tuning Sidekiq

1. Concurrency vs the database connection pool

By default one Sidekiq process runs 5 threads. Each thread that touches the database needs its own Active Record connection, so the pool in config/database.yml (max_connections, called pool before Rails 8.1) must be at least the concurrency. The RAILS_MAX_THREADS environment variable configures both:

# config/database.yml
default: &default
  max_connections: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

RAILS_MAX_THREADS=10 bundle exec sidekiq

More threads only help while jobs are waiting on I/O (HTTP calls, the database). For CPU-heavy jobs, run more processes instead of raising concurrency.

2. Queues, weights and priorities

Split work into a few queues and give important ones a higher weight in config/sidekiq.yml. A queue with weight 2 is checked twice as often as a queue with weight 1:

# config/sidekiq.yml
:concurrency: 5
:queues:
  - [critical, 2]
  - [default, 1]
  - [low, 1]

class ChargeCardJob
  include Sidekiq::Job
  sidekiq_options queue: "critical", retry: 5
end

If you list queues without weights, they are processed in strict order: a job in default only runs when critical is empty — which can starve the lower queues.

3. Idempotent, small jobs

Because a job can run more than once, design it so that a second run is harmless: look up the current state in perform (that’s another reason to pass ids), wrap multi-step writes in a database transaction, and check “already done?” before sending an email or charging a card.

4. Retries and the Dead set

By default a failing job is retried 25 times over roughly 20 days with exponential backoff, then moved to the Dead set, which keeps up to 10,000 jobs for 6 months. Lower retry for jobs that are pointless after a few hours, and use sidekiq_retries_exhausted to react when a job gives up:

class SyncInvoiceJob
  include Sidekiq::Job
  sidekiq_options retry: 3

  sidekiq_retries_exhausted do |job, exception|
    Sidekiq.logger.warn "Giving up on #{job["class"]} #{job["args"]}: #{exception.message}"
  end

  def perform(invoice_id)
    # ...
  end
end

5. Memory

Sidekiq processes are long-lived and multi-threaded, so memory bloat shows up there first:

  • On Linux/glibc, set MALLOC_ARENA_MAX=2, or run Ruby with jemalloc — the Sidekiq wiki recommends both.
  • Don’t load huge result sets into memory inside a job; process records in batches with find_each / find_in_batches, or fan the work out into many small jobs.
  • A healthy process’s memory levels off at a steady state; memory that never stops growing points to a leak worth investigating.

V. Scaling Beyond One Process

Earlier versions of this article recommended the sidekiq-pool gem for running several Sidekiq processes on one machine. The sidekiq-pool repository was archived in June 2023 and its README now says it is no longer maintained, so it’s best avoided. Instead:

  • Run multiple processes: a Sidekiq process is limited by Ruby’s GVL to roughly one CPU core, so start several bundle exec sidekiq processes per machine (via systemd, Foreman, containers, etc.), all pointing to the same Redis.

  • Use capsules for isolation: since Sidekiq 7.0 a single process can define capsules, each with its own queues and concurrency. This is handy for jobs that must run one at a time:

    # config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
  config.capsule("unsafe") do |cap|
    cap.concurrency = 1
    cap.queues = %w[unsafe]
  end
end

    Don’t declare a capsule for every queue — use them only when a group of queues genuinely needs different concurrency.

VI. Conclusion

Sidekiq is a versatile and powerful background job processing solution for Ruby and Rails. Use Sidekiq::Job with simple JSON arguments, write idempotent jobs, size the database pool to match concurrency, prioritize queues with weights, protect the Web UI, and scale out with more processes when one is not enough. Know its limits too: open-source Sidekiq is at-least-once and can lose an in-flight job on a crash, so critical work needs idempotency and, if necessary, Sidekiq Pro’s reliable fetch.