Updated October 2026: Corrected what pgBadger and pg_stat_statements actually measure and output, removed overstated Fail2ban integrations, and added OpenTelemetry/Sentry tracing, postgres_exporter and systemd Restart=.

Monitoring tools play a crucial role in ensuring the health, performance, and reliability of web systems, including those built with Rails. These tools provide insights into various aspects of system behavior, such as resource usage, response times, errors, and overall system health. Let’s explore some popular monitoring tools used in web systems and how they can be applied in a Rails environment.

I. Prometheus and Grafana

  • Prometheus: A powerful open-source monitoring and alerting toolkit designed for reliability and scalability. It collects metrics from monitored targets, stores them in a time-series database, and allows querying and visualization of these metrics.

  • Grafana: A popular open-source analytics and monitoring platform that integrates with various data sources, including Prometheus. It provides rich visualization capabilities, allowing users to create customizable dashboards to monitor and analyze system metrics.

Application in Rails: Prometheus can be integrated with Rails applications using libraries like prometheus_exporter to expose custom metrics. Grafana can then be configured to pull these metrics from Prometheus and visualize them in customizable dashboards, providing insights into the performance and behavior of the Rails application.

II. New Relic

  • New Relic: A comprehensive application performance monitoring (APM) tool that provides real-time insights into application performance, end-user experience, and infrastructure monitoring. It offers features like transaction tracing, error detection, and performance optimization recommendations.

Application in Rails: New Relic offers dedicated support for Rails applications, providing detailed performance metrics, transaction traces, and error monitoring. It can be easily integrated into Rails applications using the New Relic Ruby agent, offering deep visibility into application performance.

III. Monit

Monit is an open-source utility for monitoring and managing Unix systems. It provides automatic monitoring and maintenance features, allowing administrators to monitor various aspects of system health, such as CPU usage, memory usage, disk space, and process status. Monit can take corrective actions based on predefined conditions, such as restarting failed processes, sending alerts, or executing custom scripts.

Key Features of Monit:

  1. Process Monitoring: Monit can monitor individual processes running on a system and take actions if a process consumes too much resources or crashes unexpectedly.

  2. File System Monitoring: Monit can monitor file systems for changes, such as the creation, modification, or deletion of files, and take actions based on predefined rules.

  3. Resource Monitoring: Monit can monitor system resources like CPU usage, memory usage, and disk space, and alert administrators when resource limits are exceeded.

  4. Service Monitoring: Monit can monitor network services like web servers, databases, and email servers, and restart them if they become unresponsive or crash.

  5. Alerting: Monit can send alerts via email or other notification mechanisms when predefined conditions are met, allowing administrators to respond to issues promptly.

  6. Automatic Recovery: Monit can automatically attempt to recover from failures by restarting failed processes or services, reducing downtime and improving system reliability.

  7. Configuration: Monit uses a simple configuration file syntax that allows administrators to define monitoring rules and actions easily.

Modern alternative: On most current Linux distributions, the “restart it if it dies” part of Monit is handled by systemd itself. A service unit with Restart=on-failure (plus RestartSec=, and WatchdogSec= for hung processes) restarts Puma, Sidekiq, etc. without an extra daemon. Monit is still handy for resource checks (CPU, memory, disk) and alerting on hosts without a full metrics stack.

# /etc/systemd/system/puma.service (excerpt)
[Service]
ExecStart=/usr/local/bin/bundle exec puma -C config/puma.rb
Restart=on-failure
RestartSec=5

IV. Logrotate

Logrotate is a utility for managing log files on Unix-like systems. It automates the rotation, compression, and retention of log files to conserve disk space and ensure that log files do not grow indefinitely. Logrotate is typically used to rotate log files generated by system services, web servers, applications, and other processes.

Key Features of Logrotate:

  1. Rotation: Logrotate rotates log files based on predefined criteria, such as file size, age, or time intervals. It creates new log files and archives old log files to prevent them from becoming too large.

  2. Compression: Logrotate can compress rotated log files using gzip, bzip2, or other compression algorithms to save disk space.

  3. Retention: Logrotate can delete old log files or archive them to a specified location based on retention policies, ensuring that log files are not retained indefinitely.

  4. Post-rotation Actions: Logrotate can execute custom scripts or commands after rotating log files, allowing administrators to perform additional actions like restarting services or notifying users.

  5. Configuration: Logrotate uses a simple configuration file syntax that allows administrators to define log rotation rules and options easily. Configuration files are typically located in /etc/logrotate.d/ directory.

  6. Integration with System Services: Logrotate integrates with system services like syslogd and cron to rotate log files automatically according to predefined schedules.

  7. Flexible Configuration: Logrotate supports a wide range of configuration options, allowing administrators to customize log rotation behavior according to their specific requirements.

V. Fail2ban

Fail2ban is an open-source intrusion prevention software framework written in Python. It is designed to protect Unix-like systems from brute-force attacks by monitoring log files for suspicious activity and taking action to block or ban malicious IP addresses.

Key Features of Fail2ban:

  1. Log File Monitoring: Fail2ban continuously monitors log files, such as system logs, SSH logs, and web server logs, for patterns indicating potential malicious activity, such as repeated failed login attempts.

  2. Detection of Suspicious Behavior: Fail2ban analyzes log entries in real-time to detect patterns of suspicious behavior, such as multiple failed login attempts from the same IP address within a short period of time.

  3. Dynamic Firewall Rules: Upon detection of suspicious activity, Fail2ban dynamically updates firewall rules to block or ban the offending IP addresses. It can work with various firewall systems, including iptables, firewalld, and nftables.

  4. Automatic Unbanning: Fail2ban can automatically unban IP addresses after a certain period of time, reducing the risk of accidentally blocking legitimate users.

  5. Customizable Configuration: Fail2ban provides a flexible configuration system that allows administrators to customize detection rules, actions, and thresholds to suit their specific security requirements.

  6. Alerting: Fail2ban can send alerts to administrators via email or other notification mechanisms when suspicious activity is detected, allowing them to take further action if necessary.

Note that Fail2ban works per host: it reads the local log files and updates the local firewall. It is not a log-management or centralized-logging tool, and each server needs its own Fail2ban.

Common Use Cases for Fail2ban:

  1. SSH Brute-force Protection: Fail2ban is commonly used to protect SSH servers from brute-force attacks by blocking IP addresses that repeatedly fail to authenticate.

  2. Web Server Protection: Fail2ban can be configured to monitor web server logs for suspicious activity, such as HTTP 404 errors or repeated access attempts to restricted URLs.

  3. Email Server Protection: Fail2ban can protect email servers from spam attacks by monitoring mail server logs for patterns indicating spamming activity and blocking offending IP addresses.

  4. Application-level Protection: Fail2ban can also be used to protect custom applications by monitoring their log files for signs of unauthorized access or other security threats.

VI. pgBadger and pg_stat_statements

These two tools answer the same question — “which queries are slow or expensive?” — from different sources.

pgBadger is a PostgreSQL log analyzer. It parses the PostgreSQL log files and generates reports on slow and frequent queries, locks, temporary files, checkpoints, connections, autovacuum, and errors. It only knows what the server logged, so you have to turn logging on first (e.g. log_min_duration_statement, log_checkpoints, log_lock_waits, log_temp_files, and a log_line_prefix with a timestamp and process id — see the pgBadger README).

pg_stat_statements is a PostgreSQL extension that keeps cumulative execution statistics for every normalized query in memory. It must be added to shared_preload_libraries (restart required), then enabled with CREATE EXTENSION pg_stat_statements;.

Key Features of pgBadger and pg_stat_statements:

  1. Query Performance Analysis: Both show which queries take the most total time, run most often, or are slowest on average — the starting point for any tuning work.

  2. What pg_stat_statements measures: per query, the number of calls, planning and execution time (total/min/max/mean/stddev), rows returned, shared/local/temp blocks hit, read, dirtied and written, I/O timing (with track_io_timing), WAL generated, and JIT statistics. It does not report CPU time or memory per query; temp blocks written are the closest hint that a query is spilling to disk because work_mem is too small. See the pg_stat_statements docs.

    SELECT calls,
       round(total_exec_time) AS total_ms,
       round(mean_exec_time::numeric, 2) AS mean_ms,
       rows,
       shared_blks_read,
       temp_blks_written,
       query
FROM pg_stat_statements
ORDER BY total_exec_time DESC
LIMIT 10;

  3. Optimization is still your job: Neither tool suggests indexes or rewrites queries for you. They tell you which query to look at; you then run EXPLAIN (ANALYZE, BUFFERS) on it and decide what to change.

  4. Historical Trend Analysis: pgBadger’s incremental mode (-I) builds daily and weekly reports from the logs, which makes it easy to compare this week with last week. pg_stat_statements only keeps running totals since the last reset, so for trends you need to snapshot it periodically (or let a metrics system scrape it).

  5. Report Formats: pgBadger outputs HTML (the default), plain text, JSON, or a binary format used for incremental reports.

  6. Integration with Prometheus and Grafana: pgBadger has no Prometheus integration — it produces reports, not metrics. The usual route is postgres_exporter, the Prometheus exporter for PostgreSQL server metrics; its stat_statements collector (disabled by default, enable it with --collector.stat_statements) exposes pg_stat_statements data, which you can then graph and alert on in Grafana.

  7. Automated Report Generation: pgBadger can be scheduled with cron to run periodically and generate reports automatically, providing up-to-date information on database performance and query execution.

Common Use Cases for pgBadger and pg_stat_statements:

  1. Database Performance Tuning: Finding the handful of queries that account for most of the database time.

  2. Troubleshooting Slow Queries: Identifying slow queries and how often they run, then analyzing them with EXPLAIN.

  3. Capacity Planning: By analyzing historical reports generated by pgBadger, administrators can spot growth in query volume, connections, and temporary file usage before they become a problem.

  4. Security Auditing: pgBadger also has error, connection and session reports (connections need log_connections), and a watch mode (-w) that reports only errors — useful for spotting unexpected clients or a burst of failures.

  5. Continuous Monitoring: Real-time dashboards and alerts come from postgres_exporter + Prometheus + Grafana; pgBadger complements them with detailed periodic reports.

VII. APM and Tracing: OpenTelemetry and Sentry

Metrics tell you that something is slow; traces tell you where. A trace follows one request through Rails, the database, Redis, and any external HTTP calls.

  • OpenTelemetry is the vendor-neutral, open-source standard for traces, metrics and logs, and a CNCF project. It is not a backend itself: you instrument once and export to Jaeger, Grafana Tempo, or any commercial APM that accepts the OpenTelemetry protocol (OTLP). In Ruby, tracing is stable while metrics and logs are still in development.

    # Gemfile
gem 'opentelemetry-sdk'
gem 'opentelemetry-exporter-otlp'
gem 'opentelemetry-instrumentation-all'

# config/initializers/opentelemetry.rb
require 'opentelemetry/sdk'
require 'opentelemetry/exporter/otlp'
require 'opentelemetry/instrumentation/all'

OpenTelemetry::SDK.configure do |c|
  c.service_name = 'my-rails-app'
  c.use_all # every instrumentation whose library is present (Rails, Active Record, Net::HTTP, ...)
end

  • Sentry focuses on error tracking: every unhandled exception arrives with its stack trace, request, user and release, grouped into issues. It also offers performance tracing. In Rails, add the sentry-ruby and sentry-rails gems and set config.traces_sample_rate to enable tracing (see the Sentry Rails guide).

Application in Rails: A common setup is Prometheus + Grafana for metrics, Sentry (or your APM) for errors, and OpenTelemetry for traces, so you are not locked into a single vendor’s agent.

VIII. Conclusion

Monitoring tools are essential for maintaining the health and performance of web systems, including those built with Rails. Prometheus and Grafana give you metrics and dashboards, New Relic or OpenTelemetry-based tracing shows where time is spent inside each request, and Sentry catches the errors your users hit. On the host, Monit (or systemd’s Restart=) keeps processes running, Logrotate keeps log files from filling the disk, and Fail2ban blocks brute-force attempts by watching those logs and updating the firewall. For PostgreSQL, pg_stat_statements and pgBadger show which queries deserve your attention, and postgres_exporter brings database metrics into the same Grafana dashboards as everything else.

References: