Redis is fast. The incident meeting explaining why nobody noticed it stopped is considerably less so.
Lap one: “It’s probably the network.”
Lap two: “Has anyone checked Redis?”
Lap three: a senior engineer opens a terminal, performs one command, and ages six months.
MatrixSwarm’s redis_watchdog is the pit crew for this particular race. It watches the local service, checks for a TCP listener or configured socket path, can request restarts when the service is down, and radios the humans when something needs attention.
The lap-time improvement comes from skipping the ceremonial guessing.
Bolt on a small, useful agent
In Phoenix’s Swarm Workspace, add redis_watchdog from the Agent Palette to the Linux/systemd host running Redis. Configure it for the actual installation:
| Setting | Example | Pit-wall translation |
|---|---|---|
| Service Name | redis-server |
Match your unit; some hosts use redis
|
| Redis Port | 6379 |
The local TCP port you expect |
| Socket Path | /var/run/redis/redis-server.sock |
Match the socket path if your deployment uses one |
| Check Interval (sec) | 10 |
How often the crew checks in |
| Restart Limit | 3 |
Consecutive failed restart-command threshold |
| Alert To Role | hive.alert |
The service role for alarm delivery |
These values are an example, not a universal tuning sheet. The implementation detects Redis through conventional binary locations, so confirm that your package/layout is recognized. Custom installations deserve an actual check, not an encouraging nod.
The service account must be able to inspect systemd and local listeners. Restarting uses noninteractive sudo for the configured unit, so give it the narrow restart permission required for that job.
Then configure the alarm relay you want. One watchdog plus one reachable relay inside your existing swarm is enough for a single human notification channel. No need to put a committee on the starting grid.
Read the flags correctly
The Redis watchdog treats service state and accessibility signals separately:
- Service down: gather diagnostic context, send eligible failure alerts, optionally send a forensic report, and try the restart path.
- Service active, with a detected TCP listener or existing configured socket path: the basic checks are satisfied.
- Service active, but neither accessibility signal is present: raise an alert; this branch does not automatically restart Redis merely because those signals are missing.
Unlike the transition-based Apache, MySQL, and Nginx workers, Redis can enter its restart path again on subsequent polls while the service remains down. Consecutive failed restart commands count toward the configured limit; the disable guard then stops further restart attempts in that agent instance. A successful restart command resets that failure counter.
The recovery notice follows the service returning from a down state. Treat it as a service-state report, not a signed affidavit that every application can use the cache. An active service can still produce the separate accessibility warning.
Also, “socket exists” is a filesystem observation, and “port appears in the listener list” is a local networking observation. Neither is an authenticated Redis conversation. This worker does not use Redis PING as its health decision. Add an appropriate protocol/application probe when you need that assurance.
A parked race car still has wheels. We ask slightly more of it on race day.
Bring the black box, not just the siren
Diagnostics are best effort: systemd status, Redis CLI information where available, and recent log context. Authenticated Redis, custom ports, or TLS may need additional diagnostic handling; the basic CLI diagnostic invocation does not automatically inherit every setting from your application client.
A report_to_role consumer can receive structured forensic data if you configure one. That is separate from the human alarm relay and optional for a simple notification setup.
The Always alert on failure setting affects failure chatter, but do not assume every alert branch is governed by one universal cooldown. In particular, the active-but-inaccessible warning can repeat on each poll. Choose the interval and channel with that behavior in mind. Your phone should report incidents, not audition for percussion.
Any watchdog can use any of the alarm radios
This is a swarm-wide pattern: Apache, MySQL, Nginx, and Redis watchdogs can all be combined with Slack, Discord, Telegram, or email alarm relays. Mix the channels according to the humans you need to reach.
Use slack_relay, discord_relay, telegram_relay, or email_send. The watchdog’s alert_to_role normally targets hive.alert, and those relays advertise hive.alert@cmd_send_alert_msg.
Configure destination credentials, resolve required Registry and signing assignments, and make sure the service scope/routing connects the agents. Several matching reachable relays can receive the same alert. Your Redis alarm can reach Slack and Telegram and email; it does not need three different watchdog implementations and a branding exercise.
Encryption is an option on the relay, not a team superstition
Discord, Telegram, and email support optional protected alert envelopes. On each relay that should use them, enable encrypt_alerts and supply its assigned packet-signing/encryption keys. Configuring packet signing by itself does not turn on outgoing platform-message encryption.
The switches are labeled Encrypt Discord alert message, Encrypt Telegram alert message, and Encrypt alert email subject and body. With the proper keys, an authorized operator can use the matching relay’s Decrypt Message panel in Phoenix to read the encoded alert.
When secure wrapping fails, the relay does not quietly send that protected alert as plaintext. Email puts the alert’s subject and body inside the envelope; delivery metadata remains visible. Slack uses normal HTTPS delivery and has no equivalent MatrixSwarm alert-payload encryption switch in this implementation.
Every copy has its own setting. Encrypting Discord while sending a readable duplicate elsewhere is exactly that: one protected copy, one readable copy. Painting a padlock on the team bus does not secure the luggage.
Take one practice lap
On a test host, verify normal service/listener detection, a controlled service-down event, restart permissions and behavior, the active-but-inaccessible case, recovery reporting, and delivery through each chosen relay. If encryption is enabled, test decryption too.
And keep persistence, backups, and failover planning where they belong. A service restart does not restore lost data, elect a cluster leader, or retroactively make an untested recovery plan brilliant.
Redis supplies the speed. The watchdog supplies the tap on the shoulder before the pit wall becomes a group therapy session.
Victory Always. Keep the cache quick and the excuses shorter.
🌐 Links & Resources:
Try MatrixSwarm: https://matrixswarm.com
Join the Community / Discord: https://discord.gg/2USbWVBVV
Download Server: https://github.com/matrixswarm/matrixswarm
Youtube: https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug
Top comments (0)