<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Daniel</title>
    <description>The latest articles on DEV Community by Daniel (@matrixswarm).</description>
    <link>https://dev.to/matrixswarm</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3767433%2F93c018b2-a2a9-43b3-95fe-6dc3a947e53a.png</url>
      <title>DEV Community: Daniel</title>
      <link>https://dev.to/matrixswarm</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/matrixswarm"/>
    <language>en</language>
    <item>
      <title>Your First MatrixSwarm: Five Agents, One Railgun, and a Server That Finally Texts Back</title>
      <dc:creator>Daniel</dc:creator>
      <pubDate>Mon, 19 Oct 2026 07:18:39 +0000</pubDate>
      <link>https://dev.to/matrixswarm/your-first-matrixswarm-five-agents-one-railgun-and-a-server-that-finally-texts-back-3647</link>
      <guid>https://dev.to/matrixswarm/your-first-matrixswarm-five-agents-one-railgun-and-a-server-that-finally-texts-back-3647</guid>
      <description>&lt;p&gt;You have a Linux server. You can SSH into it. You can stare at its logs until the letters begin to rearrange themselves into career advice.&lt;/p&gt;

&lt;p&gt;What you would prefer is a small, understandable setup: send commands from Phoenix, see the agents and their logs, and get a Telegram message when someone successfully logs in over SSH.&lt;/p&gt;

&lt;p&gt;A reasonable request. Naturally, we have brought a Railgun.&lt;/p&gt;

&lt;p&gt;This walkthrough builds that setup from the beginning: &lt;strong&gt;install MatrixOS, create a Phoenix Swarm Workspace, assign its connections, deploy it through Railgun, and verify the complete path with a controlled login.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The five agents we add are &lt;code&gt;matrix_https&lt;/code&gt;, &lt;code&gt;matrix_websocket&lt;/code&gt;, &lt;code&gt;log_streamer&lt;/code&gt;, &lt;code&gt;telegram_relay&lt;/code&gt;, and &lt;code&gt;gatekeeper&lt;/code&gt;. Phoenix creates the &lt;code&gt;matrix&lt;/code&gt; root for us, so the finished tree contains six nodes.&lt;/p&gt;

&lt;p&gt;A naming note before anyone searches the palette for twenty minutes: the HTTP command-ingress agent is named &lt;strong&gt;&lt;code&gt;matrix_https&lt;/code&gt;&lt;/strong&gt; in the current catalog, and the Telegram agent is &lt;strong&gt;&lt;code&gt;telegram_relay&lt;/code&gt;&lt;/strong&gt;. Those are the names used below.&lt;/p&gt;

&lt;h2&gt;
  
  
  The mission map
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Component&lt;/th&gt;
&lt;th&gt;Responsibility&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Phoenix&lt;/td&gt;
&lt;td&gt;The operator application: vault, Registry, workspace editor, and live cockpit&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;MatrixOS&lt;/td&gt;
&lt;td&gt;The runtime installed on the Linux target&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Railgun&lt;/td&gt;
&lt;td&gt;Installs the runtime and launches deployments through SSH&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Starts the swarm, establishes its chain of trust, and routes commands&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_https&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Receives authenticated Phoenix commands over HTTPS&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_websocket&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Provides the live return connection for replies, alerts, and session traffic&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;log_streamer&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Supplies agent runtime logs to Phoenix&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;gatekeeper&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Watches authentication-log entries for successful SSH logins&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;telegram_relay&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Sends alert messages to the configured Telegram chat&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Here is the traffic flow. These arrows describe communication, not the parent-child layout of the workspace:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;INSTALL / LAUNCH
Phoenix → Railgun → SSH → MatrixOS / MatrixD

LIVE CONTROL
Phoenix → HTTPS ingress → Matrix → requested agent
Agent reply / logs → WebSocket relay → Phoenix

LOGIN NOTIFICATION
SSH authentication log → Gatekeeper → hive.alert → Telegram relay → your chat
                                          └──→ WebSocket relay → Phoenix feed
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;SSH is the installation and launch route. HTTPS and WebSocket are the live cockpit routes.&lt;/strong&gt; Successfully connecting by SSH does not prove the cockpit ports are reachable.&lt;/p&gt;

&lt;p&gt;Telegram is an alert destination here. Typing “restart everything” into the bot chat is not part of this agent’s command interface, which is probably good news for everyone in the group.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Prepare the target and open your vault
&lt;/h2&gt;

&lt;p&gt;Use a Linux machine you administer, preferably a lab VM for the first run. The current installer supports provisioning through &lt;code&gt;apt-get&lt;/code&gt; or &lt;code&gt;dnf&lt;/code&gt;. It provisions the Python 3.12 runtime it needs, including a verified source-build fallback when the required runtime is unavailable from the host’s packages.&lt;/p&gt;

&lt;p&gt;Have these ready:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Phoenix with an unlocked encrypted vault;&lt;/li&gt;
&lt;li&gt;an SSH account on the target, with root access or the passwordless-sudo access required by Railgun’s provisioning steps;&lt;/li&gt;
&lt;li&gt;the target’s independently verified SSH host-key fingerprint;&lt;/li&gt;
&lt;li&gt;an address Phoenix can reach, two available non-privileged TCP ports, and matching network/firewall access;&lt;/li&gt;
&lt;li&gt;a Telegram bot token and destination chat ID;&lt;/li&gt;
&lt;li&gt;a real authentication log that records SSH events.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The runtime service account is a separate matter from the privileged SSH account. Railgun uses provisioning privileges to prepare the host, then runs the swarm under its dedicated non-root Linux user.&lt;/p&gt;

&lt;p&gt;For the examples, we will use:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Example item&lt;/th&gt;
&lt;th&gt;Value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Target reachable address&lt;/td&gt;
&lt;td&gt;&lt;code&gt;10.20.30.40&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Phoenix workstation’s source address&lt;/td&gt;
&lt;td&gt;&lt;code&gt;10.20.30.10&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;HTTPS command port&lt;/td&gt;
&lt;td&gt;&lt;code&gt;8443&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;WebSocket return port&lt;/td&gt;
&lt;td&gt;&lt;code&gt;8765&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Universe&lt;/td&gt;
&lt;td&gt;&lt;code&gt;first-swarm&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Swarm Linux user&lt;/td&gt;
&lt;td&gt;&lt;code&gt;matrix-first-swarm&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;These are example values, not a server waiting for you. Replace them with your network’s actual values. If NAT or a VPN changes the source address seen by the target, use that observed source when configuring allowlists.&lt;/p&gt;

&lt;p&gt;Permit the two cockpit ports from the operator’s intended address or network, and keep SSH access available for installation. The target also needs outbound access for installation dependencies and Telegram delivery. A new universe on the same server needs available ports too; a different name does not let two processes own the same socket.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Put SSH in the Registry
&lt;/h2&gt;

&lt;p&gt;Open &lt;strong&gt;Registry&lt;/strong&gt;, create an &lt;strong&gt;SSH&lt;/strong&gt; record, and give it a recognizable label, such as &lt;code&gt;lab-server-ssh&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;In its &lt;strong&gt;Connection&lt;/strong&gt; tab, enter Host, Port, Username, and Trusted Fingerprint. In &lt;strong&gt;Authentication &amp;amp; Keys&lt;/strong&gt;, select the authentication method you actually use and enter the corresponding existing credentials. An encrypted private key needs its own passphrase; that is different from the server account’s password.&lt;/p&gt;

&lt;p&gt;Use &lt;strong&gt;Test Connection&lt;/strong&gt;, then &lt;strong&gt;Save to Vault&lt;/strong&gt;. Establish the host fingerprint from a trusted source, such as the server console or your provisioning inventory. Treat a fingerprint mismatch as something to investigate rather than a speed bump to flatten.&lt;/p&gt;

&lt;p&gt;The SSH record tells Railgun which machine it may contact. It does not configure the HTTPS or WebSocket agents. Those get their own records shortly.&lt;/p&gt;

&lt;p&gt;Credentials belong in the vault. A screenshot of a private key is a surprisingly efficient way to turn documentation into an incident.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Install MatrixOS with Railgun
&lt;/h2&gt;

&lt;p&gt;Open &lt;strong&gt;Railgun → Check Remote Host…&lt;/strong&gt;, select the SSH target, and run &lt;strong&gt;Run Full Check&lt;/strong&gt;. Review the results for SSH access, OS, Python, disk space, system clock, &lt;code&gt;/matrix&lt;/code&gt;, and an existing installation. This is a useful preflight, not a guarantee that every application-level requirement has been satisfied.&lt;/p&gt;

&lt;p&gt;If MatrixOS is already installed and current enough for this Phoenix build, proceed to the workspace steps. Otherwise open &lt;strong&gt;Railgun → Install MatrixOS…&lt;/strong&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  Choose the source
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Install Mode&lt;/th&gt;
&lt;th&gt;What it uses&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Install from GitHub&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;The project’s GitHub repository, fetched by the remote installer&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Local Full Install&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Your selected local MatrixOS source folder, uploaded to the target&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For Local Full Install, &lt;strong&gt;Local Source Path&lt;/strong&gt; should point to the &lt;strong&gt;MatrixOS root&lt;/strong&gt;, the directory containing its runtime source such as &lt;code&gt;agents&lt;/code&gt;, &lt;code&gt;core&lt;/code&gt;, and &lt;code&gt;scripts&lt;/code&gt;. In the monorepo, this is the &lt;code&gt;matrixos&lt;/code&gt; directory—not the Phoenix application directory or a Python virtual environment.&lt;/p&gt;

&lt;p&gt;Use matching Phoenix and MatrixOS versions. A local UI describing features that an older remote runtime has never met is not a compatibility strategy.&lt;/p&gt;

&lt;h3&gt;
  
  
  Choose the environment and target
&lt;/h3&gt;

&lt;p&gt;For a fresh installation, select &lt;strong&gt;Create new venv&lt;/strong&gt;. &lt;strong&gt;Skip Python setup&lt;/strong&gt; is for an already suitable MatrixOS environment; it is not the beginner’s “make this faster” button.&lt;/p&gt;

&lt;p&gt;Select the saved SSH target under &lt;strong&gt;Deploy To&lt;/strong&gt;, then click &lt;strong&gt;Install MatrixOS&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The installer places MatrixOS under &lt;code&gt;/matrix&lt;/code&gt;, creates or uses its isolated environment at &lt;code&gt;/matrix/.venv&lt;/code&gt;, installs dependencies, and prepares the &lt;code&gt;matrixd&lt;/code&gt; launcher. Watch the output and wait for &lt;strong&gt;Installation complete&lt;/strong&gt;. An upload reaching 100% is only an upload milestone; dependency installation and validation can still be running.&lt;/p&gt;

&lt;p&gt;If Python must be built, allow that stage to finish. The progress window has not entered a reflective phase. Compilers simply have hobbies.&lt;/p&gt;

&lt;p&gt;Installing MatrixOS prepares the runtime. It does &lt;strong&gt;not&lt;/strong&gt; deploy this swarm yet.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Create the three application connection records
&lt;/h2&gt;

&lt;p&gt;In Registry, create an &lt;strong&gt;HTTPS&lt;/strong&gt; record, a &lt;strong&gt;WSS&lt;/strong&gt; record, and a &lt;strong&gt;Telegram&lt;/strong&gt; record. You can also create or select these from the workspace’s requirement-assignment controls; preparing them now makes the later steps easy to follow.&lt;/p&gt;

&lt;h3&gt;
  
  
  HTTPS: Phoenix sends commands
&lt;/h3&gt;

&lt;p&gt;For the HTTPS record:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Field&lt;/th&gt;
&lt;th&gt;Example&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Label&lt;/td&gt;
&lt;td&gt;&lt;code&gt;first-swarm-https&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Host&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;10.20.30.40&lt;/code&gt;, without a URL scheme or path&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Port&lt;/td&gt;
&lt;td&gt;&lt;code&gt;8443&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Allowlist IPs&lt;/td&gt;
&lt;td&gt;The operator source IP actually seen by the target&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Channel&lt;/td&gt;
&lt;td&gt;&lt;code&gt;outgoing.command&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Primary Outgoing Transport&lt;/td&gt;
&lt;td&gt;Enabled for this single-command-transport example&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Directive Path&lt;/td&gt;
&lt;td&gt;Keep &lt;code&gt;config&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Use an address Phoenix can reach. &lt;code&gt;0.0.0.0&lt;/code&gt; is not a destination to enter here. The current agents listen on the target’s interfaces; the Registry Host supplies the address used to connect to them.&lt;/p&gt;

&lt;h3&gt;
  
  
  WSS: Phoenix receives live results
&lt;/h3&gt;

&lt;p&gt;For the WSS record, use the same reachable host and port &lt;code&gt;8765&lt;/code&gt;. Set the appropriate source-IP allowlist, keep the channel &lt;strong&gt;&lt;code&gt;payload.reception&lt;/code&gt;&lt;/strong&gt;, enable &lt;strong&gt;Primary Payload Reception Transport&lt;/strong&gt;, and keep Directive Path &lt;strong&gt;&lt;code&gt;config&lt;/code&gt;&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Those channel names are from Phoenix’s perspective: commands go out, payloads come back. The WebSocket agent’s “egress” role describes traffic leaving the swarm toward Phoenix.&lt;/p&gt;

&lt;p&gt;Leave the generated Serial values alone. Labels are for humans; identities and references are doing additional work behind the curtain.&lt;/p&gt;

&lt;h3&gt;
  
  
  Telegram: give the messenger an address
&lt;/h3&gt;

&lt;p&gt;For a first exercise, a new dedicated bot and a private test chat are straightforward:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Contact Telegram’s official &lt;strong&gt;&lt;a href="https://t.me/BotFather" rel="noopener noreferrer"&gt;BotFather&lt;/a&gt;&lt;/strong&gt;, send &lt;code&gt;/newbot&lt;/code&gt;, and follow its prompts to obtain the bot token.&lt;/li&gt;
&lt;li&gt;Open a private conversation with your new bot and send &lt;code&gt;/start&lt;/code&gt; or a message. A bot needs that initial user contact before ordinary private messages can be sent back.&lt;/li&gt;
&lt;li&gt;Obtain that conversation’s numeric chat ID, then save the token and chat ID in the Telegram Registry record.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Telegram documents bot creation in its &lt;a href="https://core.telegram.org/bots/tutorial" rel="noopener noreferrer"&gt;official tutorial&lt;/a&gt;. For a new bot with no webhook or other update consumer, you can retrieve an incoming update through &lt;a href="https://core.telegram.org/bots/api#getupdates" rel="noopener noreferrer"&gt;getUpdates&lt;/a&gt; and read &lt;code&gt;message.chat.id&lt;/code&gt; from your test message. Use a local API client and supply the token privately; do not put a real token into a tutorial, a shared screenshot, or a browser address you plan to share. If a webhook is already configured, getUpdates is unavailable—use the existing integration’s received update instead.&lt;/p&gt;

&lt;p&gt;The Registry fields are &lt;strong&gt;Label&lt;/strong&gt;, &lt;strong&gt;Chat ID&lt;/strong&gt;, and &lt;strong&gt;Bot Token&lt;/strong&gt;. Keep the Telegram channel selection and &lt;strong&gt;Directive Path &lt;code&gt;config/telegram&lt;/code&gt;&lt;/strong&gt;. The token is the bot credential; the chat ID is the destination. They are not interchangeable, despite both looking like something a database coughed up.&lt;/p&gt;

&lt;p&gt;For a group destination, add the bot to that group, use the group’s actual chat ID, and ensure the bot can send there. Verify the destination before connecting real login notifications to it.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Build the workspace tree
&lt;/h2&gt;

&lt;p&gt;Click &lt;strong&gt;Deploy&lt;/strong&gt; in Phoenix’s main toolbar. This opens &lt;strong&gt;Swarm Workspaces&lt;/strong&gt;. Choose &lt;strong&gt;New&lt;/strong&gt; for this tutorial, or &lt;strong&gt;Open&lt;/strong&gt; an existing workspace you intend to extend. The manager also offers &lt;strong&gt;Clone&lt;/strong&gt; when you want a separate editable copy.&lt;/p&gt;

&lt;p&gt;A new workspace starts with the &lt;code&gt;matrix&lt;/code&gt; root. In the &lt;strong&gt;Agent picker&lt;/strong&gt;, search for each of the five agents and drag it onto empty canvas space or directly onto Matrix. For this example, all five should be direct children of the root:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;matrix
├── matrix_https
├── matrix_websocket
├── log_streamer
├── telegram_relay
└── gatekeeper
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;Drop onto another agent and you can create a different parent relationship. If that happens unintentionally, right-click the misplaced node and use &lt;strong&gt;Reparent To → matrix&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;This is the agent hierarchy. You do not need to put Telegram underneath Gatekeeper to route its alerts. The &lt;strong&gt;service roles&lt;/strong&gt; supply that connection.&lt;/p&gt;

&lt;p&gt;Select a node to inspect it on the left. Double-click a configurable agent to open its configuration editor. If the picker appears to have lost an agent, use &lt;strong&gt;Clear filters&lt;/strong&gt; before filing a missing-person report.&lt;/p&gt;

&lt;p&gt;The workspace saves changes to the vault. Watch for &lt;strong&gt;Saved to Vault ✓&lt;/strong&gt;; a save failure has a &lt;strong&gt;Retry Save&lt;/strong&gt; action. Do not close an editor that is still telling you its changes have not been saved. You can rename the workspace in the manager to something recognizable, such as &lt;code&gt;First Swarm Lab&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  6. Satisfy the requirements without collecting loose certificates
&lt;/h2&gt;

&lt;p&gt;Select each node and look at &lt;strong&gt;Agent Inspector → Requirements&lt;/strong&gt;.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Node&lt;/th&gt;
&lt;th&gt;What you should see or assign&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Automatic packet-signing material&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_https&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Automatic signing and connection certificates; assign the HTTPS Registry record&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_websocket&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Automatic signing and connection certificates; assign the WSS record&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;log_streamer&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Automatic packet-signing material&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;telegram_relay&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Automatic packet-signing material; assign the Telegram record&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;gatekeeper&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;No manual Registry requirement in the current palette entry; configure its behavior separately&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;AUTOGEN&lt;/strong&gt; rows are generated during compilation. They are not missing credentials that you must hunt down on the filesystem. For assignable rows, use &lt;strong&gt;Assign or change&lt;/strong&gt;, select the correct Registry record, and confirm that the row is satisfied.&lt;/p&gt;

&lt;p&gt;Leave the supplied signing and certificate requirements intact. The deployment compiles the corresponding agent and Phoenix-side material, including the connection certificates used for mutual TLS. A normal browser visiting the port is not the intended client and may be rejected because it lacks that client identity.&lt;/p&gt;

&lt;p&gt;Missing required assignments stop deployment. This is helpful. Discovering an absent Telegram token before launch is considerably less dramatic than discovering it during an incident.&lt;/p&gt;

&lt;h2&gt;
  
  
  7. Configure the agents and keep their service roles
&lt;/h2&gt;

&lt;p&gt;For this first swarm, most defaults are useful. The important settings are the ones that let the agents find each other and do their actual jobs.&lt;/p&gt;

&lt;h3&gt;
  
  
  HTTPS and WebSocket
&lt;/h3&gt;

&lt;p&gt;Double-click each transport agent and leave &lt;strong&gt;Enable Lockdown (Disable Packet Processing)&lt;/strong&gt; unchecked for this exercise. Host, port, and allowlist settings come from the Registry records assigned above.&lt;/p&gt;

&lt;p&gt;Keep the WebSocket relay’s supplied roles, including &lt;code&gt;hive.rpc&lt;/code&gt; and &lt;code&gt;hive.alert&lt;/code&gt;. These let it carry replies and receive alerts. Removing roles to make the configuration look tidier is how a perfectly healthy connection becomes a remarkably quiet one.&lt;/p&gt;

&lt;h3&gt;
  
  
  Log streamer
&lt;/h3&gt;

&lt;p&gt;Keep &lt;code&gt;log_streamer&lt;/code&gt; and its &lt;strong&gt;&lt;code&gt;hive.log_streamer&lt;/code&gt;&lt;/strong&gt; service role. Phoenix requests an agent’s runtime log through this service when you select that agent in the live tree.&lt;/p&gt;

&lt;p&gt;It supplies visibility into the swarm. Gatekeeper itself reads the operating system’s SSH authentication log; log_streamer does not take over that job.&lt;/p&gt;

&lt;h3&gt;
  
  
  Telegram relay
&lt;/h3&gt;

&lt;p&gt;Open &lt;code&gt;telegram_relay&lt;/code&gt; and keep &lt;strong&gt;Enable Telegram alert delivery&lt;/strong&gt; enabled. Leave &lt;strong&gt;Encrypt Telegram alert message&lt;/strong&gt; off for the initial delivery test; we will cover that option below.&lt;/p&gt;

&lt;p&gt;Keep its &lt;strong&gt;&lt;code&gt;hive.alert&lt;/code&gt;&lt;/strong&gt; role. Gatekeeper sends to that role, and the relay publishes the message to the chat configured in its Registry record. WebSocket also advertises the alert role, so a connected Phoenix session can receive the event too.&lt;/p&gt;

&lt;p&gt;No hand-written destination UUID is needed for this basic arrangement.&lt;/p&gt;

&lt;h3&gt;
  
  
  Gatekeeper
&lt;/h3&gt;

&lt;p&gt;Start with these settings:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Setting&lt;/th&gt;
&lt;th&gt;First-run choice&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Alert Role&lt;/td&gt;
&lt;td&gt;&lt;code&gt;hive.alert&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Ignore IPs&lt;/td&gt;
&lt;td&gt;Empty while testing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Log Path&lt;/td&gt;
&lt;td&gt;The target’s real SSH authentication log; read the path-selection note below&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;MaxMind DB&lt;/td&gt;
&lt;td&gt;An existing GeoLite2 City database, if you want location enrichment&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Gatekeeper follows new log lines with &lt;code&gt;tail -n 0 -F&lt;/code&gt;. It recognizes successful OpenSSH-style entries containing &lt;strong&gt;Accepted&lt;/strong&gt; and &lt;strong&gt;from&lt;/strong&gt;, extracts the login account and source address, and sends an alert. It does not replay yesterday’s log when it starts.&lt;/p&gt;

&lt;p&gt;The name sounds like a bouncer. In this setup, its job is to notice who came through the door and call the operator. It does &lt;strong&gt;not&lt;/strong&gt; authenticate SSH users, block a login, ban addresses, or turn a successful login into proof of compromise.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Path selection matters:&lt;/strong&gt; the current implementation prefers &lt;code&gt;/var/log/secure&lt;/code&gt; when that file exists, then &lt;code&gt;/var/log/auth.log&lt;/code&gt;, and only then the configured fallback. Confirm the path reported in its startup log. On a host that only keeps authentication events in the journal, configure the system logger to produce a suitable text log; creating an empty file does not persuade SSH to write into it.&lt;/p&gt;

&lt;p&gt;Railgun derives log-access needs from the swarm and provisions the runtime account accordingly. Check its permission output. Initial access to an existing file is only part of the job: &lt;strong&gt;the account must still be able to read new files after log rotation&lt;/strong&gt;. The current provisioning includes a persistent rule for the supported &lt;code&gt;secure&lt;/code&gt;/rsyslog path; &lt;code&gt;auth.log&lt;/code&gt; rotation may need distro-specific log-creation or ACL configuration. Verify that on your host rather than making authentication logs world-readable.&lt;/p&gt;

&lt;p&gt;Optional MaxMind data adds a location estimate. Missing location data does not make a successful login disappear. An IP-derived city is context, not a sworn witness.&lt;/p&gt;

&lt;p&gt;A current-build detail: the editor exposes &lt;strong&gt;Enable GeoIP&lt;/strong&gt; and &lt;strong&gt;Always Alert&lt;/strong&gt;, but the successful-login path does not currently use those switches to gate its lookup or apply the cooldown helper. Do not rely on those switches to suppress lookups or throttle this flow. Ignore IPs is an exact-address exclusion list; leave it empty until the test works, and add only intentional exclusions afterward.&lt;/p&gt;

&lt;p&gt;Gatekeeper also contacts &lt;code&gt;api.ipify.org&lt;/code&gt; to obtain the server’s public IP while constructing alerts. That outbound dependency can delay processing if it stalls. This walkthrough promises a verifiable notification path, not a fictional millisecond delivery guarantee.&lt;/p&gt;

&lt;h2&gt;
  
  
  8. Deploy the workspace through Railgun
&lt;/h2&gt;

&lt;p&gt;Your workspace is the editable recipe in the vault. A deployment compiles that recipe, resolves its Registry references, generates the necessary identities, and launches an instance on the target.&lt;/p&gt;

&lt;p&gt;Click &lt;strong&gt;Deploy&lt;/strong&gt; inside the workspace. Resolve any missing requirements before proceeding.&lt;/p&gt;

&lt;p&gt;Give the deployment a recognizable &lt;strong&gt;Deployment Label&lt;/strong&gt;, then review &lt;strong&gt;Directive Deployment Options&lt;/strong&gt;:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Option&lt;/th&gt;
&lt;th&gt;Choice for this first deployment&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;SSH Target&lt;/td&gt;
&lt;td&gt;The tested SSH Registry record&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Universe Name&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;first-swarm&lt;/code&gt;, or another unused simple name&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Swarm Linux User&lt;/td&gt;
&lt;td&gt;A dedicated non-root account, such as &lt;code&gt;matrix-first-swarm&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Embed agent sources (Clown Car)&lt;/td&gt;
&lt;td&gt;Leave off when using the matching installed MatrixOS runtime&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;View the fully resolved directive after dependency to agent mappings&lt;/td&gt;
&lt;td&gt;Keep enabled so you can review the result&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;--protect-memory&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Keep enabled&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;--debug&lt;/code&gt;, &lt;code&gt;--rug-pull&lt;/code&gt;, &lt;code&gt;--clean&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;Leave off for this tutorial&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The label is for your eyes. The universe is an operational identifier: use a simple value without spaces, and check it explicitly rather than accepting a long display label copied into the field.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Review &lt;code&gt;--reboot&lt;/code&gt;: it is enabled by default in the current dialog.&lt;/strong&gt; Deploying to an existing universe is an operational change, not a harmless preview. Use a new universe and free ports for this exercise. Do not select every flag because they look lonely.&lt;/p&gt;

&lt;p&gt;The optional &lt;strong&gt;Clown Car&lt;/strong&gt; mode embeds agent source into the deployment. It does not replace installing MatrixOS, Python, or dependencies. For the beginner path here, the installed runtime supplies the agents.&lt;/p&gt;

&lt;p&gt;There is also an option to remove previous deployment records after success for the same universe and target IP. That controls saved deployment records; it is not permission to clean arbitrary files on the server. Review it if you want to retain older records.&lt;/p&gt;

&lt;h3&gt;
  
  
  Read the staging view, then launch
&lt;/h3&gt;

&lt;p&gt;With the preview option selected, Phoenix opens &lt;strong&gt;Directive Encryption Staging&lt;/strong&gt;. Check the agent tree, target addresses, ports, alert role, and resolved assignments.&lt;/p&gt;

&lt;p&gt;The resolved view can contain secrets. Review it locally; do not paste the entire thing into a support thread. “Please find attached our credentials” is not a diagnostic format.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Encrypt &amp;amp; Save&lt;/strong&gt; commits this deployment and continues into the Railgun launch flow. It is the launch action for this workflow, not a button for saving an unfinished workspace.&lt;/p&gt;

&lt;h3&gt;
  
  
  What Railgun does
&lt;/h3&gt;

&lt;p&gt;Railgun checks the pinned SSH target and the remote MatrixD launch capability, prepares the dedicated runtime user and agent-specific host access, and sends a sealed boot envelope over &lt;strong&gt;SSH standard input&lt;/strong&gt;. MatrixD receives and processes that envelope to start the selected universe.&lt;/p&gt;

&lt;p&gt;The sealed launch path avoids staging the plaintext directive and key as ordinary launch files or putting them in command-line arguments. That does not make the entire server diskless: the installed runtime, logs, and normal agent state still exist. Memory protection also does not make the process inaccessible to root.&lt;/p&gt;

&lt;p&gt;Watch the Railgun output for completion and a successful exit. A saved deployment record alone is insufficient evidence—the record can exist before remote launch is confirmed.&lt;/p&gt;

&lt;p&gt;If the SSH session times out or you cancel the local operation, inspect the target and the &lt;strong&gt;Swarms&lt;/strong&gt; view before launching again. Stopping the local wait does not prove that the remote process stopped. Distributed systems remain committed to making “did it happen?” a full-time occupation.&lt;/p&gt;

&lt;h2&gt;
  
  
  9. Connect Phoenix and prove the whole route
&lt;/h2&gt;

&lt;p&gt;Use &lt;strong&gt;Connect&lt;/strong&gt; and select the deployment you just launched. Phoenix uses that deployment’s transport records and generated credentials for the HTTPS/WSS session.&lt;/p&gt;

&lt;p&gt;Work through these checks in order:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Live tree:&lt;/strong&gt; confirm that the root and five child agents appear and report current activity.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Agent logs:&lt;/strong&gt; select Gatekeeper in the live tree and inspect &lt;strong&gt;Agent Logs&lt;/strong&gt;. The selection requests the log stream. Look for its actual authentication-log path and any read errors.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Relay health:&lt;/strong&gt; select the Telegram relay and check that delivery is enabled and there are no missing-configuration or API errors.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fresh login:&lt;/strong&gt; keep your existing administrative session open and make one new, authorized SSH login from an address absent from Gatekeeper’s ignore list. Do this after Gatekeeper has started following the log.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Source event:&lt;/strong&gt; confirm that the target recorded the corresponding successful OpenSSH authentication entry.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Delivery:&lt;/strong&gt; look for the event in the Phoenix alert feed and the intended Telegram chat. The relay’s successful-send log is useful supporting evidence; check the actual destination too.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The notification should identify the login account, source IP, authentication method, and recorded time, with location information when available. Some fields, such as terminal details, may be unknown. That is a better answer than confident fiction wearing a dashboard badge.&lt;/p&gt;

&lt;p&gt;Do not test this by attacking your own password prompt. This tutorial exercises the successful-login flow. An ordinary authorized login is sufficient.&lt;/p&gt;

&lt;p&gt;Once the message arrives, you have demonstrated the complete chain:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;real SSH login
  → real authentication entry
  → Gatekeeper
  → hive.alert
  → Telegram relay
  → the right chat
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;You also have a separate way to observe the agents through Phoenix and log_streamer. That makes the next failure diagnosable instead of merely atmospheric.&lt;/p&gt;

&lt;h2&gt;
  
  
  10. Optional: encrypt the Telegram alert body
&lt;/h2&gt;

&lt;p&gt;After ordinary delivery works, enable &lt;strong&gt;Encrypt Telegram alert message&lt;/strong&gt; in the relay’s configuration and deploy the updated workspace. Workspace edits affect the next deployment; saving the recipe does not silently rewrite a running swarm.&lt;/p&gt;

&lt;p&gt;With encryption enabled, the relay signs and encrypts the alert body using the deployment’s assigned packet keys. Telegram receives a wrapped message beginning with:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;MATRIXSWARM-ENCRYPTED-ALERT-V1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;It will look less like a login notification and more like a keyboard falling down a staircase. That is expected.&lt;/p&gt;

&lt;p&gt;In Phoenix, open the Telegram relay’s panel, paste the complete wrapped alert, and use &lt;strong&gt;Decrypt&lt;/strong&gt; with the matching deployment credentials. Phoenix verifies and displays the recovered message. Retain the appropriate deployment material if you need to read alerts from older deployments.&lt;/p&gt;

&lt;p&gt;If the relay cannot create the encrypted wrapper, it blocks that send rather than falling back to plaintext. Check its logs when testing.&lt;/p&gt;

&lt;p&gt;This option protects the alert body. Telegram still handles the bot/chat relationship and delivery metadata. The wrapper’s expiry information is not a Telegram self-destruct timer; Phoenix can mark an expired message as view-only, and the chat message is not automatically deleted by that setting.&lt;/p&gt;

&lt;h2&gt;
  
  
  11. When the server declines to text back
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Symptom&lt;/th&gt;
&lt;th&gt;Check first&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;MatrixOS installation failed&lt;/td&gt;
&lt;td&gt;Actual installer output, supported package manager, provisioning privileges, network access, and disk space&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Workspace refuses deployment&lt;/td&gt;
&lt;td&gt;Unassigned Registry requirements and invalid deployment fields&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SSH works, Phoenix does not connect&lt;/td&gt;
&lt;td&gt;Reachable Registry Host, both cockpit ports, firewall rules, actual source-IP allowlists, correct deployment credentials, and transport lockdown settings&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Agents appear, logs do not&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;log_streamer&lt;/code&gt;, its &lt;code&gt;hive.log_streamer&lt;/code&gt; role, and the WebSocket return path&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gatekeeper is alive but quiet&lt;/td&gt;
&lt;td&gt;Its selected log path, runtime-account read access, a fresh Accepted entry, and exact IP exclusions&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Alerts stop after log rotation&lt;/td&gt;
&lt;td&gt;Permissions on newly created authentication logs and persistent logging/rotation configuration&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gatekeeper processing stalls&lt;/td&gt;
&lt;td&gt;Authentication-log following, optional lookup errors, and outbound access to &lt;code&gt;api.ipify.org&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Telegram has nothing&lt;/td&gt;
&lt;td&gt;Bot token, correct chat ID, initial user contact, send permissions, delivery enabled, shared &lt;code&gt;hive.alert&lt;/code&gt; role, and outbound Telegram API access&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Encrypted Telegram text will not open&lt;/td&gt;
&lt;td&gt;Complete wrapper, matching deployment keys, verification status, and relay encryption logs&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Check one hop at a time. Reinstalling everything because a chat ID is wrong is technically activity, but so is running in circles with a clipboard.&lt;/p&gt;

&lt;h2&gt;
  
  
  The workflow you can reuse
&lt;/h2&gt;

&lt;p&gt;The pattern for the next swarm is the same:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Install the runtime once → create or clone a workspace → add agents → assign Registry requirements → configure behavior → deploy → connect → test a real event.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Keep the small transport-and-logging foundation when you want a live Phoenix cockpit. Add the next useful worker beneath Matrix, configure its service roles and requirements, and verify its output. Telegram can carry other agents’ compatible alerts too; Gatekeeper is simply the first producer in this exercise.&lt;/p&gt;

&lt;p&gt;You now have a swarm you can command, observe, and hear from when someone successfully logs in. Six nodes. Five added agents. One server that finally answers its messages.&lt;/p&gt;

&lt;p&gt;Try the controlled login test, then tell us which agent you would add next. The server has completed basic communication training. We should probably use this opportunity before it discovers meetings.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Victory Always!&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;🌐 &lt;strong&gt;Links &amp;amp; Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try MatrixSwarm: &lt;a href="https://matrixswarm.com" rel="noopener noreferrer"&gt;https://matrixswarm.com&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Join the Community / Discord: &lt;a href="https://discord.gg/2USbWVBVV" rel="noopener noreferrer"&gt;https://discord.gg/2USbWVBVV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Download Server: &lt;a href="https://github.com/matrixswarm/matrixswarm" rel="noopener noreferrer"&gt;https://github.com/matrixswarm/matrixswarm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug" rel="noopener noreferrer"&gt;https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug&lt;/a&gt;&lt;/p&gt;

</description>
      <category>devops</category>
      <category>tutorial</category>
      <category>opensource</category>
      <category>linux</category>
    </item>
    <item>
      <title>NPC Simulator: 100 NPCs, One Dashboard, and a Suspicious Amount of Cowbell.</title>
      <dc:creator>Daniel</dc:creator>
      <pubDate>Thu, 15 Oct 2026 05:39:04 +0000</pubDate>
      <link>https://dev.to/matrixswarm/npc-simulator-100-npcs-one-dashboard-and-a-suspicious-amount-of-cowbell-21c5</link>
      <guid>https://dev.to/matrixswarm/npc-simulator-100-npcs-one-dashboard-and-a-suspicious-amount-of-cowbell-21c5</guid>
      <description>&lt;p&gt;The briefing was simple: deploy a small simulation, demonstrate command flow, and do not touch production.&lt;/p&gt;

&lt;p&gt;Thirty seconds later, the scouts were wandering, the hunters were pursuing stale information, and sixty followers were heading toward the average location of management.&lt;/p&gt;

&lt;p&gt;Finally. A realistic organizational model.&lt;/p&gt;

&lt;p&gt;MatrixSwarm’s &lt;code&gt;npc_simulator&lt;/code&gt; is a compact, rule-based simulation with a Phoenix gameboard. It gives you something visible to control while learning how a swarm receives commands and returns signed state updates.&lt;/p&gt;

&lt;p&gt;There is a player marker, three kinds of NPC, several temporary effects, and a Cowbell button whose strategic importance is difficult to overstate.&lt;/p&gt;

&lt;p&gt;This walkthrough uses the current implementation as it exists: &lt;strong&gt;one agent, a 20×20 board, 100 NPC records, and one viewer session&lt;/strong&gt;. There is no language-model call behind each dot and no need to stage a real outage to make the screen interesting.&lt;/p&gt;

&lt;h2&gt;
  
  
  One agent, one crowded little world
&lt;/h2&gt;

&lt;p&gt;The default population is:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Character&lt;/th&gt;
&lt;th&gt;Board appearance&lt;/th&gt;
&lt;th&gt;Assignment&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Player&lt;/td&gt;
&lt;td&gt;Green &lt;code&gt;@&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;Move automatically around the board&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10 scouts&lt;/td&gt;
&lt;td&gt;Yellow NPC markers&lt;/td&gt;
&lt;td&gt;Wander and report a nearby player position&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;30 hunters&lt;/td&gt;
&lt;td&gt;Red NPC markers&lt;/td&gt;
&lt;td&gt;Pursue the shared last-seen player position&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;60 followers&lt;/td&gt;
&lt;td&gt;Cyan NPC markers&lt;/td&gt;
&lt;td&gt;Move toward the hunters’ average position&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Those are &lt;strong&gt;100 entities maintained by one MatrixSwarm agent&lt;/strong&gt;, not 100 independently deployed swarm agents. The distinction matters when explaining the architecture—or when someone begins multiplying an imaginary per-NPC API bill.&lt;/p&gt;

&lt;p&gt;The player is automatic too. This panel does not provide WASD movement. You influence the exercise with its command buttons while the player jitters around the grid.&lt;/p&gt;

&lt;p&gt;The world is deliberately small enough to inspect. You can see the difference between “the button changed something” and “we clicked it and everyone nodded because the demo was nearly over.”&lt;/p&gt;

&lt;h2&gt;
  
  
  Install, assemble, launch—then release the dots
&lt;/h2&gt;

&lt;p&gt;If MatrixOS is not already on the target, add and test the target’s SSH connection in Phoenix’s Registry, including its trusted host fingerprint. Open Railgun’s &lt;strong&gt;Install MatrixOS&lt;/strong&gt; flow and select &lt;strong&gt;Local Full Install&lt;/strong&gt; or &lt;strong&gt;Install from GitHub&lt;/strong&gt;. For a fresh setup, use &lt;strong&gt;Create new venv&lt;/strong&gt; with the installer’s required root or passwordless-sudo access.&lt;/p&gt;

&lt;p&gt;Open &lt;strong&gt;Deploy → Swarm Workspaces&lt;/strong&gt; and create a workspace or edit an existing one. Keep the Matrix root and add:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;npc_simulator&lt;/code&gt;;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;matrix_https&lt;/code&gt; for command ingress;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;matrix_websocket&lt;/code&gt; for the return stream;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;log_streamer&lt;/code&gt; for Agent Logs and troubleshooting.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Keep the NPC agent’s default board/population settings and supplied panel/command declarations. Resolve packet signing and the connection requirements. The return path uses the RPC relay role, normally &lt;code&gt;hive.rpc&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Save and &lt;strong&gt;Deploy&lt;/strong&gt; through Railgun with the target SSH record, universe, label, and runtime account. Railgun carries the sealed boot envelope over pinned SSH to MatrixD; MatrixD decrypts it in memory and launches the swarm. Connect Phoenix and open &lt;strong&gt;NPC Panel&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The gameboard is the interface for this exercise. Start with its controls and the supplied defaults; you do not need a separate NPC configuration panel.&lt;/p&gt;

&lt;p&gt;Briefing over. The yellow dots have already wandered away from it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What each group actually knows
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Scouts: reconnaissance with questionable cardio
&lt;/h3&gt;

&lt;p&gt;Scouts choose a random cardinal move or stay put. Under ordinary conditions, a scout detects the player when their &lt;strong&gt;Manhattan distance is less than four&lt;/strong&gt;—count horizontal plus vertical grid steps, not a straight diagonal line.&lt;/p&gt;

&lt;p&gt;A detection updates a shared last-seen player coordinate. That is the information the hunters use.&lt;/p&gt;

&lt;p&gt;The scout has not built a belief model, consulted a map, or sent an emotionally complicated radio message. It saw something close enough and updated a coordinate. Admirably concise field reporting.&lt;/p&gt;

&lt;h3&gt;
  
  
  Hunters: an address and considerable confidence
&lt;/h3&gt;

&lt;p&gt;Hunters move toward that shared last-seen coordinate. Their movement chooses the horizontal direction more often than the vertical direction. Without a target, they have nothing to pursue.&lt;/p&gt;

&lt;p&gt;The distinction between the player’s &lt;strong&gt;current&lt;/strong&gt; position and the &lt;strong&gt;last-seen&lt;/strong&gt; position is the interesting part. A hunter can be faithfully following information that is already old. The player continues moving; scouts may refresh the sighting.&lt;/p&gt;

&lt;p&gt;This is not predictive pathfinding. It is a small, visible example of agents acting on shared, imperfect information. Anyone who has followed an outdated incident ticket will recognize the terrain.&lt;/p&gt;

&lt;h3&gt;
  
  
  Followers: the center of the committee
&lt;/h3&gt;

&lt;p&gt;Followers move toward the integer average of the hunters’ coordinates. They are following the hunters as a group, rather than independently tracking the player.&lt;/p&gt;

&lt;p&gt;That makes the behavior legible: move the hunting group and the following group tends to respond. The resulting crowd is simple enough to reason about while still producing a lively board.&lt;/p&gt;

&lt;p&gt;If this reminds you of a meeting where nobody knows the objective but everyone knows where the managers are standing, please submit your findings through the proper channel.&lt;/p&gt;

&lt;h2&gt;
  
  
  The command desk
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Button&lt;/th&gt;
&lt;th&gt;Current effect&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Start&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Begin this session’s state stream and simulation stepping&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Stop&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Stop this session’s stream&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Hunt&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Set the shared target to the player’s current position at that moment&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Scatter&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Override NPC movement with random movement for three simulation ticks&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Ping&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Increase scout detection distance to less than six for two ticks&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Shield&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Reverse hunter pursuit away from the remembered target for five ticks&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Lock&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Freeze NPC movement for five ticks; the player still moves&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Cowbell&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Play a local sound if the sound asset and audio output are available&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A &lt;strong&gt;tick&lt;/strong&gt; here means a simulation step. The current stream loop uses a fixed interval of about 1.5 seconds, with processing and transport adding time. These are short-lived effects in a deliberately low-rate demo, not promises of frame-perfect timing.&lt;/p&gt;

&lt;p&gt;The palette’s &lt;code&gt;tick_interval_sec&lt;/code&gt; setting is not currently what drives that stream cadence. Changing it does not give you a 60 FPS tactical command center.&lt;/p&gt;

&lt;p&gt;Cowbell does not send an agent command or alter the simulation. It lives in the Phoenix panel. Sometimes the correct architectural boundary is between distributed state and a deeply necessary percussion instrument.&lt;/p&gt;

&lt;h2&gt;
  
  
  A field exercise you can narrate in five minutes
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;First, press Start and watch a few updates.&lt;/strong&gt; Identify the player, scouts, hunters, and followers. The exact arrangement varies because initial placement and movement are random. Do not expect someone else’s board to match yours.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Next, press Hunt.&lt;/strong&gt; This supplies an immediate target using the player’s position at that instant. Watch the hunters respond. It does not permanently attach them to a live GPS feed; later scout sightings can update the remembered position.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Press Scatter and let it run its three ticks.&lt;/strong&gt; The NPCs temporarily abandon their role movement. After the effect expires, normal role behavior returns.&lt;/p&gt;

&lt;p&gt;Scatter uses a different movement path from ordinary movement and can place multiple NPCs in the same cell. The board can show a count where characters overlap. A number on a crowded tile is not necessarily a rendering failure. Sometimes everyone really did choose the same exit.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Try Ping.&lt;/strong&gt; For two ticks, scouts can detect the player from farther away. The effect changes their detection threshold; it does not guarantee a new sighting if every scout is still too far from the player.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Try Shield when hunters have a target.&lt;/strong&gt; For five ticks, hunters reverse their pursuit direction relative to the remembered target. This affects hunters. It is not an invulnerability system, and it does not turn every NPC into a fleeing civilian.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Try Lock.&lt;/strong&gt; NPC movement freezes for five ticks, while the player continues moving. The other effect counters can continue counting down too, so a locked board does not preserve every temporary effect in amber.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Press Cowbell, purely for scientific completeness.&lt;/strong&gt; If your audio setup obliges, morale will become measurable in decibels.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Finally, press Stop.&lt;/strong&gt; Use the actual Stop control when finishing the exercise. Hiding or leaving the panel is not a reliable substitute for stopping its stream.&lt;/p&gt;

&lt;p&gt;The point is to connect each button to an observable change. You have now exercised commands, state updates, short-lived policy changes, and session lifecycle without persuading a production database to provide the entertainment.&lt;/p&gt;

&lt;h2&gt;
  
  
  What travels through the swarm
&lt;/h2&gt;

&lt;p&gt;At a high level, the control loop is:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Phoenix NPC Panel
    → command ingress
    → NPC Simulator command handler
    → simulation step and signed state frame
    → RPC/WebSocket return path
    → Phoenix board update
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;The simulation owns the positions and temporary-effect counters. Phoenix presents the controls and renders returned state. The panel consumes verified callbacks; signing requirements are part of the deployment, not an optional flourish around the demonstration.&lt;/p&gt;

&lt;p&gt;The board keeps a small frame queue and uses the latest frame instead of faithfully animating every stale update. That is sensible for a live status display: when the world has moved on, you usually want to see where it is now.&lt;/p&gt;

&lt;p&gt;It is not an event replay system, a durable simulation log, or a guarantee that every intermediate move will be drawn on screen.&lt;/p&gt;

&lt;p&gt;Likewise, &lt;code&gt;log_streamer&lt;/code&gt; is useful here for the agents’ diagnostic logs. The NPC board’s structured state frames travel through its own callback path. Agent Logs and the gameboard are complementary views, not interchangeable streams.&lt;/p&gt;

&lt;h2&gt;
  
  
  The important prototype boundary: use one viewer
&lt;/h2&gt;

&lt;p&gt;The current agent holds &lt;strong&gt;one shared world&lt;/strong&gt;. Each active viewer session starts its own stream loop, and those loops advance that same world.&lt;/p&gt;

&lt;p&gt;Consequently, opening multiple viewers is not merely adding spectators. It can change stepping speed and interleave state updates. This walkthrough deliberately uses &lt;strong&gt;one viewer session&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;With no active, healthy stream loop, this implementation is not running an independent background simulation clock. Start and Stop therefore matter to both observation and progression. Stop affects your session; another active viewer can keep advancing the world.&lt;/p&gt;

&lt;p&gt;The stream also depends on an available RPC relay and a live session connection indicator. A missing or stale connection indicator ends the stream; reconnect and start again when the connection is healthy.&lt;/p&gt;

&lt;p&gt;Stop followed by Start resumes the current in-memory world. It is not a reset button. Restarting the agent creates a new randomized world; this demo does not persist its simulation state.&lt;/p&gt;

&lt;p&gt;These details make it a useful teaching prototype. They also explain why we are calling it a small simulation exercise rather than announcing an MMO and making the networking team quietly leave the room.&lt;/p&gt;

&lt;h2&gt;
  
  
  Keep the supplied dimensions for now
&lt;/h2&gt;

&lt;p&gt;The current Phoenix board is built around a 20×20 grid. The agent’s default population is ten scouts, thirty hunters, and sixty followers.&lt;/p&gt;

&lt;p&gt;Keep those defaults for this walkthrough. Lower population values slice that role list rather than proportionally scaling the three groups, and the current list does not create arbitrary numbers above 100. Making the grid too small for its unique initial placements can also prevent initialization from completing.&lt;/p&gt;

&lt;p&gt;Changing a number in configuration is not automatically the same thing as supporting every number. This principle has prevented at least several extremely awkward demos.&lt;/p&gt;

&lt;p&gt;There is no combat system, health bar, score, win condition, navmesh, or language-model reasoning in these NPC rules. You can inspect their behavior as code, follow it visually, and use the exercise to understand the transport around it.&lt;/p&gt;

&lt;p&gt;That is already plenty of work for one hundred dots.&lt;/p&gt;

&lt;h2&gt;
  
  
  When the board refuses to perform
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Symptom&lt;/th&gt;
&lt;th&gt;First useful check&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;No frames after Start&lt;/td&gt;
&lt;td&gt;Connection, command roles, the &lt;code&gt;hive.rpc&lt;/code&gt; relay, and required signing assignments&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Stream stops unexpectedly&lt;/td&gt;
&lt;td&gt;Session connectivity and heartbeat freshness; reconnect, then Start&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;NPCs freeze but the player moves&lt;/td&gt;
&lt;td&gt;Lock may still be active&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Everything looks stale&lt;/td&gt;
&lt;td&gt;Agent Logs and the return path; do not diagnose transport from one frozen picture&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Player marker seems missing&lt;/td&gt;
&lt;td&gt;Overlapping NPCs can obscure its cell&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Simulation speeds up with another viewer&lt;/td&gt;
&lt;td&gt;Both sessions may be advancing the shared world&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cowbell is silent&lt;/td&gt;
&lt;td&gt;Local sound asset and audio output, rather than the remote agent&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;If you extend the demo, a useful next engineering step would be separating simulation time from viewer streams, then making each viewer a read-only consumer of the same world clock. A seeded random generator would also make repeatable demonstrations easier. Those are potential improvements, not features claimed by this walkthrough.&lt;/p&gt;

&lt;p&gt;For now, you have a small, interactive place to learn the swarm’s control loop. Commands have visible consequences. Return traffic has a purpose you can point at. The agents’ logs help explain a stalled demonstration.&lt;/p&gt;

&lt;p&gt;And if somebody asks what business value the Cowbell provides, tell them the distributed system has achieved eventual enthusiasm.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Victory Always. Keep the hunters informed and the briefing short.&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;🌐 &lt;strong&gt;Links &amp;amp; Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try MatrixSwarm: &lt;a href="https://matrixswarm.com" rel="noopener noreferrer"&gt;https://matrixswarm.com&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Join the Community / Discord: &lt;a href="https://discord.gg/2USbWVBVV" rel="noopener noreferrer"&gt;https://discord.gg/2USbWVBVV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Download Server: &lt;a href="https://github.com/matrixswarm/matrixswarm" rel="noopener noreferrer"&gt;https://github.com/matrixswarm/matrixswarm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug" rel="noopener noreferrer"&gt;https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug&lt;/a&gt;&lt;/p&gt;

</description>
      <category>gamedev</category>
      <category>python</category>
      <category>opensource</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>WordPress Plugin Guard: Your Plugin Changed Overnight. It Did Not Leave a Note.</title>
      <dc:creator>Daniel</dc:creator>
      <pubDate>Thu, 15 Oct 2026 05:35:32 +0000</pubDate>
      <link>https://dev.to/matrixswarm/wordpress-plugin-guard-your-plugin-changed-overnight-it-did-not-leave-a-note-26g9</link>
      <guid>https://dev.to/matrixswarm/wordpress-plugin-guard-your-plugin-changed-overnight-it-did-not-leave-a-note-26g9</guid>
      <description>&lt;p&gt;At 08:57, the website works.&lt;/p&gt;

&lt;p&gt;At 08:58, someone says, “I only updated one plugin.”&lt;/p&gt;

&lt;p&gt;At 09:03, the checkout has become a philosophical question, a new directory has appeared under &lt;code&gt;wp-content/plugins&lt;/code&gt;, and everyone is suddenly very interested in who has SSH access.&lt;/p&gt;

&lt;p&gt;Welcome to the hotel. The plugins have room keys. Some of them appear to be renovating.&lt;/p&gt;

&lt;p&gt;MatrixSwarm’s &lt;strong&gt;WordPress Plugin Guard&lt;/strong&gt; gives an operator a way to inventory plugin directories, approve a local file baseline, request integrity comparisons, and manage unexpected folders from Phoenix. It can also notify a swarm alarm relay when its directory-monitoring path detects an untracked or missing plugin folder.&lt;/p&gt;

&lt;p&gt;The useful question is simple: &lt;strong&gt;does this directory still contain the bytes we approved?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;That question deserves a better answer than “the homepage looks normal from my laptop.”&lt;/p&gt;

&lt;h2&gt;
  
  
  The guest list and the luggage check
&lt;/h2&gt;

&lt;p&gt;Plugin Guard maintains a local baseline of relative file paths and SHA-256 hashes for an approved plugin directory. A later comparison can identify added, deleted, or changed files relative to that baseline.&lt;/p&gt;

&lt;p&gt;There are two distinct inspection paths in the current implementation:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Operation&lt;/th&gt;
&lt;th&gt;What it checks&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Background directory scan&lt;/td&gt;
&lt;td&gt;Untracked plugin folders and trusted folders that have disappeared&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;strong&gt;Show Status&lt;/strong&gt; in Phoenix&lt;/td&gt;
&lt;td&gt;Current inventory plus file-content comparisons for tracked plugins&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Tracked file-content changes are compared when you request Show Status. The background scan does not continuously hash every tracked plugin.&lt;/strong&gt; A changed file can therefore appear as an integrity alert in the panel after you refresh status without having generated an automatic chat notification.&lt;/p&gt;

&lt;p&gt;This is a practical distinction. “I checked the guest list” and “I searched every suitcase” are different services, even when the receptionist is wearing a tactical vest.&lt;/p&gt;

&lt;p&gt;The panel groups entries into tracked/clean, tracked/alerting, untracked, and quarantined. A clean result means the inspected files match your saved baseline. It does not mean a malware analyst has blessed them.&lt;/p&gt;

&lt;h2&gt;
  
  
  A small swarm, with a short installation speech
&lt;/h2&gt;

&lt;p&gt;If the target does not have MatrixOS, add and test its SSH record in Phoenix’s Registry, including the trusted host fingerprint. In Railgun, use &lt;strong&gt;Install MatrixOS&lt;/strong&gt; with &lt;strong&gt;Local Full Install&lt;/strong&gt; or &lt;strong&gt;Install from GitHub&lt;/strong&gt;. A fresh installation can use &lt;strong&gt;Create new venv&lt;/strong&gt;; the installer needs the required root or passwordless-sudo access on the target.&lt;/p&gt;

&lt;p&gt;Open &lt;strong&gt;Deploy → Swarm Workspaces&lt;/strong&gt;, create a workspace or edit an existing one, and keep the Matrix root. Add:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;wordpress_plugin_guard&lt;/code&gt; for the inspection;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;matrix_https&lt;/code&gt; for command ingress;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;matrix_websocket&lt;/code&gt; for replies and telemetry;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;log_streamer&lt;/code&gt; for the agents’ runtime logs;&lt;/li&gt;
&lt;li&gt;optionally, a Slack, Discord, Telegram, or email alarm relay.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Keep the guard’s supplied &lt;code&gt;plugin.guard.*&lt;/code&gt; routes and its panel declaration. Resolve packet signing and the declared connection requirements. An existing working ingress/egress pair can be reused.&lt;/p&gt;

&lt;p&gt;Save the workspace and &lt;strong&gt;Deploy&lt;/strong&gt; through Railgun, selecting the SSH record, universe, label, and runtime account. Railgun delivers the sealed boot envelope over pinned SSH; MatrixD decrypts it in memory and launches the swarm. Connect Phoenix and open &lt;strong&gt;Plugin Guard&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;There. Deployment briefing complete. Nobody had to attend a webinar about the webinar.&lt;/p&gt;

&lt;h2&gt;
  
  
  Tell the guard which hotel it works for
&lt;/h2&gt;

&lt;p&gt;The guard runs on the host where it can access the WordPress plugin files. Configure the actual filesystem location, not a public website URL.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Setting&lt;/th&gt;
&lt;th&gt;Example or intent&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Plugin directory&lt;/td&gt;
&lt;td&gt;The real &lt;code&gt;wp-content/plugins&lt;/code&gt; directory for this site&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Site ID&lt;/td&gt;
&lt;td&gt;A distinct name such as &lt;code&gt;shop-staging&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Snapshot root&lt;/td&gt;
&lt;td&gt;A protected location such as &lt;code&gt;/opt/swarm/guard/snapshots&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Quarantine root&lt;/td&gt;
&lt;td&gt;A location outside the served website, such as &lt;code&gt;/opt/quarantine/wp_plugins&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Scan interval&lt;/td&gt;
&lt;td&gt;Your desired directory-scan interval; scan work adds to the elapsed time&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Alert role&lt;/td&gt;
&lt;td&gt;Normally &lt;code&gt;hive.alert&lt;/code&gt; for the selected relay&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The palette includes &lt;code&gt;/var/www/html/wordpress/wp-content/plugins&lt;/code&gt; as an example. Many installations live elsewhere. A default path is a suggestion, not a successful discovery expedition.&lt;/p&gt;

&lt;p&gt;The site ID separates the stored records under the snapshot and quarantine roots. The runtime account needs suitable access to inspect plugin files and write its own baseline records. Quarantine additionally requires permission to move the affected plugin directory.&lt;/p&gt;

&lt;p&gt;For the first exercise, keep &lt;strong&gt;Enforce off&lt;/strong&gt; and &lt;strong&gt;Block New off&lt;/strong&gt;. Verify the live panel state after connecting: saved manifest policy can be restored across deployments. Do not assume that a workspace checkbox is the final word about an already-running guard.&lt;/p&gt;

&lt;p&gt;Also, the current &lt;strong&gt;Read only / no writes&lt;/strong&gt; option is not a universal guardrail around every operation; it gates baseline writes. It is not a substitute for disabling enforcement, leaving Block New off, and choosing appropriate filesystem permissions.&lt;/p&gt;

&lt;h2&gt;
  
  
  Approve means “remember these bytes”
&lt;/h2&gt;

&lt;p&gt;Select an inspected plugin and use &lt;strong&gt;Approve&lt;/strong&gt; to create its baseline. Do this after establishing that the files are the version you intend to trust—such as a known release from its publisher or your reviewed deployment artifact.&lt;/p&gt;

&lt;p&gt;Approving whatever happens to be on a compromised server would faithfully preserve the attacker’s homework.&lt;/p&gt;

&lt;p&gt;For WordPress.org-hosted plugins, WP-CLI also offers an independent comparison against the checksums published by WordPress.org:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;wp plugin verify-checksums --all
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;Run it in the appropriate WordPress installation, or supply the correct path according to your WP-CLI setup. See the &lt;a href="https://developer.wordpress.org/cli/commands/plugin/verify-checksums/" rel="noopener noreferrer"&gt;official verify-checksums documentation&lt;/a&gt; for options and coverage. A custom or commercial plugin may need verification against its own trusted release artifact instead.&lt;/p&gt;

&lt;p&gt;The two comparisons answer different questions: the publisher checksum check compares with a published release; Plugin Guard’s local baseline compares with the bytes you previously approved. Neither is a complete security assessment.&lt;/p&gt;

&lt;p&gt;The stored baseline records are local JSON hash maps. Protect them from modification by accounts that should not be able to redefine “trusted.” Signed swarm messages do &lt;strong&gt;not&lt;/strong&gt; make those files cryptographically tamper-proof.&lt;/p&gt;

&lt;p&gt;And hashes are not backups. A fingerprint can tell you the suitcase changed. It cannot recreate the trousers.&lt;/p&gt;

&lt;h2&gt;
  
  
  A harmless staging exercise
&lt;/h2&gt;

&lt;p&gt;Use a staging site and a disposable directory named &lt;code&gt;swarm-guard-demo&lt;/code&gt; under its configured plugins directory. Put a harmless &lt;code&gt;README.txt&lt;/code&gt; inside containing:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;This is the original demonstration text.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;This does not need to be an activated WordPress plugin. The guard inspects directories; we are testing its file comparison, not deploying PHP that has aspirations.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Confirm &lt;strong&gt;Enforce off&lt;/strong&gt; and &lt;strong&gt;Block New off&lt;/strong&gt; before adding the demonstration folder.&lt;/li&gt;
&lt;li&gt;Open Plugin Guard and choose &lt;strong&gt;Show Status&lt;/strong&gt;. The new directory should appear as untracked.&lt;/li&gt;
&lt;li&gt;Select only that demonstration entry and &lt;strong&gt;Approve&lt;/strong&gt; it.&lt;/li&gt;
&lt;li&gt;Request &lt;strong&gt;Show Status&lt;/strong&gt; again. Its current contents should match the approved baseline.&lt;/li&gt;
&lt;li&gt;Add a second line to &lt;code&gt;README.txt&lt;/code&gt;, then request &lt;strong&gt;Show Status&lt;/strong&gt; once more.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;You should now see a tracked integrity alert for that directory. The status view indicates that its files differ; it does not provide a full file-by-file diff. Use your normal file comparison tools to inspect exactly what changed.&lt;/p&gt;

&lt;p&gt;The crucial lesson: &lt;strong&gt;this tracked-file exercise demonstrates a requested integrity check, not an automatic alarm for every file edit.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Restore the original contents of &lt;code&gt;README.txt&lt;/code&gt; and check status again. Matching the original bytes should clear that comparison. Do not press Mark Clean merely to make a warning disappear.&lt;/p&gt;

&lt;p&gt;For a separate relay exercise, create another harmless, untracked directory called &lt;code&gt;swarm-untracked-demo&lt;/code&gt; while both enforcement options remain off. That exercises the background directory detection path. Confirm that the expected notification actually reaches your chosen test destination.&lt;/p&gt;

&lt;p&gt;Use a new demonstration name when repeating an alert test: the current process suppresses repeated alerts using in-memory event keys. That suppression is not a durable delivery queue or a guarantee of repeated reminders.&lt;/p&gt;

&lt;h2&gt;
  
  
  The buttons are verbs, not decorations
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Control&lt;/th&gt;
&lt;th&gt;What it means&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Show Status&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Refresh inventory and compare tracked file contents&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Approve&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Save the selected directory’s current files as trusted baseline&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Mark Clean&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Replace the selected plugin’s baseline with its current contents&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Disapprove&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Remove its trusted baseline/bookkeeping&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Quarantine&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Move the selected plugin folder out of the active plugin tree&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Restore&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Move a quarantined folder back; inspect and verify its status afterward&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;strong&gt;Delete&lt;/strong&gt; on a quarantined entry&lt;/td&gt;
&lt;td&gt;Permanently remove that quarantined copy&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Mark Clean changes the reference. It does not repair the plugin.&lt;/strong&gt; If an update is legitimate, review the new release, verify the expected changes, then deliberately establish a new baseline. Keep the normal deployment record so the next operator knows why “trusted” changed on Thursday.&lt;/p&gt;

&lt;p&gt;Quarantine is an operational action. Removing an active plugin’s directory can affect checkout, forms, authentication, or anything else that plugin supplies. A suspicious folder and a safe time to remove it are two separate judgments.&lt;/p&gt;

&lt;p&gt;Restore also does not disinfect anything. You are putting the same luggage back in the room. Inspect it before welcoming it home.&lt;/p&gt;

&lt;h2&gt;
  
  
  Enforce and Block New are very different policies
&lt;/h2&gt;

&lt;p&gt;With &lt;strong&gt;Enforce enabled&lt;/strong&gt;, the current background path can quarantine an untracked plugin folder. That means moving it to the configured quarantine location; the implementation can fall back to copying it there and deleting the original if the move fails.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Block New is more aggressive: it deletes an untracked directory rather than preserving a quarantine copy. It takes priority over the quarantine policy.&lt;/strong&gt; Treat that toggle as a deletion policy, not as an alert-volume preference.&lt;/p&gt;

&lt;p&gt;Neither option turns the current background worker into continuous content-change enforcement for every approved plugin. That is why we separated inventory scans from Show Status at the beginning.&lt;/p&gt;

&lt;p&gt;Start by observing a staging installation. Understand how planned plugin installations and updates interact with your chosen policy before applying enforcement to a live site. Otherwise, the guard may correctly enforce a policy that your release process has never heard of.&lt;/p&gt;

&lt;p&gt;The result would be technically consistent and socially expensive.&lt;/p&gt;

&lt;h2&gt;
  
  
  Send the findings somewhere an operator will see them
&lt;/h2&gt;

&lt;p&gt;The directory-alert path can use the swarm’s &lt;strong&gt;Slack, Discord, Telegram, or email&lt;/strong&gt; relays through the configured alert role. The Discord, Telegram, and email options support optional signed payload encryption when the matching relay and receiver configuration is in place.&lt;/p&gt;

&lt;p&gt;A general alert may use urgent wording about an unauthorized plugin. Read that as “not present in this guard’s trusted records,” not as proof of hostile code. A legitimate installation can be untracked too.&lt;/p&gt;

&lt;p&gt;Make one harmless test reach the actual destination. A configured relay with invalid credentials is a very sophisticated way to notify nobody.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this guard does not inspect for you
&lt;/h2&gt;

&lt;p&gt;This implementation focuses on plugin &lt;strong&gt;directories&lt;/strong&gt; in the configured location. Do not assume coverage for standalone PHP files directly under the plugins root, must-use plugins elsewhere, themes, WordPress core, the database, or other sites on the host.&lt;/p&gt;

&lt;p&gt;There is also a current naming limitation: some baseline and restore paths normalize names by splitting at an underscore. Plugin folders containing underscores can collide or be identified unexpectedly. The demonstration uses hyphens deliberately. Check your real inventory before relying on the guard for those names.&lt;/p&gt;

&lt;p&gt;The value remains useful and specific: a visible inventory, a deliberately approved reference, an on-demand integrity check, and explicit controls for unexpected directories. Keep backups, trusted deployment artifacts, and your normal investigation tools alongside it.&lt;/p&gt;

&lt;p&gt;When a plugin changes overnight, you should be able to ask what changed and why.&lt;/p&gt;

&lt;p&gt;“Must have been housekeeping” is not a change ticket.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Victory Always. Check the guest list before handing out room keys.&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;🌐 &lt;strong&gt;Links &amp;amp; Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try MatrixSwarm: &lt;a href="https://matrixswarm.com" rel="noopener noreferrer"&gt;https://matrixswarm.com&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Join the Community / Discord: &lt;a href="https://discord.gg/2USbWVBVV" rel="noopener noreferrer"&gt;https://discord.gg/2USbWVBVV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Download Server: &lt;a href="https://github.com/matrixswarm/matrixswarm" rel="noopener noreferrer"&gt;https://github.com/matrixswarm/matrixswarm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug" rel="noopener noreferrer"&gt;https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug&lt;/a&gt;&lt;/p&gt;

</description>
      <category>wordpress</category>
      <category>security</category>
      <category>devops</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Log Monitor + Oracle: Your Error Log Has 500 Lines. Your Patience Has Three.</title>
      <dc:creator>Daniel</dc:creator>
      <pubDate>Thu, 15 Oct 2026 05:32:29 +0000</pubDate>
      <link>https://dev.to/matrixswarm/log-monitor-oracle-your-error-log-has-500-lines-your-patience-has-three-4f3f</link>
      <guid>https://dev.to/matrixswarm/log-monitor-oracle-your-error-log-has-500-lines-your-patience-has-three-4f3f</guid>
      <description>&lt;p&gt;The application has produced an error.&lt;/p&gt;

&lt;p&gt;Then another error explaining that the first error was unable to report its error.&lt;/p&gt;

&lt;p&gt;Somewhere beneath this administrative avalanche is the useful line. You have three terminals open, a cooling cup of coffee, and a colleague asking whether you have tried restarting it.&lt;/p&gt;

&lt;p&gt;You have considered restarting the colleague.&lt;/p&gt;

&lt;p&gt;MatrixSwarm’s &lt;strong&gt;Log Monitor&lt;/strong&gt;, supplied by the &lt;code&gt;log_health&lt;/code&gt; agent, gives you a bounded view of a configured server log inside Phoenix. Select relevant entries, review the excerpt, and use &lt;strong&gt;Ask Oracle&lt;/strong&gt; to request an explanation from the swarm’s configured language-model agent.&lt;/p&gt;

&lt;p&gt;The useful outcome is modest and concrete: fewer trips between windows, a smaller body of evidence, and a second opinion you can inspect before deciding what to do.&lt;/p&gt;

&lt;h2&gt;
  
  
  Meet the two members of the incident desk
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Component&lt;/th&gt;
&lt;th&gt;Job&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;log_health&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Read newly appended lines from one configured server log, classify them, and return signed snapshots&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Phoenix &lt;strong&gt;Log Monitor&lt;/strong&gt;
&lt;/td&gt;
&lt;td&gt;Display, search, filter, pause the view, and let you choose an excerpt&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;oracle&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Send the requested excerpt to its configured AI provider and return an analysis&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Oracle is optional for viewing the log. If its configuration is unavailable, the monitoring part still has a job and should continue doing it.&lt;/p&gt;

&lt;p&gt;This is also different from &lt;code&gt;log_streamer&lt;/code&gt;. Log Streamer brings &lt;strong&gt;the agents’ own runtime logs&lt;/strong&gt; into Phoenix’s Agent Logs. Log Health reads &lt;strong&gt;the particular application or service log you configured&lt;/strong&gt;, such as an Apache error log. One helps investigate the application; the other helps investigate the investigator.&lt;/p&gt;

&lt;p&gt;We try to keep the chain of confusion short.&lt;/p&gt;

&lt;h2&gt;
  
  
  Build the small swarm, then get back to the interesting part
&lt;/h2&gt;

&lt;p&gt;If MatrixOS is not installed on the target, add and test its SSH connection in Phoenix’s Registry, including its trusted host fingerprint. Open Railgun’s &lt;strong&gt;Install MatrixOS&lt;/strong&gt; flow. Choose &lt;strong&gt;Local Full Install&lt;/strong&gt; with the appropriate source directory or &lt;strong&gt;Install from GitHub&lt;/strong&gt;. For a fresh installation, use &lt;strong&gt;Create new venv&lt;/strong&gt;. The target needs the installation’s required root or passwordless-sudo access.&lt;/p&gt;

&lt;p&gt;In &lt;strong&gt;Deploy → Swarm Workspaces&lt;/strong&gt;, create a workspace or open an existing one. Keep the Matrix root and add:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;log_health&lt;/code&gt; for the application log;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;oracle&lt;/code&gt; for the optional analysis;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;matrix_https&lt;/code&gt; for Phoenix command ingress;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;matrix_websocket&lt;/code&gt; for the return path;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;log_streamer&lt;/code&gt; so Agent Logs are available when something needs inspection.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Reuse an existing working ingress/egress pair if you already have one. Resolve packet signing and the connection requirements, and assign Oracle’s provider credentials through the declared Registry constraint. Choose a model available to that account; the example does not depend on a particular model name.&lt;/p&gt;

&lt;p&gt;Save, then &lt;strong&gt;Deploy&lt;/strong&gt; with the appropriate SSH record, universe, label, and runtime account. Railgun delivers the sealed boot envelope over pinned SSH to MatrixD, which decrypts it in memory and launches the swarm. Connect Phoenix to the resulting deployment.&lt;/p&gt;

&lt;p&gt;That is the deployment tour. Please return your hard hat; the log has started talking again.&lt;/p&gt;

&lt;h2&gt;
  
  
  Give the reader one actual file
&lt;/h2&gt;

&lt;p&gt;Open the Log Health workspace configuration and set these fields:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Field&lt;/th&gt;
&lt;th&gt;What to put there&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Server log file&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;The real absolute path on the MatrixOS host&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Service name&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;A recognizable identity such as &lt;code&gt;checkout.error_log&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Forensic report role&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Usually &lt;code&gt;hive.forensics.data_feed&lt;/code&gt; if using Forensic Detective&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Oracle role (one agent)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Normally &lt;code&gt;hive.oracle&lt;/code&gt;, resolving to exactly one Oracle&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Enable Log Monitor and Ask Oracle&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Enabled for the panel workflow&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;On an Ubuntu Apache host, &lt;code&gt;/var/log/apache2/error.log&lt;/code&gt; is a common example. The palette’s &lt;code&gt;/var/log/httpd/error_log&lt;/code&gt; default is not a universal address. Your server did not sign a contract promising that directory layout.&lt;/p&gt;

&lt;p&gt;The runtime account must already be allowed to read the file. &lt;strong&gt;Check access&lt;/strong&gt; tests whether that account can open the configured path now. It does not change permissions or grant access to additional files.&lt;/p&gt;

&lt;p&gt;The path must identify a regular file. A symlink as the final path component is refused; configure the real target instead. The panel cannot browse the server, choose arbitrary files, or execute a shell command.&lt;/p&gt;

&lt;p&gt;For an older Log Health node, enabling the monitor in the editor adds the panel routes and packet-signing constraint. Save, resolve the newly required assignments, and redeploy. Replacing a Python file alone does not add the missing deployment declarations.&lt;/p&gt;

&lt;h2&gt;
  
  
  The first surprise: it starts at the end
&lt;/h2&gt;

&lt;p&gt;At the reader’s first open, it seeks to the end of the file. &lt;strong&gt;Newly appended entries&lt;/strong&gt; are what appear. This avoids turning yesterday’s incident into today’s surprise reunion.&lt;/p&gt;

&lt;p&gt;It also means an empty panel is not proof that the file is empty. The existing history may simply predate the reader. Append a harmless demonstration line after the agent has opened its file, or wait for a legitimate new event.&lt;/p&gt;

&lt;p&gt;The view is deliberately bounded:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Bound&lt;/th&gt;
&lt;th&gt;Current behavior&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Reader history&lt;/td&gt;
&lt;td&gt;Up to 500 entries in memory&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Phoenix history&lt;/td&gt;
&lt;td&gt;Up to 500 received entries&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Captured line&lt;/td&gt;
&lt;td&gt;Up to 1,024 input bytes, with truncation indicated&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Work per reader cycle&lt;/td&gt;
&lt;td&gt;At most 64 KiB and roughly 100 complete nonempty entries&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Panel polling&lt;/td&gt;
&lt;td&gt;Normally every two seconds while visible, with catch-up when needed&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oracle excerpt&lt;/td&gt;
&lt;td&gt;Up to 6,000 UTF-8 bytes, also subject to a 7,000-byte JSON-encoded limit&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oracle wait&lt;/td&gt;
&lt;td&gt;90 seconds&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The title’s 500 lines is an actual buffer bound, not a recommended daily allowance of suffering.&lt;/p&gt;

&lt;p&gt;Search and severity filtering operate on the buffered view. They do not secretly scan the server’s entire log archive. &lt;strong&gt;Pause display&lt;/strong&gt; freezes what you see while collection and forensic forwarding continue. The server is not placed in suspended animation because you wanted to read line seven.&lt;/p&gt;

&lt;h2&gt;
  
  
  Severity rules are literal-minded employees
&lt;/h2&gt;

&lt;p&gt;Classification uses the configured keyword lists in their order. Matching is case-insensitive, and the first matching rule wins. Unmatched entries are INFO.&lt;/p&gt;

&lt;p&gt;For a simple demonstration, put CRITICAL before WARNING and use &lt;code&gt;segfault&lt;/code&gt; as a critical keyword and &lt;code&gt;error&lt;/code&gt; as a warning keyword. That matches the general shape of the supplied palette defaults.&lt;/p&gt;

&lt;p&gt;A keyword rule does not know whether “error count: 0” is good news. It sees “error.” If your application writes cheerful success messages containing alarming words, tune the rules for that application.&lt;/p&gt;

&lt;p&gt;Likewise, the row’s observation time is when the reader captured the event. Treat timestamps written inside the application’s message as separate evidence. The panel is not reconstructing a globally synchronized incident timeline from every logging format ever invented.&lt;/p&gt;

&lt;h2&gt;
  
  
  A demonstration that does not require breaking production
&lt;/h2&gt;

&lt;p&gt;Use a staging host and a dedicated harmless file, such as &lt;code&gt;/var/tmp/swarm-log-demo.log&lt;/code&gt;. Create it under an account and directory arrangement the agent can read, point the agent at that file, deploy, and use &lt;strong&gt;Check access&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;After the reader is active, append these &lt;strong&gt;synthetic messages&lt;/strong&gt; to the demonstration file:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;INFO demo: checkout started
ERROR demo: database connection timed out after 3 seconds
ERROR demo: retry exhausted for database connection
segfault demo: synthetic critical marker; no process actually crashed
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;These are sample log contents. They do not crash a process, contact a database, or establish that any real service is broken. Keep the exercise separate from production alert destinations, especially if you have connected a forensic escalation path.&lt;/p&gt;

&lt;p&gt;Open Log Monitor and confirm that the new rows arrive. Search for &lt;code&gt;database&lt;/code&gt;. Select the two related errors with Ctrl or Shift. Click &lt;strong&gt;Ask Oracle…&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The dialog shows the excerpt before anything is sent by this action. Review it, edit it, and then choose &lt;strong&gt;Send to Oracle&lt;/strong&gt;. With no selection, the panel proposes recent visible entries instead.&lt;/p&gt;

&lt;p&gt;For real logs, this review is where you remove tokens, credentials, customer information, or unrelated noise. The configured provider receives the excerpt you send. Oracle’s own prompt-dump setting can also cause the excerpt to appear in its logs, so account for that setting when choosing what to submit.&lt;/p&gt;

&lt;p&gt;The assistant is invited to inspect the evidence. It does not need a complimentary copy of the production password.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a useful answer should do
&lt;/h2&gt;

&lt;p&gt;The analysis request asks Oracle to group recurring errors, explain likely causes, separate evidence from uncertainty, and suggest read-only checks. It explicitly treats the log excerpt as untrusted data rather than instructions.&lt;/p&gt;

&lt;p&gt;For the synthetic database pair, a useful response would distinguish the observed timeout and exhausted retry from possible explanations such as reachability, service availability, or connection saturation. It should not declare a proven root cause from two lines.&lt;/p&gt;

&lt;p&gt;That is an illustration of what to evaluate, not a guaranteed model response. Read the returned answer against the original entries. Missing context remains missing context even when somebody describes it confidently in complete sentences.&lt;/p&gt;

&lt;p&gt;Results are displayed as plain text. They are not executed. There is no “the model suggested a command, therefore the server has already run it” step in this panel.&lt;/p&gt;

&lt;p&gt;The agent permits one analysis request at a time. It ties the response to the expected Oracle identity and query ID, rejects unrelated or expired replies, and bounds the returned text for transport. An oversized answer may be marked &lt;strong&gt;Response shortened&lt;/strong&gt;. Request authentication establishes who answered; it does not certify that the diagnosis is correct.&lt;/p&gt;

&lt;h2&gt;
  
  
  The automatic path is a separate conversation
&lt;/h2&gt;

&lt;p&gt;Log Health can forward captured events to its configured forensic report role independently of your manual Ask Oracle action. If no forensic endpoint exists, the monitor remains available and the agent emits a diagnostic warning.&lt;/p&gt;

&lt;p&gt;If you add Forensic Detective, that agent has its own incident and Oracle policies. Pressing Pause display, or choosing not to press Ask Oracle, does not disable that separate forwarding and analysis path.&lt;/p&gt;

&lt;p&gt;For Detective’s current escalation flow, CRITICAL events can initiate investigations and notifications subject to cooldown. Connect the desired Slack, Discord, Telegram, or email relay there if that is part of your design. Discord, Telegram, and email offer optional signed payload encryption with the matching configuration. Verify actual delivery instead of assuming a drawn arrow has completed the paperwork.&lt;/p&gt;

&lt;h2&gt;
  
  
  Rotation, restarts, and honest boundaries
&lt;/h2&gt;

&lt;p&gt;On rename rotation, the reader drains the old open file before opening the replacement from its beginning. Observed truncation causes it to rewind. Partial lines wait for completion, and oversized lines are consumed with bounded memory.&lt;/p&gt;

&lt;p&gt;There are still gaps a polling reader cannot promise away: rapid copy-truncate-and-regrow between polls can lose entries, as can later writes to an old file after its handle has been closed. Restarting the agent resets its in-memory history. Cursors and a boot identifier help Phoenix recognize continuity changes; they do not turn the buffer into durable storage.&lt;/p&gt;

&lt;p&gt;Keep your normal log retention and incident evidence collection. Use this tool for live triage, with its limits visible.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Symptom&lt;/th&gt;
&lt;th&gt;Check first&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Quiet panel&lt;/td&gt;
&lt;td&gt;Was anything appended after the reader opened the file?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Missing or denied&lt;/td&gt;
&lt;td&gt;Actual path, file type, and the runtime account’s permissions&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;No panel after an upgrade&lt;/td&gt;
&lt;td&gt;Enable the monitor, resolve signing, and redeploy the saved workspace&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;No Oracle answer&lt;/td&gt;
&lt;td&gt;Exactly one endpoint for the role, provider configuration, and Agent Logs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Analysis already running&lt;/td&gt;
&lt;td&gt;One request is pending; wait for its result or timeout&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Missing older lines&lt;/td&gt;
&lt;td&gt;The bounded buffer and EOF-first behavior&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The win is a smaller, clearer incident conversation: here are the relevant lines; here is a possible explanation; here are the checks that could confirm it.&lt;/p&gt;

&lt;p&gt;Your log still has opinions. Now you have a way to interrogate them without opening a fourth terminal and naming it “final-final-actually-this-one.”&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Victory Always. Read the evidence before rebooting the witness.&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;🌐 &lt;strong&gt;Links &amp;amp; Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try MatrixSwarm: &lt;a href="https://matrixswarm.com" rel="noopener noreferrer"&gt;https://matrixswarm.com&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Join the Community / Discord: &lt;a href="https://discord.gg/2USbWVBVV" rel="noopener noreferrer"&gt;https://discord.gg/2USbWVBVV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Download Server: &lt;a href="https://github.com/matrixswarm/matrixswarm" rel="noopener noreferrer"&gt;https://github.com/matrixswarm/matrixswarm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug" rel="noopener noreferrer"&gt;https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug&lt;/a&gt;&lt;/p&gt;

</description>
      <category>devops</category>
      <category>ai</category>
      <category>opensource</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>Harvester: Your Swarm Cannot Report Its Own Funeral</title>
      <dc:creator>Daniel</dc:creator>
      <pubDate>Sat, 10 Oct 2026 01:41:43 +0000</pubDate>
      <link>https://dev.to/matrixswarm/harvester-your-swarm-cannot-report-its-own-funeral-261j</link>
      <guid>https://dev.to/matrixswarm/harvester-your-swarm-cannot-report-its-own-funeral-261j</guid>
      <description>&lt;p&gt;&lt;strong&gt;Operations memo, 09:00:&lt;/strong&gt; The production swarm will notify us if anything goes wrong.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Operations memo, 09:07:&lt;/strong&gt; The production swarm has stopped.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Operations memo, 09:08:&lt;/strong&gt; We have received no notification. Morale remains inexplicably high.&lt;/p&gt;

&lt;p&gt;A system that disappears cannot reliably announce its own disappearance. Somebody outside the room needs to take attendance.&lt;/p&gt;

&lt;p&gt;MatrixSwarm’s &lt;strong&gt;harvester&lt;/strong&gt; agent observes an assigned swarm through the target host’s &lt;strong&gt;matrixd process inventory&lt;/strong&gt;, confirms repeated failures, and sends health notifications through the usual swarm relays. It can tell you that the expected universe is absent, that its agent count is below your minimum, or that the observation itself failed.&lt;/p&gt;

&lt;p&gt;This article follows the current Phoenix workflow: &lt;strong&gt;one Harvester instance, one assigned universe, one SSH connection, observation only&lt;/strong&gt;. We are issuing a clipboard, not authorizing a remote resurrection department.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What Harvester actually asks&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The remote operation is fixed: &lt;code&gt;matrixd list --json&lt;/code&gt;, using MatrixOS’s installed Python environment and script path. Harvester reads the structured inventory, finds the configured universe, and compares its agent count with your required minimum.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Observation&lt;/th&gt;
&lt;th&gt;Meaning&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Assigned universe appears with enough agents&lt;/td&gt;
&lt;td&gt;The configured inventory check passes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Assigned universe is absent&lt;/td&gt;
&lt;td&gt;&lt;code&gt;UNIVERSE_ABSENT&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Universe appears with too few agents&lt;/td&gt;
&lt;td&gt;A status such as &lt;code&gt;AGENTS_3_BELOW_5&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SSH, command execution, or snapshot validation fails&lt;/td&gt;
&lt;td&gt;An observation error, with a diagnostic status&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The minimum is a &lt;strong&gt;count&lt;/strong&gt;, not a list of required agent identities. Five running processes are not proof that five specific jobs are working. A process can also remain alive while its application work is stuck.&lt;/p&gt;

&lt;p&gt;Use Harvester for this inventory-level question. Pair it with the appropriate service, website, or application checks when you need to know whether useful work is happening.&lt;/p&gt;

&lt;p&gt;Attendance and competence have always been separate columns on the spreadsheet.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Put the observer where it can survive&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;For a useful first deployment, put Harvester in a small observer swarm and point it at another already-deployed swarm.&lt;/p&gt;

&lt;p&gt;Two universes on the same host can demonstrate a target-swarm failure. For host-outage coverage, use a separate observer host with a route to the target. If both share the failed machine, power supply, or network path, their opinions may become unavailable together.&lt;/p&gt;

&lt;p&gt;Harvester runs on MatrixOS, so closing Phoenix does not stop its checks or the observer’s configured alert relays. Phoenix is your setup and inspection console. It does not need to spend the night staring at the process table.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Install MatrixOS, then build the observer&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If the observer host needs MatrixOS, add and test its SSH record in Phoenix’s &lt;strong&gt;Registry&lt;/strong&gt;, including the trusted host fingerprint. Open &lt;strong&gt;Railgun → Install MatrixOS…&lt;/strong&gt; and choose &lt;strong&gt;Local Full Install&lt;/strong&gt; with your source folder or &lt;strong&gt;Install from GitHub&lt;/strong&gt;. For a fresh environment, choose &lt;strong&gt;Create new venv&lt;/strong&gt;, install, and check the result. Installation requires root or passwordless sudo on that host.&lt;/p&gt;

&lt;p&gt;The target must already have MatrixOS and the current &lt;code&gt;matrixd list --json&lt;/code&gt; support. Keep its deployment record in your Phoenix vault; the assignment editor selects from those saved deployments.&lt;/p&gt;

&lt;p&gt;Open &lt;strong&gt;Deploy → Swarm Workspaces&lt;/strong&gt; and create a &lt;strong&gt;New&lt;/strong&gt; workspace or &lt;strong&gt;Open&lt;/strong&gt; an existing observer workspace. Beneath the &lt;code&gt;matrix&lt;/code&gt; root, add:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Agent&lt;/th&gt;
&lt;th&gt;Assignment&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;harvester&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Observe the selected remote universe&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_https&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Accept Phoenix commands over HTTPS&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_websocket&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Return replies and session traffic to Phoenix&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;log_streamer&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Expose Agent Logs for diagnosis&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Your chosen alert relay&lt;/td&gt;
&lt;td&gt;Deliver health notifications&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Those ingress, egress, and log agents provide the connected Phoenix workflow. Harvester’s check itself reaches the &lt;strong&gt;target host over SSH&lt;/strong&gt;. The target does not need an extra HTTPS/WebSocket pair merely to answer this inventory check.&lt;/p&gt;

&lt;p&gt;Two SSH jobs are involved: Railgun deploys the observer to its host, and Harvester later connects from that host to the watched target. Keep those roles straight. “I could reach it from my laptop” is useful testimony, but your laptop has not been hired to run this check.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The assignment lives in the Registry&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Harvester declares three constraints: &lt;strong&gt;packet_signing&lt;/strong&gt;, &lt;strong&gt;symmetric_encryption&lt;/strong&gt;, and &lt;strong&gt;harvester_assignment&lt;/strong&gt;. Resolve them in the workspace using the appropriate Registry objects.&lt;/p&gt;

&lt;p&gt;Create a &lt;strong&gt;Harvester Assignment&lt;/strong&gt; Registry object. Its editor separates the watched deployment and connection policy from the agent’s ordinary runtime settings.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Give the assignment a recognizable label.&lt;/li&gt;
&lt;li&gt;In &lt;strong&gt;Hive ↔ SSH&lt;/strong&gt;, select the target’s saved deployment and its matching SSH Registry record.&lt;/li&gt;
&lt;li&gt;Leave &lt;strong&gt;Authority&lt;/strong&gt; at &lt;strong&gt;Contact only (resurrection deferred)&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Set &lt;strong&gt;Minimum Agents&lt;/strong&gt; to the smallest acceptable count for that target.&lt;/li&gt;
&lt;li&gt;Set failure/recovery thresholds and alert cooldown.&lt;/li&gt;
&lt;li&gt;For your first deployment, enable &lt;strong&gt;Send one confirmation when monitoring is established&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Save the Registry object and assign it to Harvester’s &lt;strong&gt;harvester_assignment&lt;/strong&gt; constraint.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Selecting the deployment attempts to select its recorded SSH object too. Validation checks the match when the deployment has that reference. Check the actual host, username, and universe before deploying; a carefully signed packet can still be addressed to the wrong building if your configuration is wrong.&lt;/p&gt;

&lt;p&gt;The Registry assignment stores references and policy. When Phoenix compiles the directive, it resolves the deployment and SSH records from the unlocked vault and injects the selected connection profile and single target into Harvester’s encrypted directive.&lt;/p&gt;

&lt;p&gt;Contact-only operation does &lt;strong&gt;not&lt;/strong&gt; inject the watched swarm’s decryption key. It does require the assigned SSH authentication material needed to perform the observation.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Enable the actual checks&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Open Harvester’s workspace Config editor and enable &lt;strong&gt;Enable matrixd checks&lt;/strong&gt;. The shipped metadata starts disabled and has no assigned target.&lt;/p&gt;

&lt;p&gt;A deployed but disabled Harvester is not monitoring quietly. It is following your instructions with devastating accuracy.&lt;/p&gt;

&lt;p&gt;The runtime controls are:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Setting&lt;/th&gt;
&lt;th&gt;Default&lt;/th&gt;
&lt;th&gt;Purpose&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Enable matrixd checks&lt;/td&gt;
&lt;td&gt;Off&lt;/td&gt;
&lt;td&gt;Starts observation when the assignment is valid&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Check Interval&lt;/td&gt;
&lt;td&gt;30 seconds&lt;/td&gt;
&lt;td&gt;Delay between completed check cycles&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Matrixd Timeout&lt;/td&gt;
&lt;td&gt;60 seconds&lt;/td&gt;
&lt;td&gt;Total deadline for the supervised SSH checker&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Alert Role&lt;/td&gt;
&lt;td&gt;&lt;code&gt;hive.alert&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Role used to find notification relays&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The assignment supplies these per-target defaults:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Setting&lt;/th&gt;
&lt;th&gt;Default&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Minimum Agents&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Failure Threshold&lt;/td&gt;
&lt;td&gt;3 consecutive failed observations&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Recovery Threshold&lt;/td&gt;
&lt;td&gt;3 consecutive successful observations after a down state&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Alert Cooldown&lt;/td&gt;
&lt;td&gt;300 seconds between eligible down reminders&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Healthy Confirmation&lt;/td&gt;
&lt;td&gt;Off until selected&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Choose a meaningful minimum. A six-agent deployment with a minimum of one can lose most of its intended workforce while still passing the count test. Conversely, a minimum above the legitimate running count guarantees complaints.&lt;/p&gt;

&lt;p&gt;The current normal workflow provisions exactly one target. Multiple targets in one Harvester directive are rejected by its policy. Broader inventory fan-out is outside this release’s scope.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Pinned SSH and a bounded question&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The target SSH record needs the expected &lt;strong&gt;SHA256 host-key fingerprint&lt;/strong&gt;. Harvester verifies that identity, uses the provisioned authentication method, and runs its fixed inventory command. It does not accept an arbitrary command from a target row or roam through the network looking for swarms.&lt;/p&gt;

&lt;p&gt;The transport uses Paramiko, whose &lt;a href="https://docs.paramiko.org/en/stable/api/client.html" rel="noopener noreferrer"&gt;SSH client documentation&lt;/a&gt; explains host-key handling and authentication options. Harvester supplies its own pinned-key policy instead of silently accepting an unfamiliar server.&lt;/p&gt;

&lt;p&gt;Use credentials and routing that work &lt;strong&gt;from the observer host&lt;/strong&gt;. A hostname resolved only on your workstation, an SSH agent available only in your desktop session, or a firewall rule allowing only your laptop will not become portable through optimism.&lt;/p&gt;

&lt;p&gt;The fixed remote wrapper runs the MatrixOS inventory command as root, or uses noninteractive &lt;code&gt;sudo -n&lt;/code&gt; when connecting as another user. Provision the target account to permit the required command; a successful interactive SSH login alone does not establish that permission. The check cannot answer a sudo password prompt.&lt;/p&gt;

&lt;p&gt;The SSH work runs in a supervised checker process. The timeout covers startup, connection, authentication, and command execution. A stuck check is stopped and cleaned up before another one is allowed to proceed, while Harvester continues servicing its own heartbeat.&lt;/p&gt;

&lt;p&gt;A remote machine may be unresponsive. That does not mean the observer should join it in a silent meditation retreat.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Launch the observer through Railgun&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Save the workspace and click &lt;strong&gt;Deploy&lt;/strong&gt;. Choose the &lt;strong&gt;observer host’s&lt;/strong&gt; SSH record, a deployment label, observer universe, and runtime Linux user. Review the resolved directive and launch.&lt;/p&gt;

&lt;p&gt;Railgun verifies the deployment connection, prepares the runtime account and applicable grants, and sends the sealed boot envelope over SSH stdin to MatrixD. MatrixD decrypts it in memory and starts the observer swarm. Check the launch result, then select the deployment and &lt;strong&gt;Connect&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The Harvester assignment inside that observer points to the watched target. Your observer universe and target universe are distinct responsibilities, even if a demonstration places them on the same server.&lt;/p&gt;

&lt;p&gt;In Phoenix, use &lt;strong&gt;Agent Logs&lt;/strong&gt; through log_streamer to inspect Harvester. The current agent has workspace configuration and Registry assignment editors; it does not provide a dedicated live Harvester inventory panel.&lt;/p&gt;

&lt;p&gt;Look for the watching-target startup message and the initial observation. If you selected Healthy Confirmation and the first observation passes, verify the &lt;strong&gt;HEALTHY&lt;/strong&gt; message at your chosen destination.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Read the roll call properly&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Harvester tracks each observation through a small state machine:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Event&lt;/th&gt;
&lt;th&gt;When it happens&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;HEALTHY&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Optional one-time confirmation on the first successful observation from the unknown state&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;DOWN&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Consecutive failures reach the configured failure threshold&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;DOWN_REMINDER&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;The target remains down and the alert cooldown permits another reminder&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;RECOVERY&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Consecutive successes reach the recovery threshold after a down state&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;An isolated failed observation resets the success streak; an isolated successful observation resets the failure streak. Once down, the target needs the configured run of successful observations to be reported recovered.&lt;/p&gt;

&lt;p&gt;The 30-second interval comes &lt;strong&gt;after check work&lt;/strong&gt;. Three failures do not guarantee a precisely timed 90-second alert, especially if SSH is consuming its timeout. Treat the thresholds as consecutive observations rather than stopwatch promises.&lt;/p&gt;

&lt;p&gt;Most importantly, &lt;strong&gt;DOWN can mean “the inventory check failed.”&lt;/strong&gt; Read the accompanying status. &lt;code&gt;UNIVERSE_ABSENT&lt;/code&gt; from a valid snapshot and &lt;code&gt;SSH_AUTH_FAILED&lt;/code&gt; are very different findings. One says the selected universe was not in the inventory; the other says the observer could not establish the observation.&lt;/p&gt;

&lt;p&gt;The uniform may say Harvester. It still has to show its evidence.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Wire the notice to a human&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Harvester sends notifications directly to endpoints advertising its &lt;strong&gt;Alert Role&lt;/strong&gt;, normally &lt;code&gt;hive.alert&lt;/code&gt;. For this route you can use &lt;strong&gt;Slack, Discord, Telegram, or email&lt;/strong&gt; through &lt;code&gt;slack_relay&lt;/code&gt;, &lt;code&gt;discord_relay&lt;/code&gt;, &lt;code&gt;telegram_relay&lt;/code&gt;, or &lt;code&gt;email_send&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Configure the relay’s destination and Registry requirements. Discord, Telegram, and email support optional signed alert-payload encryption through &lt;code&gt;encrypt_alerts&lt;/code&gt;, with matching keys and compatible decryption support on the receiving side. This is separate from SSH or message-transport encryption.&lt;/p&gt;

&lt;p&gt;For selective delivery, use a dedicated role shared by Harvester and the intended receiving relay. Otherwise, it considers all endpoints discovered for its configured role.&lt;/p&gt;

&lt;p&gt;Harvester retains one current pending notification per target and retries failed swarm dispatches on later checks. It tracks successful recipient handoffs so an unavailable second relay does not cause needless repeats to the first. State changes can supersede an older pending notice; this is a current-health notification mechanism, not an unlimited historical outbox.&lt;/p&gt;

&lt;p&gt;A successful packet handoff does not prove that a human’s inbox received or displayed the message. Verify the full route during setup.&lt;/p&gt;

&lt;p&gt;Harvester’s condition counters and pending-delivery bookkeeping are in memory. Restarting the observer resets them. Do not assume a restart preserves outage history, retry history, or the once-per-run healthy confirmation state.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;About the resurrection paperwork&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Some older descriptions discuss a contact-and-resurrect capability. The current assignment editor exposes &lt;strong&gt;Contact only (resurrection deferred)&lt;/strong&gt;, disables the resurrection controls, and compiles automatic recovery as false. The running agent also keeps automatic recovery disabled.&lt;/p&gt;

&lt;p&gt;Accordingly, this walkthrough promises &lt;strong&gt;observation and notification&lt;/strong&gt;. A &lt;strong&gt;RECOVERY&lt;/strong&gt; event means Harvester observed the target return to its required state. It does not mean Harvester restarted it.&lt;/p&gt;

&lt;p&gt;Treat resurrection as deferred functionality. The existence of a form field in the filing cabinet does not mean the department has permission to operate a defibrillator.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;A drill that proves the observer stays awake&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Use a disposable target swarm with a known agent count and an observer that will remain running independently.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Deploy the observer with Healthy Confirmation enabled. Verify its initial log and the actual HEALTHY notification.&lt;/li&gt;
&lt;li&gt;Stop only the disposable target swarm through your normal lifecycle procedure.&lt;/li&gt;
&lt;li&gt;Leave the observer running and wait for its configured failure confirmations.&lt;/li&gt;
&lt;li&gt;Verify DOWN and inspect the status that explains the failed observation.&lt;/li&gt;
&lt;li&gt;Relaunch the target through Railgun using the same expected universe and enough agents.&lt;/li&gt;
&lt;li&gt;Leave the observer untouched while successful observations accumulate, then verify RECOVERY.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;If you restart the observer halfway through the drill, its state returns to unknown. That tests a new observer startup, not the down-to-recovered transition you were trying to demonstrate.&lt;/p&gt;

&lt;p&gt;No need to sabotage production to establish that the clipboard works.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;When attendance does not add up&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Symptom&lt;/th&gt;
&lt;th&gt;First check&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;“Disabled by policy”&lt;/td&gt;
&lt;td&gt;Enable matrixd checks in the workspace Config and redeploy&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;No valid target / fail closed&lt;/td&gt;
&lt;td&gt;Assignment, referenced deployment, matching SSH object, and single-target limit&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;SSH_HOST_KEY_MISMATCH&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Verify the real target host identity through a trusted channel; update the Registry only after checking the change&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;SSH_AUTH_FAILED&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Provisioned credentials and authentication available to the observer&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;SSH_DNS_ERROR&lt;/code&gt; / connection failure / timeout&lt;/td&gt;
&lt;td&gt;Name resolution and network access from the observer host&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;MATRIXD_CHECK_FAILED&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Target installation, fixed command path, noninteractive permissions, and command output&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;UNIVERSE_ABSENT&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Exact assigned universe and whether that target is actually running&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Agent count is below minimum&lt;/td&gt;
&lt;td&gt;Expected count versus legitimate target processes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Alerts remain pending&lt;/td&gt;
&lt;td&gt;Receiving role, service discovery, relay health, and packet-dispatch logs&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Start with one assigned target, one meaningful minimum, and one verified relay. Keep the observer’s own availability visible too; every watchtower is still made of computers.&lt;/p&gt;

&lt;p&gt;You now have something more useful than “I think that swarm is still up”: a repeated observation, an explicit failure policy, and a message from somewhere capable of sending it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Victory Always. Roll call is now somebody’s actual job.&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;🌐 &lt;strong&gt;Links &amp;amp; Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try MatrixSwarm: &lt;a href="https://matrixswarm.com" rel="noopener noreferrer"&gt;https://matrixswarm.com&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Join the Community / Discord: &lt;a href="https://discord.gg/2USbWVBVV" rel="noopener noreferrer"&gt;https://discord.gg/2USbWVBVV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Download Server: &lt;a href="https://github.com/matrixswarm/matrixswarm" rel="noopener noreferrer"&gt;https://github.com/matrixswarm/matrixswarm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug" rel="noopener noreferrer"&gt;https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug&lt;/a&gt;&lt;/p&gt;

</description>
      <category>devops</category>
      <category>linux</category>
      <category>opensource</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>Site Sentinel: Your Website Says 200 OK. The Customers Disagree.</title>
      <dc:creator>Daniel</dc:creator>
      <pubDate>Sat, 10 Oct 2026 01:39:51 +0000</pubDate>
      <link>https://dev.to/matrixswarm/site-sentinel-your-website-says-200-ok-the-customers-disagree-115</link>
      <guid>https://dev.to/matrixswarm/site-sentinel-your-website-says-200-ok-the-customers-disagree-115</guid>
      <description>&lt;p&gt;The server is running. The port is open. The homepage returns 200.&lt;/p&gt;

&lt;p&gt;Unfortunately, the homepage is now a maintenance apology wearing the company logo. The stylesheet has vanished. Checkout looks like a tax form transmitted by fax.&lt;/p&gt;

&lt;p&gt;“Everything is green,” says the dashboard.&lt;/p&gt;

&lt;p&gt;The customers have submitted a competing assessment.&lt;/p&gt;

&lt;p&gt;MatrixSwarm’s &lt;strong&gt;site_sentinel&lt;/strong&gt; checks the public page, required content, a sample of linked assets, an optional origin endpoint, and certificate lifetime. It also reads local access logs and measures the host’s resource pressure. Confirmed conditions become structured evidence for &lt;strong&gt;Forensic Detective&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Think of it as the building inspector who actually opens the door instead of admiring the illuminated EXIT sign from the car park.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What the inspection covers&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Evidence&lt;/th&gt;
&lt;th&gt;What it helps you notice&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Public HTTP response&lt;/td&gt;
&lt;td&gt;The endpoint is unavailable or responding unusually slowly&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Required text marker&lt;/td&gt;
&lt;td&gt;A successful response contains the wrong or incomplete page&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Linked CSS, JavaScript, and images&lt;/td&gt;
&lt;td&gt;The page responds, but selected dependencies fail&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Optional origin comparison&lt;/td&gt;
&lt;td&gt;Public and origin paths disagree&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;TLS certificate lifetime&lt;/td&gt;
&lt;td&gt;A currently valid public certificate is approaching expiration&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Access-log patterns&lt;/td&gt;
&lt;td&gt;High request rates, many distinct paths, and error-heavy or bot-like traffic&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Local CPU, memory, and load&lt;/td&gt;
&lt;td&gt;The monitoring host is under pressure&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Site Sentinel observes and reports. It does not restart Apache, ban addresses, purge caches, repair deployments, or quarantine files. Those are separate jobs. Its contribution is better evidence before somebody announces that rebooting everything is “the quickest option.”&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Install the runtime and build the inspection team&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If the host does not have MatrixOS, add and test its SSH connection in Phoenix’s &lt;strong&gt;Registry&lt;/strong&gt;, including the trusted host fingerprint. Open &lt;strong&gt;Railgun → Install MatrixOS…&lt;/strong&gt; and choose &lt;strong&gt;Local Full Install&lt;/strong&gt; with your source folder or &lt;strong&gt;Install from GitHub&lt;/strong&gt;. For a fresh environment, choose &lt;strong&gt;Create new venv&lt;/strong&gt;, install, and verify the result. Installation requires root or passwordless sudo on the target.&lt;/p&gt;

&lt;p&gt;Then open &lt;strong&gt;Deploy → Swarm Workspaces&lt;/strong&gt;. Create a &lt;strong&gt;New&lt;/strong&gt; workspace or &lt;strong&gt;Open&lt;/strong&gt; an existing one. Keep the &lt;code&gt;matrix&lt;/code&gt; root and add:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Agent&lt;/th&gt;
&lt;th&gt;Responsibility&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;site_sentinel&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Collect and classify website and host evidence&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;forensic_detective&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Receive status events, investigate critical incidents, and initiate alerts&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_https&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Accept Phoenix commands through HTTPS ingress&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_websocket&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Return swarm replies and session traffic to Phoenix&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;log_streamer&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Make Agent Logs available in the cockpit&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;An alert relay&lt;/td&gt;
&lt;td&gt;Deliver the resulting notifications to your chosen destination&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Resolve each agent’s declared Registry constraints and the HTTPS/WSS connection requirements. Forensic Detective needs its own &lt;strong&gt;Persistent State&lt;/strong&gt; profile for the encrypted incident journal. Retain that profile for later deployments.&lt;/p&gt;

&lt;p&gt;For the first exercise, open Forensic Detective’s Config and turn off &lt;strong&gt;Enable Oracle Analysis&lt;/strong&gt;. Its normal investigation and immediate critical-incident alert do not require an LLM. You can later add &lt;code&gt;oracle&lt;/code&gt; and configure the provider for an additional analysis step.&lt;/p&gt;

&lt;p&gt;The inspection should work before we invite a consultant to narrate it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Choose where the observer lives&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;URL checks run from the Site Sentinel host. Access logs and CPU/memory/load are also read from that host.&lt;/p&gt;

&lt;p&gt;For a combined view of a web server’s pages, logs, and resource pressure, deploy the agent on that server and grant the runtime account the necessary read access to the configured log files. If you put it on a separate monitoring server, its resource metrics describe the monitoring server; pointing a URL at another machine does not move the CPU sensor there.&lt;/p&gt;

&lt;p&gt;A separate observer is useful for external reachability, while an on-host observer provides local context. A monitor that loses its own host cannot report that host’s outage through sheer professional commitment.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Tell it what a healthy page looks like&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Open Site Sentinel’s workspace Config editor. Under &lt;strong&gt;Public and Origin Targets&lt;/strong&gt;, add a row:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Field&lt;/th&gt;
&lt;th&gt;Example or purpose&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;url&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Your public page, such as &lt;code&gt;https://shop.example.com/catalog&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;origin_url&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Optional direct origin URL for the equivalent page&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;host_header&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Optional virtual-host name for the origin request&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;expect&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A stable text marker that must appear in the returned page&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;note&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A short description such as “Public catalog”&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Replace the example domain with a site you operate. Pick a marker tied to useful content, not something also present on every error page. “Catalog available” is a better diagnostic than the company name if the company name appears on the maintenance screen too.&lt;/p&gt;

&lt;p&gt;The marker check is a &lt;strong&gt;case-insensitive substring search in the response text&lt;/strong&gt;, using the first two million characters. It is not a DOM selector, screenshot comparison, checksum baseline, or JavaScript execution test. Leaving &lt;code&gt;expect&lt;/code&gt; empty disables that particular content check.&lt;/p&gt;

&lt;p&gt;A heading created only after browser JavaScript runs may not exist in the HTML fetched by this agent. Choose server-returned content for the first test.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The public door and the service entrance&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The optional origin request helps compare what the public path serves with what the backend serves. When &lt;code&gt;host_header&lt;/code&gt; is empty, the origin request uses the public URL’s hostname as its HTTP Host header.&lt;/p&gt;

&lt;p&gt;For this comparison to mean something, use equivalent paths and make sure the origin URL actually reaches the origin rather than circling through the same CDN again.&lt;/p&gt;

&lt;p&gt;HTTPS still requires valid certificate trust. An HTTP Host header does not change the TLS hostname used by the URL. Use an origin hostname whose certificate and routing are valid; certificate verification remains enabled, consistent with &lt;a href="https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification" rel="noopener noreferrer"&gt;Requests’ documented behavior&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Here is the agent’s classification vocabulary:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Observation&lt;/th&gt;
&lt;th&gt;Classification&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Public response succeeds, required marker is missing&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;CRITICAL: PUBLIC_CONTENT_MISMATCH&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Public content passes, a sampled asset fails&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;WARNING: PUBLIC_ASSET_FAILURE&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Public checks pass, configured origin fails&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;WARNING: ORIGIN_DEGRADED&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Public request fails, origin checks pass&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;WARNING: EDGE_DEGRADED&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Public request fails without a healthy origin result&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;CRITICAL: SITE_UNAVAILABLE&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Otherwise healthy public response exceeds latency threshold&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;WARNING: PUBLIC_LATENCY&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Otherwise healthy target has a certificate near expiration&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;WARNING: TLS_EXPIRING&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;These labels describe observations, not a proven root cause. EDGE_DEGRADED is a reason to investigate the public path. It does not authorize a press release declaring the CDN guilty.&lt;/p&gt;

&lt;p&gt;Also, if you did not configure an origin, there is no origin evidence. Read the underlying observations before concluding both paths failed.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;“The HTML arrived” is a very low bar&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Enable &lt;strong&gt;Verify linked CSS, JavaScript, and images&lt;/strong&gt; to sample assets referenced directly in the HTML. The parser considers stylesheet links, script sources, and image sources. By default it checks up to &lt;strong&gt;8 distinct HTTP/HTTPS assets&lt;/strong&gt;; the configurable maximum is 25.&lt;/p&gt;

&lt;p&gt;This catches useful failures such as a missing stylesheet or an unavailable script. It follows referenced URLs, so an asset may live on another host. It does not run scripts, inspect their behavior, follow every dependency inside CSS, or perform a complete browser journey.&lt;/p&gt;

&lt;p&gt;A JavaScript file returning 200 can still contain a bug. Site Sentinel has an inspection clipboard, not supernatural access to the intentions of your frontend team.&lt;/p&gt;

&lt;p&gt;The latency measurement covers the public page request, not browser rendering or the complete asset pass. Choose a threshold appropriate to the page and the observer’s location.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Traffic analysis: who has been rattling the doors?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Under &lt;strong&gt;Load and Scraper Detection&lt;/strong&gt;, enable traffic analysis and list real access-log files on the deployment host, one per line. The palette suggests &lt;code&gt;/var/log/apache2/access.log&lt;/code&gt;; replace or remove it when that is not your server’s actual path.&lt;/p&gt;

&lt;p&gt;The runtime account must be able to read the files. At first open, the reader starts at the end; it does not replay yesterday’s traffic as an exciting new emergency. Subsequent cycles read appended entries, with handling for rotation and truncation.&lt;/p&gt;

&lt;p&gt;Supported inputs include common/combined-style access logs and compatible JSON records. The analysis tracks request rate, distinct paths, error ratio, and bot-like user-agent strings for the busiest addresses.&lt;/p&gt;

&lt;p&gt;The supplied settings include:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Setting&lt;/th&gt;
&lt;th&gt;Default&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Traffic window&lt;/td&gt;
&lt;td&gt;60 seconds&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Top addresses examined&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Per-address warning / critical rate&lt;/td&gt;
&lt;td&gt;120 / 300 requests per minute&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Total warning / critical rate&lt;/td&gt;
&lt;td&gt;600 / 1,500 requests per minute&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Distinct-path threshold&lt;/td&gt;
&lt;td&gt;80&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Error-ratio threshold&lt;/td&gt;
&lt;td&gt;0.35, meaning 35%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The per-address warning rate is &lt;strong&gt;not sufficient on its own&lt;/strong&gt; for the aggressive-crawler warning. It also needs many distinct paths, enough errors, or a bot-like user agent on at least half the requests. The critical per-address rate can independently produce a critical classification. Total traffic has its own thresholds.&lt;/p&gt;

&lt;p&gt;These are tuning defaults, not universal definitions of an attack. A busy API customer, a search crawler, and a broken client retry loop can all be noisy for different reasons. The agent identifies patterns; the human investigates the explanation.&lt;/p&gt;

&lt;p&gt;Rates use when the agent reads entries into its observation window, rather than reconstructing a precise event-time timeline from the log timestamps. Batched log writes can therefore affect the apparent pattern.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Do not nominate the reverse proxy as Public Enemy Number One&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The editor can prefer forwarded client addresses when compatible JSON fields or an appended forwarded-address field are available. That only helps if your proxy and logging configuration establish which upstream sources are trusted.&lt;/p&gt;

&lt;p&gt;For example, Nginx’s &lt;a href="https://nginx.org/en/docs/http/ngx_http_realip_module.html" rel="noopener noreferrer"&gt;real-IP module&lt;/a&gt; provides trusted-source configuration. Site Sentinel does not independently verify a logged forwarded address. Arbitrary client-supplied headers are not authenticated identity papers.&lt;/p&gt;

&lt;p&gt;Set the ignored-address list thoughtfully. It defaults to loopback addresses. Ignored traffic is removed from these summaries, so adding half the internet to make the graph quieter is technically effective and operationally silly.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Pressure readings and fewer false alarms&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Local resource warnings default to &lt;strong&gt;80% CPU&lt;/strong&gt;, &lt;strong&gt;90% memory&lt;/strong&gt;, or &lt;strong&gt;1.5 load per CPU&lt;/strong&gt;. Critical thresholds are &lt;strong&gt;95% CPU&lt;/strong&gt;, &lt;strong&gt;97% memory&lt;/strong&gt;, or &lt;strong&gt;3.0 load per CPU&lt;/strong&gt;. The load figure uses the one-minute system load divided by CPU count.&lt;/p&gt;

&lt;p&gt;The agent combines these readings with traffic evidence for a separate pressure condition. CPU pressure alone can trigger it, even when no access log is configured. Missing logs do not magically mean zero real traffic; they mean that traffic evidence is unavailable.&lt;/p&gt;

&lt;p&gt;The normal check interval is &lt;strong&gt;30 seconds&lt;/strong&gt;, request timeout &lt;strong&gt;8 seconds&lt;/strong&gt;, failure confirmation &lt;strong&gt;3 cycles&lt;/strong&gt;, and recovery confirmation &lt;strong&gt;2 cycles&lt;/strong&gt;. TLS warning defaults to &lt;strong&gt;21 days&lt;/strong&gt; and summary logging to &lt;strong&gt;300 seconds&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Checks run through their work and then sleep for the interval. Multiple targets, origin probes, TLS checks, and slow assets add time. Three confirmations do not promise an exact 90-second incident-detection deadline.&lt;/p&gt;

&lt;p&gt;Once a condition is active, the agent reports activation or a change in severity rather than repeating the same incident every cycle. Enough healthy observations produce an INFO &lt;strong&gt;RECOVERED&lt;/strong&gt; report. Its counters and active-condition state live in memory and reset when the agent restarts.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Connect the evidence to the alarm&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Keep &lt;strong&gt;Forensic Report Role&lt;/strong&gt; at &lt;code&gt;hive.forensics.data_feed&lt;/code&gt; for the standard arrangement. Forensic Detective advertises the corresponding status-ingestion handler.&lt;/p&gt;

&lt;p&gt;The route is:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Site Sentinel → Forensic Detective → hive.alert relays → your destination.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Do not point Site Sentinel’s forensic role straight at a generic alert relay and expect interchangeable packet handling. It sends a structured status event. Detective turns qualifying incidents into notification packets.&lt;/p&gt;

&lt;p&gt;In the current Detective implementation, confirmed &lt;strong&gt;CRITICAL&lt;/strong&gt; events trigger incident investigation and alerts, subject to its cooldown. WARNING and INFO recovery events are ingested as evidence; they do not automatically generate their own external notifications. Site Sentinel events also share Detective’s service-level alert cooldown, so several targets failing close together may not each produce a separate immediate alert.&lt;/p&gt;

&lt;p&gt;Use any of the usual relays: &lt;strong&gt;Slack, Discord, Telegram, or email&lt;/strong&gt;. Discord, Telegram, and email support optional signed payload encryption with &lt;code&gt;encrypt_alerts&lt;/code&gt; and the matching configured keys/decryption support. That is separate from transport encryption. Resolve the relay’s destination and Registry requirements, then verify actual receipt.&lt;/p&gt;

&lt;p&gt;An alert route drawn beautifully on a diagram is still a hypothesis until the intended inbox makes a noise.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Deploy and run a harmless inspection drill&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Save the workspace configuration and click &lt;strong&gt;Deploy&lt;/strong&gt;. Select the target SSH record, deployment label, universe, and runtime account; review the resolved directive and applicable access grants. Railgun sends the sealed boot envelope over pinned SSH to MatrixD, which decrypts it in memory and launches the swarm. Check the result, then &lt;strong&gt;Connect&lt;/strong&gt; to the deployment.&lt;/p&gt;

&lt;p&gt;This agent’s current workflow uses its workspace Config editor and &lt;strong&gt;Agent Logs&lt;/strong&gt; through log_streamer. It does not ship a dedicated live Site Sentinel control panel.&lt;/p&gt;

&lt;p&gt;On a staging page you control:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Serve a stable marker and a small stylesheet. Confirm normal summaries in Site Sentinel’s logs.&lt;/li&gt;
&lt;li&gt;Remove the marker while still returning HTTP 200.&lt;/li&gt;
&lt;li&gt;Wait for the configured failure confirmations and look for &lt;strong&gt;PUBLIC_CONTENT_MISMATCH&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Check Detective’s ingestion/incident logs and verify the critical alert at your chosen destination.&lt;/li&gt;
&lt;li&gt;Restore the marker. After the recovery confirmations, check for &lt;strong&gt;RECOVERED&lt;/strong&gt; in the evidence path.&lt;/li&gt;
&lt;li&gt;Separately test a missing sampled stylesheet and observe the warning classification, remembering the default external-alert policy.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;You have now demonstrated the actual reason for using this agent: a server can answer politely while delivering the wrong thing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;If the inspection report looks odd&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Symptom&lt;/th&gt;
&lt;th&gt;Check first&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;No targets checked&lt;/td&gt;
&lt;td&gt;Add target rows; the default list is empty&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Marker always fails&lt;/td&gt;
&lt;td&gt;Raw returned HTML, redirects, maintenance templates, and browser-only content&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Origin fails while public works&lt;/td&gt;
&lt;td&gt;Origin route, equivalent path, Host header, and TLS hostname/trust&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Traffic analysis is idle&lt;/td&gt;
&lt;td&gt;Real log paths, read permissions, compatible format, and new appended entries&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;CPU figures seem unrelated&lt;/td&gt;
&lt;td&gt;Which machine actually runs Site Sentinel&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;No forensic report arrives&lt;/td&gt;
&lt;td&gt;Detective’s role, running state, and Persistent State assignment&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Warning/recovery reaches logs but not chat&lt;/td&gt;
&lt;td&gt;The current critical-only escalation policy&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Start with one meaningful page, one marker, and one verified notification route. Expand the inspection as you learn what normal looks like.&lt;/p&gt;

&lt;p&gt;The dashboard may still say 200 OK. This time, somebody checked whether the building had a floor.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Victory Always. Inspect the premises before declaring the grand reopening.&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;🌐 &lt;strong&gt;Links &amp;amp; Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try MatrixSwarm: &lt;a href="https://matrixswarm.com" rel="noopener noreferrer"&gt;https://matrixswarm.com&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Join the Community / Discord: &lt;a href="https://discord.gg/2USbWVBVV" rel="noopener noreferrer"&gt;https://discord.gg/2USbWVBVV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Download Server: &lt;a href="https://github.com/matrixswarm/matrixswarm" rel="noopener noreferrer"&gt;https://github.com/matrixswarm/matrixswarm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug" rel="noopener noreferrer"&gt;https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug&lt;/a&gt;&lt;/p&gt;

</description>
      <category>devops</category>
      <category>webdev</category>
      <category>security</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Crypto Alert: Your Chart Does Not Need a Night Shift Supervisor</title>
      <dc:creator>Daniel</dc:creator>
      <pubDate>Fri, 09 Oct 2026 23:58:47 +0000</pubDate>
      <link>https://dev.to/matrixswarm/crypto-alert-your-chart-does-not-need-a-night-shift-supervisor-52bm</link>
      <guid>https://dev.to/matrixswarm/crypto-alert-your-chart-does-not-need-a-night-shift-supervisor-52bm</guid>
      <description>&lt;p&gt;At 02:14, the chart moves.&lt;/p&gt;

&lt;p&gt;At 02:15, an operator refreshes it.&lt;/p&gt;

&lt;p&gt;At 02:16, the operator refreshes it harder, as though the browser has been withholding liquidity out of spite.&lt;/p&gt;

&lt;p&gt;We can automate this shift.&lt;/p&gt;

&lt;p&gt;MatrixSwarm’s &lt;strong&gt;crypto_alert&lt;/strong&gt; agent watches public cryptocurrency prices and individual Bitcoin addresses, evaluates the rules you give it, and hands notifications to your swarm’s alert relays. Phoenix provides the &lt;strong&gt;Crypto Watches&lt;/strong&gt; panel for creating, editing, pausing, and deleting those rules.&lt;/p&gt;

&lt;p&gt;The agent takes observation duty. You can stop impersonating a cron job with caffeine dependency.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What is on the duty roster?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The current agent supports price thresholds, percentage and absolute moves, derived asset ratios, conversion-value thresholds, and confirmed Bitcoin address changes. Each active watch has its own worker and state; one shared Phemex connection supplies the market prices. An agent accepts up to &lt;strong&gt;64 watches&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;It uses public data. There is no exchange account to connect and no exchange API key to supply. It does not place orders or sign blockchain transactions. Bitcoin watches need public addresses, never seed phrases, private keys, or extended wallet keys.&lt;/p&gt;

&lt;p&gt;Our examples below use invented numbers to explain the controls. They are not current market quotes. The crystal ball has been removed from the dependency list.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Install once, assemble a small swarm&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If MatrixOS is missing from your server, add and test its SSH connection in Phoenix’s &lt;strong&gt;Registry&lt;/strong&gt;, including the trusted host fingerprint. Open &lt;strong&gt;Railgun → Install MatrixOS…&lt;/strong&gt;, choose the host, then use &lt;strong&gt;Local Full Install&lt;/strong&gt; with your MatrixOS source folder or &lt;strong&gt;Install from GitHub&lt;/strong&gt;. For a fresh environment, select &lt;strong&gt;Create new venv&lt;/strong&gt;, install, and check the completion result. Installation needs root or passwordless sudo access on the target.&lt;/p&gt;

&lt;p&gt;Already installed? Carry on. The runtime does not need a ceremonial reinstallation every time we give it a new job.&lt;/p&gt;

&lt;p&gt;Open &lt;strong&gt;Deploy → Swarm Workspaces&lt;/strong&gt;. Create a &lt;strong&gt;New&lt;/strong&gt; workspace or &lt;strong&gt;Open&lt;/strong&gt; an existing one. Keep the &lt;code&gt;matrix&lt;/code&gt; root and add these agents beneath it:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Agent&lt;/th&gt;
&lt;th&gt;Job on this shift&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;crypto_alert&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Observes prices and Bitcoin addresses, evaluates saved rules&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_https&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Accepts Phoenix commands through HTTPS ingress&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_websocket&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Returns configuration and live watch updates to Phoenix&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;log_streamer&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Provides Agent Logs for troubleshooting&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;An alert relay&lt;/td&gt;
&lt;td&gt;Delivers notifications to Slack, Discord, Telegram, or email&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The current HTTP ingress is named &lt;strong&gt;matrix_https&lt;/strong&gt; in the palette. Resolve the signing assignments and the HTTPS/WSS connection and certificate requirements, using endpoints Phoenix can reach. Keep the supplied service roles for your first run.&lt;/p&gt;

&lt;p&gt;Crypto Watches sends its own live updates through the RPC/WebSocket return path. &lt;strong&gt;log_streamer is for the agents’ operational logs&lt;/strong&gt;; it does not carry the price feed. Also, the external Phemex WebSocket and your swarm’s &lt;code&gt;matrix_websocket&lt;/code&gt; agent are separate connections with separate jobs.&lt;/p&gt;

&lt;p&gt;One socket watches the market. The other reports to the cockpit. Neither needs to become a podcast.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Give the watchkeeper a memory&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Resolve &lt;strong&gt;both&lt;/strong&gt; of crypto_alert’s required constraints: &lt;code&gt;packet_signing&lt;/code&gt; and &lt;code&gt;persistent_state&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Create a dedicated Persistent State profile in the Registry, then assign it to the agent’s persistent_state constraint in the workspace. Keep that same profile for future deployments of this crypto agent on the same host and universe. Give separate running crypto agents separate profiles.&lt;/p&gt;

&lt;p&gt;That profile supplies the stable identity and encryption material for saved watches and their trigger state. With the state directory and profile retained, the agent can restore them after a restart or redeployment, even when its agent ID changes.&lt;/p&gt;

&lt;p&gt;A fresh key and an empty directory are not a backup strategy. They are amnesia with excellent branding.&lt;/p&gt;

&lt;p&gt;In the agent’s workspace Config editor, check &lt;strong&gt;Bitcoin Esplora API (HTTPS)&lt;/strong&gt; and &lt;strong&gt;Alert delivery role&lt;/strong&gt;. The defaults are &lt;code&gt;https://blockstream.info/api&lt;/code&gt; and &lt;code&gt;hive.alert&lt;/code&gt;. API endpoint changes require a restart or redeployment. Everyday watch edits happen later in the live panel.&lt;/p&gt;

&lt;p&gt;For an older workspace, make sure it includes and resolves the current persistent-state constraint. Updating Python source alone does not provision that assignment.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Launch through Railgun&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Click &lt;strong&gt;Deploy&lt;/strong&gt; inside the workspace. Supply a deployment label, select the Registry SSH target, set a universe and runtime Linux user, review the resolved directive, and launch. Use a separate demo universe for your first exercise.&lt;/p&gt;

&lt;p&gt;Phoenix prepares the encrypted deployment data. Railgun verifies the SSH host, prepares the runtime account and applicable grants, and sends the sealed boot envelope to MatrixD over SSH stdin. MatrixD decrypts it in memory and starts the swarm without leaving loose directive/key boot files behind.&lt;/p&gt;

&lt;p&gt;Check the remote launch result, then choose the deployment and &lt;strong&gt;Connect&lt;/strong&gt;. Deployment starts the swarm; Connect opens its cockpit. Select the crypto agent and open &lt;strong&gt;Crypto Watches&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The briefing is over. We may now stop configuring the chair and sit in it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Your first price watch&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Start with one rule so you can follow the entire route from observation to notification.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Let the panel load the agent’s saved watch list.&lt;/li&gt;
&lt;li&gt;Click &lt;strong&gt;＋ Price / Pair Watch&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Give it a useful label, such as &lt;code&gt;BTC observation test&lt;/code&gt;, and use &lt;code&gt;BTC/USDT&lt;/code&gt; as the pair.&lt;/li&gt;
&lt;li&gt;Choose &lt;code&gt;price_above&lt;/code&gt; and enter a positive target in &lt;strong&gt;Alert price (USDT)&lt;/strong&gt;. The new card deliberately leaves the target blank.&lt;/li&gt;
&lt;li&gt;Leave &lt;strong&gt;Active&lt;/strong&gt;, &lt;strong&gt;Alert Enabled&lt;/strong&gt;, and &lt;strong&gt;Stream Enabled&lt;/strong&gt; checked. Select &lt;strong&gt;One Shot&lt;/strong&gt; for this first delivery test.&lt;/li&gt;
&lt;li&gt;Click &lt;strong&gt;Save Changes&lt;/strong&gt; and wait for the saved encrypted-state revision acknowledgment.&lt;/li&gt;
&lt;li&gt;Check the feed status, the card’s observations, its hit count, and the actual receiving relay destination.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A threshold rule can fire on its &lt;strong&gt;first fresh observation&lt;/strong&gt; if the condition is already satisfied. For an intentional test on a destination you control, choose a target already satisfied by the fresh quote. One Shot limits that watch to one trigger. No purchase, sale, or transfer is needed to test the alert route.&lt;/p&gt;

&lt;p&gt;Until Save Changes succeeds, the card is a draft and the agent continues using its last saved configuration. Looking extremely satisfied with the form does not transmit it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The rule menu, translated into human&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Trigger&lt;/th&gt;
&lt;th&gt;Meaning&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;price_above&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Price or ratio is at or above the target&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;price_below&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Price or ratio is at or below the target&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;price_change_above&lt;/code&gt; / &lt;code&gt;price_change_below&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;Positive percentage magnitude upward/downward from this watch’s armed baseline&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;price_delta_above&lt;/code&gt; / &lt;code&gt;price_delta_below&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;Positive absolute magnitude upward/downward from the baseline, in the pair’s quote units&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;asset_conversion&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The calculated value of a source amount reaches a target amount of another asset&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;wallet_change&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A watched Bitcoin address’s confirmed balance or confirmed transaction count changes&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For a downward percentage or delta rule, enter a &lt;strong&gt;positive magnitude&lt;/strong&gt;. Choose the &lt;code&gt;below&lt;/code&gt; trigger to specify the direction.&lt;/p&gt;

&lt;p&gt;Percentage and delta rules measure from the watch’s armed baseline, &lt;strong&gt;not a rolling 24-hour window&lt;/strong&gt;. After a trigger, that watch’s baseline resets to the observed price.&lt;/p&gt;

&lt;p&gt;For example, a hypothetical baseline of 100 and an upward move of 5% triggers at 105 or more. If the observation that fires it is 106, the new baseline becomes 106. A second watch on the same pair keeps its own baseline.&lt;/p&gt;

&lt;p&gt;The fact that the upstream ticker includes 24-hour market data does not turn your custom rule into a 24-hour rule. The paperwork matters, even when the candles are excited.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Reset Band: please stop ringing the same doorbell&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Threshold and conversion rules latch after firing. &lt;strong&gt;Reset Band (%)&lt;/strong&gt; determines how far the observed value must retreat before the rule can arm again. The default is &lt;strong&gt;0.5%&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Using an invented threshold of 100:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Rule&lt;/th&gt;
&lt;th&gt;Fires&lt;/th&gt;
&lt;th&gt;With a 0.5% band, rearms when&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;price_above&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;At 100 or higher&lt;/td&gt;
&lt;td&gt;Value falls to 99.5 or lower&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;price_below&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;At 100 or lower&lt;/td&gt;
&lt;td&gt;Value rises to 100.5 or higher&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Rearming permits a later qualifying observation to trigger again, subject to cooldown and limits. Remaining beyond the target does not produce a new threshold alert every second. With a zero reset band, the value still has to move strictly to the other side of the threshold to rearm.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Cooldown&lt;/strong&gt; adds a minimum interval between triggers. It does not replace the reset band, and it does not promise a reminder every cooldown period. Conditions are evaluated from observations; a brief move between observations can be missed.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;One Shot&lt;/strong&gt; allows one trigger, then leaves the watch complete until you change and save it. &lt;strong&gt;Trigger Limit&lt;/strong&gt; sets a maximum hit count; &lt;strong&gt;0 means unlimited&lt;/strong&gt;, unless One Shot is selected. Reaching a limit leaves the card visible.&lt;/p&gt;

&lt;p&gt;A fired watch has done its assigned job. It is not sulking because you looked away.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Pairs, ratios, and conversion arithmetic&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;BTC/USDT&lt;/code&gt; is a Phemex spot price in USDT per BTC. A cross-pair such as &lt;code&gt;BTC/ETH&lt;/code&gt; is &lt;strong&gt;derived&lt;/strong&gt; by dividing BTC/USDT by ETH/USDT. Its units are ETH per BTC.&lt;/p&gt;

&lt;p&gt;Suppose the observed source prices were 60,000 USDT per BTC and 3,000 USDT per ETH. The derived ratio would be 20 ETH per BTC. Reversing the pair gives 0.05 BTC per ETH. Read the dynamically labeled &lt;strong&gt;Alert ratio&lt;/strong&gt; field before typing the target.&lt;/p&gt;

&lt;p&gt;For &lt;code&gt;asset_conversion&lt;/code&gt;, set &lt;strong&gt;From Asset&lt;/strong&gt;, &lt;strong&gt;To Asset&lt;/strong&gt;, the source amount, and the target amount. With those same invented prices, 0.1 BTC calculates to 2 ETH:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Source amount × source USDT price ÷ target USDT price.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;These values describe arithmetic using spot observations. They are not executable swap quotes and do not include fees, spreads, slippage, or market depth. The agent observes the relationship; it does not execute the conversion.&lt;/p&gt;

&lt;p&gt;Both required price legs must be fresh. Unsupported markets or stale data produce a waiting status instead of an imaginary number with impressive decimal places.&lt;/p&gt;

&lt;p&gt;The agent uses Phemex’s public &lt;code&gt;spot_market24h.subscribe&lt;/code&gt; feed. Its endpoint follows the &lt;a href="https://phemex.com/announcements/phemex-updates-api-entry-endpoints" rel="noopener noreferrer"&gt;official public endpoint migration notice&lt;/a&gt;; the &lt;a href="https://github.com/phemex/phemex-api-docs/blob/master/Public-Spot-API-en.md#subscribe-24-hours-ticker" rel="noopener noreferrer"&gt;spot protocol reference&lt;/a&gt; describes the ticker messages. The implementation checks freshness, rejects quotes older than 15 seconds, and clears cached quotes during reconnects. Keep the host’s clock accurate.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Bitcoin addresses: the blockchain has a visitor log&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Click &lt;strong&gt;＋ Bitcoin Address&lt;/strong&gt;, enter an individual Bitcoin mainnet public address, and choose a check interval. The default is &lt;strong&gt;60 seconds&lt;/strong&gt;, with a minimum of &lt;strong&gt;30 seconds&lt;/strong&gt;. Save and wait for acknowledgment.&lt;/p&gt;

&lt;p&gt;The first successful lookup establishes the baseline. It does not notify you about the address’s entire history as a welcoming ceremony.&lt;/p&gt;

&lt;p&gt;The card can show confirmed BTC, pending net BTC, confirmed transaction count, and a USDT valuation when a fresh BTC price is available. Confirmed-change monitoring can continue while the Phemex price feed is unavailable; the valuation simply depends on that separate feed.&lt;/p&gt;

&lt;p&gt;An alert can fire when either the confirmed balance or confirmed transaction count changes. A count change can therefore trigger even when the net balance is unchanged. Pending activity is displayed but does not trigger this rule. During cooldown, the next eligible notice reports the net confirmed change since the previous notice.&lt;/p&gt;

&lt;p&gt;This is periodic address-state monitoring. It can miss intermediate changes between polls, and chain reorganizations can change the observed state. It is not a complete transaction ledger or a configurable multi-confirmation settlement policy.&lt;/p&gt;

&lt;p&gt;Watch each address you care about. One address does not represent every receive/change address in an HD wallet. This agent does not scan xpubs or index Ethereum/ERC-20 wallet activity.&lt;/p&gt;

&lt;p&gt;The default explorer is Blockstream’s Esplora API. The agent reads the address statistics documented by &lt;a href="https://github.com/Blockstream/esplora/blob/master/API.md#addresses" rel="noopener noreferrer"&gt;Esplora&lt;/a&gt;. A public explorer can see the queried addresses and the request’s network origin. You can configure your own HTTPS Esplora endpoint; keep valid certificate trust in place.&lt;/p&gt;

&lt;p&gt;Publicly visible on a blockchain does not mean your personal address collection needs its own press release.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Send the report where somebody will read it&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Crypto Alert uses the configured alert role, normally &lt;strong&gt;hive.alert&lt;/strong&gt;, to find notification destinations. You can combine it with any of the swarm’s usual alarm relays:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Destination&lt;/th&gt;
&lt;th&gt;Agent&lt;/th&gt;
&lt;th&gt;Optional signed alert-payload encryption&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Slack&lt;/td&gt;
&lt;td&gt;&lt;code&gt;slack_relay&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Not provided by this relay’s current alert configuration&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Discord&lt;/td&gt;
&lt;td&gt;&lt;code&gt;discord_relay&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Telegram&lt;/td&gt;
&lt;td&gt;&lt;code&gt;telegram_relay&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Email&lt;/td&gt;
&lt;td&gt;&lt;code&gt;email_send&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Configure the relay’s destination and credentials through its usual Registry/workspace assignments. For Discord, Telegram, or email, optional &lt;code&gt;encrypt_alerts&lt;/code&gt; protects the alert payload using the assigned signing/encryption material. The receiving side needs the matching keys and compatible decryption support to read that envelope. This is a separate setting from HTTPS/WSS or SMTP transport security.&lt;/p&gt;

&lt;p&gt;Every discovered endpoint advertising the chosen role is considered. If you need selective delivery, use a dedicated role shared by this crypto agent and the intended relay instead of assuming the first relay wins.&lt;/p&gt;

&lt;p&gt;Address alert text abbreviates the address, but labels and balance details still belong to the report. Choose labels, recipients, and encryption accordingly. Authorized Phoenix panels can display the full address.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Three switches and a save button walk into a cockpit&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Control&lt;/th&gt;
&lt;th&gt;Effect after saving&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;strong&gt;Active&lt;/strong&gt; off&lt;/td&gt;
&lt;td&gt;Stops that watch’s worker&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;strong&gt;Alert Enabled&lt;/strong&gt; off&lt;/td&gt;
&lt;td&gt;Keeps observation running while suppressing notifications&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;strong&gt;Stream Enabled&lt;/strong&gt; off&lt;/td&gt;
&lt;td&gt;Stops that watch’s periodic panel updates while remote monitoring continues&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Closing or hiding the panel releases its live subscription. &lt;strong&gt;It does not stop the saved remote watches.&lt;/strong&gt; The panel’s subscription lease expires if the client disappears; MatrixOS continues the monitoring work.&lt;/p&gt;

&lt;p&gt;To pause a watch, clear Active and &lt;strong&gt;Save Changes&lt;/strong&gt;. To remove it, delete its card and save. Saving an empty list removes all watches from this agent.&lt;/p&gt;

&lt;p&gt;Any &lt;strong&gt;changed watch that you save&lt;/strong&gt; resets that watch’s baseline, hit count, and crossing state—even a label or toggle change. Unchanged watches retain theirs. Clicking Save on an identical completed watch does not rearm it; make an intentional edit and save when you want a fresh run.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Reload Saved&lt;/strong&gt; fetches the agent’s authoritative list and asks before discarding unsaved edits. Revision checks prevent another panel from silently overwriting a newer list. If a save times out, its outcome is unknown: reload before retrying.&lt;/p&gt;

&lt;p&gt;The panel also offers &lt;strong&gt;Clean Cockpit&lt;/strong&gt;, &lt;strong&gt;Fusion Reactor&lt;/strong&gt;, and &lt;strong&gt;Arcane Oracle&lt;/strong&gt; display styles. Pick your preferred level of theatrical instrumentation. The oracle still uses the same rules; the robe grants no additional foresight.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Persistence and delivery: read the receipt&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Watches and their saved runtime state use authenticated encrypted storage through the Persistent State profile. Retain the encrypted files and securely back up the profile’s key material. Keeping only ciphertext is an ambitious way to archive something you can never read again.&lt;/p&gt;

&lt;p&gt;The deployment’s initial watch list seeds a new store only. It does not overwrite an existing saved list, including an intentionally empty one. Invalid or undecryptable saved state fails closed. Older experimental watch stores are not automatically migrated.&lt;/p&gt;

&lt;p&gt;A trigger’s state is persisted &lt;strong&gt;before&lt;/strong&gt; notification is attempted. The card’s delivery field has a precise meaning:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Status&lt;/th&gt;
&lt;th&gt;What it tells you&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;pending&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The trigger was recorded; a completed delivery result is not recorded&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;queued&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The notification was handed to at least one swarm endpoint&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;failed&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;No successful handoff was recorded&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Queued does not confirm arrival in Discord, Telegram, Slack, or an inbox.&lt;/strong&gt; Check the intended relay and recipient. Failed or interrupted events are not automatically retried; that avoids duplicate replays but means a recorded trigger can lack a delivered notification. One Shot and hit limits count triggers, not messages you personally read.&lt;/p&gt;

&lt;p&gt;This is best-effort monitoring and alerting. The receipt says what happened, rather than what we hoped happened.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;When the watchtower is quiet&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Symptom&lt;/th&gt;
&lt;th&gt;First place to look&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Waiting for fresh prices&lt;/td&gt;
&lt;td&gt;Phemex connectivity, supported USDT markets for both legs, and host clock&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Card says unsaved draft&lt;/td&gt;
&lt;td&gt;Complete the fields, Save Changes, and read the acknowledgment&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Price is past the target but no new alert&lt;/td&gt;
&lt;td&gt;One Shot, hit limit, reset band, cooldown, Active, and Alert Enabled&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Panel values look stale&lt;/td&gt;
&lt;td&gt;Phoenix connection, Stream Enabled, and the RPC/WebSocket return route&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Bitcoin watch reports an SSL error&lt;/td&gt;
&lt;td&gt;Host CA trust, proxy configuration, and the HTTPS explorer certificate&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Save fails or revision conflicts&lt;/td&gt;
&lt;td&gt;Persistent State assignment, directory write access, and Reload Saved&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Hit count rises but no external message arrives&lt;/td&gt;
&lt;td&gt;Delivery status, alert role, relay configuration, and recipient&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Use &lt;strong&gt;Agent Logs&lt;/strong&gt; through log_streamer for the remaining clues. Start with one saved watch and one intended relay, verify the whole route, then expand the roster.&lt;/p&gt;

&lt;p&gt;Install, assemble, resolve, deploy, connect, save, verify. The same small-swarm habit now gives you a remote watchkeeper with a memory, a notification route, and clearly defined rules.&lt;/p&gt;

&lt;p&gt;The chart may continue making questionable life choices. You no longer have to supervise every candle in person.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Victory Always. The night shift has been assigned.&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;🌐 &lt;strong&gt;Links &amp;amp; Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try MatrixSwarm: &lt;a href="https://matrixswarm.com" rel="noopener noreferrer"&gt;https://matrixswarm.com&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Join the Community / Discord: &lt;a href="https://discord.gg/2USbWVBVV" rel="noopener noreferrer"&gt;https://discord.gg/2USbWVBVV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Download Server: &lt;a href="https://github.com/matrixswarm/matrixswarm" rel="noopener noreferrer"&gt;https://github.com/matrixswarm/matrixswarm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug" rel="noopener noreferrer"&gt;https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug&lt;/a&gt;&lt;/p&gt;

</description>
      <category>devops</category>
      <category>crypto</category>
      <category>opensource</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>Terminal Streamer: Please Hold While We Ask the Server What’s Wrong</title>
      <dc:creator>Daniel</dc:creator>
      <pubDate>Fri, 09 Oct 2026 18:40:48 +0000</pubDate>
      <link>https://dev.to/matrixswarm/terminal-streamer-please-hold-while-we-ask-the-server-whats-wrong-2nd7</link>
      <guid>https://dev.to/matrixswarm/terminal-streamer-please-hold-while-we-ask-the-server-whats-wrong-2nd7</guid>
      <description>&lt;p&gt;Thank you for calling the Production Incident Hotline.&lt;/p&gt;

&lt;p&gt;Press 1 if someone thinks it is DNS.&lt;/p&gt;

&lt;p&gt;Press 2 if the dashboard is green but the customers are red.&lt;/p&gt;

&lt;p&gt;Press 3 if you would like to ask the actual server a question before scheduling a cross-functional guessing session.&lt;/p&gt;

&lt;p&gt;We are pressing 3.&lt;/p&gt;

&lt;p&gt;MatrixSwarm’s &lt;strong&gt;terminal_streamer&lt;/strong&gt; agent runs Linux commands on the swarm host and returns their output to the Terminal panel in Phoenix. Run a command once, or repeat a short diagnostic at a chosen interval. The palette supplies a command whitelist, and the panel can show you what the running agent has configured.&lt;/p&gt;

&lt;p&gt;It is a handy way to answer “How full is the disk?” while everyone else is still establishing the incident’s preferred emoji.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Get the host ready, then assemble a small swarm&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If MatrixOS is not installed, add and test the target server’s SSH connection in Phoenix’s &lt;strong&gt;Registry&lt;/strong&gt;, including its trusted host fingerprint. Open &lt;strong&gt;Railgun → Install MatrixOS…&lt;/strong&gt;, select the target, and choose &lt;strong&gt;Local Full Install&lt;/strong&gt; with your MatrixOS source folder or &lt;strong&gt;Install from GitHub&lt;/strong&gt;. For a fresh setup, select &lt;strong&gt;Create new venv&lt;/strong&gt;, click &lt;strong&gt;Install MatrixOS&lt;/strong&gt;, and check the completion result. Installation requires root or passwordless sudo access on the target.&lt;/p&gt;

&lt;p&gt;Already installed? Excellent. We have successfully avoided reinstalling the building to use the telephone.&lt;/p&gt;

&lt;p&gt;Open &lt;strong&gt;Deploy → Swarm Workspaces&lt;/strong&gt; and choose &lt;strong&gt;New&lt;/strong&gt;, or &lt;strong&gt;Open&lt;/strong&gt; an existing workspace. Keep the &lt;code&gt;matrix&lt;/code&gt; root and add the following agents beneath it from the Agent Palette:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Agent&lt;/th&gt;
&lt;th&gt;Responsibility&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;terminal_streamer&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Runs commands and returns their results&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_https&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Ingress for Phoenix commands over HTTPS&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_websocket&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Egress for replies back to Phoenix&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;log_streamer&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Makes the agents’ own logs available in the cockpit&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The HTTP ingress is named &lt;strong&gt;&lt;code&gt;matrix_https&lt;/code&gt;&lt;/strong&gt; in the current palette. Resolve the required packet-signing assignments and the HTTPS/WSS connection and certificate requirements. Set addresses and ports that Phoenix can reach. Keep the supplied service roles for this first walkthrough.&lt;/p&gt;

&lt;p&gt;Five agents including Matrix. Enough staff to answer the phone without forming a steering committee.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Two kinds of output, two different jobs&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Terminal command results and agent logs take different paths through the application:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Terminal output:&lt;/strong&gt; &lt;code&gt;terminal_streamer&lt;/code&gt; executes the requested command and sends the result through the reply route to the Terminal panel.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Agent Logs:&lt;/strong&gt; &lt;code&gt;log_streamer&lt;/code&gt; reads the selected agent’s log and sends it back for operational diagnosis.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Both use the configured return transport in this swarm, provided by &lt;code&gt;matrix_websocket&lt;/code&gt;. &lt;strong&gt;You do not need &lt;code&gt;log_streamer&lt;/code&gt; to carry command stdout.&lt;/strong&gt; We include it so you can inspect the agents when troubleshooting the command and reply path.&lt;/p&gt;

&lt;p&gt;The WebSocket relay advertises &lt;code&gt;hive.rpc&lt;/code&gt;, which the terminal agent uses for replies. The terminal agent’s service roles cover starting a run, stopping repeats, and listing allowed commands. You can keep those details at their supplied defaults and get on with the job.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The command menu is already written&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Yes, the commands are whitelisted. The current palette sets &lt;code&gt;safe_shell_mode&lt;/code&gt; to &lt;code&gt;true&lt;/code&gt; and supplies this list:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Command&lt;/th&gt;
&lt;th&gt;The question it helps answer&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;whoami&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Which account is actually running this?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;uptime&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;How long has the host been up, and what does its load look like?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;df -h&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;How much filesystem space is available?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;free -m&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;What does memory usage look like in MiB?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;top -b -n 1 -c&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Can I get one batch-mode process snapshot?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;ps -eo pid,user,%cpu,%mem,etime,cmd --sort=-%cpu&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Which processes deserve a closer look?&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For the first deployment, use that supplied configuration. &lt;strong&gt;List Allowed&lt;/strong&gt; asks the running agent for its configured list; that is more useful than assuming every old workspace has the newest palette values.&lt;/p&gt;

&lt;p&gt;Commands run with the terminal agent’s OS permissions on the remote host. The account you use to install MatrixOS and the account running the swarm are separate concerns. &lt;code&gt;whoami&lt;/code&gt; is an excellent first call because Linux rarely accepts “but I thought I was root” as supporting documentation.&lt;/p&gt;

&lt;p&gt;There is one important boundary: the current filter checks command prefixes, and execution uses a shell. &lt;strong&gt;Treat this as a facility for trusted operators, not a security sandbox or exact-command isolation.&lt;/strong&gt; Keep safe shell mode enabled, use the intended diagnostic commands, and run the swarm with appropriately limited permissions. The word “safe” in a setting does not negotiate with the kernel on your behalf.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Launch through Railgun&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Click &lt;strong&gt;Deploy&lt;/strong&gt; inside the workspace. Supply a deployment label, choose the Registry SSH target, set a universe name and runtime Linux user, review the resolved directive, and complete the launch flow. Use a distinct demo universe for the first exercise, and verify Railgun’s remote result.&lt;/p&gt;

&lt;p&gt;Phoenix prepares the encrypted runtime directive and keeps the deployment record in its encrypted vault. Railgun verifies the SSH host, prepares the runtime account and applicable grants, and sends the sealed boot envelope to MatrixD over SSH stdin. MatrixD decrypts it in memory and starts the swarm, without leaving loose directive/key boot files behind.&lt;/p&gt;

&lt;p&gt;Then select the deployment and click &lt;strong&gt;Connect&lt;/strong&gt;. Deployment launches the swarm; Connect opens its cockpit. The hold music is optional.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Your first call: identify the caller&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Select &lt;code&gt;terminal_streamer&lt;/code&gt; in the live agent tree and open its &lt;strong&gt;Terminal&lt;/strong&gt; specialty panel.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Click &lt;strong&gt;List Allowed&lt;/strong&gt; and read the returned command list.&lt;/li&gt;
&lt;li&gt;Enter &lt;code&gt;whoami&lt;/code&gt; in the command field.&lt;/li&gt;
&lt;li&gt;Set the adjacent refresh field to &lt;strong&gt;0&lt;/strong&gt; for one execution.&lt;/li&gt;
&lt;li&gt;Click &lt;strong&gt;Run&lt;/strong&gt; and confirm the returned account name.&lt;/li&gt;
&lt;li&gt;Repeat with &lt;code&gt;uptime&lt;/code&gt; or &lt;code&gt;df -h&lt;/code&gt;, keeping refresh at &lt;strong&gt;0&lt;/strong&gt; while you check the basics.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;These results come from the machine running the agent. Phoenix is the console you are using to ask the question; your workstation’s disk is not secretly auditioning for the server’s role.&lt;/p&gt;

&lt;p&gt;With the stock list, a benign command such as &lt;code&gt;hostname&lt;/code&gt; is outside the supplied prefixes and should return a &lt;strong&gt;[BLOCKED] Command not allowed&lt;/strong&gt; message. That is a useful functional check of the configured filter, not proof of a hardened sandbox.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Put one useful question on repeat&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try &lt;code&gt;free -m&lt;/code&gt; with a refresh value of &lt;strong&gt;5&lt;/strong&gt; and click &lt;strong&gt;Run&lt;/strong&gt;. The agent executes the command, returns the completed result, waits five seconds, and runs it again.&lt;/p&gt;

&lt;p&gt;That timing matters: &lt;strong&gt;execution time plus the refresh delay&lt;/strong&gt; determines the next run. It is not a promise of samples exactly five seconds apart.&lt;/p&gt;

&lt;p&gt;A refresh value of &lt;strong&gt;0&lt;/strong&gt; runs once. Use positive whole seconds for repeats. The panel’s &lt;strong&gt;Auto-Refresh&lt;/strong&gt; display control is separate from this execution interval; toggling how output is displayed does not replace the refresh field or stop the remote loop.&lt;/p&gt;

&lt;p&gt;Click &lt;strong&gt;Stop&lt;/strong&gt; when you have enough samples. Starting a new run in the same cockpit session replaces that session’s previous repeat schedule.&lt;/p&gt;

&lt;p&gt;A useful favorite might be &lt;code&gt;df -h&lt;/code&gt; with a modest refresh interval. Enter the command and interval, then choose &lt;strong&gt;Add to Favorites&lt;/strong&gt;. Double-clicking a favorite populates the fields; click &lt;strong&gt;Run&lt;/strong&gt; to execute it. Saving a favorite does not add a command to the agent’s whitelist.&lt;/p&gt;

&lt;p&gt;Your own little diagnostic menu. Finally, a menu where pressing 2 does something.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What “streaming” means here&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The implementation collects a command’s output before sending the result. It uses Python’s &lt;a href="https://docs.python.org/3/library/subprocess.html#subprocess.check_output" rel="noopener noreferrer"&gt;subprocess.check_output&lt;/a&gt;, then broadcasts the completed output through the swarm’s reply machinery. Repeated snapshots make the panel useful for observing changes over time.&lt;/p&gt;

&lt;p&gt;This is not a persistent interactive terminal session. There is no PTY keyboard conversation with &lt;code&gt;vim&lt;/code&gt;, no interactive password prompt, and no lasting shell state between separate runs. A working-directory change in one invocation is not carried into the next.&lt;/p&gt;

&lt;p&gt;Choose commands that finish and produce manageable output. That is why the supplied &lt;code&gt;top&lt;/code&gt; command uses batch mode and a single iteration. A never-ending command can keep the result waiting indefinitely; this execution path does not set a command timeout.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Stop cancels future repetitions; it does not terminate a subprocess already running.&lt;/strong&gt; The current invocation may still finish and return output. If it is stuck, use your normal authorized host-management path to inspect it. Repeatedly clicking Run is not a medically recognized treatment for impatience.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;If the line goes quiet&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Start with what you can observe:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Symptom&lt;/th&gt;
&lt;th&gt;First check&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;List Allowed produces no reply&lt;/td&gt;
&lt;td&gt;Connection, signing assignments, service discovery, and the WebSocket return route&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Command returns [BLOCKED]&lt;/td&gt;
&lt;td&gt;The running agent’s allowed list and the command you entered&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Command returns [ERROR]&lt;/td&gt;
&lt;td&gt;Command availability and permissions on the swarm host&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A command appears to hang&lt;/td&gt;
&lt;td&gt;Whether it finishes without input and returns bounded output&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Terminal results work, but Agent Logs are empty&lt;/td&gt;
&lt;td&gt;The separate &lt;code&gt;log_streamer&lt;/code&gt; agent and its route&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The terminal agent also monitors relay session freshness and can remove stale repeat sessions. Keep this first example to one WebSocket reply relay, and use &lt;strong&gt;Stop&lt;/strong&gt; intentionally when your observation is complete. A disconnected panel is not a durable job scheduler.&lt;/p&gt;

&lt;p&gt;Use the agents’ own logs for the remaining clues. A command failure may be presented as an error summary rather than every detail you would see in a normal terminal, so keep your usual host-level diagnostic tools available too.&lt;/p&gt;

&lt;p&gt;You now have the same repeatable habit as the other small swarms: install the runtime once, build or edit the workspace, resolve its requirements, deploy through Railgun, connect, and verify a useful result.&lt;/p&gt;

&lt;p&gt;This time the useful result is the server answering for itself.&lt;/p&gt;

&lt;p&gt;Thank you for calling the Production Incident Hotline. Your next incident may still involve DNS, but at least we have asked the disk.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Victory Always. Please stop the refresh loop before leaving the call.&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;🌐 &lt;strong&gt;Links &amp;amp; Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try MatrixSwarm: &lt;a href="https://matrixswarm.com" rel="noopener noreferrer"&gt;https://matrixswarm.com&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Join the Community / Discord: &lt;a href="https://discord.gg/2USbWVBVV" rel="noopener noreferrer"&gt;https://discord.gg/2USbWVBVV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Download Server: &lt;a href="https://github.com/matrixswarm/matrixswarm" rel="noopener noreferrer"&gt;https://github.com/matrixswarm/matrixswarm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug" rel="noopener noreferrer"&gt;https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug&lt;/a&gt;&lt;/p&gt;

</description>
      <category>devops</category>
      <category>linux</category>
      <category>opensource</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>Tripwire Lite: Your Uploads Folder Did Not Hire a PHP Developer</title>
      <dc:creator>Daniel</dc:creator>
      <pubDate>Fri, 09 Oct 2026 08:37:06 +0000</pubDate>
      <link>https://dev.to/matrixswarm/tripwire-lite-your-uploads-folder-did-not-hire-a-php-developer-3cfa</link>
      <guid>https://dev.to/matrixswarm/tripwire-lite-your-uploads-folder-did-not-hire-a-php-developer-3cfa</guid>
      <description>&lt;p&gt;02:13. A file called &lt;code&gt;maintenance.php&lt;/code&gt; walks into the uploads directory.&lt;/p&gt;

&lt;p&gt;It says it is with IT.&lt;/p&gt;

&lt;p&gt;IT says it has never seen this file before.&lt;/p&gt;

&lt;p&gt;The file would prefer we stop asking questions and grant it a corner office.&lt;/p&gt;

&lt;p&gt;This is a good job for MatrixSwarm’s &lt;strong&gt;Tripwire Lite&lt;/strong&gt;, the &lt;code&gt;tripwire_lite&lt;/code&gt; agent: a filesystem watcher with an extension policy, a dry-run mode, optional quarantine, and a way to call the operator. We will give it a small swarm, connect Phoenix, and watch a harmless test event come back.&lt;/p&gt;

&lt;p&gt;The setup is short. The interesting part is teaching the guard what belongs in the building.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;First, put MatrixOS on the target&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Already installed? Move straight to the workspace.&lt;/p&gt;

&lt;p&gt;Otherwise, add and test the Linux server’s SSH connection in Phoenix’s &lt;strong&gt;Registry&lt;/strong&gt;, including its trusted host fingerprint. Open &lt;strong&gt;Railgun → Install MatrixOS…&lt;/strong&gt;, choose the SSH target and an install source: &lt;strong&gt;Local Full Install&lt;/strong&gt; for your local MatrixOS folder, or &lt;strong&gt;Install from GitHub&lt;/strong&gt;. For a fresh installation, choose &lt;strong&gt;Create new venv&lt;/strong&gt;, then &lt;strong&gt;Install MatrixOS&lt;/strong&gt; and wait for successful completion.&lt;/p&gt;

&lt;p&gt;Installation needs root or passwordless sudo access on that target. Railgun handles the remote installation; you still check the result. A progress bar moving enthusiastically is not a completion certificate.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Build the little swarm&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Use Phoenix’s &lt;strong&gt;Deploy&lt;/strong&gt; button to open &lt;strong&gt;Swarm Workspaces&lt;/strong&gt;. Choose &lt;strong&gt;New&lt;/strong&gt;, or &lt;strong&gt;Open&lt;/strong&gt; an existing workspace to edit it. A new workspace starts with the &lt;code&gt;matrix&lt;/code&gt; root. Add these four agents beneath it from the Agent Palette:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Agent&lt;/th&gt;
&lt;th&gt;Job in this walkthrough&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;tripwire_lite&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Watches the selected filesystem paths and applies the policy&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_https&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Ingress: receives Phoenix commands over HTTPS&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;matrix_websocket&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Egress: carries live replies and alerts back to Phoenix&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;log_streamer&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Supplies the selected agent’s log stream through the reply route&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;That is five agents including Matrix. A small team. Everybody has a job. Nobody exists solely to forward meeting invitations.&lt;/p&gt;

&lt;p&gt;The current palette names the HTTP ingress agent &lt;strong&gt;&lt;code&gt;matrix_https&lt;/code&gt;&lt;/strong&gt;. Use that exact name when looking for it.&lt;/p&gt;

&lt;p&gt;Resolve the agents’ required Registry assignments before deployment: packet signing, HTTPS/WSS connections and certificates, and Tripwire’s symmetric-encryption key. Configure reachable ingress/egress addresses and ports for your host. Keep the supplied service roles unless you intend to change the routing.&lt;/p&gt;

&lt;p&gt;Tripwire can watch locally without a cockpit connection. &lt;strong&gt;For this connected walkthrough, include both ingress and egress, plus &lt;code&gt;log_streamer&lt;/code&gt; for live agent logs.&lt;/strong&gt; WebSocket provides the return channel; it does not tail Tripwire’s log file by itself.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Give the guard one door to watch&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Begin with a dedicated test directory, such as &lt;code&gt;/srv/tripwire-demo&lt;/code&gt;, outside a publicly served web root. Create it on the target and give the swarm’s runtime Linux account the directory access it needs. The path must exist on the server—not on the laptop running Phoenix.&lt;/p&gt;

&lt;p&gt;In Tripwire’s configuration editor, replace the broad example watch paths with that directory. Start with directory event watches: &lt;code&gt;watch_dirs: true&lt;/code&gt; and &lt;code&gt;watch_files: false&lt;/code&gt;. Enable &lt;code&gt;recursive&lt;/code&gt; if you want subdirectories covered. Directory watches report events for their entries; direct file watches additionally require file read permission. The underlying mechanism is Linux &lt;a href="https://man7.org/linux/man-pages/man7/inotify.7.html" rel="noopener noreferrer"&gt;inotify&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Here is a &lt;strong&gt;Tripwire config fragment&lt;/strong&gt;, not a whole swarm directive. Keep the palette’s service-manager and resolved security configuration around it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"watch_paths"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"path"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"/srv/tripwire-demo"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"recursive"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"watch_dirs"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"watch_files"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"ignore_paths"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"/srv/tripwire-demo/cache"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"allowed_extensions"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;".txt"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;".json"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;".jpg"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;".png"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"suspicious_extensions"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;".php"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;".phtml"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;".sh"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;".py"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;".exe"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"quarantine_root"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"dry_run"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"enforce"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"cooldown"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;60&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"interval"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"alert_to_role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"hive.alert"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"rpc_router_role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"hive.rpc"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Both mode switches are intentional. &lt;strong&gt;Enforce on plus Dry run on rehearses enforcement and can send “would quarantine” alerts while leaving files in place.&lt;/strong&gt; The palette’s detect-only starting state is a different behavior, so set the values explicitly.&lt;/p&gt;

&lt;p&gt;For production, replace the test policy with one that fits the watched location. A PHP file appearing in an image-upload directory deserves attention. A PHP file appearing in your PHP application may simply be reporting for work.&lt;/p&gt;

&lt;p&gt;Do not point this sample policy at all your application code—or at MatrixOS’s Python agents—and then act surprised when the guard starts escorting employees outside.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What gets stopped at reception&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The current implementation responds to relevant create, modify, and move-in events, then checks the path against its policy:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Ignored paths and Tripwire’s own quarantine directory are excluded.&lt;/li&gt;
&lt;li&gt;An extension in &lt;code&gt;suspicious_extensions&lt;/code&gt; is suspicious, even if also listed as allowed.&lt;/li&gt;
&lt;li&gt;With a nonempty &lt;code&gt;allowed_extensions&lt;/code&gt; list, another nonempty extension outside that list is also suspicious.&lt;/li&gt;
&lt;li&gt;An extensionless name is not rejected merely for being absent from the allowed list.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Use lowercase extensions with their leading dots. A harmless file called &lt;code&gt;visitor.php&lt;/code&gt; is enough to exercise the rule; no malicious payload is necessary. This guard checks the badge, not the visitor’s inner monologue.&lt;/p&gt;

&lt;p&gt;That last point matters: this is an &lt;strong&gt;event-driven extension policy&lt;/strong&gt;, not a content malware scanner or a hash-baseline integrity system. It does not establish that an allowed &lt;code&gt;.jpg&lt;/code&gt; contains a safe image. It does not retroactively classify every existing file at startup, and its enforcement path does not reconstruct deleted files. Keep upload validation, execution restrictions, and backups doing their own jobs.&lt;/p&gt;

&lt;p&gt;The editor’s interval setting should not be mistaken for a full-disk malware scan every five seconds. File events arrive through inotify; the regular worker also maintains the agent’s operational heartbeat.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Three shifts, two switches&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Enforce&lt;/th&gt;
&lt;th&gt;Dry run&lt;/th&gt;
&lt;th&gt;Result for an eligible suspicious event&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Off&lt;/td&gt;
&lt;td&gt;Either&lt;/td&gt;
&lt;td&gt;Detect-only: logs what would happen; ordinary detection does not dispatch the quarantine alert&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;On&lt;/td&gt;
&lt;td&gt;On&lt;/td&gt;
&lt;td&gt;Logs and can alert “would quarantine”; leaves the file in place&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;On&lt;/td&gt;
&lt;td&gt;Off&lt;/td&gt;
&lt;td&gt;Attempts to move the item into quarantine; can alert the outcome&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Alerts are subject to cooldown. In the current implementation, the cooldown uses one timestamp for the agent, so separate paths can share that quiet period. It is not an independent timer for every file.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Cooldown limits notifications, not active quarantine.&lt;/strong&gt; A quiet phone does not mean the guard has gone on break.&lt;/p&gt;

&lt;p&gt;Leave &lt;strong&gt;Quarantine Root&lt;/strong&gt; blank for the agent’s automatic quarantine directory in its universe-specific static storage. The resolved location appears in status/log output, and the guard excludes it from its own watches. Active enforcement needs permission to move items out of the watched directory and into quarantine; merely entering a path in Phoenix grants no filesystem permissions.&lt;/p&gt;

&lt;p&gt;Tripwire encrypts its stored quarantine-history records. The quarantined files themselves are moved by the filesystem operation; do not confuse that with encryption of their contents.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Launch it through Railgun&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Click &lt;strong&gt;Deploy&lt;/strong&gt; inside the workspace. Give the deployment a label, select the Registry SSH target, choose a universe name and runtime Linux user, review the resolved directive, and complete the launch flow. For a first exercise, use a distinct demo universe. Wait for Railgun’s remote result, then confirm the agents in the cockpit.&lt;/p&gt;

&lt;p&gt;Behind the button, Phoenix builds and encrypts the runtime directive and retains the deployment record in its encrypted vault. Railgun verifies the SSH host, prepares the runtime account and applicable grants, and streams the sealed boot envelope to MatrixD over SSH stdin. MatrixD decrypts it in memory and starts the swarm. The directive and swarm key are not left as loose remote boot files.&lt;/p&gt;

&lt;p&gt;Installing MatrixOS prepares the host. Deploying launches this swarm. &lt;strong&gt;Connect opens the cockpit session; it does not start the swarm.&lt;/strong&gt; Three jobs, three verbs. We can all go home without inventing a fourth wizard.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Connect, then make the guard earn its chair&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Select the new deployment in Phoenix and click &lt;strong&gt;Connect&lt;/strong&gt;. Choose Tripwire in the live agent tree. That selection requests its logs from &lt;code&gt;log_streamer&lt;/code&gt;; the replies return through &lt;code&gt;matrix_websocket&lt;/code&gt; into &lt;strong&gt;Agent Logs&lt;/strong&gt;. Tripwire’s panel also exposes &lt;strong&gt;Refresh Status&lt;/strong&gt;, &lt;strong&gt;Toggle Enforce&lt;/strong&gt;, &lt;strong&gt;Toggle Dry-Run&lt;/strong&gt;, and &lt;strong&gt;Reset Watches&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Check the reported modes, watched paths, watcher setup, and resolved quarantine location. Missing paths, permissions failures, or inotify resource limits need attention before the first test. “Agent exists” and “watch is installed” are different statements.&lt;/p&gt;

&lt;p&gt;Now rehearse in that dedicated test directory:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;With &lt;strong&gt;Enforce on / Dry run on&lt;/strong&gt;, create a new harmless &lt;code&gt;hello.txt&lt;/code&gt;. It is allowed by this sample policy and should not generate a suspicious-file alert.&lt;/li&gt;
&lt;li&gt;Create a new &lt;code&gt;visitor.php&lt;/code&gt; containing only plain text such as “Tripwire training file.” Look for &lt;strong&gt;Would quarantine&lt;/strong&gt; in the live log and the eligible alert in Phoenix. The file stays put.&lt;/li&gt;
&lt;li&gt;If a repeat test is quiet, check the cooldown and live logs before declaring the radio haunted. A simulated event does not create an actual quarantined-file record.&lt;/li&gt;
&lt;li&gt;After verifying the policy and move permissions, keep &lt;strong&gt;Enforce on&lt;/strong&gt; and switch &lt;strong&gt;Dry run off&lt;/strong&gt; for a disposable quarantine test. Create another new harmless file with a suspicious extension. Confirm the move in the log and refresh the quarantine list.&lt;/li&gt;
&lt;li&gt;Use &lt;strong&gt;Restore&lt;/strong&gt; only for the known-safe test item, then verify the result. Restoring an item does not permanently allow its extension; later events can trigger the policy again. Return to dry-run while tuning.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Quarantine is reactive. It cannot promise that a file was never accessed before being moved. The guard is useful; the guard is not a time machine with a pension plan.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Give it another phone, if you want&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The included WebSocket agent advertises &lt;code&gt;hive.alert&lt;/code&gt; for Phoenix alerts. Tripwire can also send to reachable Slack, Discord, Telegram, or email relays using that same role, so you can add notifications without replacing the watcher. The relay agents are &lt;code&gt;slack_relay&lt;/code&gt;, &lt;code&gt;discord_relay&lt;/code&gt;, &lt;code&gt;telegram_relay&lt;/code&gt;, and &lt;code&gt;email_send&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Discord, Telegram, and email support optional alert-payload encryption configured on each relay with its assigned keys. That is separate from Tripwire’s encrypted history and from HTTPS/WSS transport. You can keep this first swarm small and add the extra phone numbers once the basic route works.&lt;/p&gt;

&lt;p&gt;You have now installed the runtime, built a workspace, launched a swarm, connected the cockpit, and watched a filesystem event turn into an operator-visible result. Repeat that same short deployment habit for the next agent; change the job, keep the routine.&lt;/p&gt;

&lt;p&gt;As for &lt;code&gt;maintenance.php&lt;/code&gt;: reception would like to see some identification.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Victory Always. Night shift has the door.&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;🌐 &lt;strong&gt;Links &amp;amp; Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try MatrixSwarm: &lt;a href="https://matrixswarm.com" rel="noopener noreferrer"&gt;https://matrixswarm.com&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Join the Community / Discord: &lt;a href="https://discord.gg/2USbWVBVV" rel="noopener noreferrer"&gt;https://discord.gg/2USbWVBVV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Download Server: &lt;a href="https://github.com/matrixswarm/matrixswarm" rel="noopener noreferrer"&gt;https://github.com/matrixswarm/matrixswarm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug" rel="noopener noreferrer"&gt;https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug&lt;/a&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>devops</category>
      <category>linux</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Redis Watchdog: Fast Cache, Slow Excuses</title>
      <dc:creator>Daniel</dc:creator>
      <pubDate>Thu, 08 Oct 2026 05:17:43 +0000</pubDate>
      <link>https://dev.to/matrixswarm/redis-watchdog-fast-cache-slow-excuses-a66</link>
      <guid>https://dev.to/matrixswarm/redis-watchdog-fast-cache-slow-excuses-a66</guid>
      <description>&lt;p&gt;Redis is fast. The incident meeting explaining why nobody noticed it stopped is considerably less so.&lt;/p&gt;

&lt;p&gt;Lap one: “It’s probably the network.”&lt;/p&gt;

&lt;p&gt;Lap two: “Has anyone checked Redis?”&lt;/p&gt;

&lt;p&gt;Lap three: a senior engineer opens a terminal, performs one command, and ages six months.&lt;/p&gt;

&lt;p&gt;MatrixSwarm’s &lt;strong&gt;redis_watchdog&lt;/strong&gt; is the pit crew for this particular race. It watches the local service, checks for a TCP listener or configured socket path, can request restarts when the service is down, and radios the humans when something needs attention.&lt;/p&gt;

&lt;p&gt;The lap-time improvement comes from skipping the ceremonial guessing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Bolt on a small, useful agent&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;In Phoenix’s Swarm Workspace, add &lt;strong&gt;redis_watchdog&lt;/strong&gt; from the Agent Palette to the Linux/systemd host running Redis. Configure it for the actual installation:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Setting&lt;/th&gt;
&lt;th&gt;Example&lt;/th&gt;
&lt;th&gt;Pit-wall translation&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Service Name&lt;/td&gt;
&lt;td&gt;&lt;code&gt;redis-server&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Match your unit; some hosts use &lt;code&gt;redis&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Redis Port&lt;/td&gt;
&lt;td&gt;&lt;code&gt;6379&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The local TCP port you expect&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Socket Path&lt;/td&gt;
&lt;td&gt;&lt;code&gt;/var/run/redis/redis-server.sock&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Match the socket path if your deployment uses one&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Check Interval (sec)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;10&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;How often the crew checks in&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Restart Limit&lt;/td&gt;
&lt;td&gt;&lt;code&gt;3&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Consecutive failed restart-command threshold&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Alert To Role&lt;/td&gt;
&lt;td&gt;&lt;code&gt;hive.alert&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The service role for alarm delivery&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;These values are an example, not a universal tuning sheet. The implementation detects Redis through conventional binary locations, so confirm that your package/layout is recognized. Custom installations deserve an actual check, not an encouraging nod.&lt;/p&gt;

&lt;p&gt;The service account must be able to inspect systemd and local listeners. Restarting uses noninteractive sudo for the configured unit, so give it the narrow restart permission required for that job.&lt;/p&gt;

&lt;p&gt;Then configure the alarm relay you want. One watchdog plus one reachable relay inside your existing swarm is enough for a single human notification channel. No need to put a committee on the starting grid.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Read the flags correctly&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The Redis watchdog treats service state and accessibility signals separately:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Service down:&lt;/strong&gt; gather diagnostic context, send eligible failure alerts, optionally send a forensic report, and try the restart path.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Service active, with a detected TCP listener or existing configured socket path:&lt;/strong&gt; the basic checks are satisfied.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Service active, but neither accessibility signal is present:&lt;/strong&gt; raise an alert; this branch does not automatically restart Redis merely because those signals are missing.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Unlike the transition-based Apache, MySQL, and Nginx workers, Redis can enter its restart path again on subsequent polls while the service remains down. Consecutive failed restart commands count toward the configured limit; the disable guard then stops further restart attempts in that agent instance. A successful restart command resets that failure counter.&lt;/p&gt;

&lt;p&gt;The recovery notice follows the service returning from a down state. Treat it as a service-state report, not a signed affidavit that every application can use the cache. An active service can still produce the separate accessibility warning.&lt;/p&gt;

&lt;p&gt;Also, “socket exists” is a filesystem observation, and “port appears in the listener list” is a local networking observation. Neither is an authenticated Redis conversation. This worker does not use &lt;a href="https://redis.io/docs/latest/commands/ping/" rel="noopener noreferrer"&gt;Redis PING&lt;/a&gt; as its health decision. Add an appropriate protocol/application probe when you need that assurance.&lt;/p&gt;

&lt;p&gt;A parked race car still has wheels. We ask slightly more of it on race day.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Bring the black box, not just the siren&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Diagnostics are best effort: systemd status, Redis CLI information where available, and recent log context. Authenticated Redis, custom ports, or TLS may need additional diagnostic handling; the basic CLI diagnostic invocation does not automatically inherit every setting from your application client.&lt;/p&gt;

&lt;p&gt;A &lt;code&gt;report_to_role&lt;/code&gt; consumer can receive structured forensic data if you configure one. That is separate from the human alarm relay and optional for a simple notification setup.&lt;/p&gt;

&lt;p&gt;The &lt;strong&gt;Always alert on failure&lt;/strong&gt; setting affects failure chatter, but do not assume every alert branch is governed by one universal cooldown. In particular, the active-but-inaccessible warning can repeat on each poll. Choose the interval and channel with that behavior in mind. Your phone should report incidents, not audition for percussion.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Any watchdog can use any of the alarm radios&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;This is a swarm-wide pattern: &lt;strong&gt;Apache, MySQL, Nginx, and Redis watchdogs can all be combined with Slack, Discord, Telegram, or email alarm relays.&lt;/strong&gt; Mix the channels according to the humans you need to reach.&lt;/p&gt;

&lt;p&gt;Use &lt;code&gt;slack_relay&lt;/code&gt;, &lt;code&gt;discord_relay&lt;/code&gt;, &lt;code&gt;telegram_relay&lt;/code&gt;, or &lt;code&gt;email_send&lt;/code&gt;. The watchdog’s &lt;code&gt;alert_to_role&lt;/code&gt; normally targets &lt;code&gt;hive.alert&lt;/code&gt;, and those relays advertise &lt;code&gt;hive.alert@cmd_send_alert_msg&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Configure destination credentials, resolve required Registry and signing assignments, and make sure the service scope/routing connects the agents. Several matching reachable relays can receive the same alert. Your Redis alarm can reach Slack and Telegram and email; it does not need three different watchdog implementations and a branding exercise.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Encryption is an option on the relay, not a team superstition&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Discord, Telegram, and email support optional protected alert envelopes. On each relay that should use them, enable &lt;code&gt;encrypt_alerts&lt;/code&gt; and supply its assigned packet-signing/encryption keys. Configuring packet signing by itself does not turn on outgoing platform-message encryption.&lt;/p&gt;

&lt;p&gt;The switches are labeled &lt;strong&gt;Encrypt Discord alert message&lt;/strong&gt;, &lt;strong&gt;Encrypt Telegram alert message&lt;/strong&gt;, and &lt;strong&gt;Encrypt alert email subject and body&lt;/strong&gt;. With the proper keys, an authorized operator can use the matching relay’s &lt;strong&gt;Decrypt Message&lt;/strong&gt; panel in Phoenix to read the encoded alert.&lt;/p&gt;

&lt;p&gt;When secure wrapping fails, the relay does not quietly send that protected alert as plaintext. Email puts the alert’s subject and body inside the envelope; delivery metadata remains visible. Slack uses normal HTTPS delivery and has no equivalent MatrixSwarm alert-payload encryption switch in this implementation.&lt;/p&gt;

&lt;p&gt;Every copy has its own setting. Encrypting Discord while sending a readable duplicate elsewhere is exactly that: one protected copy, one readable copy. Painting a padlock on the team bus does not secure the luggage.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Take one practice lap&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;On a test host, verify normal service/listener detection, a controlled service-down event, restart permissions and behavior, the active-but-inaccessible case, recovery reporting, and delivery through each chosen relay. If encryption is enabled, test decryption too.&lt;/p&gt;

&lt;p&gt;And keep persistence, backups, and failover planning where they belong. A service restart does not restore lost data, elect a cluster leader, or retroactively make an untested recovery plan brilliant.&lt;/p&gt;

&lt;p&gt;Redis supplies the speed. The watchdog supplies the tap on the shoulder before the pit wall becomes a group therapy session.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Victory Always. Keep the cache quick and the excuses shorter.&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;🌐 &lt;strong&gt;Links &amp;amp; Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try MatrixSwarm: &lt;a href="https://matrixswarm.com" rel="noopener noreferrer"&gt;https://matrixswarm.com&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Join the Community / Discord: &lt;a href="https://discord.gg/2USbWVBVV" rel="noopener noreferrer"&gt;https://discord.gg/2USbWVBVV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Download Server: &lt;a href="https://github.com/matrixswarm/matrixswarm" rel="noopener noreferrer"&gt;https://github.com/matrixswarm/matrixswarm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug" rel="noopener noreferrer"&gt;https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug&lt;/a&gt;&lt;/p&gt;

</description>
      <category>devops</category>
      <category>database</category>
      <category>linux</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Nginx Watchdog: Tower, We’ve Lost the Reverse Proxy</title>
      <dc:creator>Daniel</dc:creator>
      <pubDate>Thu, 08 Oct 2026 05:16:25 +0000</pubDate>
      <link>https://dev.to/matrixswarm/nginx-watchdog-tower-weve-lost-the-reverse-proxy-1m6d</link>
      <guid>https://dev.to/matrixswarm/nginx-watchdog-tower-weve-lost-the-reverse-proxy-1m6d</guid>
      <description>&lt;p&gt;&lt;strong&gt;Tower:&lt;/strong&gt; Nginx, confirm you are receiving traffic.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Nginx:&lt;/strong&gt; …&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Tower:&lt;/strong&gt; Nginx, say again.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Application team:&lt;/strong&gt; We have seventeen dashboards.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Tower:&lt;/strong&gt; Splendid. Do any of them answer the radio?&lt;/p&gt;

&lt;p&gt;MatrixSwarm’s &lt;strong&gt;nginx_watchdog&lt;/strong&gt; gives your local Nginx service a small control tower: regular service/listener checks, alarms on health transitions, diagnostic context, and a restart path when the service becomes unhealthy.&lt;/p&gt;

&lt;p&gt;It is a useful job for a simple agent. Nobody needs a forty-slide observability strategy to notice the runway has gone missing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;File the flight plan&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;In Phoenix’s Swarm Workspace, add &lt;strong&gt;nginx_watchdog&lt;/strong&gt; from the Agent Palette to the Linux/systemd host running Nginx. Then configure the actual service and listening ports.&lt;/p&gt;

&lt;p&gt;For a machine that really listens locally on ports 80 and 443, an example config fragment is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"service_name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"nginx"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"ports"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;80&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;443&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"check_interval_sec"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"restart_limit"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"alert_cooldown"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;300&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"alert_to_role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"hive.alert"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Set &lt;code&gt;ports&lt;/code&gt; explicitly. The palette metadata inspected for this article contains &lt;code&gt;[85]&lt;/code&gt;; your own installation may use 80, 443, 8080, or something else. Defaults do not know where you built the airport.&lt;/p&gt;

&lt;p&gt;Every configured port must appear among local TCP listeners for the port check to pass. Do not demand 443 from a backend host where TLS terminates somewhere else.&lt;/p&gt;

&lt;p&gt;The agent also checks whether the systemd unit is &lt;strong&gt;enabled&lt;/strong&gt;. If it is disabled, this implementation waits rather than proceeding with its normal monitoring work. Confirm that this gate matches your deployment. “Installed,” “running,” and “enabled at boot” are three different passengers with suspiciously similar luggage.&lt;/p&gt;

&lt;p&gt;Automatic restarts require the agent service account’s narrowly scoped, noninteractive sudo permission for the configured unit. Configure that as part of deployment; a restart request cannot talk its way past a missing permission.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What the tower sees&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The healthy verdict combines active systemd state with the configured ports appearing in &lt;code&gt;ss -ltn&lt;/code&gt; output. It is local service monitoring, not an external request through your whole application stack.&lt;/p&gt;

&lt;p&gt;A running Nginx with a dead upstream may happily return 502s while passing this service/listener test. TLS problems, bad application responses, and remote network failures need additional checks. A green runway light does not prove there is a pilot in the plane.&lt;/p&gt;

&lt;p&gt;Nginx’s &lt;a href="https://nginx.org/en/docs/http/ngx_http_stub_status_module.html" rel="noopener noreferrer"&gt;stub_status module&lt;/a&gt; offers additional connection/request statistics if you choose to build more monitoring, but this watchdog does not poll it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Loss of signal&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The first health probe establishes a baseline and returns. After that, a healthy-to-unhealthy transition can collect service status and recent Nginx error-log lines, send an eligible alarm, optionally report structured evidence, and request a restart.&lt;/p&gt;

&lt;p&gt;After a successful restart command, the implementation waits briefly and checks the configured ports again, warning if they still are not listening. Later health transitions can produce recovery notifications.&lt;/p&gt;

&lt;p&gt;There is a restart failure counter and a disable guard. This is not three immediate retries on every bad poll: an unchanged down state does not repeatedly enter the transition-based recovery path. Starting the agent while Nginx is already down likewise records that first state instead of immediately running that failure path.&lt;/p&gt;

&lt;p&gt;Commission against a healthy service, then exercise a controlled failure on a test host. The best time to discover what a watchdog does is before everyone is typing “any update?” into the incident channel.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Choose your radio frequency&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The alarm destination is modular. &lt;strong&gt;Any Apache, MySQL, Nginx, or Redis watchdog can pair with any of the swarm-alarm relays: Slack, Discord, Telegram, or email.&lt;/strong&gt; The agent name does not reserve a particular channel.&lt;/p&gt;

&lt;p&gt;For a small deployment, use the watchdog plus one configured relay inside your existing swarm. For several destinations, add several reachable relays advertising the alarm role.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Channel&lt;/th&gt;
&lt;th&gt;Agent to add&lt;/th&gt;
&lt;th&gt;Payload-encryption option&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Slack&lt;/td&gt;
&lt;td&gt;&lt;code&gt;slack_relay&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;No equivalent MatrixSwarm toggle here&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Discord&lt;/td&gt;
&lt;td&gt;&lt;code&gt;discord_relay&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Optional&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Telegram&lt;/td&gt;
&lt;td&gt;&lt;code&gt;telegram_relay&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Optional&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Email&lt;/td&gt;
&lt;td&gt;&lt;code&gt;email_send&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Optional&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Keep the watchdog’s &lt;code&gt;alert_to_role&lt;/code&gt; set to &lt;code&gt;hive.alert&lt;/code&gt; unless you are deliberately changing the route. The relays advertise &lt;code&gt;hive.alert@cmd_send_alert_msg&lt;/code&gt;. Resolve their credentials and required Registry/signing assignments, and ensure service scope makes them discoverable from the watchdog.&lt;/p&gt;

&lt;p&gt;The watchdog can fan an alarm out to matching endpoints. A separate &lt;code&gt;report_to_role&lt;/code&gt; consumer is optional for structured forensic reports. It is not required just to tell someone the proxy has left the building.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Secure the radio traffic&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Discord, Telegram, and email each have an optional &lt;code&gt;encrypt_alerts&lt;/code&gt; switch. Enable it on the specific relay and supply its assigned packet-signing/encryption keys. Merely configuring packet signing does not enable outgoing alert-payload encryption.&lt;/p&gt;

&lt;p&gt;The Phoenix controls are &lt;strong&gt;Encrypt Discord alert message&lt;/strong&gt;, &lt;strong&gt;Encrypt Telegram alert message&lt;/strong&gt;, and &lt;strong&gt;Encrypt alert email subject and body&lt;/strong&gt;. Recipients with the proper keys can paste the protected envelope into the matching relay’s &lt;strong&gt;Decrypt Message&lt;/strong&gt; panel.&lt;/p&gt;

&lt;p&gt;If secure wrapping fails, those relays do not quietly fall back to plaintext. Email keeps the alert subject and body inside the protected envelope, while delivery metadata remains visible. Slack still uses HTTPS for normal delivery; it does not have this additional MatrixSwarm payload-encryption option.&lt;/p&gt;

&lt;p&gt;Protect each copy that needs protection. An encrypted Telegram alarm and a plain Slack duplicate are two different transmissions. Confidentiality does not spread through the swarm by enthusiasm.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Preflight before production&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Verify the service name, enabled state, real ports, and restart permission. Then rehearse the healthy baseline, controlled outage, recovery, every alarm destination, and decryption where enabled.&lt;/p&gt;

&lt;p&gt;Now the proxy has a tower that checks in, takes notes, and calls the right people. Your seventeen dashboards may continue looking expensive in peace.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Victory Always. Tower out.&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;🌐 &lt;strong&gt;Links &amp;amp; Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try MatrixSwarm: &lt;a href="https://matrixswarm.com" rel="noopener noreferrer"&gt;https://matrixswarm.com&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Join the Community / Discord: &lt;a href="https://discord.gg/2USbWVBVV" rel="noopener noreferrer"&gt;https://discord.gg/2USbWVBVV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Download Server: &lt;a href="https://github.com/matrixswarm/matrixswarm" rel="noopener noreferrer"&gt;https://github.com/matrixswarm/matrixswarm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug" rel="noopener noreferrer"&gt;https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug&lt;/a&gt;&lt;/p&gt;

</description>
      <category>devops</category>
      <category>webdev</category>
      <category>linux</category>
      <category>opensource</category>
    </item>
    <item>
      <title>MySQL Watchdog: The Database Kitchen Is Closed, and Nobody Told the Waiters</title>
      <dc:creator>Daniel</dc:creator>
      <pubDate>Thu, 08 Oct 2026 05:15:15 +0000</pubDate>
      <link>https://dev.to/matrixswarm/mysql-watchdog-the-database-kitchen-is-closed-and-nobody-told-the-waiters-43bg</link>
      <guid>https://dev.to/matrixswarm/mysql-watchdog-the-database-kitchen-is-closed-and-nobody-told-the-waiters-43bg</guid>
      <description>&lt;p&gt;The frontend is taking orders. The API is smiling professionally. Somewhere in the back, MySQL has locked the kitchen, turned off the lights, and left a note that says “connection refused.”&lt;/p&gt;

&lt;p&gt;The load balancer would like table seven to know their request is very important to us.&lt;/p&gt;

&lt;p&gt;MatrixSwarm’s &lt;strong&gt;mysql_watchdog&lt;/strong&gt; is the shift manager who actually checks the kitchen. It watches the local MySQL or MariaDB service, collects useful clues when health changes, can request a restart, and gets an alarm to the humans who can do something about it.&lt;/p&gt;

&lt;p&gt;No inspirational poster required. Just an agent with a job.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The two-item health menu&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The current watchdog calls the database healthy when the configured systemd service is active &lt;strong&gt;and&lt;/strong&gt; its configured TCP port appears among local listeners. The checks run on the Linux/systemd host where the database lives.&lt;/p&gt;

&lt;p&gt;This is deliberately a small menu: service state and a listening port. It does not log in, execute a query, measure replication lag, or establish that your schema migration was a sound life choice. You do not need database credentials for those basic process/listener checks.&lt;/p&gt;

&lt;p&gt;The configuration also exposes a socket path, but the current worker’s health decision uses the TCP listener check; the socket helper is not its fallback health path. A socket-only database needs a different or extended probe. Putting a socket path in a field does not make the worker use it. That would be configuration by wishful thinking, a surprisingly popular framework.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Open the kitchen in Phoenix&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Add &lt;strong&gt;mysql_watchdog&lt;/strong&gt; from the Agent Palette to your Swarm Workspace. Start with the facts on the database host:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Setting&lt;/th&gt;
&lt;th&gt;Example&lt;/th&gt;
&lt;th&gt;What to verify&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Service Name&lt;/td&gt;
&lt;td&gt;&lt;code&gt;mariadb&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Use the actual unit: it may instead be &lt;code&gt;mysql&lt;/code&gt; or &lt;code&gt;mysqld&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;MySQL Port&lt;/td&gt;
&lt;td&gt;&lt;code&gt;3306&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Match the local TCP listener&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Check Interval (sec)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;20&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Choose a sensible polling interval&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Alert Cooldown (sec)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;300&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Controls eligible repeated failure notifications&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Alert To Role&lt;/td&gt;
&lt;td&gt;&lt;code&gt;hive.alert&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Match the alarm relays’ advertised service&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;These are example settings, not a complete deployment file. Configure the service account’s access to service status and listener inspection. For automatic recovery, its noninteractive sudo permission must cover restart of the exact database unit.&lt;/p&gt;

&lt;p&gt;Give it a narrow permission for its job. Handing a dishwasher the keys to the entire shopping center is not operational elegance.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What happens when service goes sideways&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The first probe records a baseline and returns. Once a healthy baseline exists, a transition to unhealthy triggers diagnostic collection, eligible alert delivery, optional forensic reporting, and a service restart request.&lt;/p&gt;

&lt;p&gt;The agent can collect systemd status and recent lines from common MySQL/MariaDB error logs. This is useful evidence when “database unavailable” turns out to mean a full disk, a bad setting, or a permissions problem wearing a fake mustache.&lt;/p&gt;

&lt;p&gt;A successful restart command is not proof that queries work. Subsequent health probes determine when the service/listener combination is healthy again, and the recovery path can send a notice. If the state stays down, the transition-based worker does not launch a new restart on every poll.&lt;/p&gt;

&lt;p&gt;That also means &lt;strong&gt;starting this watchdog against an already-down database does not immediately exercise the failure-transition path&lt;/strong&gt;. Commission it with the database healthy, then test a controlled outage in a test environment. Keep separate SQL/application checks for the meal actually reaching the table. MySQL’s &lt;a href="https://dev.mysql.com/doc/refman/8.4/en/mysqladmin.html" rel="noopener noreferrer"&gt;mysqladmin documentation&lt;/a&gt; is a useful reference for additional database administration probes; those are not automatically performed by this watchdog.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The alarm menu is à la carte&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Here is the part that saves unnecessary architecture meetings: &lt;strong&gt;Apache, MySQL, Nginx, and Redis watchdogs can each work with any of the swarm-alarm relays—Slack, Discord, Telegram, or email.&lt;/strong&gt; There is no “MySQL only speaks email” rule hidden behind the refrigerator.&lt;/p&gt;

&lt;p&gt;Pick the relay agent for your destination:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;slack_relay&lt;/code&gt; for Slack.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;discord_relay&lt;/code&gt; for Discord.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;telegram_relay&lt;/code&gt; for Telegram.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;email_send&lt;/code&gt; for email alerts through your SMTP configuration.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The watchdog sends to the role in &lt;code&gt;alert_to_role&lt;/code&gt;, normally &lt;code&gt;hive.alert&lt;/code&gt;. Alarm relays advertise &lt;code&gt;hive.alert@cmd_send_alert_msg&lt;/code&gt;. Configure the selected relay’s credentials and required Registry/signing assignments, and check that service discovery scope and routing connect the agents.&lt;/p&gt;

&lt;p&gt;One reachable relay is enough for one destination. Several matching relays can receive the same alarm, so the operator on Telegram and the team in Slack can both get the news. A separate &lt;code&gt;report_to_role&lt;/code&gt; consumer is optional if you also want structured forensic data.&lt;/p&gt;

&lt;p&gt;The watchdog owns the health check. The relay owns getting the message to people. This division of labor is considerably healthier than making the database server learn everyone’s vacation schedule.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Put the sensitive order slip in an envelope&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Discord, Telegram, and email support &lt;strong&gt;optional alert-payload encryption&lt;/strong&gt;. Set &lt;code&gt;encrypt_alerts&lt;/code&gt; to true on each relay that should use it and provision its assigned packet-signing/encryption keys. Packet signing alone does not turn on encryption of the outgoing platform message.&lt;/p&gt;

&lt;p&gt;Phoenix labels the switches &lt;strong&gt;Encrypt Discord alert message&lt;/strong&gt;, &lt;strong&gt;Encrypt Telegram alert message&lt;/strong&gt;, and &lt;strong&gt;Encrypt alert email subject and body&lt;/strong&gt;. Authorized recipients can open the matching relay’s &lt;strong&gt;Decrypt Message&lt;/strong&gt; panel and decrypt the encoded envelope with the proper keys.&lt;/p&gt;

&lt;p&gt;If secure wrapping fails, the relay does not downgrade that alert to plaintext. Email’s protected content contains the alert subject and body; delivery metadata still exists. Slack has normal HTTPS delivery but no equivalent MatrixSwarm alert-payload encryption toggle in this implementation.&lt;/p&gt;

&lt;p&gt;And encryption is per destination. A locked envelope on Telegram does not protect the duplicate you sent openly to Slack. That is not encryption; that is a takeaway order with a second receipt taped to the window.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Run a dinner rehearsal&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Before relying on it, test a healthy baseline, a controlled failure, restart permissions, recovery detection, delivery to every chosen channel, and decryption where enabled. Do this on a test host, not by surprising the production dinner rush.&lt;/p&gt;

&lt;p&gt;The goal is simple: when the database kitchen closes, the people holding the orders should hear about it before the customers start reviewing the restaurant.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Victory Always. Yes, chef.&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;🌐 &lt;strong&gt;Links &amp;amp; Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Try MatrixSwarm: &lt;a href="https://matrixswarm.com" rel="noopener noreferrer"&gt;https://matrixswarm.com&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Join the Community / Discord: &lt;a href="https://discord.gg/2USbWVBVV" rel="noopener noreferrer"&gt;https://discord.gg/2USbWVBVV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Download Server: &lt;a href="https://github.com/matrixswarm/matrixswarm" rel="noopener noreferrer"&gt;https://github.com/matrixswarm/matrixswarm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug" rel="noopener noreferrer"&gt;https://www.youtube.com/channel/UCMjiY4_-W2KP5fHXO0eC2ug&lt;/a&gt;&lt;/p&gt;

</description>
      <category>devops</category>
      <category>database</category>
      <category>linux</category>
      <category>opensource</category>
    </item>
  </channel>
</rss>
