DEV Community

Cover image for Seeing Your Bot's Crash Logs Without SSHing Into the Server
John Wick
John Wick

Posted on

Seeing Your Bot's Crash Logs Without SSHing Into the Server

How StayPresent v1.6.0 captures bot stdout/stderr into a live log tail on the status page, so you can debug a crash without shell access.

Seeing Your Bot's Crash Logs Without SSHing Into the Server

When a bot crashes on a managed PaaS platform, your options for seeing why are usually limited to whatever log viewer the platform itself provides — which can be slow to load, hard to search, or scrolled past by the time you notice the crash happened. StayPresent v1.6.0 changes what's available directly from the status page itself, by capturing bot output into a queryable buffer rather than only streaming it to the console.

Table of Contents

  1. What Changed
  2. Why This Wasn't Possible Before
  3. How the Ring Buffer Works
  4. Where the Logs Show Up
  5. The isatty() Side Effect
  6. When This Matters
  7. Full Example
  8. Best Practices
  9. Common Mistakes
  10. FAQs
  11. Conclusion

What Changed

Previously, a bot's stdout/stderr was passed straight through to the parent process's own console — useful if you're watching the deploy logs live, much less useful after the fact, especially for a crash that already happened and scrolled past. As of v1.6.0, bot output is captured into a ring buffer in addition to being echoed live, and that buffer feeds directly into the status page's admin view, giving crash incidents real surrounding context instead of just a bare exit code.

Why This Wasn't Possible Before

A subprocess's stdout/stderr can only be captured by the parent if the parent explicitly sets up pipes for it at launch time — otherwise the subprocess just inherits the parent's own file descriptors and writes straight through, with nothing else able to intercept or store what was written. Making this available on the status page required restructuring how bot subprocesses are launched.

How the Ring Buffer Works

Output is captured into a fixed-size ring buffer per bot — recent lines are retained, older ones roll off as new output comes in. This keeps memory usage bounded regardless of how chatty a bot's logging is or how long it's been running, while still preserving enough recent context to actually be useful when diagnosing a crash.

Where the Logs Show Up

The captured output feeds the admin section of the status page (/status), gated behind the api_key login described in the status-page deep dive. When a bot crashes, its incident entry now includes a tail of recent stdout/stderr alongside the exit code — so you can often see exactly what your bot was doing right before it died, without needing shell access to the host at all:

staypresent.web.status(
    api_key=os.getenv("STATUS_ADMIN_KEY"),
)
Enter fullscreen mode Exit fullscreen mode

The isatty() Side Effect

One behavioral change worth knowing about explicitly: because bot subprocesses are now launched with piped output streams (required to capture them at all), a bot calling sys.stdout.isatty() now sees False, where it might previously have seen the parent terminal's actual value if you were running things locally in an interactive shell.

# inside bot.py
import sys
print(sys.stdout.isatty())  # now always False when run via staypresent.run()
Enter fullscreen mode Exit fullscreen mode

If your bot branches on this (for example, to decide whether to print colored output or a progress bar), this is worth checking after upgrading — logic gated on isatty() returning True will now consistently take the non-interactive path.

When This Matters

This capture is most valuable in exactly the scenario that's hardest to debug otherwise: a bot crashes intermittently, on a platform where you don't have convenient shell access, hours or days after the crash actually happened. Previously your options were limited to whatever log retention the platform itself provides. Now, StayPresent's own status page retains recent output tied directly to the specific incident, independent of the platform's own logging retention policy.

Full Example

import os
import staypresent

staypresent.web.status(
    title="Bot Fleet",
    api_key=os.getenv("STATUS_ADMIN_KEY"),
)

staypresent.run(
    "bot.py",
    port=int(os.getenv("PORT", 8080)),
    restart_on_crash=True,
    max_restarts=5,
)
Enter fullscreen mode Exit fullscreen mode

No additional configuration is needed to enable log capture itself — it's automatic as of v1.6.0. The only thing worth explicitly setting is api_key, so you (and only you) can actually view it.

Best Practices

  • Set api_key explicitly via an environment variable rather than relying on the auto-generated per-session key, specifically so you can always find it when you need to check a crash after the fact.
  • If your bot has color-coded or progress-bar output that depends on isatty(), verify it degrades gracefully now that it always sees False under StayPresent.
  • Keep your bot's own logging reasonably concise per line — the ring buffer retains a bounded number of recent lines, so extremely verbose per-iteration logging can push earlier, potentially more relevant context out faster than expected.

Common Mistakes

  • Assuming logs are retained forever. The ring buffer is bounded by design — for genuinely long-term log retention, pair this with your platform's own logging or an external log aggregator.
  • Not setting api_key, then losing track of the auto-generated one. Check your deploy logs for it immediately after your first v1.6.0 deploy, or set your own to avoid the issue entirely.
  • Bot code that behaves differently based on isatty() without accounting for it now always returning False under StayPresent's subprocess launching.

FAQs

Does this slow down my bot?
Capturing output into a bounded ring buffer is a lightweight operation; it shouldn't introduce meaningful overhead for typical bot logging volumes.

Can I disable log capture if I don't want it?
It's tied to how bot subprocesses are launched as of v1.6.0 and isn't independently toggleable — if you don't want the admin log tail visible at all, disabling the admin view entirely via api_key="" is the relevant control.

Does this replace proper external logging for a production deployment?
Not necessarily — it's most useful as an immediate, zero-setup way to see recent context around a specific crash; a dedicated logging/observability stack is still worth having for anything with serious production requirements.

Conclusion

Captured bot output turns StayPresent's status page from a bare uptime dashboard into something genuinely useful for debugging — recent stdout/stderr tied directly to the incident it happened around, visible without shell access to the host at all. The tradeoff is the isatty() behavior change, which is worth a quick check in any bot with output that branches on interactive-terminal detection.

pip install --upgrade "staypresent[prod]"
Enter fullscreen mode Exit fullscreen mode

Top comments (0)