DEV Community

Daniel Ioni
Daniel Ioni

Posted on

From a GitHub OAuth 404 to a Healthy Runtime: Rebuilding MyZubster's DevOps Foundation

#devops #github #mongodb #node

From a GitHub OAuth 404 to a Healthy Runtime: Rebuilding MyZubster's DevOps Foundation

What started as a GitHub OAuth 404 turned into a much deeper engineering session.

We ended up reviewing and improving almost every operational layer around MyZubster:

  • GitHub identity and repository ownership
  • OAuth configuration
  • VPS runtime management
  • systemd and PM2
  • MongoDB initialization
  • multiple Mongoose runtimes
  • Metaverse health checks
  • realtime authorization
  • Zorgax AI access and budgeting
  • Bitcoin payment verification
  • legacy Jest tests
  • deployment safety

And we are still working through the final part: aligning the older test suite with the architecture that now exists in production.

What is MyZubster?

MyZubster is an evolving open-source ecosystem that connects digital experiences, community participation, Marketplace/Seller workflows, AI assistance, experimental environmental work, a Metaverse interface, contribution systems, and other research-oriented components.

Some useful entry points:

The project is actively evolving, so an important part of the work is making sure the infrastructure, runtime behavior, tests, and public product all describe the same reality.

1. It started with a GitHub OAuth 404

The first visible problem was relatively simple:

GitHub authentication was not behaving reliably, and an OAuth flow could end in a 404.

But authentication problems were only part of the situation.

The previous GitHub environment had also accumulated operational problems around API access, OAuth applications, Actions, and repository workflows.

Instead of continuing to build on top of an unreliable development identity, we separated two concepts:

  • the historical/public MyZubster identity;
  • the operational identity used for development and repository administration.

A new GitHub account became the operational account for development.

The important point was that we did not rewrite the product identity or merge user identities inside MyZubster.

We changed the engineering control plane, not the history of the project.

2. Migrating the Git repository without destroying local state

The active project on the VPS lives in:


bash
/root/myzubster
The machine also contains worktrees, experimental copies, verifier directories, and other local state.
That immediately ruled out destructive shortcuts.
We deliberately avoided commands such as:
git reset --hard
and we also avoided:
git add .
because there are local untracked directories that are not supposed to become part of the repository.
Instead, the Git migration was done incrementally.
The new repository became the canonical origin, while older remotes were preserved as references.
Conceptually:
origin
  -> new operational repository

old-origin
  -> previous repository

old-ecosystem
  -> historical ecosystem remote
The current public development repository is:
github.com/danieldirimini-myzubster/myzubster
3. Updating the feature branch without rewriting history
The active development branch is:
feat/myz-188-university-developer-community-e2e
At one point the branch was many commits ahead of main, but also one commit behind it.
Rather than rebasing or rewriting published history, we merged the latest origin/main into the feature branch.
That was especially useful because main already contained an OAuth-related correction.
After the merge, the branch included both:
- the current feature work;
- the latest GitHub OAuth fix from main.
The OAuth tests remained green:
8 tests passed
4. Then we discovered two runtimes trying to own the same application
Once Git was stable, we turned to the VPS.
MyZubster had pieces managed by both:
- PM2
- systemd
That is not automatically wrong.
The problem was that there was also a PM2 process named myzubster, while the actual gateway was being served through systemd.
Even more importantly, the two systems were not using exactly the same startup path.
This is the kind of situation that creates confusing bugs:
developer edits repository A

systemd runs repository B

PM2 launches server.js

server.js exports Express app

another bootstrap performs app.listen()
Everything looks almost correct.
Almost.
We removed the duplicate PM2 ownership of the gateway and made systemd the canonical supervisor for the main MyZubster gateway.
PM2 remains in use for separate services where appropriate.
5. The production service was running from the wrong project copy
There was another important mismatch.
The systemd service was still configured around an older project directory:
/root/myzubster-ahp
while the active repository was:
/root/myzubster
That is a dangerous configuration because you can successfully:
git pull
git commit
git push
and still have production execute different code.
We moved the systemd working directory to the active repository.
The rule is simple:
Always verify the actual WorkingDirectory and ExecStart of the process serving production.

Do not assume the repository you are editing is the repository the process is running.
6. The homepage was healthy, but the Metaverse was not
After aligning the runtime, the homepage responded correctly:
HTTP 200
But the Metaverse health endpoint reported something different:
{
  "success": false,
  "status": "degraded",
  "transport": "unavailable",
  "mongodb": "disconnected"
}
That looked like a MongoDB problem.
It was not.
The MongoDB credentials were valid and the main application could connect.
The real issue was more subtle.
7. MyZubster intentionally had two Mongoose installations
The project contains two Node dependency contexts:
/root/myzubster/node_modules/mongoose
/root/myzubster/backend/node_modules/mongoose
They were also using different major versions.
Conceptually:
root application
  -> Mongoose 7

backend
  -> Mongoose 8
These are not the same JavaScript object.
So this:
await rootMongoose.connect(...)
does not make this true:
backendMongoose.connection.readyState === 1
That was the key.
The root application was connected to MongoDB.
The backend Metaverse code was checking its own Mongoose connection, which had never been initialized.
8. MongoDB was fine. The bootstrap sequence was wrong.
The backend already had the correct database startup logic.
It exposed functionality equivalent to:
async function connectDatabase() {
  await mongoose.connect(MONGODB_URI);
}
and its normal server bootstrap connected MongoDB before listening.
But the systemd gateway startup path bypassed that backend bootstrap.
The runtime effectively looked like this:
systemd
  |
  v
root server
  |
  v
app.listen()

backend.connectDatabase()
  -> never called
So we created a dedicated systemd entrypoint.
Simplified:
'use strict';

const app = require('../server');
const backend = require('../backend/src');

async function main() {
  await backend.connectDatabase();

  const port = Number(process.env.PORT || 5003);
  const host = '127.0.0.1';

  const server = app.listen(port, host, () => {
    console.log(`MyZubster Gateway listening on ${host}:${port}`);
    console.log('Backend MongoDB initialized');
  });

  let shuttingDown = false;

  async function shutdown(signal) {
    if (shuttingDown) return;
    shuttingDown = true;

    console.log(`${signal}: shutting down MyZubster Gateway`);

    server.close(async () => {
      try {
        await backend.disconnectDatabase();
      } finally {
        process.exit(0);
      }
    });

    setTimeout(() => process.exit(1), 10000).unref();
  }

  process.on('SIGTERM', () => shutdown('SIGTERM'));
  process.on('SIGINT', () => shutdown('SIGINT'));
}

main().catch((error) => {
  console.error('Failed to start MyZubster Gateway:', error);
  process.exit(1);
});
systemd now launches that explicit bootstrap.
The important sequence is:
load application
      |
      v
connect backend MongoDB
      |
      v
start HTTP listener
After the change, the Metaverse health endpoint became healthy:
{
  "success": true,
  "status": "healthy",
  "transport": "shared-polling",
  "mongodb": "connected"
}
That was one of the most important fixes in this migration.
9. We deliberately did not "fix" Mongoose by deleting dependencies
A tempting response would have been:
rm -rf backend/node_modules
or forcing the entire repository onto one Mongoose version immediately.
We did not do that.
The two dependency trees currently represent separate application contexts.
The immediate bug was not:
there are two Mongoose installations

The bug was:
only one of them was initialized during the production startup lifecycle

Those are very different problems.
Fix lifecycle first.
Refactor dependencies later, if there is a good reason.
10. We started separating regressions from existing technical debt
Once production was healthy, we went back to Jest.
The tests specifically associated with the MYZ-188 feature were green:
7 suites passed
37 tests passed
But the complete repository suite still contained failures.
Instead of assuming our branch introduced all of them, we tested selected failures against the main baseline.
Several existed there too.
This distinction matters:
feature regression
is not the same thing as:
pre-existing failing test
Without that comparison, it is easy to spend hours "fixing" code that your branch never broke.
11. Updating Zorgax access without duplicating production logic
One failure involved the Zorgax access layer.
The newer access implementation was centered around:
- entitlement access;
- sponsored access.
Some older tests still expected a subscription access adapter.
We added an optional compatibility source rather than restoring the old subscription system as a first-class production dependency.
Conceptually:
subscriptionAccessFn = null
The important part is that subscription compatibility can still be injected when needed, while the entitlement service remains the canonical production authority.
After the change:
Zorgax access tests: 6/6 passed
Assistant access contract: 13/13 passed
12. A failing test does not automatically mean the production code is wrong
We then found an AI budget test expecting a key like:
openai:gpt-5.6-sol:2026-09
while production generated:
openai:all:2026-09
It would have been easy to modify production until the test passed.
Instead, we checked the actual architecture.
The monthly spend calculation aggregates all OpenAI usage.
The budget is a single monthly Astra/OpenAI cap.
And the provider can fall back from one OpenAI model to another.
That means a model-specific budget key would actually be less correct.
The test was outdated.
We updated the expectation to match the intended global monthly budget:
openai:all:YYYY-MM
Result:
6/6 passed
The lesson:
Tests are executable documentation, but documentation can become stale too.

13. Realtime authorization now fails closed
Another test took more than five seconds:
community channels fail closed without membership authority
The implementation performed a real membership query:
CommunityMembership.findOne(...).lean()
When the backend database connection was unavailable, Mongoose buffered the query.
Jest eventually timed out.
We changed authorization to explicitly check database availability first:
if (mongoose.connection.readyState !== 1) {
  return {
    allowed: false,
    reason: 'community_membership_authority_unavailable'
  };
}
and wrapped the membership lookup so database failures also return a denied result.
This changed the behavior from:
database unavailable
  -> wait
  -> timeout
to:
database unavailable
  -> deny immediately
That is also the safer authorization model.
Fail closed.
The realtime tests became:
6/6 passed
14. Then we found a real Bitcoin production bug
The full suite exposed this:
TypeError:
Cannot read properties of undefined (reading 'includes')
The failing expression was effectively:
SUPPORTED_ASSETS.includes(asset)
The strange part was that SUPPORTED_ASSETS was obviously declared as:
['BTC']
So how could it be undefined?
The answer was a CommonJS circular dependency.
The dependency graph looked roughly like this:
zorgaxLegacyMonetizationService
            |
            v
zorgaxUnifiedCheckoutService
            |
            +--------------------+
            |                    |
            v                    v
zorgaxQuoteService      zorgaxChainVerifierService
            |                    |
            +----------+---------+
                       |
                       v
         zorgaxLegacyMonetizationService
During module initialization, one side of the cycle was reading exports before the legacy module had finished constructing them.
That produced:
SUPPORTED_ASSETS === undefined
15. Breaking the dependency cycle properly
We did not hide the bug with:
SUPPORTED_ASSETS || []
Instead, we extracted payment constants into a dependency-free module:
'use strict';

const SUPPORTED_ASSETS = Object.freeze(['BTC']);

module.exports = {
  SUPPORTED_ASSETS
};
Now both services import from:
zorgaxPaymentConstants
instead of importing back through the legacy monetization service.
The dependency graph is now acyclic.
The BTC production tests went from:
3 failed
2 passed
to:
5 passed
This included:
- BTC operational configuration;
- BTC/EUR quoting;
- satoshi conversion;
- Esplora verification;
- fail-closed behavior for unconfirmed transactions.
16. The next problem was not production code — it was legacy mocks
After the payment architecture had evolved, some tests were still mocking old classes such as:
ZorgaxPaymentIntent
ZorgaxSubscription
But the active checkout architecture now uses:
PaymentIntent
ZorgaxPurchase
Entitlement
This creates a very confusing Jest failure.
A test appears fully mocked.
But it is mocking an object that production no longer touches.
The real model executes instead.
Mongoose tries to reach the database.
Five seconds later:
Exceeded timeout of 5000 ms
17. Checkout renewal test: 10 seconds to 1 second
The old renewal test mocked:
ZorgaxPaymentIntent
ZorgaxSubscription
The unified checkout now creates:
PaymentIntent
ZorgaxPurchase
After updating the test to mock the actual architecture:
2/2 passed
and the runtime dropped from around ten seconds to roughly one second.
That was strong evidence that the timeout was not a slow implementation.
It was an accidental database call caused by a stale mock.
18. Payment replay protection moved from subscriptions to purchases
The same thing happened with replay protection.
The old test expected replay detection through ZorgaxSubscription.
The current implementation uses a purchase record keyed through an external payment intent identifier.
The current behavior is closer to:
payment reference
      |
      v
existing purchase?
   /       \
 yes       no
  |         |
  |      create purchase
  |
same owner?
 /       \
yes       no
 |         |
reuse     reject
          "Payment already used"
Then, and only then, the entitlement is granted.
We updated the test around ZorgaxPurchase and the entitlement boundary.
The result changed from:
~11 seconds
2 timeouts
to:
2/2 passed
~0.7 seconds
This is exactly the kind of test modernization we are doing now.
19. What we are working on right now
At the time of writing, the production/runtime foundation is much healthier.
We have already completed:
- GitHub operational account migration;
- new canonical repository remote;
- branch synchronization with main;
- OAuth regression verification;
- systemd working-directory cleanup;
- removal of the duplicate PM2 gateway process;
- explicit backend MongoDB startup;
- graceful shutdown handling;
- restored Metaverse health;
- Zorgax access compatibility cleanup;
- global AI budget test alignment;
- realtime authorization fail-closed behavior;
- Bitcoin payment circular-dependency fix;
- migration of several outdated payment tests to the unified checkout architecture.
The current focus is the remaining legacy Jest suite.
A number of older tests still encode assumptions from previous MyZubster architectures.
Some inspect old source files directly.
Some mock models that are no longer used.
Some expect functionality to live in compatibility wrapper modules even though it has moved into unified services.
So the work now is not:
make every red test green at any cost.

It is:
determine which behavior represents the current contract, then either fix the implementation or modernize the test.

That distinction is critical.
20. We are also improving architectural boundaries
One recurring theme in this work has been the removal of implicit behavior.
We are moving toward clearer boundaries.
For example:
systemd
  -> explicit gateway bootstrap

gateway bootstrap
  -> explicit backend Mongo lifecycle

authorization
  -> explicit database availability check

payments
  -> shared payment constants

checkout
  -> unified PaymentIntent + ZorgaxPurchase model

access
  -> canonical entitlement service

tests
  -> mock the actual current boundary
The goal is to make the system easier to understand when something fails.
A good architecture should not require knowing a hidden historical path through five compatibility layers.
21. Small commits made this debugging possible
We intentionally kept fixes isolated.
Examples from this work include commits conceptually like:
fix(runtime): initialize backend Mongo before gateway listen

fix(zorgax): include optional legacy subscription access source

test(zorgax): align AI budget expectation with global monthly cap

fix(realtime): fail closed when community membership authority is unavailable

fix(zorgax): break payment asset circular dependency
Small commits are not just a Git preference.
They are an operational debugging tool.
When a behavior changes, we can identify exactly which decision introduced it.
22. Things we deliberately avoided
A large part of safe production debugging is knowing what not to do.
We avoided destructive Git operations.
We did not delete worktrees blindly.
We did not force two Mongoose installations into one just because they looked redundant.
We did not increase Jest timeouts to hide real database calls.
We did not add fallback values to hide a CommonJS circular dependency.
We did not commit secrets.
We did not dump environment files into logs or public documentation.
When credentials needed rotation, they were rotated rather than treated as harmless development data.
23. Current MyZubster architecture, simplified
The operational picture is becoming much clearer:
GitHub
  |
  v
danieldirimini-myzubster/myzubster
  |
  v
VPS: /root/myzubster
  |
  +--> systemd
  |      |
  |      v
  |   MyZubster Gateway
  |      |
  |      +--> Root Mongo context
  |      |
  |      +--> Backend Mongo context
  |      |
  |      +--> Metaverse
  |      |
  |      +--> Realtime
  |      |
  |      +--> Zorgax
  |
  +--> PM2
         |
         +--> independent supporting services
The public application remains available at:
https://myzubster.com
and development continues openly at:
https://github.com/danieldirimini-myzubster/myzubster
24. What comes next
The immediate engineering roadmap is:
1. finish migrating the remaining legacy payment tests;
2. review the source-inspection tests that still point at compatibility wrappers;
3. run the complete Jest suite again;
4. separate real regressions from intentionally changed contracts;
5. verify production health after the final changes;
6. prepare the feature branch for review and merge into main.
After that, the same cleanup approach can be applied to other areas of MyZubster.
The goal is not only a green test suite.
The goal is to make every green test correspond to something we actually believe about the current system.
25. The biggest lessons from this debugging session
A few engineering lessons became especially clear.
A GitHub problem can expose a runtime problem
The original symptom was OAuth.
The deeper issues involved process management, database lifecycle, stale test architecture, and module dependencies.
The repository you edit may not be the repository production executes
Always inspect:
systemctl show ...
and confirm the actual working directory and executable.
Two Mongoose packages mean two connection states
Never assume a database connection is shared across different installed copies of a library.
A Jest timeout often means an unexpected external dependency
Before increasing the timeout, ask:
Did this test accidentally reach MongoDB, Redis, HTTP, or another real service?

Circular dependencies are architectural warnings
An undefined CommonJS export is often not a random Node.js bug.
Draw the dependency graph.
Authorization should fail closed
If the authority required to make an access decision is unavailable, denial is safer than guessing.
Tests can become legacy too
Production code evolves.
Tests must evolve with it.
A stale test can be just as misleading as stale documentation.
Conclusion
This work began with a simple question:
Why is GitHub OAuth returning a 404?

It turned into:
GitHub migration
        |
        v
repository stabilization
        |
        v
runtime ownership cleanup
        |
        v
systemd bootstrap
        |
        v
dual-Mongoose lifecycle fix
        |
        v
Metaverse recovery
        |
        v
Zorgax access hardening
        |
        v
realtime fail-closed authorization
        |
        v
Bitcoin circular-dependency fix
        |
        v
legacy test modernization
And that is probably the most useful part of the story.
We did not rebuild the entire platform from scratch.
We followed the real system, one boundary at a time.
Observe.
Reproduce.
Understand.
Change the smallest correct thing.
Test it.
Commit it.
Then move to the next boundary.
That is what we are doing with MyZubster now.
You can follow the project here:
MyZubster:
https://myzubster.com
GitHub:
https://github.com/danieldirimini-myzubster/myzubster
Marketplace:
https://myzubster.com/marketplace
Metaverse:
https://myzubster.com/metaverse
LIFE Pilot:
https://myzubster.com/life-pilot
The next milestone is simple to describe, even if the work behind it is not:
finish aligning the legacy test suite with the current architecture, validate the complete branch, and prepare it for merge into main.
Build carefully. Verify everything. Keep moving.
Enter fullscreen mode Exit fullscreen mode

Top comments (0)