DEV Community: italojs

Meteor 3.5 is out!

italojs — Mon, 06 Jul 2026 19:42:52 +0000

Meteor 3.5 is a broad release, and it earns the word. Its center of gravity is the reactive data layer (the part of your app that runs in production around the clock), but the work reaches from the database driver out to your auth middleware and the Node runtime underneath. Here is the map, and everything below is covered in detail further down. For the complete, line-by-line list of what shipped, see the full changelog.

The headline is change streams by default: reactivity now works on managed and serverless MongoDB (Atlas Shared, Atlas Serverless) where the oplog was never available, so each instance does less work per subscription, your fleet gets smaller, and so does the bill. Around that sits the rest of the real-time modernization, with DDP session resumption that survives a dropped connection, a pluggable DDP transport with a uws option, a fully functional DISABLE_SOCKJS, and EJSON optimizations that trim allocations on the hot path. If you build APIs and auth, there is accounts-express for first-class authenticated routes, async DDPRateLimiter matchers, and client logins that are fully promise-based end to end. Underneath it all, the platform moves to Node.js 24, with a handful of dependencies (MongoDB collation, OTPAuth for 2FA, http-proxy-3) quietly modernized so you don't have to think about them.

None of these are vendor benchmarks; the numbers in this post come from developers testing their own apps in public. And 3.5 is the release where the performance thread that ran through the 3.x line finally reaches runtime.

Change streams are where it starts.

The problem we kept running into

For years, scaling real-time in Meteor came with a quiet catch, one a lot of teams hit without talking about it much.

Meteor's real-time updates rode on Mongo's oplog: to know when data changed, every app server tailed the database's replication log (a running record of every write to every collection) and filtered it, in your Node process, down to just the queries it cared about. It worked, but it carried two costs.

The first is that the oplog is the whole database's write log, not just yours. Meteor tails all of it and throws away what it doesn't need. The moment something outside Meteor wrote to the same database (a nightly ETL job, a data migration, a third-party service injecting records), every app server still had to read and discard all of that traffic. Your Meteor CPU bill went up because of writes your app never even made.

The second is that it tied you to self-managed Mongo. On managed and serverless tiers like Atlas Shared, the oplog is off-limits. So the moment you moved to the hosting most teams actually want, real-time broke, and your only fallback was polling: asking the database the same questions over and over, burning CPU and money to fake what should be instant.

Meteor 3.5 changes that. Change streams are now the default, real-time works on managed and serverless Mongo, and there's no config to write.

And real apps are already reporting the difference, in public, with numbers.

The headline: MongoDB Change Streams

Do more with fewer servers. In Meteor 3.5, reactivity is powered by MongoDB change streams by default, and the practical result is simple: each app instance does far less work per subscription because change detection now lives on the database server instead of your Node process. That translates into more concurrent users, subscriptions, and methods per instance, smaller fleets, and a smaller bill.

There is a second payoff that matters just as much. Real-time now works on managed and serverless MongoDB tiers, like Atlas Shared and Atlas Serverless, where oplog access simply is not available. Until now, users on those tiers were pushed into polling, the slowest and most expensive fallback. With change streams, they get efficient real-time updates without paying for a dedicated cluster just to tail the oplog.

There is one requirement, and Meteor handles it for you: change streams need MongoDB 6+ running as a replica set or a sharded cluster. On older or standalone MongoDB, Meteor detects that it can't use them and automatically falls back (to oplog, and then to polling). The same fallback covers cursors that use skip or limit and selectors change streams cannot serve. So the default is safe: you get the fast path where it works and a working path everywhere else, with no code changes and nothing to configure.

If you ever need the previous behavior, you can pin the pre-3.5 path for the whole app by setting packages.mongo.reactivity to ["oplog", "polling"] in settings.json (more on that, and the rest of the driver's tuning, below).

For the full picture, including how the observer driver selects and switches strategies, see the change streams observer driver docs. The feature landed in PR#13787.

The proof: real-world numbers

We could tell you change streams are faster. Instead, here is what the community measured. These are not vendor benchmarks run on hand-picked hardware to make a launch look good. They come from Meteor developers testing their own apps and load harnesses, and posting the results in public. Read them that way.

More connection capacity under load. In the official Meteor 3.5 announcement, forum member @italojs shared a load test comparing the two reactivity backends on the same app. Change streams sustained roughly 40% higher connection capacity than oplog. Under the same ramp, oplog hit out-of-memory crashes around 1,200 virtual users, while change streams stayed stable up to about 1,680 virtual users, degrading only to timeouts rather than crashing. The post includes a montiAPM chart of both runs; see the charts in the thread for the full picture.

A real app, not a benchmark. One of the most striking reports came from a production user. In the same thread, @mvogt22 (Michael) reported that after switching his app to change streams, average Event Loop Delay dropped from 97ms on v3.4 to 18ms on v3.5-beta.4, roughly a 5x (about 81%) reduction (post 32, post 37). Event Loop Delay is a direct measure of how starved your Node process is; cutting it that far means far more headroom before an instance falls over. He also posted monitoring screenshots marking the v3.5-beta.4 deploy, with the improvement visible right at the deploy line.

Directional, but pointing the right way. @dupontbertrand built a community benchmark harness (a Compare Runs dashboard that pits release-3.4.1 against release-3.5 across several scenarios), documented in the Meteor performance: a to-do list thread. The resource picture is striking. Across the ddp-reactive-light, fanout-light, and reactive-crud scenarios, 3.5 cut average app-server CPU by roughly 32–67%, average RAM by up to about 73%, and total garbage-collector pause time by 81–94%, with GC event counts down a comparable amount. Treat these as directional rather than final (the author noted that server configuration changed between runs, and wall-clock time was mixed across scenarios), but the direction is unmistakable: with change streams, the app server simply does far less work.

Numbers like these depend on running change streams well, so it is worth understanding the driver you are now running by default.

Change streams under the hood: tuning and trade-offs

Change streams are the default, but the whole driver pipeline is configurable. Meteor picks a reactivity driver in order, and you control that order under packages.mongo.reactivity in settings.json. The default is effectively ["changeStreams", "oplog", "polling"]: try change streams first, fall back to oplog, then polling. To pin the pre-3.5 behavior, set it to ["oplog", "polling"].

A few optional knobs live under packages.mongo.changeStream. delay.error (default 100 ms) is how long the driver waits before restarting a stream after an error; delay.close (default 100 ms) is the equivalent delay after a clean close. waitUntilCaughtUpTimeoutMs (default 1000 ms) is the upper bound Meteor waits for the change stream to catch up when coordinating with DDP write fences. That timeout is a genuine trade-off: if it elapses, the write fence proceeds anyway, so a client can briefly miss its own write (read-your-writes). Raise it for stricter consistency, lower it to keep methods snappy under load.

{
  "packages": {
    "mongo": {
      "reactivity": ["changeStreams", "oplog", "polling"],
      "changeStream": {
        "delay": { "error": 100, "close": 100 },
        "waitUntilCaughtUpTimeoutMs": 1000
      }
    }
  }
}

Now the trade-off worth knowing: change streams are not free everywhere. They are extremely efficient for targeted queries, but a very broad selector or a highly-mutated collection pushes matching work onto the Mongo cluster, which can become the bottleneck. Oplog shifts that load back to the app server, better when Mongo is your constraint, but it scales poorly under very high global write volume and needs oplog permissions. Polling is the expensive last resort.

The guidance is boring and effective: narrow your selectors, add the matching indexes, and force oplog for specific broad-filter collections by reordering their drivers. Full details are in the change streams observer driver docs.

How a feature that was ready took four and a half months to ship

Change streams were essentially done before the first beta. When 3.5-beta.0 went out on 2026-02-11, the feature worked. The four and a half months that followed were not spent finishing it. They were spent earning the right to make it the default.

None of this was for lack of wanting it. Change streams had come up in the community for years as the obvious answer to the oplog's limits. Issue #11578, "Slow oplog tailing on ATLAS" is one of many threads where the pain of tailing the oplog on managed Mongo ran straight into "so why not change streams?". Wanting a feature and shipping it as a safe default, though, are very different problems.

Three forces stretched the calendar.

The first is the nature of the change itself. Change streams rewire the observe driver, which is the reactive core of Meteor. Every subscription, every live query, every observeChanges call ultimately flows through it. You do not flip the heart of a framework to a new default on a hunch. That confidence has to be manufactured in production-like conditions, over and over, which is why 3.5 ran through twelve betas and two release candidates instead of one or two.

The second is timing. A large backlog of performance work landed in the same window: the EJSON serialization improvements, the DDP transport rewiring, the session-resumption machinery. Each of those PRs needed careful review, and reviewing them alongside a driver rewrite meant the same small group of maintainers was under load on multiple fronts at once.

The third is context, not a complaint: this cycle also saw a surge in incoming PRs, driven in large part by the community's growing use of LLM-assisted contributions. That is a healthy signal, and it broadened the review surface the maintainers were carrying while everything above was in flight. (What that shift means for Meteor is worth its own note, at the end.)

Then there was the long tail of correctness, which is where the calendar really went. Making change streams the default meant grinding through every place the Meteor data model meets MongoDB's native types. PR#14238 fixed ObjectID fields being delivered to the client as binary when a query used a projection. PR#14389 closed a set of race conditions in ChangeStreamObserveDriver where events could be dropped during snapshot, restart, or watch setup, added a polling fallback for skip/limit cursors, and preserved uws listeners across restarts.

Beyond those two, the work was quieter and harder to headline. Meteor-to-native BSON and ObjectID translation had to be handled correctly at every selector and matcher boundary, not just once at the edge. And a standalone-Mongo mis-detection could drop the driver into a change-stream restart loop, which had to be tracked down and fixed.

The pattern is the theme. The code was done early. Getting it right, in public, at production scale, is what took the calendar.

The rest of the real-time modernization

Change streams get the headline, but they are not the only reason 3.5 is faster. The whole real-time layer was reworked this release (how sessions survive a dropped connection, how DDP moves bytes over the wire, and how EJSON serializes them), and each piece contributes its own share of the speedup.

Sessions survive the network, not just the request. With DDP Session Resumption (PR#14051), a client that drops and reconnects within a grace period (default 15s) picks up its existing session instead of tearing everything down. Pending method calls are not lost, and active subscriptions resume where they left off. The payoff is felt on mobile handoffs, sleeping tabs, and flaky Wi-Fi: smoother UX for users, and far less server CPU during reconnect storms because you are not re-running every subscription from scratch. The grace period and message-queue limit are tunable per app.

Pick your transport. DDP is now pluggable (PR#14231). Keep sockjs, the default, for maximum compatibility behind strict proxies, or switch to uws (uWebSockets) for lower latency and higher throughput on internal or controlled deployments. You flip it via an env var or a settings flag, with no app code changes. See the DDP transport docs.

Drop SockJS entirely when you don't need it. DISABLE_SOCKJS is now fully functional (PR#14206). Where you don't rely on the polling fallback, you get a smaller client bundle and fewer handshakes.

Fewer allocations on every message. A batch of EJSON optimizations trims the cost of serialization on the hot path: zero-clone stringifyDDP (PR#14213), copy-on-write toJSONValue/fromJSONValue (PR#14209), fast-path primitive comparisons and early bail-out in EJSON.equals (PR#14208, PR#14205), and no array allocation in lengthOf (PR#14204). The net effect on a busy server: fewer allocations in DDP serialization, and so less GC pressure.

For the API and auth builders

If you have ever bolted a REST endpoint onto a Meteor app, you know the drill: pull the token off the Authorization header yourself, hash it, look up the user in services.resume.loginTokens, check expiry, and hope you got it right. Every protected route reinvented that plumbing, and none of it talked to the rest of Meteor's account system.

The new accounts-express package (PR#14091), shipped out of the box, makes authenticated routes first-class. createAuthMiddleware resolves the login token from an Authorization: Bearer header or the meteor_login_token cookie, validates it, and sets up the invocation context so that inside your handler Meteor.userId() and Meteor.userAsync() work exactly like they do in a method.

import express from "express";
import { WebApp } from "meteor/webapp";
import { createAuthMiddleware } from "meteor/accounts-express";

const app = express();

// required: true -> 401 on missing/invalid/expired token
app.use("/api", createAuthMiddleware({ required: true }));

app.get("/api/me", async (req, res) => {
  const user = await Meteor.userAsync();
  res.json({ userId: Meteor.userId(), email: user?.emails?.[0]?.address });
});

WebApp.handlers.use(app);

A wrapped fetch attaches the token automatically, and TypeScript types landed in PR#14433.

Two more wins for this crowd. DDPRateLimiter rule matchers can now be async (PR#14182), so you can gate methods and subscriptions on data-driven conditions, role, billing tier, or a feature flag pulled straight from the database. And client logins are fully promise-based: Meteor.loginWithPasswordAsync and Meteor.loginWithTokenAsync (PR#14070) round out the asyncified client accounts (PR#14069), so await works end to end.

Platform: Node 24 and the invisible work

The payoff is a single line in your terminal: run meteor update --release 3.5 and your app is on Node.js 24.15.0 and NPM 11.12.1 (PR#14176, PR#14399). No drama on your end. That was the goal, and it took more than a version number to hit it.

This was not a trivial bump. Node 24 surfaced OS-level compatibility issues with parts of Meteor's own build and release infrastructure, forcing the team to update it before the new runtime could ship cleanly. Most of that happened below the waterline, and it is deliberately not detailed here. The point is that the friction landed on the maintainers, not on you, so that meteor update just works on Node 24.

While the platform moved forward, a few dependencies got quietly modernized too:

Two-factor auth now runs on OTPAuth instead of the unmaintained node-2fa in accounts-2fa (PR#14321), a healthier foundation with no change to your code.
The dev proxy moved from http-proxy to the maintained http-proxy-3 (PR#13916).
MongoDB Collation support (PR#14188) brings reliable case-insensitive search and locale-aware sorting, consistent across client and server, without silently falling back to polling.

None of these will change how you write an app. That is exactly the idea: the platform stays current so you do not have to think about it.

Where 3.5 fits in the Meteor 3 story

Meteor 3.0 (July 2024) was about survival: tearing out Fibers and moving the entire framework to async/await, the deepest migration in Meteor history. 3.1 (November 2024) stabilized that new foundation: Node 22, the v6 MongoDB driver, Express 5, roles folded into core.

What came next is easy to miss as a pattern, but it is the whole point. From 3.2 on, every release chipped away at performance. 3.2 (March 2025) shipped the meteor profile command and fixed a long-standing oplog data-loss bug. 3.3 (June 2025) went after build and dev speed: the SWC transpiler and minifier, parcel/watcher, a modern-by-default architecture. 3.4 (January 2026) modernized the build stack itself with the Rspack integration and tools-core, cutting both bundle sizes and build times.

Notice where all of that work lived: in the build and the tooling. Faster rebuilds, smaller bundles, better profiling. The one place it had not reached was the part of Meteor that runs in production around the clock, the reactive data layer, and specifically how Meteor watches MongoDB for changes.

That is what 3.5 is. Change streams move the performance story from build time to runtime. It is the same multi-release obsession, now made measurable and public through the meteor/performance benchmark suite and the community performance to-do list, finally aimed at the single most expensive thing a real-time app does.

3.5 is not a new direction. It is the moment the performance thread that started with meteor profile in 3.2 reaches the core.

A note on how 3.5 got built

There is one more thing worth naming, because it quietly shaped this entire release: how it came together.

Over the past year, the number of pull requests landing on Meteor has climbed sharply. A large part of that surge is LLM-assisted contribution: people who might once have filed an issue and waited now arrive with a working patch. The barrier between "I noticed something" and "here is a fix" is lower than it has ever been, and it shows. More contributors, more ideas in flight, more of the codebase being touched by more hands.

That is a genuinely good problem to have. A livelier community is the whole point, and the throughput of ideas 3.5 absorbed would have been hard to imagine a couple of years ago.

Throughput on one side of the table is capacity on the other. Every incoming change still has to be read, understood, tested against edge cases, and reconciled with everything else in flight, real work, gladly done. This is not a question of contribution quality; it is a shared resource called review bandwidth, and 3.5 invested heavily in it while re-wiring the heart of Meteor's reactivity.

The right response is not to slow contribution down; it is to make correctness cheaper to prove. That is why 3.5 leans harder than any release before it on measurable performance, and why the community now has a public benchmark suite to point real numbers at real claims.

That is one answer, aimed at the code. The other answer, aimed at the organization around the code, has been playing out in the open in PR#14426, a proposed GOVERNANCE.md that starts from exactly the surge this section describes: when AI makes a diff nearly free, a contribution's value stops being its line count and becomes judgment, ownership, and follow-through. As Antonio Soto puts it in "Open Source in the AI Age", "AI made PRs, issues, and security reports cheap. Maintainer trust, review time, and ownership stayed expensive." The proposal, the first governance document Meteor has ever had, tries to make that expensive part legible: named areas of specialization with a clear owner for review (Accounts, DDP, MongoDB, Reactivity, the build system), and explicit credit for the non-code work (reviewing, triaging, answering in the forum, writing docs) that holds a project together.

It is worth being honest that this is unresolved, and that the disagreement is the point. The sharpest pushback argued that a publicly documented, vote-gated path to write access on core is not a perk but a supply-chain risk: Meteor sits under thousands of production systems, defenders have to be right on every vote, and an adversary only has to pass once, which is roughly the shape of the xz/liblzma backdoor. Others pushed to separate recognition from merge rights and to treat areas as stewardship rather than ownership boxes. That argument left a mark: the proposal has since moved toward giving Core Committers triage permissions rather than merge rights. None of it has landed yet, and it may change again. Doing that reasoning in public, warmly and in disagreement, is itself part of how 3.5 got built.

More people building, a shared way to check the result, and a place, still being argued over in the open, for that effort to land. That is the direction, and 3.5 is the first release shaped by it.

Upgrade, what is next, and thanks

Ready to try it? Upgrade with a single command:

meteor update --release 3.5

Change streams are on by default, so there is nothing else to configure. One requirement to keep in mind: change streams need MongoDB 6+ running as a replica set or a sharded cluster. If you are on older or standalone Mongo, Meteor automatically falls back to oplog or polling, so meteor update stays safe either way.

Prefer the legacy behavior for now? Set packages.mongo.reactivity to ["oplog", "polling"] in your settings.json and you are back to how 3.4 worked.

What is next. The work is already underway. A 3.5.1 patch release is in progress (it sharpens the change-stream driver with safer projection fallbacks and opt-in journal-commit tuning, moves to Node 24.16, and clears a batch of fixes), and a 3.5.2 will likely follow to keep the polish coming. Further out, Meteor 3.6 is just getting started, and it points where the roadmap does: built-in OpenTelemetry and observability, a modern successor to Cordova for mobile, faster release CI/CD, and more Rspack and TypeScript polish. The performance thread does not stop either. If you have a workload to share, the community benchmark effort is the place.

Thanks. Meteor 3.5 is the product of many hands. Our deep gratitude to @italojs, @nachocodoner, @Grubba27, @radekmie, @9Morello, @alextaaa, @dupontbertrand, @Eshaan-byte, @harryadel, @mvogttech, @OleksandrChekhovskyi, @storytellercz, and @vlasky. Real-time on managed Mongo (long asked for, finally the default) is yours because of them.

Enhancing Performance in MeteorJS bundler: Complete Guide to CPU Profiling

italojs — Fri, 28 Mar 2025 19:20:50 +0000

Performance is a fundamental concern for any modern web application, and Meteor apps are no exception. Currently, one of our main focuses is optimizing the Meteor bundler, which can be a significant bottleneck in larger projects. Although Meteor already offers built-in tools for performance analysis, using native CPU profiling can provide much deeper insights into performance bottlenecks. In this article, we'll explore how to effectively use CPU profiling in Meteor, including the difference between Meteor's current profiling system and native profiling based on Node.js Inspector, with special emphasis on how this can help improve the bundling process.

What is CPU Profiling?

CPU profiling is a performance analysis technique that tracks exactly which functions are being executed by the CPU, for how long, and how frequently. This type of analysis allows you to identify precisely where your code is spending the most processing time.

Unlike simple time measurement techniques (like basic timers), CPU profiling creates a comprehensive view of code execution, including:

Function call tree - shows which functions call other functions
Execution time for each function
Frequency of calls for each function
Memory allocation (in some cases)
Call stack visualization

Current Meteor Profiling vs Native CPU Profiling

Meteor's Built-in Profiling System

Meteor has a built-in profiling system that can be activated by setting the METEOR_PROFILE environment variable. This system uses a timer-based measurement approach:

METEOR_PROFILE=1 meteor <command>

This profiler is designed specifically for the Meteor ecosystem and works through manual code instrumentation. The result is a report that shows the time spent on different Meteor operations, such as:

| (#1) Profiling: ProjectContext prepareProjectForBuild
| ProjectContext prepareProjectForBuild..........9,207 ms (1)
|    _initializeCatalog.............................24 ms (1)
|       files.readFile                               7 ms (2)
|       runJavaScript package.js                     2 ms (1)
|       files.rm_recursive                           4 ms (4)
|       other _initializeCatalog                    11 ms
|    _resolveConstraints.........................6,702 ms (1)
|       bundler.readJsImage.........................42 ms (1)
...
| (#1) Total: 9,544 ms (ProjectContext prepareProjectForBuild)

Native CPU Profiling with Node.js Inspector

Recent versions of Meteor have included support for native CPU profiling using Node.js's inspector module. This approach generates .cpuprofile files that provide much more detailed data and can be analyzed with advanced tools such as Chrome DevTools or cpupro.

To enable native CPU profiling:

METEOR_INSPECT=bundler.bundle,compiler.compile meteor <command>

Advantages of Native CPU Profiling:

Complete execution view: Captures all functions being executed, not just those manually instrumented
Interactive analysis: Allows you to visually explore and filter performance data using graphical tools
Hidden bottleneck discovery: Identifies problems in third-party libraries or Node.js code
Superior precision: Uses CPU sampling directly, providing more accurate data about actual CPU usage
Multiple metrics: Beyond time, can provide data on memory allocation and other aspects

How to Use Native CPU Profiling in Meteor

Basic Configuration

To generate a basic CPU profile, use the METEOR_INSPECT environment variable specifying which functions you want to profile:

# Profile a single function
METEOR_INSPECT=bundler.bundle meteor build ./output-build

# Profile multiple functions
METEOR_INSPECT=bundler.bundle,compiler.compile meteor build ./output-build

Available Functions for Profiling

Meteor supports profiling for several critical functions:

bundler.bundle - Application packaging process
compiler.compile - General compilation
Babel.compile - Babel-specific compilation
_readProjectMetadata - Reading project metadata
initializeCatalog - Package catalog initialization
_downloadMissingPackages - Downloading missing packages
_saveChangeMetadata - Saving change metadata
_realpath - File path resolution
package-client - Package client operations

Advanced Configuration Options

You can customize the profiling behavior with additional environment variables:

# Identifier for profile files
METEOR_INSPECT_CONTEXT=context_name

# Directory where .cpuprofile files will be saved (default: ./profiling)
METEOR_INSPECT_OUTPUT=/path/to/directory

# Sampling interval in ms - lower values = more details but uses more memory
METEOR_INSPECT_INTERVAL=500

# Maximum profile size in MB (default: 2000)
METEOR_INSPECT_MAX_SIZE=1000

For Large Applications

Larger applications may require more memory for profiling. To avoid out-of-memory (OOM) errors:

NODE_OPTIONS="--max-old-space-size=4096" METEOR_INSPECT=bundler.bundle meteor <command>

Analyzing CPU Profiling Results

With Chrome DevTools

Open Chrome DevTools
Go to the "Performance" or "Profiler" tab
Click "Load Profile" and select the generated .cpuprofile file

With cpupro (Open-Source Visualizer)

For more advanced analysis, you can use cpupro:

Visit https://discoveryjs.github.io/cpupro/ in your browser
Drag and drop your .cpuprofile file onto the interface
Use the interactive visualization to explore your profile data

cpupro offers several advantages over Chrome DevTools:

Better handling of large profiles
More flexible filtering options
Advanced search capabilities
Multiple visualization modes
Ability to compare different profiles

Practical Use Cases

Identifying Bottlenecks in Template Compilation

If your Meteor application is taking a long time to restart after template changes, you can use:

METEOR_INSPECT=compiler.compile meteor run

Analyze the resulting profile to identify which templates are causing the most overhead.

Diagnosing Slow Build Problems

For apps with slow builds:

METEOR_INSPECT=bundler.bundle meteor build ./output

Look in the profile for functions that are consuming excessive time, such as CSS processing, code transpilation, or minification.

Optimizing Package Loading

If you suspect packages are affecting performance:

METEOR_INSPECT=_downloadMissingPackages,package-client meteor update

When to Use Each Type of Profiling

Use Meteor's Built-in Profiling (`METEOR_PROFILE`) when:

You need a quick and general overview of Meteor tool performance
You're analyzing specific issues in Meteor operations
You prefer simple text-based reports
You have limited system resources (less memory usage)

Use Native CPU Profiling (`METEOR_INSPECT`) when:

You need more detailed analysis than the standard profiler provides
You're investigating complex performance issues
You want to identify specific bottlenecks in heavy functions like bundler or compiler
You need interactive visualization and in-depth analysis
You're looking to understand issues in third-party libraries or Node.js runtime

Real life demo

When using CPU profiling in real Meteor applications, we can identify specific bottlenecks that significantly impact performance. Let's analyze two practical examples:

Case 1: Babel as the main CPU consumer

When examining a Meteor CPU profile, we can clearly observe that Babel is the module consuming most of the processing time, both in self time (time spent within the function itself) and total time (time including calls to other functions):

This is a known issue in the Meteor community and fortunately it's already being addressed by PRs #13675 and #13674, which aim to replace Babel with SWC, a much faster transpiler written in Rust.

PR #13675, in particular, implements a smart approach: it uses SWC for initial transpilation, but preserves reify to maintain Meteor-specific features, such as reify modules, nested imports, and top-level await support. This allows gaining speed without losing important functionalities of the Meteor ecosystem.

Case 2: Analyzing by Self Time - The bottleneck in compiler-plugin

When sorting the same profile by "self time", we find the "script" module appearing as the one consuming the most processing time.

Digging deeper, we see that the compiler-plugin.js file is responsible for much of this consumption. And digging even deeper, the _linkJS function stands out as the most expensive of all.

Examining the flame graph of this function, we can see that there is especially intensive processing in the generation of hashes for the cache system:

const cacheKeyPrefix = sha1(JSON.stringify({
  linkerOptions,
  files: await Promise.all(
    jsResources.map(async (inputFile) => {
      fileHashes.push(await inputFile.hash);
      return {
        meteorInstallOptions: inputFile.meteorInstallOptions,
        absModuleId: inputFile.absModuleId,
        sourceMap: !!(await inputFile.sourceMap),
        mainModule: inputFile.mainModule,
        imported: inputFile.imported,
        alias: inputFile.alias,
        lazy: inputFile.lazy,
        bare: inputFile.bare,
      };
    })
  )
}));
const cacheKeySuffix = sha1(JSON.stringify({
  LINKER_CACHE_SALT,
  fileHashes
}));

This snippet shows that for each JavaScript file processed, the function executes:

Multiple await operations to access properties like hash and sourceMap
Conversions to JSON (via JSON.stringify)
Multiple SHA-1 hash calculations
Promise operations that occur for each file individually

The impact of this code is significant because it is executed for each JavaScript file in each module/package, becoming a serious bottleneck in larger applications with many files. Each call of this function processes all files again to generate cache keys, even when most files haven't changed.

This is an area where focused optimization could bring significant benefits.

Conclusion

While Meteor's built-in profiling system offers a more practical solution for performance analysis, native CPU profiling based on Node.js Inspector provides deeper and more detailed insights into your application's build performance. By combining both approaches, Meteor developers can identify and solve performance problems with unprecedented precision.

Try these techniques in your next performance debugging session and see how CPU profiling can transform your approach to optimizing Meteor applications.

🇧🇷 child_process: spawn, fork ou exec, qual usar?

italojs — Thu, 02 Nov 2023 21:31:10 +0000

Introdução

Uma das características valiosas do Node.js é sua capacidade de gerenciar processos filhos através de métodos como spawn, fork e exec. Embora sirvam propósitos semelhantes, há diferenças cruciais entre eles que podem impactar a eficiência e a facilidade de implementação em diferentes cenários. Este artigo visa elucidar as distinções entre spawn, fork e exec, e fornecerá exemplos práticos e casos de uso para uma compreensão aprofundada.

Spawn

Descrição

O método spawn, disponível através do módulo child_process no Node.js, é utilizado para criar processos filhos. Ele é capaz de executar qualquer comando no sistema operacional e é ideal para tarefas que demandam a execução de comandos externos.

Exemplo de Código

const { spawn } = require('child_process');
const ls = spawn('ls', ['-lh', '/usr']);

ls.stdout.on('data', (data) => {
  console.log(`stdout: ${data}`);
});

ls.stderr.on('data', (data) => {
  console.error(`stderr: ${data}`);
});

ls.on('close', (code) => {
  console.log(`child process exited with code ${code}`);
});

Caso de Uso: Processamento de Imagens

Em uma aplicação web, pode ser necessário converter formatos de imagem ou vídeo. Utilizando o spawn, é possível executar comandos externos para realizar estas conversões.

const { spawn } = require('child_process');
const ffmpeg = spawn('ffmpeg', ['-i', 'input.mp4', 'output.avi']);
ffmpeg.stderr.on('data', data => console.log(data.toString()));
ffmpeg.on('close', code => console.log(`Process exited with code ${code}`));

Quando não usar

Embora o método spawn seja extremamente útil para executar comandos externos, ele não é recomendado em cenários onde:

A saída do comando não é relevante, pois pode consumir recursos desnecessariamente.
A comunicação bidirecional entre o processo principal e o filho é necessária. Neste caso, o fork seria mais apropriado.
Comandos simples do shell são necessários, pois exec poderia ser mais conciso.

Fork

Descrição

O método fork é uma variação especial do spawn que é utilizada para criar um novo processo Node.js. Ele facilita a comunicação entre o processo pai e o processo filho através de um canal IPC (Inter-process Communication).

Exemplo de Código

const { fork } = require('child_process');
const child = fork('child.js');

child.on('message', (msg) => {
  console.log('Message from child', msg);
});

child.send({ hello: 'world' });

Caso de Uso: APIs de Longa Execução

Em cenários onde há APIs que levam muito tempo para serem processadas, pode-se utilizar o fork para processar essas requisições em um processo separado, mantendo o servidor principal livre para atender outras requisições.

const { fork } = require('child_process');
const app = require('express')();

app.post('/long-running-api', (req, res) => {
  const process = fork('longRunningApi.js');
  process.send(req.body);
  process.on('message', result => res.send(result));
});

app.listen(3000, () => console.log('Server is running on port 3000'));

Quando não usar

O método fork é uma ferramenta poderosa, mas não é a melhor escolha quando:

A tarefa não exige um novo processo Node.js, pois pode causar sobrecarga desnecessária.
Não há necessidade de comunicação IPC entre os processos.
Se deseja executar comandos que não são específicos do Node.js. Nesse caso, spawn ou exec seria mais apropriado.

Exec

Descrição

O método exec é utilizado para executar comandos no shell. Ele é semelhante ao spawn, mas difere em que ele gera um shell e executa o comando dentro desse shell, bufferizando qualquer saída produzida.

Exemplo de Código

const { exec } = require('child_process');

exec('ls -lh /usr', (error, stdout, stderr) => {
  if (error) {
    console.error(`Error: ${error.message}`);
    return;
  }

  if (stderr) {
    console.error(`Stderr: ${stderr}`);
    return;
  }

  console.log(`Stdout: ${stdout}`);
});

Caso de Uso: Execução de Scripts Shell

Em uma aplicação web, pode haver uma necessidade de executar scripts shell para automação ou administração do sistema. O exec facilita a execução desses scripts diretamente do código Node.js.

Quando não usar

O método exec é excelente para comandos simples do shell, mas deve ser evitado quando:

A saída do comando é extensa, pois ele bufferiza a saída e pode levar a problemas de memória.
A execução de comandos externos de longa duração é necessária, pois pode bloquear o Event Loop.
Precisão e controle detalhado sobre o processo filho são necessários. Neste caso, o spawn seria a melhor escolha.

Conclusão

A escolha entre spawn, fork e exec depende das necessidades específicas do projeto. O entendimento profundo desses métodos e de suas aplicações práticas permite uma programação mais eficaz e eficiente em Node.js.

Gostou desse conteúdo? Aproveita pra se inscrever na nossa newsletter https://highperformanceletterbr.substack.com/
Postamos conteúdos sobre performance e otimização todas as semanas!

Primeiro curso de performance backend js do brasil(ou do mundo)

🚀 Domine o Desempenho Backend com JavaScript! 🚀

🎓 Apresentamos o PRIMEIRO Curso de Performance Backend em JavaScript no Brasil! 🇧🇷

🔥 Seja um dos pioneiros a explorar o mundo do desenvolvimento backend com foco em eficiência e velocidade. Este curso exclusivo é sua porta de entrada para se tornar um especialista em otimização e desempenho de aplicações JavaScript.

🌟 O Que Oferecemos:

Instrutores Especialista: Aprenda com um profissional que atua com esse assunto no dia a dia.
Conteúdo Avançado: Desde fundamentos até técnicas avançadas de otimização.
Projetos Práticos: Coloque suas habilidades à prova com desafios reais.
Comunidade e Networking: Junte-se a uma comunidade vibrante de desenvolvedores.
🛠️ Domine Ferramentas e Técnicas Essenciais: Conteúdo prático sobre as melhores tools de monitoramento e análise de recursos

Análise de Performance;

Diagnóstico;
Otimização de Código;
Gerenciamento Eficiente de Memória;
Eventos e streaming;

🚀 Transforme a forma que os outros devs te veem:

Esteja na vanguarda do desenvolvimento backend. As habilidades que você adquirirá são inestimáveis e altamente procuradas no mercado de trabalho para posições senior.

✅ Acesse: https://highperformanceschool.hotmart.host/

🇧🇷 Multi-Threading deixando a app mais lenta? como?

italojs — Thu, 02 Nov 2023 21:31:03 +0000

Introdução 🧐

O multi-threading é uma técnica comum para melhorar a performance de aplicativos, mas não é isenta de desafios. Este artigo se propõe a explorar, de maneira objetiva, as circunstâncias sob as quais o multi-threading pode não oferecer os benefícios esperados, utilizando um exemplo prático em Node.js.

Contextualização

A capacidade de executar várias threads simultaneamente pode, teoricamente, reduzir o tempo de execução de tarefas complexas. No entanto, a realidade é que o overhead de gerenciamento de threads e a contenção de recursos podem impactar negativamente a performance.

Exemplo Prático 👩🏾‍💻

Para ilustrar isso, consideremos um exemplo específico em Node.js, onde o objetivo é calcular números primos. O código abaixo foi utilizado para comparar a eficiência entre a execução single-threaded e multi-threaded.

const { 
    Worker, 
    isMainThread, 
    parentPort, 
    workerData } = require('worker_threads');
const { performance } = require('perf_hooks');

// Função para encontrar números primos em um intervalo
const findPrimes = (start, end) => {
    const primes = [];

    // Itera por todos os números no intervalo [start, end]
    for (let i = start; i <= end; i++) {
        let isPrime = true;

        // Verifica se o número 'i' é divisível por algum número 
        // entre 2 e sua raiz quadrada
        for (let j = 2, sqrt = Math.sqrt(i); j <= sqrt; j++) {
            if (i % j === 0) {
                isPrime = false;
                break;
            }
        }

        // Se 'i' for primo (e maior que 1), adiciona-o ao 
        // array de primos
        if (isPrime && i > 1) {
            primes.push(i);
        }
    }

    return primes;
}

// Função para calcular números primos em um único thread
function singleThread(max) {
    // Marca o início da medição de tempo
    const stStart = performance.now(); 

    // Chama a função para encontrar números primos no 
    // intervalo [2, max]
    findPrimes(2, max); 

     // Marca o final da medição de tempo
    const stEnd = performance.now();
    // Imprime o tempo decorrido em milissegundos
    console.log('Single-threaded:', stEnd - stStart, 'ms'); 
}

// Função para calcular números primos em vários threads
function multiThread(max, numbersPerThread, threadCount) {
    const mtStart = performance.now(); // Marca o início da medição de tempo
    let completed = 0;

    for (let i = 0; i < threadCount; i++) {
        const start = i * numbersPerThread + 2;
        const end = i === threadCount - 1 ? max : start + numbersPerThread - 1;

        // Cria um novo thread para calcular primos em um subintervalo
        const worker = new Worker(__filename, {
            workerData: { start, end }
        });

        // Quando o thread conclui, incrementa o contador 'completed'
        // e verifica se todos os threads terminaram para medir o tempo total
        worker.on('message', () => {
            completed++;
            if (completed === threadCount) {
                // Marca o final da medição de tempo quando todos os threads terminam
                const mtEnd = performance.now(); 
                 // Imprime o tempo decorrido em milissegundos
                console.log('Multi-threaded:', mtEnd - mtStart, 'ms');
            }
        });
    }
}

// Se estiver no thread principal
if (isMainThread) {
    // Define o número máximo no intervalo
    const max = 1e5; 
    // Define o número total de threads a serem usadas
    const threadCount = 4; 
    // Calcula quantos números cada thread deve processar
    const numbersPerThread = Math.ceil(max / threadCount); 
    // Calcula primos em um único thread
    singleThread(max);

    // Calcula primos em múltiplos threads
    multiThread(max, numbersPerThread, threadCount);
} else {
    // Se estiver em um thread
    // Recebe os dados de start e end do intervalo a ser processado
    const { start, end } = workerData; 
    // Chama a função para encontrar primos no intervalo especificado
    findPrimes(start, end); 
    // Envia uma mensagem de conclusão após o cálculo
    parentPort.postMessage(true); 
}

Quando você executar o script completo, os resultados que você irá obter serão parecidos com:

`
Single-threaded: 17.95 ms

Multi-threaded: 75.41 ms`

😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱

Contrariando as expectativas iniciais, a execução multi-threaded foi mais lenta.

Análise 👀

A razão para isso pode ser atribuída ao overhead associado à criação e gerenciamento de threads. Cada thread requer recursos para ser iniciada, executada e, posteriormente, encerrada. Em tarefas que não são intensivamente paralelizáveis ou que são relativamente rápidas, o custo de gerenciamento de threads pode superar os benefícios do processamento paralelo.

Além disso, a contenção de recursos, onde várias threads competem pelos mesmos recursos do sistema, pode levar a um aumento no tempo de execução, especialmente em sistemas com recursos limitados.

Considerações

É essencial, portanto, avaliar a adequação do multi-threading para cada tarefa específica. Fatores como a natureza da tarefa, o ambiente de execução e os recursos disponíveis desempenham um papel crucial na determinação da eficácia do multi-threading.

A Necessidade de Benchmarking

Um passo essencial e muitas vezes subestimado na implementação do multi-threading é a realização de testes de benchmark rigorosos. Estes testes não são apenas uma formalidade, mas uma necessidade para avaliar objetivamente o impacto real na performance. Os desenvolvedores devem se equipar com dados concretos para tomar decisões informadas, evitando suposições e generalizações.

Volume de Dados: Um Fator Crítico

O volume de dados é um elemento inseparável desta discussão. Testes de benchmark realizados com um volume de dados baixo podem não refletir a realidade operacional e, consequentemente, oferecer uma visão distorcida da eficácia do multi-threading. É imperativo que os testes sejam conduzidos em um ambiente que simule as condições reais de operação, com volumes de dados significativos.

A razão para isso é clara: o overhead de gerenciamento de threads pode ser minimizado ou justificado quando lidamos com volumes de dados grandes. Em contrapartida, para volumes de dados menores, o custo de inicialização, execução e terminação de threads pode superar os benefícios percebidos, como observado em nosso exemplo prático.

Recomendação Final

Portanto, a recomendação é clara. Antes de embarcar na jornada do multi-threading, invista tempo e recursos em testes de benchmark detalhados, com um volume de dados que reflita as condições reais de uso. Este passo é fundamental para garantir que a implementação de multi-threading traga benefícios tangíveis em termos de eficiência e performance, e para evitar surpresas indesejadas após a implementação.

Ao adotar uma abordagem metódica, baseada em dados e adaptada às especificidades de cada aplicação e conjunto de dados, os desenvolvedores podem maximizar os benefícios do multi-threading, transformando-o em um aliado valioso na otimização contínua da performance de aplicativos.

Gostou desse conteúdo? Aproveita pra se inscrever na nossa newsletter https://highperformanceletterbr.substack.com/
Postamos conteúdos sobre performance e otimização todas as semanas!

Primeiro curso de performance backend js do brasil(ou do mundo)

🚀 Domine o Desempenho Backend com JavaScript! 🚀

🎓 Apresentamos o PRIMEIRO Curso de Performance Backend em JavaScript no Brasil! 🇧🇷

🌟 O Que Oferecemos:

Análise de Performance;

Diagnóstico;
Otimização de Código;
Gerenciamento Eficiente de Memória;
Eventos e streaming;

🚀 Transforme a forma que os outros devs te veem:

Esteja na vanguarda do desenvolvimento backend. As habilidades que você adquirirá são inestimáveis e altamente procuradas no mercado de trabalho para posições senior.

✅ Acesse: https://highperformanceschool.hotmart.host/

🇧🇷 Otimizando buscas em mais de 1000x de maneira rápida e fácil

italojs — Thu, 02 Nov 2023 21:30:54 +0000

`Map.get` vs `Array.find`

Quer fazer a boa no seu time e trazer uma baita performance pra sua aplicação com pouco esforço? se liga nessa dica! 🤯

De um ctrl+f e busque pelo #find e se possível os substitua por uma estrutura de dados #Map

Sabe por que? 🧐

BuscaLinear (find):

Percorre o array objeto por objeto até encontrar o que procura.

Complexidade de tempo: O(n), onde n é o número de objetos no array.

Índices:

Complexidade de tempo: Aproximadamente O(1) (em média, para a maioria dos casos).

Para buscas frequentes, pode ser benéfico usar índices com base na propriedade que você está pesquisando.

🔎 Busca linear: #find

As formas mais tradicionais de busca em arrays no #javascript é o #find, mas ele apesar de muito útil, ele tem um problema, ele é linear O(n), onde pode percorrer todo o seu arrau até encontrar o que deseja.

const product = products.find(p => p.id === 1);

⚡️ Índices: #Map

Uma forma de otimizar muitos cenários de busca é usar a estrutura de dados #Map

Nesse link eu deixo um #benchmark com um #exemplo de busca de produtos onde você pode ver um resultaod semelhante a esse aqui :

Busca Linear: 12.647450000047684ms

Busca via Map: 0.005816996097564697ms

Conclusão

Usar a estrutura de dados correta na hora de fazer uma busca pode impactar muito no tempo de resposta e consumo de recurso da sua aplicação, claro que nem sempre iremos poder usar uma estrutura de dados do tipo Map ou Object, mas tenho certeza que em muitos casos você consegue substituir a criaçao da sua lista por um Map e ganhar um baita ganho de performence!

Gostou desse conteúdo? Aproveita pra se inscrever na nossa newsletter https://highperformanceletterbr.substack.com/
Postamos conteúdos sobre performance e otimização todas as semanas!

Primeiro curso de performance backend js do brasil(ou do mundo)

🚀 Domine o Desempenho Backend com JavaScript! 🚀

🎓 Apresentamos o PRIMEIRO Curso de Performance Backend em JavaScript no Brasil! 🇧🇷

🌟 O Que Oferecemos:

Análise de Performance;

Diagnóstico;
Otimização de Código;
Gerenciamento Eficiente de Memória;
Eventos e streaming;

🚀 Transforme a forma que os outros devs te veem:

Esteja na vanguarda do desenvolvimento backend. As habilidades que você adquirirá são inestimáveis e altamente procuradas no mercado de trabalho para posições senior.

✅ Acesse: https://highperformanceschool.hotmart.host/

🇧🇷 Como Evitar Vazamentos de Memória das closures

italojs — Thu, 02 Nov 2023 21:30:48 +0000

Vazamentos de Memória

Vazamentos de memória acontecem quando um programa usa mais memória do que deveria e não a libera de volta para o sistema. Agora, com essa ideia em mente, vamos falar sobre como isso pode acontecer quando usamos closures no Node.js.

O que são Closures?

Closures são funções em JavaScript que retêm acesso às variáveis de funções pai onde elas foram criada. Isso significa que um closure pode acessar variáveis de uma funções externas que já terminaram sua execução.

Exemplo de Código com Vazamento de Memória ⚠️

Vamos usar um exemplo onde criamos uma função que retorna informações de um produto baseado em seu ID.

function infoProduto(id) {
    const produto = {
        id: id,
        nome: 'Produto ' + id,
        detalhes: new Array(5000000).fill('detalhe')
    };

    return function() {
        console.log(produto.nome);
    }
}

const produto5 = infoProduto(5);
produto5(); // Logará "Produto 5"

Neste exemplo, a função infoProduto(id) cria um objeto de produto que é mantido na memória pelo closure, mesmo após a execução da função, levando a um possível vazamento de memória se muitos produtos forem carregados.

Como Resolver o Problema ✅

Podemos ajustar o código para garantir que apenas as informações necessárias sejam mantidas e que o restante seja liberado.

function infoProduto(id) {
     let produto = {
        id: id,
        nome: 'Produto ' + id,
        detalhes: new Array(5000000).fill('detalhe')
    };
     const name = produto.name
     produto = null // libera objeto grande da memoria

    return function() {
        console.log(nome);
    }
}

const produto5 = infoProduto(5);
produto5(); // Logará "Produto 5"

Agora, o array pesado detalhes é liberado da memória, enquanto o closure mantém referência apenas ao nome do produto, que é o que realmente precisamos para essa operação.

Conclusão 🎓

Closures são uma ferramenta poderosa, mas é preciso manejá-los com cuidado para evitar vazamentos de memória. Ser consciente das referências que seus closures mantêm pode ajudar a manter sua aplicação Node.js rápida e eficiente.

Gostou desse conteúdo? Aproveita pra se inscrever na nossa newsletter https://highperformanceletterbr.substack.com/
Postamos conteúdos sobre performance e otimização todas as semanas!

Primeiro curso de performance backend js do brasil(ou do mundo)

🚀 Domine o Desempenho Backend com JavaScript! 🚀

🎓 Apresentamos o PRIMEIRO Curso de Performance Backend em JavaScript no Brasil! 🇧🇷

🌟 O Que Oferecemos:

Análise de Performance;

Diagnóstico;
Otimização de Código;
Gerenciamento Eficiente de Memória;
Eventos e streaming;

🚀 Transforme a forma que os outros devs te veem:

Esteja na vanguarda do desenvolvimento backend. As habilidades que você adquirirá são inestimáveis e altamente procuradas no mercado de trabalho para posições senior.

✅ Acesse: https://highperformanceschool.hotmart.host/

🇧🇷 A Ineficiência do Regex e Como Otimizá-lo

italojs — Thu, 02 Nov 2023 21:29:58 +0000

Regex, o vilão da performance

Você sabia que expressões regulares (regex) podem ser extremamente ineficientes em certos cenários? Isso ocorre principalmente devido a 'backtracking' excessivo, que pode levar a tempos de execução exponenciais!

Por exemplo, considere o regex (a+)+b e a string a...a! (onde ... são milhares de 'a'). Esse regex tentará diversas combinações antes de falhar, tornando-se muito lento.

Mas a boa notícia é: o V8 (motor JavaScript do Chrome e Node.js) introduziu uma flag experimental para lidar com isso: --enable-experimental-regexp_engine-on-excessive-backtracks.

🔍 Teste Prático:

const regex = /(a+)+b/;
const input = 'a'.repeat(5000);
console.time('Regex Sem Otimização');
input.match(regex);
console.timeEnd('Regex Sem Otimização');

o resultado será algo em torno de 70.885ms

Para usar a flag no Node.js:

node --enable-experimental-regexp_engine-on-excessive-backtracks seuScript.js

Com isso, ao detectar backtracks excessivos, o V8 mudará para um mecanismo de regex diferente e mais eficiente!

Aqui o resultado caiu pra 1.977ms 😲

Lembre-se: regex é uma ferramenta poderosa, mas com grandes poderes vêm grandes responsabilidades. Use com sabedoria e sempre monitore o desempenho!

Olhe https://v8.dev/blog/non-backtracking-regexp para mais informações.

Gostou desse conteúdo? Aproveita pra se inscrever na nossa newsletter https://highperformanceletterbr.substack.com/
Postamos conteúdos sobre performance e otimização todas as semanas!

Primeiro curso de performance backend js do brasil(ou do mundo)

🚀 Domine o Desempenho Backend com JavaScript! 🚀

🎓 Apresentamos o PRIMEIRO Curso de Performance Backend em JavaScript no Brasil! 🇧🇷

🌟 O Que Oferecemos:

Análise de Performance;

Diagnóstico;
Otimização de Código;
Gerenciamento Eficiente de Memória;
Eventos e streaming;

🚀 Transforme a forma que os outros devs te veem:

Esteja na vanguarda do desenvolvimento backend. As habilidades que você adquirirá são inestimáveis e altamente procuradas no mercado de trabalho para posições senior.

✅ Acesse: https://highperformanceschool.hotmart.host/

🇧🇷 FlameGraphs, uma introdução

italojs — Thu, 26 Oct 2023 14:19:00 +0000

FlameGraphs

Ei, devs! 🚀 Se vocês estão procurando uma forma de identificar gargalos de performance nos seus projetos Node.js, eu tenho uma dica quente para vocês: Flamegraphs!

🔍 O que é um Flamegraph? Um Flamegraph é uma visualização que representa a pilha de chamadas (stack trace) de um programa em execução. Cada barra horizontal representa um arquivo ou função, e sua largura indica o tempo que essa arquivo/função consumiu. Isso pode ser extremamente útil para identificar onde seu código está gastando mais tempo!

🛠 Como criar um Flamegraph no Node.js?

Rode sua aplicação com npx 0x index.js

Feche a aplicação e 0x irá gerar um arquivo HTML com um Flamegraph interativo. Você pode abri-lo em seu navegador e visualizar as áreas do código que consumiram mais tempo.

💡 Como se beneficiar disso?

**
Identificação Rápida:** Com um olhar, você pode identificar as funções que estão consumindo mais tempo e se tornar mais eficaz ao otimizar.

Melhoria Contínua: Ao fazer ajustes no código e comparar os Flamegraphs antes e depois, você tem uma visão clara do impacto das suas mudanças.

Performance em Produção: Você pode até usar ferramentas, como o 0x, para criar Flamegraphs de aplicativos Node.js em produção.

🔑 Conclusão

Se você está comprometido em otimizar a performance do seu projeto, o Flamegraph é uma ferramenta essencial. Não só ajuda a identificar os gargalos, mas também a entender a interação entre diferentes partes do código.

Então, da próxima vez que você se deparar com um desafio de performance, 🕶️ coloque seus óculos de detetive e use o Flamegraph para iluminar seu caminho!

E você? Já usou o #Flamegraph em seus projetos Node.js? Compartilhe sua experiência! #NodeJS #Optimization

Gostou desse conteúdo? Aproveita pra se inscrever na nossa newsletter https://highperformanceletterbr.substack.com/
Postamos conteúdos sobre performance e otimização todas as semanas!

Primeiro curso de performance backend js do brasil(ou do mundo)

🚀 Domine o Desempenho Backend com JavaScript! 🚀

🎓 Apresentamos o PRIMEIRO Curso de Performance Backend em JavaScript no Brasil! 🇧🇷

🌟 O Que Oferecemos:

Análise de Performance;

Diagnóstico;
Otimização de Código;
Gerenciamento Eficiente de Memória;
Eventos e streaming;

🚀 Transforme a forma que os outros devs te veem:

Esteja na vanguarda do desenvolvimento backend. As habilidades que você adquirirá são inestimáveis e altamente procuradas no mercado de trabalho para posições senior.

✅ Acesse: https://highperformanceschool.hotmart.host/

Saving time when starting a project

italojs — Mon, 11 Oct 2021 12:30:54 +0000

How much time do you spend setting up the basis of a new project?

No matter how simple or complex is your service, we have to build the base, that basic base:

Create entities
Create the folder structure
Then create some repositories
Some usecases/services for each entity
Configure the api, routes, midlewares
etc

Come oooooooonnnn, who likes to do this?

How to save time configuring the base's project?

Today we will create a Postgres-based api with two presentation layers: Graphql and REST in less then 1 minute.

For today's dish, we use the ingredients:

To start our "fast project" we need first of all to modeling our base. Imagine we are creating an e-commerce backend app, in an e-commerce we would have 3 entities: user, product and sale.

The first step to build our project will be to create the entities using the @herbs/herbs lib that will provide some metadata to build everything else based in our entities: usecases, repositories, migrations, graphql, rest etc.

Building an entity with Herbs is very simple, just import the entity module and pass the parameters entity("entityName", { fields }).

All our entities will be inside a folder /entities

// entities/customer.js
const { entity, field } = require('@herbsjs/herbs')


const Customer = entity('Customer', {
    id: field(Number),
    cellphone: field(String),
    email: field(String),
    gender: field(String),
    age: field(Number),
    discountClub: field(Boolean),
    city: field(String),
    state: field(String),
    address: field(String),
    zipcode: field(String),
    lastName: field(String),
    fisrtName: field(String),
    civilDocument: field(String),
})

module.exports = Customer

After create our simple entity, what about add some validations? To do this with Herbs is very simple too, just add an option object with some rules as second parameter field(Type, { options }).

If you'd like to see more about validation types, you can check out the Herbs documentation here.

// entities/customer.js
const { entity, field } = require('@herbsjs/herbs')

const requiredString = {
    validation: {
        presence: true,
        length: { minimum: 3, maximum: 255 },
    }
}

const Customer = entity('Customer', {
    id: field(Number, {
        // Optional option
        validation: {
            presence: true,
            numericality: {
                greaterThan: 0,
                onlyInteger: true
            }
        }
    }),
    cellphone: field(String, {
        validation: {
            format: /^\+[1-9]{1}[0-9]{3,14}$/
        }
    }),
    email: field(String, {
        validation: { ...requiredString.validation, email: true } 
    }),
    gender: field(String, {
        validation: { ...requiredString.validation, contains: { allowed: ["M", "W", "X"] } } 
    }),
    age: field(Number),
    discountClub: field(Boolean),
    city: field(String, requiredString),
    state: field(String, requiredString),
    address: field(String, requiredString),
    zipcode: field(String, requiredString),
    lastName: field(String, requiredString),
    fisrtName: field(String, requiredString),
    civilDocument: field(String, requiredString),
})

module.exports = Customer

// entities/product.js
const { entity, field } = require('@herbsjs/herbs')

const required =  { validation: { presence: true } }
const requiredString = {
    validation: {
        ...required.validation,
        length: { minimum: 3, maximum: 255 },
    }
}

module.exports = entity('Product', {
    id: field(Number, {
        // Optional option
        validation: {
            presence: true,
            numericality: {
                greaterThan: 0,
                onlyInteger: true
            }
        }
    }),
    limitDiscount: field(Number, { 
        validation: { 
            greaterThan: 0,
            lessThan: 100,
            onlyInteger: true 
        }
    }),
    size:  field(String, required),
    stock: field(Number, required),
    color: field(String, required),
    name: field(String, requiredString),
    brand: field(String, requiredString),
    category: field(String, requiredString),
    description: field(String, requiredString),
    price: field(Number, { validation: { presence: true, greaterThan: 0 } }),
})

// entities/sale.js
const { entity, field } = require('@herbsjs/herbs')

const required =  { validation: { presence: true } }

module.exports = entity('Sale', {
    id: field(Number, {
        // Optional option
        validation: {
            presence: true,
            numericality: {
                greaterThan: 0,
                onlyInteger: true
            }
        }
    }),
    productID: field(Number, required),
    customerID: field(Number, required),
    total: field(Number, required),
    quantity: field(Number, required)
})

In the end we will have this result:

generating the project

Finally our dreamed come to be true, where we generated our project based on our entities!

The Herbs CLI was created to help you create and maintain your project in a faster way when you uses the entire Herbs ecosystem.

When we type herbs new herbs will ask a some things about your project to generate our project.

Note the last question is an optional question, if you don't specify your entities folder path, the project will be generated based on a default “User” entity, but we're not here to generate a “boilerplate” are we? So let's fill it in with out entities folder path.

When you press Enter our app will be generated and you will find a folder with the project name.

look this! How much automatically generated stuff! How much time we saved here! Very satisfying! Note that the entire project is done: migrations, repositories, api(graphl and rest) create, read, update and delete usecases and everything else!

Explaining the project

Okay, but what we built here?

The diagram below shows a little bit how the project is organized (which you can check on the Herbs doc home), the generated app is based on the clean architecture, there we have our domain with our entities and our usecases, around from this we have the rest of our application (api, database, clients, documentation, etc.) and in all these other layers use the herbsjs "glues" that read the entities and usecases metadatas to facilitate our day-a-day software development, remembering that no glue is a code generator, are just tools to speed up the development of your code.

now let's be a little practical and talk about the our project's folder structure.

NOTE: remembering that architecture is not a folder structure!

Inside domain folder we have our entities and usecases that provide some metadata as usecase's name, request and output schema, entity fields, types and more.

Above we have herbs2gql glue example, the herbs2gql receive a usecase and a resolver so it return the Graphql schemas string.

The concept of “glue” runs throughout the application, from the presentation layer to the repository, remembering it isn't mandatory and this libraries are only meant to help you develop and maintain your project. :)

Inside our usecases we can see that exists not anly a simples creation, but there have a validations too, so it means the generated project is done to production!

Running the project

Once we've generated and understood the code, let's put this code to work! Just give the npm run start command and watch the magic happen. Note that all REST routes are printed on the terminal and that they are all based on our usecases.

Also, if you open your browser on the http://localhost:3000/graphql route you will see all generated schemas for the Graphql presentation layer with the Graphql playground documentation

Beside everything we generated here, the herbs generates a dynamic documentation too, so imagine you'll never need to worry about your API's documentation since everything will be automatically generated via usecase metadata and your Readme.md file! To do that we use the HerbsShelf glue

So now just implement your business rule and go up to production!

DEV Community: italojs

Meteor 3.5 is out!

The problem we kept running into

The headline: MongoDB Change Streams

The proof: real-world numbers

Change streams under the hood: tuning and trade-offs

How a feature that was ready took four and a half months to ship

The rest of the real-time modernization

For the API and auth builders

Platform: Node 24 and the invisible work

Where 3.5 fits in the Meteor 3 story

A note on how 3.5 got built

Upgrade, what is next, and thanks

Enhancing Performance in MeteorJS bundler: Complete Guide to CPU Profiling

What is CPU Profiling?

Current Meteor Profiling vs Native CPU Profiling

Meteor's Built-in Profiling System

Native CPU Profiling with Node.js Inspector

How to Use Native CPU Profiling in Meteor

Basic Configuration

Available Functions for Profiling

Advanced Configuration Options

For Large Applications

Analyzing CPU Profiling Results

With Chrome DevTools

With cpupro (Open-Source Visualizer)

Practical Use Cases

Identifying Bottlenecks in Template Compilation

Diagnosing Slow Build Problems

Optimizing Package Loading

When to Use Each Type of Profiling

Use Meteor's Built-in Profiling (METEOR_PROFILE) when:

Use Native CPU Profiling (METEOR_INSPECT) when:

Real life demo

Case 1: Babel as the main CPU consumer

Case 2: Analyzing by Self Time - The bottleneck in compiler-plugin

Conclusion

🇧🇷 child_process: spawn, fork ou exec, qual usar?

Introdução

Spawn

Fork

Exec

Conclusão

Newsletter

Primeiro curso de performance backend js do brasil(ou do mundo)

🚀 Domine o Desempenho Backend com JavaScript! 🚀

🌟 O Que Oferecemos:

Análise de Performance;

🚀 Transforme a forma que os outros devs te veem:

🇧🇷 Multi-Threading deixando a app mais lenta? como?

Introdução 🧐

Contextualização

Exemplo Prático 👩🏾‍💻

Análise 👀

Considerações

A Necessidade de Benchmarking

Volume de Dados: Um Fator Crítico

Recomendação Final

Newsletter

Primeiro curso de performance backend js do brasil(ou do mundo)

🚀 Domine o Desempenho Backend com JavaScript! 🚀

🌟 O Que Oferecemos:

Análise de Performance;

🚀 Transforme a forma que os outros devs te veem:

🇧🇷 Otimizando buscas em mais de 1000x de maneira rápida e fácil

Map.get vs Array.find

Sabe por que? 🧐

BuscaLinear (find):

Índices:

🔎 Busca linear: #find

⚡️ Índices: #Map

Conclusão

Newsletter

Primeiro curso de performance backend js do brasil(ou do mundo)

🚀 Domine o Desempenho Backend com JavaScript! 🚀

🌟 O Que Oferecemos:

Análise de Performance;

🚀 Transforme a forma que os outros devs te veem:

🇧🇷 Como Evitar Vazamentos de Memória das closures

Vazamentos de Memória

Use Meteor's Built-in Profiling (`METEOR_PROFILE`) when:

Use Native CPU Profiling (`METEOR_INSPECT`) when:

`Map.get` vs `Array.find`