Prepare your Meteor.js project for the big 3.0 release!

Meteor.js 3.0 is an upcoming major release with lots of breaking changes. The platform is moving away from Fibers to use Node's native async await implementation.

In this article I'm going to show you a few techniques to detect code that requires changes in order to be ready for this upcoming Meteor.js 3.0 release. 🧙‍♂️

Photo by Susan Q Yin on Unsplash

Note, that this will not be a complete or perfect analysis, because you may end up with some false-positives or missing pieces. Consider it as a helping hand to aid the start for the transition. 🤝

Another primary goal is to be non-intrusive to your current application. This means, there should be no breaking effect to your current application by implementing the techniques, presented in this article.

This is important, because any breaking effect (server crashes etc.) creates frustration and decreases the chance of people finally starting to migrate now.

Background / why this is important? ☝

If you know all about Meteor.js 3.0 then you can safely skip this section.

Meteor.js is around for over 10 years now, and it's core API-structure did not change significantly. I previously showed this in another article on migrating an ancient Meteor.js project.

However, while Meteor.js pioneered the fullstack technologies until about 2016 it eventually lost its amazing momentum in about 2017 (read here on the why and how).

The JavaScript ecosystem evolved and like in the cambrian explosion we saw a multitude of tools, frameworks and libraries emerging.
A vast majority seemingly adopted the async/await pattern,
while asynchronous code execution in Meteor.js was implemented using coroutines via Fibers.

Finally, the release of Node 16 became basically incompatible with fibers and thus, Meteor.js got stuck on Node 14. This implied a major rewrite of a large potion of the platform's internals as it all depended on fibers. Further, it implied that all Meteor.js projects and packages required a major rewrite as well.

This is where YOU come into play.

How do I know, if I need to migrate? 🤔

This answer is simple: you will have to! 🤓

This is, because all pre-3.0 projects run on a security-patched Node 14 (read here how to apply the patched version) and you will need Meteor 3.0 to be back on the safe side of the Node game by being able to use the latest Node LTS for your deployments.

How do I know what to migrate? 🤔

This is a rather complex situation with no simple answer. There are basically three big parts of your codebase to review:

• Mongo Collection and Cursor usage
• Meteor Methods + Publications
• Meteor Packages that depend on the "old" code style

There is also an official list of core functionality that
is to be migrated. It's part of the "How to migrate to Meteor Async in 2.x" guide.

Each of these come with a different approach, different complexity and constraints. Implementation also varies, whether a static analysis is desired or if runtime analysis is okay, too. Let's take a look at each of them, one by one.

Mongo Collection and Cursor usage 💽

In classic mode (using fibers) the Mongo collection calls to the MongoDB driver were executed within a Fiber and thus could be written in sync-style code, while being magically async under the hood:

const document = collection.findOne({ foo: 'bar' })

In the new async mode (using async/await) we need to change this to an async counterpart:

const document = await collection.findOneAsync({ foo: 'bar' })

This change targets the following Mongo.Collection methods:

Class mode	Async Mode	Notes
insert	insertAsync	on Mongo.Collection
update	updateAsync	on Mongo.Collection
remove	removeAsync	on Mongo.Collection
findOne	findOneAsync	on Mongo.Collection
createIndex	createIndexAsync	on Mongo.Collection
count	countAsync	on Mongo.Cursor
forEach	forEachAsync	on Mongo.Cursor
map	mapAsync	on Mongo.Cursor
fetch	fetchAsync	on Mongo.Cursor

Additional info

Note, that Cursor.count is deprecated and you should consider migrating to Collection.countDocuments or Collection.estimatedDocumentCount.

Static analysis 🔬

If you want to analyze their sync usage statically you would have to either write an isobuild plugin or a babel plugin.

The hardest challenge here is to get rid of the ambivalence due to the very common naming of the methods. For example: forEach and map are typical Array methods and may create lots of false-positives. You will likely have to track, if the variable.

Additional Info

There seems to be a discussion in the Meteor forums about an ESLINT plugin for static analysis. I suggest you to keep watching the topic, if this is an alternative to you.

Runtime analysis 🔬

This approach is rather easy to implement and produces no false-positives. However, detection requires the code to run, which therefore requires a great test-coverage in order to really cover all possible calls. We can do that by monkey patching the methods mentioned above.

Big plus: we can also detect usage in dependent Meteor packages, if we make sure the pathing occurs before the package code runs on startup. This is fortunately very easy as you just need to follow these steps:

1. Create a local package 📦

If your Meteor project does not contain a packages folder then you need to create one:

$ cd your-great-project
$ mkdir -p packages # needs to be on the same level as server, client, imports etc.
$ cd packages

Then you create a new local package with a unique name:

$ meteor create --package jkuester:migration-helper
$ cd ../ # back to project root
$ meteor add jkuester:migration-helper

Let's change the following method in package.js in order to immediately execute the code on startup (not to be confused with Meteor.startup, which runs after all startup code):

Package.onUse(function (api) {
  api.versionsFrom('2.3')
  api.use('ecmascript')
  api.use('mongo')
  api.addFiles('migration-helper.js', 'server')
})

Finally, open the Meteor packages list at your-great-project/.meteor/packages and move the entry for the package at the very top of the file in order to apply the patches before any other package code runs.

2. Create a new patch function 🔨

Stay in the package and edit the main file migration-helper.js which the following code (yes, you can copy/paste):

import {Meteor} from 'meteor/meteor'
import {Mongo} from 'meteor/mongo'
import {ValidatedMethod } from 'meteor/mdg:validated-method'

// some common pattern
const newLine = /\n/g
const whiteSpace = /\s+/g

// return the current location
// of function execution, considering
// multiple levels of wrappers
const getLocation = () => {
  const e = new Error('')
  const lines = e.stack.split(newLine)
  return lines[3].replace(whiteSpace, ' ').trim()
}

// monkey-patch a Collection/Cursor proto function
// to inject some analysis code without altering
// the original behavior
const patch = (proto, name) => {
  const original = proto[name]
  const className = proto.constructor.name

  proto[name] = function (...args) {
    const self = this
    const location = getLocation()
    const isWrappedAsync = location.includes(`as ${name}Async`)

    if (!isWrappedAsync) {
      console.warn(`Deprecated: ${className}.${name} needs to be migrated to ${name}Async in collection "${self._name}"!`)
      console.warn('=>', location)
    }
    return original.call(self, ...args)
  }
}

// apply patching to Mongo.Collection functions
const mNames = ['insert', 'update', 'remove', 'findOne', 'createIndex']
const mProto = Mongo.Collection.prototype
mNames.forEach(name => patch(mProto, name))

// applying patches Mongo.Cursor functions
const cNames = ['count', 'forEach', 'map', 'fetch']
const cProto = Mongo.Cursor.prototype
cNames.forEach(name => patch(cProto, name))

Start your Meteor project or run your tests and look for the deprecation messages. You can also replace the console.warn with throw new Error to make the tests fail but that will also likely prevent your app from running - this is up to you how to handle things here.

Meteor Methods ☄️

Most of the time you will define Methods and Publications statically on startup. This allows us to create a neat little tool that provides a summary of all our methods and publications.

Consider the following example, which inserts a new document into a Mongo.Collection (similar to what happens in the Meteor tutorials):

// server/main.js

Meteor.methods({
  'createTodo' (text) {
    const userId = this
    const docId = TodosCollection.insert({ text, userId })
    return TodosCollection.findOne(docId)
  }
})

Building on top of the previous section, we continue to extend our migration helper package in order to detect the migration needs for this method. Let's start with Meteor Methods by patching the Meteor.methods function. You can continue in migration-helper.js by simply adding the following code at the end of the file:

// ...continueing in migration-helper.js

const asyncLine = /\s*return Promise.asyncApply\(\(\) => {\n/g

// scans a function body for the above pattern
// to detect async functions
const analyze = ({ name, fn, location, type }) => {
  const source = fn.toString()
  const lines = source.split(byNewline)
  const isAsync = asyncLine.test(lines[1])

  if (!isAsync) {
    console.warn(`Deprecated (${type}): ${name} is not async, consider migrating now.`)
  }
}

What is this actually good for?

Let me explain: In Meteor.js 2.x (using Fibers) any async declared function gets recompiled by the Meteor build tool, transforming async functions into this Promise.asyncApply. It basically connects the given function to a Fiber that keeps track of the execution stack.

In turn - every Method or Publication that is not transformed into this Promise.asyncApply will not contain this pattern on line 1. Thus, we can detect that this function is not async yet.

Let's apply this to Meteor.methods; you can simply continue at the end of the file:

// ...continueing in migration-helper.js

const originalMethods = Meteor.methods

Meteor.methods = options => {
  const location = getLocation()
  const entries = Object.entries(options)
  const type = 'Method'
  entries.forEach(([name, fn]) => {
    analyze({ name, fn, location, type })
  })

  return originalMethods(options)
}

Restarting the project yields to the following output:

W20231102-15:28:12.775(1)? (STDERR) Deprecated (Method): createTodo is not async, consider migrating now.
W20231102-15:28:12.776(1)? (STDERR)   => at module (server/main.js:15:8)

And during tests or runtime we will see a message like this:

W20231102-15:32:31.550(1)? (STDERR) Deprecated: Collection.insert needs to be migrated to insertAsync in collection "todos"!
W20231102-15:32:31.551(1)? (STDERR) => at Collection.Mongo.Collection.<computed> [as insertAsync] (packages/mongo/collection.js:1004:46)
W20231102-15:32:31.560(1)? (STDERR) Deprecated: Collection.findOne needs to be migrated to findOneAsync in collection "todos"!
W20231102-15:32:31.561(1)? (STDERR) => at MethodInvocation.createTodo (server/main.js:26:28)

Both indicating the createTodo method needs some migration efforts. Let's do that:

Meteor.methods({
  'createTodo': async function (text) {
    const userId = this
    const docId = await TodosCollection.insertAsync({ text, userId })
    return TodosCollection.findOneAsync(docId)
  }
})

The warnings are gone and you have safely prepared this method for the major 3.0 update.

Meteor Publications ☄️

If you have already implemented the functions from the Meteor Methods section before then patching publications is as easy as it can get:

const originalPub = Meteor.publish

Meteor.publish = (name, fn) => {
  const location = getLocation()
  const type = 'Publication'
  analyze({ name, fn, location, type })
  return originalPub(name, fn)
}

Note, that publications return cursors using .find which does not need to be migrated to async. Therefore this may create some false-positives for functions that use no other Mongo.Collection or Mongo.Cursor functions that have to be async.

Validated Methods ☄️

A special case is, when using ValidatedMethod via mdg:validated-method. If you don't know the package, I highly suggest to check it out as it's the recommended way (add guide link) to create method with builtin argument validation.

We will not be able to detect non-async methods passed to the ValidatedMethod constructor as with the Meteor.methods pattern, described above. This is, because we basically can't override/monkey-patch an ES6 constructor function the same way we do with any other function.

Fortunately we still have two alternative ways for detection.

Detect using a mixin 🔍

This works by creating a mixin function that detects the options.run function the same way we detect the Meteor.methods. However, there is one disadvantage - you have to assign the mixin to every ValidatedMethod construction options and at that point you'll likely check each method manually.

Therefore, this approach makes only sense if you use a factory-function to create your Methods, which acts as a single point of construction. In that case, it's rather easy to inject the mixin into your validated methods:

export const createMethod = options => {
  options.mixins = options.mixins ?? []
  options.mixins.push(checkAsyncMixin)
  return new ValidatedMethod(options)
}

const checkAsyncMixin = options => {
  const { name, run } = options
  analyze({
    name: name,
    location: getLocation(),
    type: 'ValidatedMethod',
    fn: run
  })
  return options
}

Detect at runtime 🔬

If you don't use a pattern, similar to the above-mentioned factory-function, then you still have one more option here. However, this will only detect the methods at runtime, and you will either have to have a decent test-coverage or other ways to make sure that every of your methods are called.

It works by overriding the _execute prototype method, which will run on every method invocation for methods, declared using ValidatedMethod:

const originalExec = ValidatedMethod.prototype._execute

ValidatedMethod.prototype._execute = function (...args) {
  const self = this

  analyze({
    name: self.name,
    location: getLocation(),
    type: 'ValidatedMethod',
    fn: self.run
  })
  return originalExec.call(self, ...args)
}

Summary and outlook 🔭

I showed in this article how to easily detect parts of your Meteor 2.x server environment that need change in order to be 3.0-ready. I purposefully leaved out the client side as this easily exceeds the scope of a short and simple article.

Since Meteor.js supports many client side libraries and frameworks this may be subject to specific articles for each of them, and I encourage everyone to step up and start writing on their experience
with them.

You can use this code and refactor / modify it to your needs or simply use the package jkuester:migration-helper.
It's listed on Packosphere and available on GitHub, too:

Packosphere Link 📦

packosphere.com

GitHub repository ⌨️

jankapunkt / meteor-migration-helper

Detect which parts of your Meteor.js server environment need to be migrated in your current 2.x code.

Meteor.js Migration Helper

Detect which parts of your Meteor.js server environment need to be migrated in your current 2.x code.

No need to upgrade to 3.0 now to find out, what's still using Fibers.

There is also an article, which covers this packages functionality https://dev.to/jankapunkt/prepare-your-meteorjs-project-for-the-big-30-release-14bf

Installation

$ meteor add jkuester:migration-helper

Now open in your Meteor.js project the file .meteor/packages and move the entry jkuester:migration-helper to the top, in order to also detect dependency packages that still use Fibers.

Run detection

This is a runtime detection. In order to cover all detectable structures you need to either run your Meteor.js application or the tests.

The more your tests cover of your code (test-coverage), the better you will be able to detect these.

Detect validated methods using mixins

This package also provides a mixin to be used with mdg:validated-method

You can import it via

import { checkAsyncMixin } from 'meteor/jkuester:migration-helper'
// ...

…

View on GitHub

---

About me 👋

I regularly publish articles here on dev.to about Meteor.js and JavaScript. Recently I also co-host the weekly Meteor.js community podcast, which contains the newest from Meteor.js and the community.

You can also find (and contact) me on GitHub, Twitter/X and LinkedIn.

If you like what you are reading and want to support me, you can sponsor me on GitHub, send me a tip via PayPal or sponsor me a book form my Amazon wishlist.

Comments

[Jan Dvorak]

For the count, the migration should be more advanced, from documentation:

This method is deprecated since MongoDB 4.0; see Collection.countDocuments and Collection.estimatedDocumentCount for a replacement.

[Jan Küster]

Thanks for the hint, I will update this one

[Carlos Alvidrez]

Trying to use the ValidationMethod override, but stomped with this error... not sure if the "analyze" method requires another import?

Exception while invoking method 'XXX' ReferenceError: analyze is not defined

Thanks!

[Jan Küster]

Did you use the jkuester:migration-helper package or did you used the code from this article?

[Carlos Alvidrez]

Hi! Both... here's the snippet. Puzzled because I read the code and figure out a couple of imports might be required... but the helper still outputs to the console that my method is not async yet.

//import { getLocation, analyze } from 'meteor/jkuester:migration-helper';
import { checkAsyncMixin } from 'meteor/jkuester:migration-helper';
import { getLocation } from 'meteor/jkuester:migration-helper';
import { analyze } from 'meteor/jkuester:migration-helper';

const originalExec = ValidatedMethod.prototype._execute;
ValidatedMethod.prototype._execute = function (...args) {
    const self = this;
    console.log("--> self.name:", self.name);
    //console.log("getLocation():", getLocation());
    analyze({
        name: self.name,
        location: getLocation(),
        type: 'ValidatedMethod',
        fn: self.run
    });
    return originalExec.call(self, ...args);
};

And this is the method code (very simple case):

export const pubVersesPaginationPassageRateLimiter = new ValidatedMethod({
    name: 'pubVersesPaginationPassageRateLimiter',
    validate: new SimpleSchema({
        source: { type: String },
    }).validator(),
    async run({}) {
        return true;
    }
});

Any advice will be appreciated... working hard on refactoring my webapp to migrate to v3 ASAP.

Cheers

[Jan Küster]

Thanks for the examples I will try to reproduce this. In the meantime, there has also been an eslint-plugin being developed by quave, which can also help you to detect parts that need migration (it is specifically designed for this task):

github.com/quavedev/eslint-plugin

Will get back to you when I find the cause for the issue.

[Carlos Alvidrez]

Thank you!

[Carlos Alvidrez]

Also, wanted to pick someone's brain around how to fix these issues... must I find new packages for these? (debugging results from the suggested migration helper package):

Deprecated (Publication): meteor_autoupdate_clientVersions is not async, consider migrating now.
  => at packages/autoupdate/autoupdate_server.js:109:8

Deprecated (Publication): _roles is not async, consider migrating now.
  => at module (packages/alanning:roles/roles/roles_server.js:30:8)

Deprecated (Method): slingshot/uploadRequest is not async, consider migrating now.
  => at packages/edgee_slingshot.js:404:8

Deprecated: Collection.findOne needs to be migrated to findOneAsync in collection "__dummy_coll_4MqSQRvCKueAzPu9m"!
=> at module (packages/mdg:meteor-apm-agent/lib/hijack/meteorx.js:34:12)

[Carlos Alvidrez]

Another one:

Deprecated: Collection.update needs to be migrated to updateAsync in collection "meteor_accounts_loginServiceConfiguration"!
=> at Collection.upsert (packages/mongo/collection.js:785:17)

[Carlos Alvidrez]

Is there a pagination package you'd recommend to use for Meteor 3?
I am using this one but it's not been maintained and JKuster's migration helper reports the underlying methods not being async.

github.com/Kurounin/Pagination/tre...

[Carlos Alvidrez]

I believe this library was causing me trouble as it trips the migration helper... as soon as I removed it, all my methods stopped reporting not being async.

mdg:meteor-apm-agent

@jankapunkt FYI