DEV Community

Jan Küster
Jan Küster

Posted on • Edited on

Microservices with Meteor

#meteor #javascript #microservices

Introduction

Meteor is well known for it's full-scale solution to rapidly create JavaScript applications with many flavors (SPA, PWA, Mobile native, Desktop website and more).

If you are totally new to Meteor or you know it "from the old days" then this article may give you a fresh update on what Meteor is nowadays and what it's not:

jankapunkt

Why choose Meteor (or not) for your next project?

Jan Küster ・ Jan 18 '21

#javascript #meteor #stateofjs

Meteor comes with a very detailed and sophisticated developer guide. It guides you through all architectural aspects and provides best-practice suggestions for architecture and design decisions.

However, it does not teach you about how to create Microservices with Meteor. This is because Meteor as a framework is very flexible and to cover every potential architectural decision would go beyond the scope of the guide.

This is why this post is here to guide you through the most important aspects of Microservices with Meteor.

Covered topics

In order to get everyone on board, we will go through the most important aspects to get a running, usable Microservice example:

  • Why Microservices with Meteor
  • How to create a "headless" Nanoservice with Meteor
  • How to create a fullstack Microservice with Meteor
  • Connect apps and service with each other (HTTP / DDP)
  • Security considerations
  • Deployment

All the code is also put in a repository, which I link at the end of the article.

What's not covered

The field of Microservices is very broad. Thus, I want to keep this article focused and only scratch the surface of architectural constrains or things playing a role when communicating between services.

If you are new to Microservices and are interested in learning about them, you may start with some good standard literature:

On language and symbols

I am often switching between I/me, you or we/use and by using those words I am referring to different contexts:

  • I/me - Reflecting my choices, intentions or experience
  • you/yours - Provoking you to think about a certain aspect of the article or topic
  • we/us - Hands-on situation or practical task, where you should think of us as a small team that currently works together
  • 🤓 - These paragraphs add background details for those who exactly want to know what's going on and why. If it's too much information you can skip them for now and read them later.

The context

To make this much more accessible we should think in a concrete use-case for such a service. Let's say we want to implement an online-shop that has some kind of connection to a warehouse.
At the same time there should be a catalog of products, where a person (catalog-manager) can insert and update new product entries.
Finally, the availability of a product should be updated, based on the physical availability in the warehouse.

Derived from this we can split our application and services into the following:

  • shop (application)
  • catalog-service (Microservice)
  • warehouse-status-service (Nanoservice)

The architecture could look like this:

Example arcitecture of our catalog service

Why Microservices with Meteor

This should always be the very first question: why using a certain tech or stack to solve a specific problem. If you can't answer this question then you may rethink your decision. Here are some examples on why I chose Meteor:

Well-established stack

Meteor offers a full stack out of the box. It brings bundling, package management (NPM/Meteor packages), transport (server/client) and zero config required. Additionally it fully supports TypeScript, as well as the most popular frontends like React, Angular, Vue and Svelte (plus it's own client engine "Blaze").

If we can control the full stack with nearly no integration effort we can easily create a new Microservice in a few steps.

One language to rule them all

Furthermore, since Meteor uses one language (JavaScript) for this whole stack we can easily onboard newcomers into a project and assign them one service. This maximizes the focus, because there is one language, one framework and one Microservice to cover.

DB-Integration

As already mentioned Meteor comes with a tight integration for MongoDB. While this is often criticized for being less flexible, it actually allows us to easily implement "data ownership", where services have their own database: even if we have one MongoDB provider we can assign every service a database, simply by putting the MONGO_URL to the application environment variables with the respective database name. This allows us to keep services separated not only in terms of code but also in terms of data.

Time-to-market

The step from development to deployment is very fast, since there is no bundler, minifier, babel and whatnot to be configured. It's all there already, so you just need to deploy in one step to Galaxy (Meteor-optimized hosting from the official Developers) or use Meteor-Up to deploy to any other service provider you can imagine.

This all leads to a very short time-to-market and allows you to rapidly add new Microservices to your infrastructure or update them without fiddling into complex configurations.

For the next steps we will get our hands-on Meteor and create our own Microservice example in about 15 Minutes.

Install Meteor

If you haven't installed Meteor on your machine, just follow the steps from the official install website:

curl https://install.meteor.com/ | sh
Enter fullscreen mode Exit fullscreen mode

or on Windows:

npm install -g meteor
Enter fullscreen mode Exit fullscreen mode

There is also another post that can help you decide, which frontend framework you might use for your next Meteor apps:

jankapunkt

Meteor 5 minutes setup for newcomers

Jan Küster ・ Jan 14 '21

#javascript #beginners #meteor

How to create a "headless" Nanoservice with Meteor

Step 1 - Create the most minimal Meteor app

For our warehouse we will create the most minimal Meteor app possible. In order to do so, let's create a bare project:

$ meteor create --bare warehouse
$ cd warehouse
$ meteor npm install
Enter fullscreen mode Exit fullscreen mode

This warehouse service now contains no code and only a very minimal list of Meteor packages (see .meteor/packages in the warehouse project):

meteor-base@1.5.1             # Packages every Meteor app needs to have
mobile-experience@1.1.0       # Packages for a great mobile UX
mongo@1.12.0                   # The database Meteor supports right now
static-html             # Define static page content in .html files
reactive-var@1.0.11            # Reactive variable for tracker
tracker@1.2.0                 # Meteor's client-side reactive programming library

standard-minifier-css@1.7.3   # CSS minifier run for production mode
standard-minifier-js@2.6.1    # JS minifier run for production mode
es5-shim@4.8.0                # ECMAScript 5 compatibility for older browsers
ecmascript@0.15.2              # Enable ECMAScript2015+ syntax in app code
typescript@4.3.2              # Enable TypeScript syntax in .ts and .tsx modules
shell-server@0.5.0            # Server-side component of the `meteor shell` command
Enter fullscreen mode Exit fullscreen mode

Well, we can squeeze this even further! This service is "headless" (contains no client-side code), thus we can remove a few unnecessary packages here:

$ meteor remove mobile-experience static-html reactive-var tracker standard-minifier-css es5-shim shell-server
Enter fullscreen mode Exit fullscreen mode

Now this is the smallest possible set of packages for our headless nano-service:

meteor-base@1.5.1             # Packages every Meteor app needs to have
mongo@1.12.0                   # The database Meteor supports right now

standard-minifier-js@2.6.1    # JS minifier run for production mode
ecmascript@0.15.2              # Enable ECMAScript2015+ syntax in app code
typescript@4.3.2              # Enable TypeScript syntax in .ts and .tsx modules
Enter fullscreen mode Exit fullscreen mode

Since our warehouse service will make some HTTP requests to the catalog service (to update some product availability), we add one more package here:

$ meteor add http
Enter fullscreen mode Exit fullscreen mode

🤓 Why http and not fetch

Note: we could instead use the fetch package, which is basically a wrapper for node-fetch but I love the ease of use of http, which is why I chose it here.

Step 2 - Implement the warehouse service

First, we create a new main server file:

$ mkdir -p server
$ touch ./server/main.js
Enter fullscreen mode Exit fullscreen mode

Then we add the following code:

import { Meteor } from 'meteor/meteor'
import { HTTP } from 'meteor/http'

// fake data for some products
const productIds = [
  '012345',
  'abcdef',
  'foobar'
]

const randomProductId = () => productIds[Math.floor(Math.random() * 3)]
const randomAvailable = () => Math.random() <= 0.5

Meteor.startup(() => {
  Meteor.setInterval(() => {
    const params = {
      productId: randomProductId(),
      available: randomAvailable()
    }

    const response = HTTP.post('http://localhost:3000/warehouse/update', { params })

    if (response.ok) {
      console.debug(response.statusCode, 'updated product', params)
    } else {
      console.error(response.statusCode, 'update product failed', params)
    }
  }, 5000) // change this value to get faster or slower updates
})
Enter fullscreen mode Exit fullscreen mode

What's happening here?

When the application start has completed (Meteor.startup) we want to safely execute an interval (Meteor.setInterval), where we call our remote endpoint http://localhost:3000/warehouse/update with some productId and available parameters.

That's it.

🤓 More background

The product id's are random from a fixed set of hypothetical ids - we assume these ids exist. In a real service setup you might either want to synchronize the data between warehouse and catalog or - as in this example - use an implicit connection, based on the productId, which requires the product manager to enter when updating the catalog.

With the first example you ensure a high data integrity, while you also introduce a soft step towards coupling the services. The second option is free of any coupling but it requires the catalog to contain the products before the warehouse can update them.

Step 3 - Run the service

Finally, let's run the warehouse on port 4000:

$ meteor --port=4000
Enter fullscreen mode Exit fullscreen mode

We can ignore the error messages for now, since our catalog service is not established yet. It will be the subject of focus in the next section.

How to create a fullstack Microservice with Meteor

Step 1 - Create a normal Meteor app

A normal app? Yes, a Microservice can be an app that covers the full stack! The scope is not architectural but domain driven.

Therefore let's go back to our project root and create a new Meteor app:

$ cd .. # you should be outside of warehouse now
$ meteor create --blaze catalog-service
$ cd catalog-service
$ meteor npm install --save bcrypt body-parser jquery mini.css simpl-schema
$ meteor add reactive-dict accounts-password accounts-ui aldeed:autoform communitypackages:autoform-plain leaonline:webapp jquery@3.0.0!
Enter fullscreen mode Exit fullscreen mode

🤓 What are these packages for?

name description
brypt Used with accounts for hashing passwords
body-parser Used to proper decode json from post request body that are not using application/x-www-form-urlencoded
jquery Makes life easier on the client
mini.css Minimal css theme, optional
simpl-schema Used by aldeed:autoform to create forms from schema and validate form input
reactive-dict Reactive dictionary for reactive states
accounts-password Zero config accounts system with passwords
accounts-ui Mock a register/login component for fast and easy creation of accounts
aldeed:autoform Out-of-the-box forms from schemas
communitypackages:autoform-plain Plain, unstyled forms theme
leaonline:webapp Drop-in to enable body-parser with webapp
jquery@3.0.0! Force packages to use latest npm jquery

Step 2 - Create the backend

For our backend we mostly need a new Mongo Collection that stores our products and some endpoints to retrieve them (for the shop) and update their status (for the warehouse).

Step 2.1 - Create products

First we create a new Products collection that we will use in isomoprhic fashion on server and client:

$ mkdir -p imports
$ touch ./imports/Products.js
Enter fullscreen mode Exit fullscreen mode

The Products.js file contains the following

import { Mongo } from 'meteor/mongo'

export const Products = new Mongo.Collection('products')

// used to automagically generate forms via AutoForm and SimpleSchema
// use with aldeed:collection2 to validate document inserts and updates
Products.schema = {
  productId: String,
  name: String,
  description: String,
  category: String,
  price: Number,
  available: Boolean
}
Enter fullscreen mode Exit fullscreen mode

If you are too lazy to enter the products by yourself (as I am) you can extend this file by the following code to add some defaults:


const fixedDocs = [
  {
    productId: 'foobar',
    name: 'Dev Keyboard',
    description: 'makes you pro dev',
    category: 'electronics',
    price: 1000,
    available: true
  },
  {
    productId: '012345',
    name: 'Pro Gamepad',
    description: 'makes you pro gamer',
    category: 'electronics',
    price: 300,
    available: true
  },
  {
    productId: 'abcdef',
    name: 'Pro Headset',
    description: 'makes you pro musician',
    category: 'electronics',
    price: 800,
    available: true
  }
]

// to make the start easier for you, we add some default docs here
Meteor.startup(() => {
  if (Products.find().count() === 0) {
    fixedDocs.forEach(doc => Products.insert(doc))
  }
})
Enter fullscreen mode Exit fullscreen mode

Step 2.2 - Create HTTP endpoint for warehouse

Now we import Products in our server/main.js file and provide the HTTP POST endpoint that will later be called by the warehouse nanoservice. Therefore, we remove the boilerplate code from server/main.js and add our endpoint implementation here:

import { Meteor } from 'meteor/meteor'
import { WebApp } from 'meteor/webapp'
import bodyParser from 'body-parser'
import { Products } from '../imports/Products'

const http = WebApp.connectHandlers

// proper post body encoding
http.urlEncoded(bodyParser)
http.json(bodyParser)

// connect to your logger, if desired
const log = (...args) => console.log(...args)

// this is an open HTTP POST route, where the
// warehouse service can update product availability
http.use('/warehouse/update', function (req, res, next) {
  const { productId, available } = req.body
  log('/warehouse/update', { productId, available })

  if (Products.find({ productId }).count() > 0) {
    const transform = {
      productId: productId,
      available: available === 'true' // http requests wrap Boolean to String :(
    }

    // if not updated we respond with an error code to the service
    const updated = Products.update({ productId }, { $set: transform })
    if (!updated) {
      log('/warehouse/update not updated')
      res.writeHead(500)
      res.end()
      return
    }
  }

  res.writeHead(200)
  res.end()
})
Enter fullscreen mode Exit fullscreen mode

🤓 More background

For those of you who look for an express route - Meteor comes already bundled with connect, which is a more low-level middleware stack. It's express compatible but works perfect on it's own.
Furthermore, our endpoint skips any updates on products that are not found. In reality we might return some 404 response but this will be up to your service design.
Note, that even with body-parser we still need to parse the Boolean values, that have been parsed to strings during the request ("true" and "false" instead of true and false).x

Step 2.3 - Create DDP endpoints for the shop

In order to provide some more powerful service with less coding effort we actually also want to have some Data available the Meteor way.
Our shop will then be able to subscript to data and "automagically" resolve the response into a client-side Mongo Collection.

Extend your server/main.js file by the following code:

// We can provide a publication, so the shop can subscribe to products
Meteor.publish('getAvailableProducts', function ({ category } = {}) {
  log('[publications.getAvailableProducts]:', { category })
  const query = { available: true }

  if (category) {
    query.category = category
  }

  return Products.find(query)
})

// We can provide a Method, so the shop can fetch products
Meteor.methods({
  getAvailableProducts: function ({ category } = {}) {
    log('[methods.getAvailableProducts]:', { category })
    const query = { available: true }

    if (category) {
      query.category = category
    }

    return Products.find(query).fetch() // don't forget .fetch() in methods!
  }
})
Enter fullscreen mode Exit fullscreen mode

That's all for our backend right now. We will not implement any authentication mechanisms as this will totally blow the scope of this article.

In the next step we will create a minimal frontend for the catalog manager, including a login and a form to insert new products.

Step 3 - Create the frontend

Step 3.1 - Add HTML Templates

The frontend code is located in the client folder. First, let's remove the boierplate code from client/main.html and replace it with our own:

<head>
    <title>catalog-service</title>
</head>

<body>
<h1>Catalog service</h1>

{{#unless currentUser}}
    {{> loginButtons}}
{{else}}
    {{> products}}
{{/unless}}
</body>

<template name="products">
    <ul>
    {{#each product in allProducts}}
        <li>
            <div>
                {{product.productId}} - {{product.name}}
                {{#if product.available}})(available){{else}}(not available){{/if}}
            </div>
            <div>{{product.description}}</div>
        </li>
    {{else}}
        <li>No products yet!</li>
    {{/each}}
    </ul>

    <button class="addProduct">Add product</button>

    {{#if addProduct}}
        {{> quickForm id="addProductForm" schema=productSchema type="normal"}}
    {{/if}}
</template>
Enter fullscreen mode Exit fullscreen mode

🤓 What's going on here?

This template renders all our products in a list (ul) and also displays their current status. If the user is logged in. Otherwise it renders the login screen. If the user clicks on the "Add product" button, she can acutally enter new products using the quickForm generated from the Product.schema that is passed by the productSchema Template helper.

Step 3.2 - Add Template logic

The above Template code relies on some helpers and events, which we implement in client/main.js:

/* global AutoForm */
import { Template } from 'meteor/templating'
import { Tracker } from 'meteor/tracker'
import { ReactiveDict } from 'meteor/reactive-dict'
import { Products } from '../imports/Products'
import SimpleSchema from 'simpl-schema'
import { AutoFormPlainTheme } from 'meteor/communitypackages:autoform-plain/static'
import 'meteor/aldeed:autoform/static'
import 'mini.css/dist/mini-dark.css'
import './main.html'

// init schema, forms and theming
AutoFormPlainTheme.load()
AutoForm.setDefaultTemplate('plain')
SimpleSchema.extendOptions(['autoform'])

// schema for inserting products,
// Tracker option for reactive validation messages
const productSchema = new SimpleSchema(Products.schema, { tracker: Tracker })

Template.products.onCreated(function () {
  const instance = this
  instance.state = new ReactiveDict()
})

Template.products.helpers({
  allProducts () {
    return Products.find()
  },
  productSchema () {
    return productSchema
  },
  addProduct () {
    return Template.instance().state.get('addProduct')
  }
})

Template.products.events({
  'click .addProduct' (event, templateInstance) {
    event.preventDefault()
    templateInstance.state.set('addProduct', true)
  },
  'submit #addProductForm' (event, templateInstance) {
    event.preventDefault()

    const productDoc = AutoForm.getFormValues('addProductForm').insertDoc
    Products.insert(productDoc)

    templateInstance.state.set('addProduct', false)
  }
})
Enter fullscreen mode Exit fullscreen mode

🤓 What's going on here?

At first we initialize the AutoForm that will render an HTML form, based on Products.schema.

Then we create a new state variable in the Template.products.onCreated callback. This state only tracks, whether the form is active or not.

The Template.products.helpers are reactive, since they are connected to reactive data sources (Products.find and Template.instance().state.get).

The Template.products.events simply handle our buttons clicks to switch the state or insert a new Product into the collection.

Step 4 - Run the service

Now with these few steps we created a full-working Microservice. Let's run it on localhost:3000 (we agreed in warehouse to use this port, use Meteor.settings to easily configure those dynamically).

$ meteor
Enter fullscreen mode Exit fullscreen mode

Then open your browser on localhost:3000 and register a new user / log in with the user and with the warehouse service update the availability status of our products. 🎉

Create the shop app

Now the last part of our hands-on is to create a minimal shop that uses Meteor's DDP connection to subscribe to all available products LIVE!

The shop itself doesn't contain any backend code so it won't take much time to get it running:

$ cd .. # you should be outside catalog-service now
$ meteor create --blaze shop
$ cd shop
$ meteor npm install --save jquery mini.css
Enter fullscreen mode Exit fullscreen mode

Then, as with catalog-service, replace the client/main.html with our own template code:

<head>
    <title>shop</title>
</head>

<body>
<h1>Welcome to our Shop!</h1>

{{> products}}
</body>

<template name="products">

    <h2>Subscribed products (live)</h2>
    <ul>
        {{#each product in subscribedProducts}}
            <li>{{product.name}}</li>
        {{else}}
            <li>Currently no products available</li>
        {{/each}}
    </ul>

    <h2>Fetched products (not live)</h2>
    <ul>
        {{#each product in fetchedProducts}}
            <li>{{product.name}}</li>
        {{else}}
            <li>Currently no products available</li>
        {{/each}}
    </ul>
</template>
```



Do the same with `client/main.js`:



```js
import { Template } from 'meteor/templating'
import { Mongo } from 'meteor/mongo'
import { ReactiveVar } from 'meteor/reactive-var'
import { DDP } from 'meteor/ddp-client'
import 'mini.css/dist/mini-dark.css'
import './main.html'

// at very first we establish a connection to our catalog-service
// in a real app we would read the remote url from Meteor.settings
// see: https://docs.meteor.com/api/core.html#Meteor-settings
const remote = 'http://localhost:3000'
const serviceConnection = DDP.connect(remote)

// we need to pass the connection as option to the Mongo.Collection
// constructor; otherwise the subscription mechanism doesn't "know"
// where the subscribed documents will be stored
export const Products = new Mongo.Collection('products', {
  connection: serviceConnection
})

Template.products.onCreated(function () {
  // we create some reactive variable to store our fetch result
  const instance = this
  instance.fetchedProducts = new ReactiveVar()

  // we can't get our data immediately, since we don't know the connection
  // status yet, so we wrap it into a function to be called on "connected"
  const getData = () => {
    const params = { category: 'electronics' }

    // option 1 - fetch using method call via remote connection
    serviceConnection.call('getAvailableProducts', params, (err, products) => {
      if (err) return console.error(err)

      // insert the fetched products into our reactive data store
      instance.fetchedProducts.set(products)
    })

    // options 2 - subscribe via remote connection, documents will be
    // added / updated / removed to our Products collection automagically
    serviceConnection.subscribe('getAvailableProducts', params, {
      onStop: error => console.error(error),
      onReady: () => console.debug('getAvailableProducts sub ready')
    })
  }

  // we reactively wait for the connected status and then stop the Tracker
  instance.autorun(computation => {
    const status = serviceConnection.status()
    console.debug(remote, { status: status.status })

    if (status.connected) {
      setTimeout(() => getData(), 500)
      computation.stop()
    }
  })
})

Template.products.helpers({
  subscribedProducts () {
    return Products.find({ available: true })
  },
  fetchedProducts () {
    return Template.instance().fetchedProducts.get()
  }
})
```



Now run the app on a different port than 3000 or 4000 and see the avialable products getting magically appear and the non-available ones disappear:



```bash
$ meteor --port=5000
```



We have finished our example project :tada:

**:nerd_face: What's going on here?**

The shop uses a DDP-connection to the running `catalog-service` app and subscribes to the publication we created in [Step 2.3](#create-ddp). Since we add this connection the client Mongo Collection, Meteor knows that the received documenmts have to be placed in this collection. Since queries on the client are reactive our Template engine detects changes of these updates and re-renders, based on the new data.

## Security considerations

We have created some services that communicate with each other by given endpoints. However, these services do neither verify the integrity of the data nor authenticate the source of the requests. This is an advanced topic and may be covered in future articles.

Also note, that the `catalog-service` contains the `autoupdate` package to automatically return any data to any client and the `insecure` package, allowing client-side inserts to be synchronized to the server collection.
These packages are super nice for mocking new prototypes of projects but **you should remove them and implement authentication and verification procedures**.

Many of these topics are covered in the [Meteor guide's security section](https://guide.meteor.com/security.html).

## Deployment

The deployment of these apps is a topic for itself. With more services added to the infrastructure the complexity of deployment increases, too.

In general you can rely on Meteor Software's Galaxy solution, which allows you to deploy your apps and services in one step. It also deploys them on a Meteor-optimized AWS configuration and brings APM tools out-of-the-box.

If you run your own infrastructure or want to use a different provider then you may check out [Meteor-up](https://meteor-up.com/), which allows you to deploy to any server in one step with a few configurations added to a JSON file.

In general you should read on the [deployment guide](https://guide.meteor.com/deployment.html) which covers both solutions and many more topics, like settings files, CDN or SEO.

## Summary and outlook

This article was a short introduction to Microservices with Meteor and should provide enough insights to get you something running.

From here you can extend the example or create your own ones. Notice, that security measures were not part of the article and should therefore be taken seriously, before getting your services out in the wild.

## Further resources

All the code of the hands-on is located on this repository:



  

    

      GitHub logo
      
        jankapunkt
       / 
        microservices-with-meteor
      
    

    

      An example setup to show how to use Microservices with Meteor
    

  

  

    






  



  
View on GitHub







More of my articles on Meteor:



Beginners





Advanced





<hr>

I regularly publish articles here on dev.to about Meteor and JavaScript. If you like what you are reading and want to support me, you can send me a tip via PayPal.



You can also find (and contact) me on GitHub, Twitter and LinkedIn.



Keep up with the latest development on Meteor by visiting &utm_medium=online&utm_campaign=Q2-2022-Ambassadors">their blog and if you are the same into Meteor like I am and want to show it to the world, you should check out the &utm_medium=online&utm_campaign=Q2-2022-Ambassadors">Meteor merch store.
Enter fullscreen mode Exit fullscreen mode

Top comments (1)

Collapse
 
focus97 profile image
Michael Lee •

Just want to express appreciation for this great, detailed write-up. I have two Meteor apps that connect to the same db (one is for all users while the other is limited to administrators), and your article provides more ideas for how data could be shared across apps.

Read next

hamzakhan profile image

🛠️ React Server Components vs. Client Components: When to Use Which? ⚡

Hamza Khan -

harshalshah26 profile image

MarkoJS Frontend Development with Generative AI: A Modern Approach

Harshal Shah -

imsushant12 profile image

Logging and Monitoring in Node.js: Best Practices

Sushant Gaurav -

midhul profile image

Handling Async Errors in JavaScript: A Quick Guide

Midhul P -