Mathieu

Posted on Aug 2, 2022 • Originally published at blog.meta-api.io

One step towards the revolution of APIs usage

#showdev #javascript #api #programming

Hey there 👋,

I hope you are doing well in these complicated times.

I’ll try to cheer you up and, maybe, inspire you with this project which is the most ambitious one I’ve started as a side project eight years ago and now a tech startup with a team of 10 people.

Let’s talk about APIs, JavaScript and a platform to bind them all: Meta API.

Description

Meta API is a platform for developers to integrate and automate APIs in the cloud.

It can make you think of a Zapier or a Make (ex-Integromat) for developers, but with major differences:

It’s open to any APIs (thanks to the OpenAPI standard), so you never need to rely on someone to make the API you need available. That also mean you can import your private APIs 😉
It’s a pro-code project, because we trust that code is the most adapted way to create advanced integration and the most efficient way to work with data.

Our goal is also to cover the whole dev cycle, including testing, deployment and monitoring.

Motivation

Everything started eight years ago when I wanted to go on a weekend trip. I’ve lost all my weekend planning this trip (so, no trip) and I was so frustrated about the work required: open hundreds of browser’s tabs, comparing possibilities between Booking, Airbnb, train lines, flights, etc… to find a “great match” satisfying both travel, accomodations and budget.

As an engineer and a web developer, I thought that there is a much more optimize way to solve this kind of problem with an algorithm. So I started to write some code and, the golden way to retrieve all the data is through an API.

After thousands of hours of work, coding, refactor and abstraction, we came (with my co-founder) with this idea of a platform to connect everything, write easily your own algorithm and deploy it to be accessible. Due to the Covid crisis, we dropped the original idea to plan trip and holidays to pivot and focus on B2B needs.

This experience make me believe that Web APIs are one of the most fundamental pillar for Internet and assuring a universal interoperability of all APIs will change the Web forever, switching from closed and competitive ecosystems to a collaborative universe of specialized tools and services. That our final goal: creating a protocol or a new open technology to make all APIs interoperable.

Design

The first design part was to ensure any APIs can be plugged inside the platform, in no time and with the less possible friction. The OpenAPI became quickly an evidence: it’s used by a vast majority of projects and it’s allow to describe any API. That’s our entry point.

The second part was the algorithm: how can we allow our user (and us) to write the algorithm they want, without restriction? Providing a code editor and an engine to run this code safely and isolated was our solution.

Now, with these two major concept, we built a whole interface and automate a lot of work to make it more pleasant and efficient than doing it in your own code editor or servers. We introduce auto-deployment on serverless (for performance and scalability), automatic monitoring, versioning, parallel environment for staging and production, etc…

Step by step, feature by feature, we are making our platform more easy to use and more accessible.

How does it work?

The Spell

The central part of the platform is the Spell where APIs will meet your code.

A Spell example where Stripe’s invoices are pushed to a Google Spreadsheet

The main part is a code editor, where you can write Javascript and Typescript code and use NPM dependencies.

On the left, you can manage your connectors, which are API’s endpoint to retrieve and send actions to the target API.

Each connector has its own configuration panel (on the right) to set up authentication, parameters, etc.

The link between your code and the connectors are done by a snippet of code, looking like this:

/* Start of Stripe connector */
const invoices: Datagetvinvoices = await connectorService.config("6182bd7f00c78e583df01f18", {body: {}});
/* End of Stripe connector */

We automate a lot of boring stuff related to connectors with this snippet and our backend : authentication format (especially OAuth), formatting values, ensuring valid types, etc.

The run

When you hit Run, everything is compiled on our backend, pushed into a dedicated development server (run at the fly on a Docker container) and executed.

We can see the response and all console.log done inside the code.

Example of a Spell run

You can check the data, fix errors, run again, and so on.

It’s fast and almost feel like your local dev environment.

Deploy and use

When your Spell is ready, you can check the changes and deploy it.

A spell, ready to be deployed

We’ll deploy your code inside a Serverless function and give you a unique URL to trigger the Spell like you want:

with a webhook, to automation action in real time
inside your code, with a request to integrate external services
with a scheduler, to periodically process or sync data

Your Spell will become an API of API (so a Meta API 😉)

Monitoring

Once deployed, every request are monitored by our backend, ensuring the code is well executed each time.

We only collect requests metadata and the data you choose to log. Everything is happening in streaming and destroyed after the run.

Seeing some successful requests on my freshly released Spell

If an error happen, a notification is automatically sent to you.

Usage

If you have read the How does it work? part and are a coder, you have figured out that Meta API can solve a lot of developer’s problems.

But to be more practical, here are the two main usage of our users.

Automate APIs

Many of our users want to automate their daily workflows or business processes. From generating invoices to ease their CI/CD workload, any service can be automated.

The best part of this kind of workload is the running done on the cloud, without the need to involve your own servers. You won't have to worry about servers, maintenance, and security.

As a startup, we are our first platform users because we need so much to create this kind of automated process without bloating our code.

Here are some examples of real usage for us:

Invoice generation on Quickbooks, based on our Stripe’s invoices because the Quickbooks’ Stripe integration is broken and can’t manage properly things like multiple tax. A scheduler run it every day at 10am and can be run on demand.
Notify our product team on Discord each time a new release is done on GitHub: we can follow the changes (thanks to webhooks) and inform our users of new features and bug fixes.
Follow our user signup inside Hubspot: each time a user signup, we’ll create a new entry inside Hubspot to follow their usage and see if they expect help. We trigger this Spell directly with a request coming from our backend.
Following our backlog: we use Notion to manage our backlog, follow the dev process and notify the team about pull request taking too much time or feature getting some delay and require help.

Integrate APIs

If you are working on a SaaS platform, you probably already have considered integration with others tools: messaging, payments, CRM, project management tools… Besides creating new features, these integrations will lower your churn and increase your users’ adoption.

Integration implies a data exchange between your platform and a third-party service, with mostly a storage of data in your side.

You can create a unified API, easy to use inside your code and letting all the hard stuff being processed outside.

Here are some real use cases:

Have access to your end-user's Google Calendar (using OAuth) to create and update events for them. This result of adding one button inside the user interface and one line of code to add inside the backend to create and update the events.
Send notification to their Slack, Discord, or Microsoft Teams
Integrate your marketing tools to generate media visuals with Bannerbear and collect campaign data from Google Ads and Facebook Campaigns

Near future

We want to push the dev experience even further. Here is a sneak peek of the features we are working on:

Workspaces to make the platform even more collaborative with the capability to invite external users, create spaces dedicated to subteams, manage permissions…
Runners: our Open Source project to run your Spells inside your own server for powerful integration and zero-trust security
Shared Spells and APIs: we have a great community on our Discord server, and we want to make Spells and APIs even more sharable and interactive.

Final words

Thanks for taking the time to read this. I hope you have found some inspiring thought inside.

I’m eager to have your feedback about our approach and the concept of this platform, either shared opinions or criticisms. You can try the platform and create a free account.