DEV Community: Jan Vlnas

Get started with Greenhouse APIs: Overview and authentication

Jan Vlnas — Thu, 11 May 2023 12:12:08 +0000

Our goal at Superface is to simplify API integrations, so you can focus on building your app instead of reading the API docs. We’ve built ready-made integrations for Greenhouse, and in this article we’re sharing what we’ve learned through the process.

The first step to integrating any API is obtaining access. When it comes to Applicant Tracking Systems (ATS), the API for Greenhouse is among the most developer-friendly, with useful and straightforward documentation.

Still, there are six separate APIs for Greenhouse, some of them with overlapping concerns (such as Harvest and Job Board APIs). We have prepared a little crash course on Greenhouse APIs with pointers on when to use which one, how to obtain credentials for each, and how to authenticate your API calls. You will also find examples in JavaScript, which should work in all modern runtimes (particularly Node.js version 18 and newer and Deno).

This article focuses on Greenhouse Recruiting APIs. There are also APIs for Greenhouse Onboarding and Assessment. We will cover these in future posts.

Summary

The Job Board API is intended for building custom careers pages.
- It’s publicly accessible without authentication, cached and not rate limited.
- It only uses HTTP Basic authentication for submitting candidates through the API.
The Harvest API provides full access to Greenhouse Recruiting data.
- It’s useful for building advanced automation and internal productivity tools.
- It requires HTTP Basic authentication using API keys with granular permissions, and an On-Behalf-Of header for auditing of write operations.
- The number of requests is limited within 10-second windows.
The Candidate Ingestion API is intended for recruiting partners, like agencies and job portals.
- Access is authenticated either with HTTP Basic authentication (with API key provided by Greenhouse customer), or OAuth 2.0 with granular scopes.
- Requests authenticated by HTTP Basic require an On-Behalf-Of header, which identifies the Greenhouse user and sets the integration’s permissions.

Job Board API for custom careers pages

You can use the Job Board API to build a custom careers page for your company. It provides data about published jobs and the company hierarchy (offices and departments). Since job boards only work with published and public data, you don’t need any special credentials for read access – only a “board token” which corresponds to the URL path of a job board (and can be customized).

For example, if the job board is accessible on the URL https://boards.greenhouse.io/acme, the board_token is acme. Anyone can access the respective API endpoints, for example https://boards-api.greenhouse.io/v1/boards/acme/jobs to list published jobs.

The only exception is posting job applications, which requires an API key. Greenhouse recommends using their embedded application form, but if you decide to build one on your own, you will need to create a Job Board API key, which you’ll use in HTTP Basic authentication as the username, with no password. For example, in JavaScript with Node.js (v18+) or Deno, you can submit an application as follows:

// Set according to your Job Board settings
const JOB_BOARD_TOKEN = `acmeinc`;
// Create Job Board key in Configure > Dev Center > API Credentials
const GREENHOUSE_API_KEY = `2f11da80ea73b20b4d15bfab0ee73257-1`;
// ID of the job where the application is submitted
const JOB_ID = '4043584006';

const CANDIDATE_DATA = {
  first_name: 'Jane',
  last_name: 'Doe',
  email: 'j.doe@example.com',
  phone: '12345678',
  data_compliance: {
    gdpr_consent_given: true,
  },
};

// base64 encode credentials
const basicAuth = btoa(`${GREENHOUSE_API_KEY}:`);

fetch(
  `https://boards-api.greenhouse.io/v1/boards/${JOB_BOARD_TOKEN}/jobs/${JOB_ID}`,
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Basic ${basicAuth}`,
    },
    body: JSON.stringify(CANDIDATE_DATA),
  }
)
  .then((response) => {
    if (response.ok) {
      return response.json();
    } else {
      throw new Error(
        `Unexpected response from the server: ${response.status}`
      );
    }
  })
  .then((responseData) => {
    console.log('Application submitted!', responseData);
  })
  .catch((error) => {
    console.error('Error submitting application:', error);
  });

Harvest API for full access to recruiting data

A step above the public Job Board API is the Harvest API, which provides access to “most” Greenhouse Recruiting data. This API gives you full access to read and modify candidates, interviews, jobs, and various other organization’s resources.

Harvest API keys have granular permissions. You can choose what API endpoints and operations are available, so you can limit the access scope (and potential security risks) of your application.

Similarly to the Job Board API, Harvest API keys are used with HTTP Basic authentication, where the username is the API key and the password remains empty.

In the following JavaScript example, we’re listing all candidates from the Harvest API using the GET Candidates endpoint. The pagination is handled through an async iterable. Each API response has a Link header for navigating to the next page of results, so we’re parsing the header in the sample code:

// Create Harvest API key in Configure > Dev Center > API Credentials
const GREENHOUSE_API_KEY = `c8ee19deadbeef5466d92e643916316-2`;

// base64 encode credentials
const basicAuth = btoa(`${GREENHOUSE_API_KEY}:`);

// Parse Link header:
// Example: <https://harvest.greenhouse.io/v1/candidates?page=2&per_page=2>; rel="next", <https://harvest.greenhouse.io/v1/candidates?page=474&per_page=2>; rel="last"
function getNextPageUrl(linkHeader) {
  if (!linkHeader) {
    return '';
  }
  const links = linkHeader.split(',');
  const nextLink = links.find((link) => link.includes('rel="next"'));
  if (!nextLink) {
    return '';
  }
  const nextUrl = nextLink.match(/<(.+)>/)[1];
  return nextUrl;
}

async function* fetchCandidatesPage() {
  // TIP: add ?per_page=1 to test iteration or ?per_page=500 to get a maximum of results
  let nextPageUrl = 'https://harvest.greenhouse.io/v1/candidates';

  while (nextPageUrl) {
    const response = await fetch(nextPageUrl, {
      headers: {
        Authorization: `Basic ${basicAuth}`,
      },
    });
    if (response.ok) {
      nextPageUrl = getNextPageUrl(response.headers.get('Link'));
      const data = await response.json();
      yield data;
      console.log(nextPageUrl);
    } else {
      throw new Error(
        'Unexpected response from the server: ',
        await response.text()
      );
    }
  }
}

async function iterateCandidates() {
  let page = 0;
  for await (const candidates of fetchCandidatesPage()) {
    console.log(`### Listing candidates, page ${page}`);
    console.log(candidates);
    page++;
  }
}

iterateCandidates();

Since the API key is global, there is no straightforward way to identify who performed which operations through the API. Therefore, for auditing purposes, write operations (creating, updating, and deleting resources) require an On-Behalf-Of HTTP header containing the Greenhouse ID of the user performing the operation. Your application can get the ID of the user from the GET List Users endpoint. If you know the email of the user logged into your app, you can use the email query parameter to obtain the user’s Greenhouse record, including their ID.

Job Board API vs. Harvest API

The Job Board API is clearly a subset of the Harvest API, so you may be asking why should you deal with the Job Board API at all?

This depends on the type of application you’re building. The Job Boards API is mostly useful for custom career sites, so if you’re building that, consider the following advantages:

Security: Since the Job Board API is limited only to public data such as published job posts, there’s no risk of leaking sensitive information (for example if the API key is compromised).
Rate Limits: The Job Board API is heavily cached and there are no hard rate limits, while requests to the Harvest API are throttled within a 10-second window.
Simplicity: Since the Job Board API is publicly accessible, you can use it with purely client-side applications. For the Harvest API, you will need to implement a backend to keep the API key secure and to filter out internal data.

On the other hand, the Job Board API doesn’t expose some data, which may be helpful for building custom integrations. Particularly lacking are external IDs of offices and departments (useful for mapping office or department descriptions to an external CMS), or the time when a job was first published (for displaying jobs which were recently added, not updated).

Candidate Ingestion API for sourcing partners

The Candidate Ingestion API is intended for sourcing partners, like external job portals and agencies. It provides limited access to jobs and candidates, as well as the ability to post new candidates.

Similarly to the Harvest API, the Candidate Ingestion API requires HTTP Basic authentication with the API key as the username, an empty password, and the On-Behalf-Of header. However, the On-Behalf-Of must be set to the user’s e-mail. The permissions of the integration are limited to the user’s role. Typically, you will have a dedicated “service account” for the integration.

In this JavaScript sample, we’re listing jobs using the Candidate Ingestion API

// Create Candidate Ingestion API key in Configure > Dev Center > API Credentials; in Partners, select Resource (dev)
const GREENHOUSE_API_KEY = `321a2ff8cdcdeadbeefd372a9c1a69e9-3`;
// Your email or for a dedicated service account
const ON_BEHALF_OF = 'demo@example.com';

// base64 encode credentials
const basicAuth = btoa(`${GREENHOUSE_API_KEY}:`);

fetch(`https://api.greenhouse.io/v1/partner/jobs`, {
  headers: {
    Authorization: `Basic ${basicAuth}`,
    'On-Behalf-Of': ON_BEHALF_OF,
  },
})
  .then((response) => {
    if (response.ok) {
      return response.json();
    } else {
      throw new Error(
        `Unexpected response from the server: ${response.status}`
      );
    }
  })
  .then((responseData) => {
    console.log(responseData);
  })
  .catch((error) => {
    console.error(error);
  });

The Candidate Ingestion API also provides OAuth 2.0 authorization, with granular scopes for viewing jobs, candidates, and creating candidates. Consumer key and secret are provided by Greenhouse. The access token is bound to the user who authorized the application, and therefore the On-Behalf-Of header is not needed.

Time to build the integration

Picking an API and getting the right API key is just the start. Now you’ll need to read the documentation, figure out the resources, properties, API calls… And once you build the integration, you also need to keep checking the API for changes and breakages.

Maybe there’s a better way. We've already went through multiple ATSs and distilled their APIs into ready-made ATS connectors, so you don’t need to read the docs and can focus on building your app instead.

With Superface, you’ll get unified integration logic which shields your application from API changes, and provides enhanced monitoring. Our SDK provides direct, proxy-less integration with the external API, so it’s faster and respects the privacy of your candidates. And you’re not reliant on our ready-made connectors – you can modify the integration logic to suit your needs, and build your own connectors.

If that sounds interesting, take a look at our Greenhouse ATS integrations – but we also provide other integrations, like geolocation, sending emails, Slack, and more. In case you have any questions or troubles with Greenhouse (or any other) API, meet us on our Discord server.

We’re posting more tutorials for APIs regularly. To stay in touch, subscribe to our newsletter or follow us here on DEV, LinkedIn, or Twitter.

LinkedIn's New Posts API: The Good, The Bad, and The Ugly

Jan Vlnas — Tue, 11 Apr 2023 09:00:00 +0000

Last summer, LinkedIn announced new API versioning and plans to migrate existing API endpoints to the new versioning scheme along with some other improvements. The first set of endpoints to be migrated by LinkedIn were Posts API, responsible for working with users' and business profiles' posts.

There was also a deadline: by February 2023, existing API users must migrate to the new Posts API and the old endpoints will stop functioning. That gave integration partners around 8 months for migration, but apparently that was not sufficient. So on the last day of February, LinkedIn announced a deadline extension to June 30.

Since I've migrated LinkedIn's integration through Superface, I wrote down a few notes about the overall experience and frustrations with LinkedIn's new API and this particular deprecation.

The Good Parts

The API is better

I used to show the LinkedIn API as an example of poor API design. There were three endpoints for handling users' and organizations' posts (UGC Posts, Shares, and Posts API which was in beta for a long time), with seemingly overlapping features.
With weird, ad-hoc syntax for field projections and annoyingly long property names, it wasn't the most pleasant API to work with.

I'll have to find another poorly designed API now because the new Posts API is definitely an overall improvement. Deeply nested structures with confusing properties and arbitrary nesting of arrays – it's all gone.

To illustrate the difference, here is the same post as represented by the legacy ugcPosts API and the new versioned Posts API:

Post from ugcPosts API (legacy)

{
  "lifecycleState": "PUBLISHED",
  "specificContent": {
    "com.linkedin.ugc.ShareContent": {
      "shareCommentary": {
        "inferredLocale": "en_US",
        "attributes": [],
        "text": "Don't forget the image."
      },
      "media": [
        {
          "description": {
            "attributes": [],
            "text": "Image"
          },
          "media": "urn:li:digitalmediaAsset:D4E10AQE71V5w_-aalA",
          "thumbnails": [],
          "overlayMetadata": {
            "tapTargets": [],
            "stickers": [],
            "overlayTexts": []
          },
          "status": "READY"
        }
      ],
      "shareFeatures": {
        "hashtags": []
      },
      "shareMediaCategory": "IMAGE"
    }
  },
  "visibility": {
    "com.linkedin.ugc.MemberNetworkVisibility": "PUBLIC"
  },
  "created": {
    "actor": "urn:li:person:bDTsVFtMTq",
    "time": 1679663763610
  },
  "author": "urn:li:organization:2414183",
  "clientApplication": "urn:li:developerApplication:208506072",
  "versionTag": "0",
  "id": "urn:li:share:7045020441609936898",
  "firstPublishedAt": 1679663764088,
  "lastModified": {
    "actor": "urn:li:csUser:7",
    "time": 1679663764133
  },
  "distribution": {
    "externalDistributionChannels": [],
    "distributedViaFollowFeed": true,
    "feedDistribution": "MAIN_FEED"
  },
  "contentCertificationRecord": "{\"originCountryCode\":\"nl\",\"modifiedAt\":1679663763588,\"spamRestriction\":{\"classifications\":[],\"contentQualityClassifications\":[],\"systemName\":\"MACHINE_SYNC\",\"lowQuality\":false,\"contentClassificationTrackingId\":\"F00EA9A4CF8AA4C780241D4CE87D5E87\",\"contentRelevanceClassifications\":[],\"spam\":false},\"contentHash\":{\"extractedContentMd5Hash\":\"7871A7EF3ADBC18955073E68D2203F27\",\"lastModifiedAt\":1679663763587}}"
}

Post from the Posts API (new, versioned)

{
  "isReshareDisabledByAuthor": false,
  "createdAt": 1679663763610,
  "lifecycleState": "PUBLISHED",
  "lastModifiedAt": 1679663764133,
  "visibility": "PUBLIC",
  "publishedAt": 1679663764088,
  "author": "urn:li:organization:2414183",
  "id": "urn:li:share:7045020441609936898",
  "distribution": {
    "feedDistribution": "MAIN_FEED",
    "thirdPartyDistributionChannels": []
  },
  "content": {
    "media": {
      "altText": "Image",
      "id": "urn:li:image:D4E10AQE71V5w_-aalA"
    }
  },
  "commentary": "Don't forget the image.",
  "lifecycleStateInfo": {
    "isEditedByAuthor": false
  }
}

Clear versioning and deprecation policy

The legacy APIs were all “version 2” as to be distinguished from even older “version 1 endpoints”. However, there was no further granularity in these versions. It seems to me that new features were introduced on new endpoints, which is probably why they ended up with three different endpoints for posts.

For the versioned API “reboot” LinkedIn chose a calendar-based versioning scheme. Each version is identified by year and month (e.g., 202303) and no guarantees about breaking changes between versions (as opposed to semantic versioning). Per documentation, each API version is supported for one year and specifying the version is mandatory for each call. Therefore, one can quickly see how much their integration code is behind the current versions.

I think calendar-based versioning is a right call for quickly evolving APIs, and it seems to be an ever more popular choice. GitHub announced a similar versioning scheme in November. And combined with a stable support window (something what Facebook is doing with Graph API), it provides a predictability and stable pace for API consumers.

However, it's not clear to me how exactly the deprecation will be handled, which I address below.

Communication to integration partners

Maybe my expectations about LinkedIn API were too low, but I was pleasantly surprised how well the communication about deprecation was handled. There was a relatively long transition period of eight months (now extended), and all consecutive email communication and documentation pages contained a big reminder about the deprecation.

Still, it's pretty usual the emails go amiss and no one checks the API documentation. The integration is working now, so why'd I need to check the docs? But LinkedIn did use the communication channels they have with partners to get the message across.

Responsive support

Even more important was responsive support. While previously LinkedIn recommended asking questions using linkedin-api tag on Stack Overflow (which usually went unanswered), now they provide a support portal. I've submitted a ticket, and to my surprise, I've received a helpful answer from a support representative in less than 24 hours. While I'd prefer a public forum where I could search for existing solutions first, having a working support channel is an improvement in itself.

The Bad Parts

While LinkedIn got many things right, there are a few things which bug me.

The clean shut-off

At this point, it's clear that the 8-month transition period was either too optimistic, or a planned “soft deadline” from the start. API deprecation is constant pain, since you need to wait on your integration partners to make the changes. If your partners are paying for your product, you want to avoid pulling the rug from them. But if your partners are big enterprises, you can't expect them to react quickly to your changes. But I think LinkedIn could have done a few things to hasten the migration.

At this point, it's not clear what actually happens when LinkedIn shuts off the deprecated endpoints. Will they return an HTTP status error 410 Gone with a JSON message explaining the situation? Will they return a proxy error as HTML page? Or will they redirect to a Rick Roll? (Probably not the last option.)

One way to test the migration readiness is to schedule planned “brownouts”. For example, GitHub uses this strategy for API deprecation (see authentication changes notice for an example). GitHub schedules multiple 12 to 48 hour outages over the months before the deprecation, to simulate the final removal of the API. This is a great way to check whether the migration is complete as typically there's that one more call no one migrated yet.

I'm uncertain if this is an acceptable strategy for LinkedIn, both from a technical and business standpoint. But it seems more sensible to me than just to pull the plug on the final day.

Removed features in favor of simplicity

I mentioned that the new API removed field projections with weird and poorly documented syntax. The downside is that there's no equivalent feature in the new API.

My typical use case for projections was to grab images and videos in posts with a single API request. To achieve the same functionality now, I need to collect media IDs from posts and resolve them with separate API calls. I believe this leads to much simpler implementation on LinkedIn's side, and it can encourage clients to cache referenced media, but it still shifts some complexity on the client's side.

Not-so-opaque object IDs

And speaking of media resolution, here's another catch.

Take a look at these objects from the Posts API response:

[
  {
    "id": "urn:li:ugcPost:7044823133844885504",
    "commentary": "Post A",
    "content": {
      "media": {
        "title": "Some title",
        "id": "urn:li:video:C5605AQHzRSAmLcHkTA"
      }
    }
  },
  {
    "id": "urn:li:share:7046413622230614016",
    "commentary": "Post B",
    "content": {
      "media": {
        "id": "urn:li:image:D5622AQHy2GLswHBoSg"
      }
    }
  }
]

Now, can you tell which post contains a video, and which contains an image?

Obviously, you can tell by the id property (urn:li:video vs. urn:li:image) but you shouldn't have to. IDs should be opaque values.

LinkedIn has separate endpoints to resolve images and videos, so you need to do string match the ID to figure out which endpoint to call.

Here are a few approaches how this could be improved:

Provide a new endpoint for resolving any media, where I can pass both video and image IDs (e.g., /rest/media?ids=List(urn:li:video:...,urn:li:image:...)).
Add a new property identifying the type of media:

  {
    "id": "urn:li:share:7046413622230614016",
    "commentary": "Post B",
    "content": {
      "media": {
        "type": "image",
        "id": "urn:li:image:D5622AQHy2GLswHBoSg"
      }
    }
  }

Or provide me with a link to the resource (although it doesn't match the style of this API):

  {
    "id": "urn:li:share:7046413622230614016",
    "commentary": "Post B",
    "content": {
      "media": {
        "self": "https://api.linkedin.com/rest/images/urn:li:image:D5622AQHy2GLswHBoSg",
        "id": "urn:li:image:D5622AQHy2GLswHBoSg"
      }
    }
  }

The Ugly Parts

Some API changes are painful and ugly, but they have their reasons and maybe they'll be resolved in time.

Scrape it yourself, will you?

Most social media, like Facebook or Twitter, automatically generate a “preview card” from a link contained in a post. There are some slight differences when publishing with API, for example, Facebook accepts a custom title, description, and thumbnail for the preview card – as long as the link points to a domain with verified ownership. Twitter, on the other hand, doesn't allow any preview customization during publishing.

LinkedIn used to generate a link preview automatically when a post was published through the legacy ugcPosts API. In the Posts API, this functionality has been removed:

Posts API does not support URL scraping for article post creation as it introduces level of unpredictability in how a post is going to look when API partners create it. Instead, API partners need to set article fields such as thumbnail, title and description within the post when creating an article post.

Fundamentally, I agree with this approach. I've experienced first-hand customer complaints about articles with missing or incorrect thumbnails. Usually these were caused by a disparity between the preview generated by a 3rd-party application, and LinkedIn's preview scraper. Furthermore, LinkedIn's scraped previews were impossible to refresh, so sometimes I had to instruct customers to add dummy query string to the links they share just to get a correct preview.¹

So putting the responsibility for generating a link preview fully on the API clients' side makes sense. Still, it adds an extra complexity to the publishing process.

Sure, you could just willy-nilly scrape arbitrary pages you want to publish. But every so often that won't work. I've dealt with websites whose owners were paranoid about any scraping, and blocked any requests coming from unknown bots. In the end, they allowlisted our link preview scraper, but it took some negotiation.

LinkedIn could simplify this process, and help both developers and paranoid site owners, by providing a separate API endpoint for scraping URL previews. Similar to Facebook, which has this feature.

The migration guide is somewhat useless

LinkedIn provides convenient migration guides for individual APIs.
Unfortunately, the Content APIs migration guide left me struggling with the new API. Sure, it describes how individual fields in schemas were renamed, simplified, or removed (although a direct JSON to JSON comparison would be probably more descriptive) and it briefly describes how the workflow changed. But it's still too brief.

If you need me to change the workflow, show me step by step how it differs from the old one. Even better, show me some code. The guide also doesn't mention anything about the removal of field projections, but maybe the usage of this feature was far too marginal.

Not-so-clear deprecation strategy

The versioning guide mentions that LinkedIn expects their partners to “keep with them”:

LinkedIn expects that our LinkedIn Marketing API Program API partners work to deliver the latest and most valuable experiences to our customers within a reasonable time of their availability. As a result, we will sunset our API versions as early as one (1) year after release.²

Since no versioned API reached its end-of-life yet, I have yet to see what “sunsetting an API version” means. I think it could be one of these options:

The requests start immediately return an error on the first day of the 13th month. (But what error? What status code?)
The requests will probably work for some time, but can break at any time – it's your risk to call an outdated API.
We will automatically redirect your outdated calls to a newer API version, and it will work as long as we don't introduce any breaking changes to the request schema.

The third option is something of what Facebook does. I occasionally run into code using 5+ years old API versions,³ and it still works. I suspect LinkedIn could go with the first or second option. If this is the case, I hope they'll provide developers with an early warning that their integrations are about to break. In other words:

Provide developers with API versions usage in app analytics.
Describe in gory details what exactly happens after the API reaches the end-of-life.

Conclusion

So, that's probably far too many words about the new LinkedIn API. Despite my criticism, I think it's still an overall improvement, and I'm glad LinkedIn takes the developer experience seriously, unlike other social media (ahem).

Unlike Facebook, which provides a convenient tool for debugging and refreshing link previews. ↩
Emphasis mine. ↩
In passport-facebook, for example. ↩

Twitter API Changes: The missing FAQ for Free & Basic access

Jan Vlnas — Fri, 07 Apr 2023 08:11:22 +0000

Last week, the new Twitter API access tiers were finally announced. Unfortunately, some important details were left out from the announcement, leaving many developers confused and stressed out as the deprecation deadline is getting closer.

At Superface, we maintain social media integrations, including Twitter’s, and we’ve built an authorization library for Twitter API, so we’ve been closely observing the recent developments around Twitter API. (I’ve even made a site for that.)

In this article, I have collected observations and recommendations about Twitter’s new API. In summary:

If possible, don’t migrate to the new plans yet
You can use Twitter Login for free both with OAuth 2.0 and OAuth 1.0a
You need to migrate your app to API v2
You can post tweets with media
You can still embed tweets
If you need to do anything else than login users or post tweets, you'll have to pay a monthly fee (maybe even a large sum)
Don’t rely on official support

Here’s a warning, though: Twitter is in constant flux, so any information in this article can become outdated at any time. I will do my best to keep it up to date – check the changelog and the original article for updates. If you notice any false or outdated information, please let me know.

What do we know from the announcement?

The changes were announced on the Twitter Dev account and the community forums. Here's what we can learn from these announcements:

All existing API access tiers (Standard, Premium, Essential, and Elevated) are being replaced by the new Free and Basic tiers.
Both tiers allow users to log in with Twitter, read the profile of an authorized user, and post tweets on behalf of users.
Only the paid Basic tier provides read access to user profiles and tweets at a much lower rate than previous tiers (10,000 tweets per month, compared to 500,000 on Essential and 2 million on Elevated).
The Basic tier costs $100 / month.
The v1.1 API is being deprecated in favor of the v2 API (with an exception for media uploads, see below).
The previous plans and legacy API will be deprecated by April 29th, 2023 at the latest – so technically the changes can happen any time sooner.
The Twitter Ads API is unaffected by these changes.
There are no special access plans for researchers and academics at this time.

Do I need to pay so the users of my app can log in with Twitter?

No, Twitter Login is available on the Free plan.

You can also read the information about a logged-in user through the GET /2/users/me endpoint. It is rate limited to 25 requests per 24 hours per user, so just make sure your integration code doesn’t read this endpoint too frequently (or fails gracefully when you hit the limit).

During my testing, I also frequently encountered a “Something went wrong” error on Twitter’s authorization page.

After a few retries, the authorization flow was successful. If your users encounter a similar issue, instruct them to just retry logging in a few times.

Do I need to use OAuth 2.0 for login?

No, both OAuth 1.0a and OAuth 2.0 (with app-only and user contexts) are supported on both access tiers.

You must use OAuth 1.0a if you want to publish tweets with images or videos. On the other hand, newer API features, like Bookmarks or Spaces are available only with OAuth 2.0. Check the Twitter v2 Authentication Mapping to see what features are supported in respective authentication contexts.

Do I need to migrate to the Twitter API v2?

Yes. According to the announcement, both Standard v1.1 and Premium v1.1 endpoints will be deprecated. The Basic tier is described as:

Rate limited access to suite of v2 endpoints

The only exceptions are media upload endpoints, which are not available in the v2 API.

Can I post tweets with media (images, GIFs, videos)?

Yes, that’s possible even on the Free plan, but you need to combine v2 API endpoints with v1.1 media upload endpoints. You must use OAuth 1.0a with read+write access, as media upload endpoints don’t support OAuth 2.0 access tokens.

Follow these steps to post a tweet with media attachments:

Upload media using the Upload media endpoints: POST media/upload for images or chunked upload endpoints for videos.
You will receive media_id for the uploaded objects.
Post a tweet using the POST /2/tweets endpoint and reference the uploaded media objects in a media property like this:

   {
     "text": "Tweet with media",
     "media": { "media_ids": ["1455952740635586573"] }
   }

Can I only post tweets with Free access?

Mostly, yes. It’s possible only to manage a user’s tweets (i.e., create, delete), upload media, and look up information about the authorized user.

If you hit a paid endpoint, you will get a non-descriptive error message like this:

{
  "title": "Forbidden",
  "type": "about:blank",
  "status": 403,
  "detail": "Forbidden"
}

Notably, you can’t read user’s tweets timeline or the mentions timeline. So if you are building an application that tracks users’ mentions (e.g., social media care or analytics, or a bot that replies to tweets), you will have to pay for Basic access.

I need to read more than 10,000 tweets per month, what should I do?

The next access tier after the Basic is Enterprise, which, according to the leaked sales documents, starts at $42,000 per month. You can apply through the Twitter Developer Portal.

No doubt there are other ways to get the data for cheaper or for free, but that’s outside the scope of this article.

Can I embed tweets?

Yes, Twitter for Websites features remain unaffected by these changes, including Embedded Tweets.

While there are reports of broken embeds, they are usually caused by suspended access to the Twitter API. For example, Substack reported issues with embeds, however, they use custom embeds unrelated to the official widgets. If you embed tweets on your own by fetching them from API, you will need to pay at least for the basic plan.

Should I migrate to the new plans now?

No. If you currently have Essential or Elevated access, I recommend stalling the migration to the Free and Basic plans at least until the limits enforcement stabilizes a bit. On the community forums, some users report losing access after purchasing the Basic access, probably because they were over the 10,000 tweets/month limit at the time of the purchase. Other users report issues with rate limits. These bugs seem to be symptoms of rushed development. We can hope that they will be fixed before the April 29th deadline.

Consider setting up a separate developer account with either Free or Basic access and test your application there first, before migrating your main account.

If you're using API v1.1, you should migrate your application to API v2. The new API is accessible both on the Essential and Elevated plans. Check Twitter’s migration guides.

And of course, be prepared that Twitter can suspend your application at any time, migrated or not.

Where can I get help with Twitter API?

If you run into issues with migration to the new access plans, don’t expect any help or refund from Twitter. As Ryan Barrett on the Twitter Developers forum points out, you can treat the Twitter API as effectively unmaintained.

Still, the Twitter Developers forum is probably the best place where you can get help from community volunteers and a good place to search for known issues.

If you’re migrating a Node.js application to API v2, take a look at our ready-made social media integrations, and our Twitter OAuth 2.0 strategy for Passport.js. We'll be happy to help you with Twitter API on Superface Discord or you can reach directly to us.

Your article is wrong or outdated / I still have questions

Please – let me know! Leave a comment or reach out to me on Mastodon or Discord.

I’m also sending out a monthly Superface Newsletter, so if you’d like to stay in touch and get more articles like this one with a mix of API and AI news, feel free to subscribe.

Is Twitter API (still) Free? I've built a website to find out

Jan Vlnas — Wed, 08 Feb 2023 02:22:36 +0000

Last week, Twitter announced the end of free access to its public API. The announcement came in a Tweet, a single week before the change, lacked any details about the pricing, and only vaguely promised more information.

Today is February 8th, one day before the announced deadline. No further information was provided on the TwitterDev account, community forums, or developer newsletter. Only more vague promises of a free “write-only API for bots providing good content” in a reply tweet by Mr. “Chief Twit” himself.

To help fellow developers, I've made a website to provide a definitive answer to the question: Is Twitter API Free?

A single-serving site

istwitterapifree.com as of February 8th.

There's a long tradition of single-serving sites, like Tired.com or Zombo.com. I wanted to build a single-serving site of my own, and this seemed like a good opportunity to make some fun out of this chaotic situation.

It's also telling that Ryan King, Twitter employee number 33, relaunched his Is Twitter Down? site after more than a decade, reminding the old days of Fail Whale.

Behind the scenes

I could've just made a static HTML site with “Yes”, and changed it to “No” once it'd be clear Twitter pulled the plug. That'd be the simplest thing to do, so I overengineered the whole thing, obviously.

The website is periodically regenerated by GitHub Actions and deployed to GitHub Pages. During the build, a single Twitter API call is made for the existence of the announcement tweet.

I don't know how exactly the Twitter API will communicate the paid access, but I assume that the API call will fail somehow. It's also possible that Twitter might remove the tweet if Mr. “Chief Twit” changes his mind. Therefore the success or failure of the call decides what is shown on the page.

I challenged myself to avoid any extra packages, so the site generator logic is written in vanilla JavaScript with a few Node.js built-in libraries. I'm using the built-in fetch function, which is available without any flags since Node.js 18.

I'm also curious about how many visitors my silly site gets. Goat Counter is a perfect web analytics tool for this kind of project, free and privacy-friendly. You can even view the traffic to the site on a public dashboard.

The feed and the bots

The site wouldn't be complete without a Twitter bot and a Mastodon bot (of course). I wanted to avoid implementing the posting logic, so I used the most unappreciated technology on the web: the good old RSS feed (well, technically an Atom feed).

The site generator creates a feed with a single entry. The entry is identified by the current date, so feed readers will display a new item at most once a day, regardless of how many times the site is regenerated.

This feed is passed to automation. For Twitter, I've used IFTTT to post a new feed item to the Twitter account. I assume that a larger provider like IFTTT already pays for Twitter API, so it won't be affected.

The whole magic behind the IsTwApiFree bot.

For posting to Mastodon, I originally tried Mastofeed, but I haven't figured out a way to include the content of the feed entry in the post. I've turned to GitHub Actions, particularly Mastofeedbot, which runs as a separate action after the site is updated.

Lesson learned: Intl API

The generated site displays how many days remain until the deadline. What helped me was JavaScript's built-in Internationalization API, particularly Intl.RelativeTimeFormat to format the number of days to a string like “in 2 days” or “yesterday”. I'm calculating the number of days between now and the assumed deadline, which is set to midnight on February 9 in Pacific time.

const deadline = new Date(1675929600 * 1000); // Feb 9 midnight PST
const now = new Date();

// Calculate millisecond difference between the dates
const diffMs = deadline.getTime() - now.getTime();
// Round the difference to number of days
const diffDays = Math.round(diffMs / (1000 * 3600 * 24));

const rtf = new Intl.RelativeTimeFormat("en", {
  localeMatcher: "best fit",
  numeric: "auto",
  style: "long",
});

// 3 → "in 3 days"
// 0 → "today"
// -1 → "yesterday" etc.
const daysRelative = rtf.format(diffDays, "days");

return diffDays > 0
  ? `The deadline is ${daysRelative}`
  : `The deadline was ${daysRelative}`;

While the manipulation with dates is still painful, Intl APIs simplify at least formatting. I'm looking forward to the stable support for Temporal API.

What's next

My hope for the Is Twitter API Free? site is that it becomes irrelevant in a few weeks. I might add some recommendations for developers who stumble upon the site, like resources for building ActivityPub bots or scraping Twitter's private API. No matter whether Twitter implements the API paywall in the end, or retracts the plan, it has already lost any remaining credibility as a platform for developers. It's time to move on.

Meanwhile, keep checking the site, follow the Twitter bot or the Mastodon bot, and check the site's source.

Exploring Pocket API: Authorization

Jan Vlnas — Wed, 18 Jan 2023 22:59:16 +0000

I've been using Pocket for quite some time. Recently, I wanted to build something on top of their API. I've collected my notes and thoughts on Pocket API as a future reference for myself. Perhaps it will be useful to you.

The first thing I needed to figure out was authorization. The good news is the authorization flow is well documented. The bad news is immediately in the first sentence:

The Pocket Authentication API uses a variant of OAuth 2.0 for authentication.

(Emphasis mine.)

“Variant of OAuth 2.0” reeks of custom authorization schemes, which usually spells trouble. But while Pocket's authentication scheme is non-standard, it's actually closer to OAuth 1.0 flow with “temporary credentials”. Minus all the request signing characteristic for OAuth 1.0.

Pocket authorization vs. OAuth 2.0 Authorization Code Flow

My simple understanding of a typical OAuth 2.0 Authorization Code Flow is this:

The consumer application identifies itself by the client ID. The provider also keeps a list of allowed callback URLs, so it's not possible to steal the authorization code by redirecting the user to a malicious app.

Pocket's authorization flow is a bit different:

The consumer app asks for a request token (“temporary credentials”) at the beginning of the flow, which it later exchanges for an access token. It's like getting a blank ticket and later validating it.

In the Authorization Code Flow, the provider adds the authorization code to the callback URL, so there's no need to store any state during the authorization. In case of Pocket's flow, the request token needs to be stored somewhere, typically in a session or in a cookie.

On the other hand, Pocket doesn't need to know a list of allowed URLs. Even if the user were redirected to a malicious client app, it wouldn't know the original request token and couldn't exchange it for an access token.

The current version of Pocket API was introduced in 2012 which is the same year when OAuth 2.0 was finished. So, I think the authorization scheme ended up somewhere in between OAuth 1.0 and 2.0: it's mostly OAuth 1.0 flow without requests signing, which was also removed in OAuth 2.0.

Pocket authorizatiom in Node.js

Since no API client tool like Postman or Hoppscotch can handle Pocket's authorizatiom scheme, I had to implement it on my own.

Luckily, there are a few Node.js libraries handling the scheme, but most of them are over 5 years old. I've picked pocket-auth by Michael Heap, and got my access token with this code modified from the library's example:

const auth = require("pocket-auth");
const consumerKey = "<redacted>";
const redirectUri = "https://example.com";

async function main() {
  try {
    let code = await auth.fetchToken(consumerKey, redirectUri, {});
    let uri = auth.getRedirectUrl(code.code, redirectUri);
    console.log(
      "Visit the following URL and click approve in the next 10 seconds:"
    );
    console.log(uri);

    setTimeout(async function () {
      try {
        let r = await auth.getAccessToken(consumerKey, code.code);
        console.log(r);
      } catch (err) {
        console.error(err);
        console.log(
          "You didn't click the link and approve the application in time"
        );
      }
    }, 20000);
  } catch (err) {
    console.log(err);
  }
}

main();

The script will show a URL with the request token, and after 20 seconds it attempts to grab the access key – meanwhile, you need to authorize access to Pocket.

Only later I've found that Michael also built a CLI tool for pocket-auth, which is much more convenient. Just run the tool with consumer key as argument, and it will handle the whole flow.

$ npx pocket-auth-cli <Pocket consumer key>
Opening web browser to authorize application
Press CTRL+C to cancel
{ access_token: '<redacted>', username: '<redacted>' }

Onto retrieval

This was a distracting but necessary step to get access to Pocket's API. Now it's time to retrieve some articles – but let's keep it for another time.

Instagram Graph API Explained: How to log in users

Jan Vlnas — Tue, 03 Jan 2023 13:16:21 +0000

There are two ways how to access the Instagram API:

Through Instagram’s Basic Display API
Through Instagram Graph API

Instagram Basic Display API is limited to read-only access to a user’s profile and isn’t intended for user authentication. It’s useful, for example, to get up-to-date information about the user’s profile or display their recent posts. It also works with any Instagram account type: personal and professional.

On the other hand, Instagram Graph API allows you to publish posts, moderate comments, search hashtags, and access insights. However, the Graph API can be used only by “professional” accounts which have been paired with a Facebook page. The authentication flow is handled through Facebook Login, and the user can pick which Facebook pages and Instagram accounts are available to the application.

This article is part of a series about Instagram Graph API. Previously, I covered test account and app setup and finding the correct Instagram account ID. In this part, I will show you how to implement the authentication flow for Instagram Graph API with Facebook Login. I will use Node.js, Express, and Passport with the passport-facebook strategy, but the basic ideas are applicable to any language and framework.

Facebook Login Setup

This tutorial assumes you have a Facebook application set up, and an Instagram business account paired with a Facebook page for testing. If not, check my previous tutorial.

We will need to set up Facebook Login for our testing application.

Find your application on the Facebook Developer Portal, and on the Dashboard, set up Facebook Login for Business. If you didn’t select a “Business app type” when creating the application, you may see Facebook Login instead, so choose that – both options will work.

Under Facebook Login Settings, make sure to enable both Client OAuth login and Web OAuth login.

This screen is also where you can add allowed redirect URIs for deployed application. We will run the application only locally, and Facebook allows localhost URLs by default, so there’s no need to add any URL.

Finally, visit application Settings > Basic, and copy the App ID and App Secret. We will need them for the Passport strategy.

Instagram Graph API authentication flow with Passport

With Facebook Login now set up, we can build an authentication flow. I will use the following npm packages:

express server
express-session to handle user data persistence
passport for authentication
passport-facebook for handling Facebook Login flow
dotenv to read secrets from .env file
@superfaceai/one-sdk to get the list of accessible Instagram profiles

Let’s set up the project and install the dependencies:

mkdir instagram-facebook-login-passport
cd instagram-facebook-login-passport
npm install express express-session passport passport-facebook dotenv @superfaceai/one-sdk

Now create a .env file, and paste in the App ID and App Secret values you obtained in Facebook app settings:

BASE_URL=http://localhost:3000
FACEBOOK_CLIENT_ID="App ID from Settings"
FACEBOOK_CLIENT_SECRET="App Secret from Settings"

I have also prepared the BASE_URL variable, since we will be passing the callback URL during the authentication process. Keeping this value configurable makes it easier to take your app to production.

And now, let's create a server.js file with the following content:

const express = require('express');
const session = require('express-session');
const passport = require('passport');
const { Strategy } = require('passport-facebook');
const { SuperfaceClient } = require('@superfaceai/one-sdk');

require('dotenv').config();

const sdk = new SuperfaceClient();

// <1> Serialization and deserialization
passport.serializeUser(function (user, done) {
  done(null, user);
});
passport.deserializeUser(function (obj, done) {
  done(null, obj);
});

// Use the Facebook strategy within Passport
passport.use(
  // <2> Strategy initialization
  new Strategy(
    {
      clientID: process.env.FACEBOOK_CLIENT_ID,
      clientSecret: process.env.FACEBOOK_CLIENT_SECRET,
      callbackURL: `${process.env.BASE_URL}/auth/facebook/callback`,
    },
    // <3> Verify callback
    (accessToken, refreshToken, profile, done) => {
      console.log('Success!', { accessToken, profile });
      return done(null, { profile, accessToken });
    }
  )
);

const app = express();

// <4> Session middleware initialization
app.use(
  session({ secret: 'keyboard cat', resave: false, saveUninitialized: true })
);
app.use(passport.initialize());

// <5> Start authentication flow
app.get(
  '/auth/facebook',
  passport.authenticate('facebook', {
    // <6> Scopes
    scope: ['pages_show_list', 'instagram_basic', 'instagram_content_publish'],
  })
);

// <7> Callback handler
app.get(
  '/auth/facebook/callback',
  passport.authenticate('facebook'),
  async function (req, res, next) {
    try {
      // <8> Obtaining profiles
      const accessToken = req.user.accessToken;
      const sdkProfile = await sdk.getProfile(
        'social-media/publishing-profiles@1.0.1'
      );
      const result = await sdkProfile
        .getUseCase('GetProfilesForPublishing')
        .perform(
          {},
          {
            provider: 'instagram',
            parameters: {
              accessToken,
            },
          }
        );
      const profiles = result.unwrap();

      res.send(
        `
        <h1>Authentication succeeded</h1>
        <h2>User data</h2>
        <pre>${JSON.stringify(req.user, undefined, 2)}</pre>
        <h2>Instagram profiles</h2>
        <pre>${JSON.stringify(profiles, undefined, 2)}</pre>
        `
      );
      next();
    } catch (err) {
      next(err);
    }
  }
);

app.listen(3000, () => {
  console.log(`Listening on ${process.env.BASE_URL}`);
});

Run the server with npm start and visit http://localhost:3000/auth/facebook. You will be redirected to the Facebook login, where you select the Facebook pages and Instagram profiles the application can have access to. Make sure to select the correct Facebook page with the associated Instagram profile. Otherwise, you won’t be able to access that profile.

After passing the flow, you should see the basic data about your profile and a list of Instagram profiles you have access to.

Explaining the example code

If you read my tutorial on Twitter OAuth 2.0 authentication, the example will seem very familiar. The most significant difference is the handling of access tokens and additional logic for obtaining Instagram profiles.

I will explain the example in individual parts:

User serialization and deserialization

// <1> Serialization and deserialization
passport.serializeUser(function (user, done) {
  done(null, user);
});
passport.deserializeUser(function (obj, done) {
  done(null, obj);
});

These functions serialize and deserialize the user to and from a session. In our example application, we keep all sessions in memory with no permanent storage, so we just pass the whole user object.

Typically, you will persist the data in a database. In that case, you will store the user ID in the session, and upon deserialization, find the user in your database using the serialized ID, for example:

passport.serializeUser(function (user, done) {
  done(null, user.id);
});
passport.deserializeUser(function (id, done) {
  User.findOrCreate(id).then(user => done(null, user));
});

The deserialized user object is then accessible through the req.user property in middleware functions. You can find more in the Passport documentation on sessions.

Strategy initialization

// Use the Facebook strategy within Passport
passport.use(
  // <2> Strategy initialization
  new Strategy(
    {
      clientID: process.env.FACEBOOK_CLIENT_ID,
      clientSecret: process.env.FACEBOOK_CLIENT_SECRET,
      callbackURL: `${process.env.BASE_URL}/auth/facebook/callback`,
    }
    // ...
  )
);

This code registers the passport-facebook strategy with credentials obtained from the application settings. The callback URL must be absolute, and registered as a valid OAuth redirect URL (except for localhost).

Success callback

passport.use
  new Strategy(
    // ...
        ,
    // <3> Verify callback
    (accessToken, refreshToken, profile, done) => {
      console.log('Success!', { accessToken, profile });
      return done(null, { profile, accessToken });
    }
  )
);

The second argument to the strategy constructor is a verify function, which is called at the end of the successful authorization flow. The user has authorized your application, and you will receive their access token and basic information about their profile (in this case just the ID and full name of the user). Facebook API doesn’t provide refreshToken (you can, instead, turn the access token into a long-lived token), therefore that value will be always empty.

Here, you might want to update or create the user in your database and store the access token, so that you can access the API on behalf of the user. The done callback should receive a user object, which is later available through req.user. To keep things simple, I only passed the access token along with the profile data.

Passport and Session middlewares initialization

// <4> Session middleware initialization
app.use(
  session({ secret: 'keyboard cat', resave: false, saveUninitialized: true })
);
app.use(passport.initialize());

Passport needs to be initialized as middleware as well. And it requires a session middleware for storing state and user data. The most common session middleware is express-session.

By default, express-session stores all data in memory, which is good for testing, but not intended for production: if your server gets restarted, all users will be logged out. There is a wide selection of compatible session stores – pick one which fits with the rest of your stack.

Before you put this code into production, make sure to change the value of secret. This value is used to sign the session cookie. Keeping it easily guessable increases the risk of session hijacking. Check the express-session docs.

Start the authentication flow

Now we are getting to the actual route handlers where the authentication happens.

// <5> Start authentication flow
app.get(
  '/auth/facebook',
  passport.authenticate('facebook', {
    // <6> Scopes
    scope: ['pages_show_list', 'instagram_basic', 'instagram_content_publish'],
  })
);

passport.authenticate creates a middleware for the given strategy. It redirects the user to Facebook with URL parameters, so that Facebook knows what application is the user authorizing and where the user should be then redirected back. The authenticate function accepts a second parameter with additional options, where the most important is scopes.

Authorization scopes (permissions)

passport.authenticate('facebook', {
  // <6> Scopes
  scope: ['pages_show_list', 'instagram_basic', 'instagram_content_publish'],
});

OAuth scopes define what the application is allowed to do on behalf of the user (Facebook calls them “permissions”). The user can then review and approve these permissions.

In this case, we request the following permissions:

pages_show_list to list Facebook pages the user manages and allowed for our app
instagram_basic to read basic information about an Instagram profile
instagram_content_publish to allow publishing content; this is not needed now, but may be useful in the next tutorial.

You can find additional possible permissions in the Permissions Reference. Keep in mind that if you’d like your application to be publicly accessible, you will have to submit it for an app review, and explain how permissions are used.

Callback handler

// <7> Callback handler
app.get(
  '/auth/facebook/callback',
  passport.authenticate('facebook'),
  async function (req, res, next) {
    // ...
  }
);

This is the final step in the authentication flow. After the user authorized your application, they are redirected to the /auth/twitter/callback route. The passport.authenticate middleware is here again, but this time it checks the query parameters Facebook provided on redirect, and obtains access and refresh tokens.

If the authentication succeeds, the next middleware function is called – typically you will display some success message to the user, or redirect them back to your application. Since the authentication passed, you can now find the user data in the req.user property.

Obtaining Instagram profiles

async function (req, res, next) {
  try {
    // <8> Obtaining profiles
    const accessToken = req.user.accessToken;
    const sdkProfile = await sdk.getProfile(
      'social-media/publishing-profiles@1.0.1'
    );
    const result = await sdkProfile
      .getUseCase('GetProfilesForPublishing')
      .perform(
        {},
        {
          provider: 'instagram',
          parameters: {
            accessToken,
          },
        }
      );
    const profiles = result.unwrap();

    res.send(
      `
      <h1>Authentication succeeded</h1>
      <h2>User data</h2>
      <pre>${JSON.stringify(req.user, undefined, 2)}</pre>
      <h2>Instagram profiles</h2>
      <pre>${JSON.stringify(profiles, undefined, 2)}</pre>
      `
    );
    next();
  } catch (err) {
    next(err);
  }
}

The route handler uses Superface OneSDK to fetch basic data about Instagram profiles we have access to. I use GetProfilesForPublishing for that, which also works with Facebook, LinkedIn, Pinterest, and Twitter. The logic is the same as in the previous Find the right account ID tutorial, except we are passing the access token stored in session from req.user.accessToken (see the code for the Success callback).

Next steps

One issue with the example code is that both the user data and the access token are stored in memory. The user needs to go through the authentication flow every time the server is restarted. For a real-world use, you will need to persist the data using, for example, a configuration file or a database.

In future tutorials, I will look at publishing of images and videos, searching hashtags, and retrieving posts and comments. If you don’t want to miss them, subscribe to our newsletter and follow our blog 👇

Superface Follow

Superface allows GPT builders to easily connect to any API, enabling them to create, retrieve, and manage data from external platforms.

Did you run into any problems? I will be happy to help you on our Discord!

How to use Twitter OAuth 2.0 and Passport.js for user login

Jan Vlnas — Thu, 24 Nov 2022 13:26:43 +0000

tl;dr Use the Node.js package @superfaceai/passport-twitter-oauth2 to handle user authentication with Twitter's v2 API and Passport.js.

superfaceai / passport-twitter-oauth2

Twitter OAuth 2.0 Strategy for Passport for accessing Twitter API v2

Twitter OAuth 2.0 Strategy for Passport

@superfaceai/passport-twitter-oauth2

Passport strategy for authenticating with Twitter using OAuth 2.0.

This module lets you authenticate using Twitter in your Node.js applications By plugging into Passport, Twitter authentication can be integrated into any application or framework that supports Connect-style middleware, including Express.

Twitter announced OAuth 2.0 general availability on December 14 2021 and encourages developers to use Twitter API v2.0 with OAuth 2.0 authentication.

Twitter OAuth 2.0 implementation specifics:

PKCE is required
OAuth2 client credentials must be passed via Authorization header for confidential client types

Install

npm install @superfaceai/passport-twitter-oauth2

Usage

Note Check our blog for a complete tutorial with code explanation.

Create an Application

Before using @superfaceai/passport-twitter-oauth2, you must register a project and an application with Twitter by following these steps:

go to https://developer.twitter.com/ and either sign up for a new account or sign in with existing one
sign up…

View on GitHub

Read on for the background, how to obtain Twitter credentials, example Express app, and code explanation.

Why use OAuth 2.0 for Twitter API

When you want to interact with Twitter's API or just provide a “social login” through Twitter, until last year you had to use legacy OAuth 1.0a authentication. While it still has its uses, Twitter is transitioning to the new version of their API (v2) and OAuth 2.0 protocol, which is also used by other providers, like Facebook, Google, and LinkedIn.

Besides being more widespread, Twitter's OAuth 2.0 has other advantages compared to OAuth 1.0a:

It gives you full access to Twitter's v2 API (including the endpoints for Twitter Spaces and editing tweets)
It lets you specify exactly what your application needs from the API through scopes, so users have a clear idea what your app can and cannot do with their account; OAuth 1.0a has only three levels of access: “read”, “read + write”, and “read + write + access DMs”
It is future-proof, as the development effort is focused on v2 API, for which OAuth 2.0 is the default authentication protocol.

When we wanted to build social media integrations, we couldn't find any up-to-date Node.js library to handle the authentication flow. So, we built one as a strategy for the popular Passport.js authentication middleware.

The strategy is available in @superfaceai/passport-twitter-oauth2 package. Recently we released a new minor version 1.2, which conducts a full rewrite to TypeScript.

Getting OAuth 2.0 Client ID and Secret from Twitter

To start using Twitter API, you need to register for a developer account (including phone number verification). Once you register, you will be prompted to create the first application. You will immediately receive API Key and API Key Secret – but ignore them, since these are only for OAuth 1.0. Instead, go to your project's dashboard, there go to the App Settings of the only application you created.
In application settings, find User authentication settings and click Set up.

In User authentication settings make sure to select Type of App as Web App, Automated App or Bot.

In App info enter the following URL under Callback URI / Redirect URL:

http://localhost:3000/auth/twitter/callback

This is a URL where the user can be redirected after approving the application. The callback will be handled by the example server we will build in the next section.

Fill in also the Website URL, since it is a required value. Feel free to ignore other fields.

Once you save the form, you will get OAuth 2.0 Client ID and Client Secret. Make sure to write these values down, since you can't reveal the secret later, only regenerate it.

But in case you forget to save the secret, you can always regenerate it under Keys and Tokens section of your app.

Authenticating with Passport.js and Twitter in Express.js

Let's create a simple Express server with Passport.js to show the authentication flow in action. Passport requires some session handling mechanism, usually through express-session. I will also use the dotenv package to load the credentials from .env file to process.environment object.

First, let's create a project and install dependencies:

mkdir twitter-oauth2-passport-demo
cd twitter-oauth2-passport-demo
npm install express express-session passport @superfaceai/passport-twitter-oauth2 dotenv

Now put your Client ID and Client Secret into .env file in the root of your project. Use the following template and make sure to paste in the correct values from the developer portal:

BASE_URL=http://localhost:3000
TWITTER_CLIENT_ID="OAuth 2.0 Client ID from Twitter Developer portal"
TWITTER_CLIENT_SECRET="OAuth 2.0 Client Secret from Twitter Developer portal"

I have also prepared the BASE_URL variable, since we will be passing the callback URL during the authentication, and keeping this value configurable makes it easier to take your app to production.
And now, let's create server.js file, and put in the following contents:

const express = require('express');
const passport = require('passport');
const { Strategy } = require('@superfaceai/passport-twitter-oauth2');
const session = require('express-session');
require('dotenv').config();

// <1> Serialization and deserialization
passport.serializeUser(function (user, done) {
  done(null, user);
});
passport.deserializeUser(function (obj, done) {
  done(null, obj);
});

// Use the Twitter OAuth2 strategy within Passport
passport.use(
  // <2> Strategy initialization
  new Strategy(
    {
      clientID: process.env.TWITTER_CLIENT_ID,
      clientSecret: process.env.TWITTER_CLIENT_SECRET,
      clientType: 'confidential',
      callbackURL: `${process.env.BASE_URL}/auth/twitter/callback`,
    },
    // <3> Verify callback
    (accessToken, refreshToken, profile, done) => {
      console.log('Success!', { accessToken, refreshToken });
      return done(null, profile);
    }
  )
);

const app = express();

// <4> Passport and session middleware initialization
app.use(passport.initialize());
app.use(
  session({ secret: 'keyboard cat', resave: false, saveUninitialized: true })
);

// <5> Start authentication flow
app.get(
  '/auth/twitter',
  passport.authenticate('twitter', {
    // <6> Scopes
    scope: ['tweet.read', 'users.read', 'offline.access'],
  })
);

// <7> Callback handler
app.get(
  '/auth/twitter/callback',
  passport.authenticate('twitter'),
  function (req, res) {
    const userData = JSON.stringify(req.user, undefined, 2);
    res.end(
      `<h1>Authentication succeeded</h1> User data: <pre>${userData}</pre>`
    );
  }
);

app.listen(3000, () => {
  console.log(`Listening on ${process.env.BASE_URL}`);
});

Now run your server with npm start and visit http://localhost:3000/auth/twitter. You will be redirected to Twitter authorization page:

And once you authorize the app, you should see the user data from your profile and, in addition, accessToken and refreshToken will be logged into console.

Breaking down the example

Let's break down the example code above a bit.

User serialization and deserialization

// <1> Serialization and deserialization
passport.serializeUser(function (user, done) {
  done(null, user);
});
passport.deserializeUser(function (obj, done) {
  done(null, obj);
});

These functions serialize and deserialize user to and from session. In our example application, we keep all sessions in the memory with no permanent storage, so we just pass the whole user object.

Typically, you will persist data in a database. In that case, you will store the user ID in the session, and upon deserialization find the user in your database using the serialized ID, for example:

passport.serializeUser(function (user, done) {
  done(null, user.id);
});
passport.deserializeUser(function (id, done) {
  User.findOrCreate(id).then(user => done(null, user));
});

The deserialized user object is then accessible through req.user property in middleware functions.

Strategy initialization

// Use the Twitter OAuth2 strategy within Passport
passport.use(
  // <2> Strategy initialization
  new Strategy(
    {
      clientID: process.env.TWITTER_CLIENT_ID,
      clientSecret: process.env.TWITTER_CLIENT_SECRET,
      clientType: 'confidential',
      callbackURL: `${process.env.BASE_URL}/auth/twitter/callback`,
    }
    // ...
  )
);

To use an authentication strategy, it must be registered with Passport through passport.use. Here, the Twitter OAuth 2.0 strategy is initialized with credentials from Twitter developer portal. The callback URL must be absolute and registered with Twitter – it's where the user is redirected after authorizing the application.

clientType can be either confidential or public, but in case of server-side applications you will usually use confidential. As explained in OAuth specification, confidential clients can keep the secret safe, while public clients, like mobile applications, can't.

Success callback

passport.use(
  new Strategy(
    //...
    ,
    // <3> Verify callback
    (accessToken, refreshToken, profile, done) => {
      console.log('Success!', { accessToken, refreshToken });
      return done(null, profile);
    }
  )
);

The second argument to the strategy constructor is a verify function. In case of OAuth-based strategies, it is called at the end of successful authorization flow. The user has authorized your application, and you will receive their access token and (optionally) refresh token and user's profile (username, display name, profile image etc.).

Now it's your turn: typically you will want to update or create the user in your database and store the tokens, so you can call the API with them. The done callback should receive a user object, which is passed in req.user property.

Passport and Session middlewares initialization

// <4> Passport and session middleware initialization
app.use(passport.initialize());
app.use(
  session({ secret: 'keyboard cat', resave: false, saveUninitialized: true })
);

Passport needs to be initialized as middleware as well. And it requires a session middleware for storing state and user data. The most common session middleware is express-session.

Start the authentication flow

Now we get to the juicy part, routes where the authentication happens. The first route is /auth/twitter:

// <5> Start authentication flow
app.get(
  '/auth/twitter',
  passport.authenticate('twitter', {
    //...
  })
);

passport.authenticate creates a middleware for the given strategy. It redirects the user to Twitter with URL parameters, so Twitter knows what application the user is authorizing and where the user should be then redirected back. authenticate function accepts a second parameter with additional options, where the most important is scopes.

Authorization scopes

passport.authenticate('twitter', {
  // <6> Scopes
  scope: ['tweet.read', 'users.read', 'offline.access'],
});

OAuth scopes define what the application is allowed to do on behalf of the user. The user can then review and approve these permissions.

In this case, the following scopes are requested:

tweet.read – allows reading of user's and others' tweets (including tweets from private accounts the user follows)
users.read – allows reading information about users profiles
offline.access – allows access even when the user isn't signed in; in practice, the application receives the refresh token

Both tweet.read and users.read are necessary for accessing information about the signed-in user. offline.access is useful when you want to do something when the user isn't directly interacting with your application, for example if you post scheduled tweets, build a bot, or monitor mentions on Twitter.

For a detailed overview of what scopes are used in Twitter's API, check Twitter's authentication mapping.

Callback handler

// <7> Callback handler
app.get(
  '/auth/twitter/callback',
  passport.authenticate('twitter'),
  function (req, res) {
    const userData = JSON.stringify(req.user, undefined, 2);
    res.end(
      `<h1>Authentication succeeded</h1> User data: <pre>${userData}</pre>`
    );
  }
);

This is the final step in the authentication flow. After the user authorized your application, they are redirected to /auth/twitter/callback route. The passport.authenticate middleware is here again, but this time it checks query parameters Twitter provided on redirect, checks if everything is correct, and obtains access and refresh tokens.

If the authentication succeeds, the next middleware function is called – typically you will display some success message to the user or redirect them back to your application. Since the authentication passed, you can now find the user data in req.user property.

Next steps

Passport.js isn't limited to just Express, you can use our strategy with other frameworks with Passport compatibility like NestJS, Fastify, or Koa.

With the obtained access token, you can start integrating Twitter's API. Check out our twitter-demo repository with CLI scripts showing how to list followers, lookup posts by hashtag, or publish a tweet.

We also have a bit more complex social-media-demo which demonstrates authentication and integrations with Facebook, Instagram, LinkedIn, and Twitter.

I will be diving into more hands-on examples with Twitter, Instagram, and other social media in the future posts. Subscribe to our blog or sign up for our monthly newsletter, so you don't miss it.

Superface Follow

Superface allows GPT builders to easily connect to any API, enabling them to create, retrieve, and manage data from external platforms.

Try our strategy and let me know what you are building with it!

Resources

Announcement of OAuth 2.0 General Availability by Twitter
Twitter's guide Getting Access to the Twitter API is useful for the general overview of required steps and use of various credentials.
Twitter API v2 authentication mapping for overview of what scopes are needed where.
OAuth 2.0 Authorization Code Flow with PKCE explains details of Twitter's OAuth 2.0 implementation, like lifetime of access and refresh tokens, glossary, and overview of all scopes.

How do you glue APIs together?

Jan Vlnas — Fri, 11 Nov 2022 13:56:44 +0000

Say you want to connect two or more APIs, for example:

Whenever someone submits a form on our website, add their contact to HubSpot (or other CRM) and email them.
When someone gives a star to my repository on GitHub, send me a Telegram message.
Post a tweet when there's a new post on our blog.

How do you connect these different API integrations together?

Do you use some low-code platform like Zapier, n8n, or Node-RED? Or do you prefer to “glue” the code manually? Or do you have a magical framework which does the hard work for you?

And importantly, how much you like/hate your current approach, and how do you think it should work?

I am curious to know!

Instagram API: Find the right account ID

Jan Vlnas — Mon, 26 Sep 2022 07:17:02 +0000

To manage your Instagram account through an API, you need two things: a user access token and a business account ID. Getting an access token is easy, but figuring out the account ID takes a few steps and sometimes causes a confusion.

Prerequisites: This tutorial assumes you have an Instagram business account connected with a Facebook page and Facebook application with Instagram Graph API enabled. Check my previous article on how to set up a test account for Instagram API.

Get started with Instagram API: The Setup

Jan Vlnas for Superface ・ Sep 23 '22

#tutorial #instagram #api #facebook

If you are currently troubleshooting requests to Instagram's API and can't wrap your head around their IDs, skip right away to common ID confusions.

Get an access token

For the following steps, I will use Graph API Explorer. This is a useful tool for trying out Facebook's APIs and also to obtain access tokens for authorized API access.

In the right sidebar, select the previously created application under “Meta App”. Make sure “User Token” is selected and add the following permissions: instagram_basic, pages_show_list, and instagram_content_publish (this will come handy in later tutorials).

Finally, click “Generate Access Token”. Facebook will display a pop-up with prompts to authorize access to your Instagram and Facebook pages. Make sure to select both your testing Instagram account and the associated Facebook page.

Find your Instagram account ID

If you authorized the application correctly, you should be able to list Facebook pages the application has access to.

Enter the following path to the Graph API Explorer: me/accounts. After submitting, you should see Facebook pages you gave your application access to.

Now, copy the value id of your page and enter the following path:

<page ID>?fields=instagram_business_account

In the response, you will see both the Facebook's page ID and the ID of your Instagram account. Copy the value of id under instagram_business_account – this ID is necessary for further interactions with your Instagram account via API.

Get account details

With this ID, we can retrieve basic information about our Instagram account, like its username, name, and profile picture. Try querying the following path with your Instagram business account ID:

<business account ID>?fields=id,name,username,profile_picture_url

All in single request

There is also an undocumented way to retrieve Instagram account details in a single request from me/accounts. This way, we can skip intermediary requests and retrieve authorized Instagram accounts directly:

me/accounts?fields=instagram_business_account{id,name,username,profile_picture_url}

In Facebook's Graph API, it is possible to traverse some edges within a single request – this is what the curly braces in the fields query parameter are for. In other words, we are telling the API, “give me these fields for instagram_business_account edge under me/accounts edge”.

Common ID confusions

A common source of mistakes when dealing with Graph API is use of an incorrect type ID. If the API doesn't behave like you expect, check if you have the right ID. In case of Instagram Graph API, we are dealing with the following IDs:

Facebook Page ID – retrieved from me/accounts endpoint identifies Page node and acts as an entry point to get an Instagram account associated with the Page.
Instagram (Business) Account ID – retrieved from Page node under instagram_business_account edge. Documentation refers to it as IG User node. It must be accessed with a user access token.
ig_id or Instagram User ID – this is an ID of the Instagram account from the legacy, deprecated API. It is intended for migration of applications using the pre-graph API, but it's not used anywhere else. It's also represented as a number, while Graph API IDs are strings.

Get Instagram account easier way

Picking a correct Instagram account is a pretty basic integration task, so we've built an easier way to do that. If you use Node.js, before you grab fetch and start building your custom abstraction, try OneSDK. We have an integration ready to get a list of authorized Instagram accounts. And the same interface works also for Facebook, LinkedIn, Pinterest, and Twitter – but let's keep it for another time.

First, install OneSDK into your project:

npm i @superfaceai/one-sdk

And paste the following code into profiles.js file:

const { SuperfaceClient } = require('@superfaceai/one-sdk');

// Replace the value with the token from Graph Explorer
const ACCESS_TOKEN = 'YOUR USER ACCESS TOKEN HERE'

const sdk = new SuperfaceClient();

async function getProfiles() {
  const sdkProfile = await sdk.getProfile('social-media/publishing-profiles@1.0.1');
  const result = await sdkProfile
    .getUseCase('GetProfilesForPublishing')
    .perform({}, {
      provider: 'instagram',
      parameters: {
        accessToken: ACCESS_TOKEN
      }
    });

  try {
    return result.unwrap();
  } catch (error) {
    console.error(error);
  }
}

getProfiles().then((result) => {
  console.log(result.profiles);
});

Don't forget to insert your actual user access token as a value of the ACCESS_TOKEN variable. You can copy it from Graph API Explorer:

When you run this code, you will get an array with authorized accounts, their ID, username, and profile image:

$ node profiles.js
[
  {
    id: '17841455280630860',
    name: 'Dev Testing App IG Acct',
    username: 'dev_testing_app',
    imageUrl: 'https://scontent.fprg5-1.fna.fbcdn.net/...'
  }
]

The code behind integration is open-source and your application communicates with Instagram API directly without intermediary servers.

Conclusion

With the Instagram account ID, we can start managing the account via API. In the following articles I will cover publishing of images and videos, authorization flow, and retrieving posts and comments. If you don't want to miss it, follow Superface on Dev.

Superface Follow

Superface allows GPT builders to easily connect to any API, enabling them to create, retrieve, and manage data from external platforms.

Was this tutorial useful or did you run into any problems? Let me know in the comments! 👇

Get started with Instagram API: The Setup

Jan Vlnas — Fri, 23 Sep 2022 15:11:09 +0000

Instagram Graph API is Facebook's official way to access Instagram from your application. The API allows you to manage your account, publish content, and access some public data from Instagram, but only a subset of features is available compared to Instagram's official applications.

That's no coincidence. Facebook severely restricted its APIs after the Cambridge Analytica scandal. Instagram Graph API was designed with these privacy concerns in mind, replacing former Instagram API which was much more open.

Some design decisions and limitations around Instagram Graph API are so confusing, developers often ask how to perform even basic tasks. For example, how to get access to the API for their account, or figure out the correct account ID. I've answered numerous questions like this, which led me to start this series.

This is a first dive into Instagram Graph API, covering the initial prerequisites: creating an Instagram business account and setting up a Facebook app. In the next article I will cover how to get an access token and basic information about Instagram account, including the business account ID required for further interactions. In future posts, I will cover authorization in Node.js, content publishing, and retrieval of posts and comments.

Pair Instagram account with Facebook Page

Instagram Graph API can be used only by “Instagram Professional accounts” – this includes Business accounts (intended for companies), and Creator accounts (intended for individuals, like influencers). The good news is you don't need to pay anything for a Professional account.

Before you proceed, you need three things:

A Facebook account (can be your personal)
A Facebook page – ideally a dedicated testing page, so create one now
An Instagram account – ideally a dedicated account for testing (sign up for one)

For development purposes, we will need to turn the Instagram account into a Business account and pair it with the Facebook page.

Once you log into your Instagram account, go to account settings. Here, click “Switch to professional account”.

You have two options: Creator and Business. Select Business, pick a category (can be anything) and skip the step to add contact information.

Next, on the Facebook side, go to your Facebook page's settings. On “Professional dashboard” select “Linked Accounts” and you should see a button to “Connect account” from Instagram.

This will show you a pop-up to enter Instagram credentials, and once you log in, you should see a success message.

Create a Facebook application

Next, we will need to create a Facebook app. Visit My apps on ~~Facebook~~ Meta for Developers site and click “Create App”. This will take you to app type selection.

Select “Business”, enter name and contact e-mail, and finally, you should see various products to add to your app. Find Instagram Graph API, click “Set up”, and that's it for now.

Next steps

In the next article, we will get to work with the API itself.

Instagram API: Find the right account ID

Jan Vlnas for Superface ・ Sep 26 '22

#tutorial #api #instagram #node

If you don't want to miss it, follow Superface on Dev.

Superface Follow

Superface allows GPT builders to easily connect to any API, enabling them to create, retrieve, and manage data from external platforms.

Let me know if this post is useful to you and if you encounter any problems with my instructions!

Resources

Facebook is constantly changing their products, so it's possible that by the time you read this article, some flows are entirely different. You can refer to the following official resources, although in some cases these may also be outdated.

Set Up a Business Account on Instagram
Add or Change the Facebook Page Connected to Your Instagram Business Account – as of September 2022 this guide seems to be outdated; it describes the possibility to connect a Facebook page from Instagram side, but I didn't find the described functionality in Instagram's web interface.
Instagram Graph API: Getting Started

API is like a box of chocolates, you never know what you GET

Jan Vlnas — Fri, 16 Sep 2022 14:48:44 +0000

Imagine you get a task to integrate an API. It's a partner API, the code is outside your control, and it's not very widely used. But there is at least some documentation. You quickly dive in. There's a list of endpoints, required parameters, authorization, but something's missing. There is absolutely no information about API responses, not even how a regular response should look like, not to mention possible errors. So, it's time to pull out an HTTP client, and start poking the API with requests to see how it behaves.

Dealing with undocumented responses isn't such a big deal, we have tools for visualizing JSON (for example JSON Crack) and inferring its schema (like MakeTypes). It's those edge cases which completely throw off any assumptions you already made.

Recently, I worked with an API which was like that. The documentation only mentioned that endpoints exist, and some particular parameters are required. Once I figured API's weird, custom-made authorization schema, I started to poke around the endpoints.

First thing I noticed was responses’ content type. While the API responds with JSON, the content-type header marks the response as HTML. This breaks automatic parsing in most HTTP clients (and also syntax highlighting and formatting), but the response can still be parsed manually. The fun started when some responses included HTML warnings with the data, for example:

<br />
<b>Warning</b>: Undefined array key "TEST" in <b>/some/source/file</b> on line <b>123</b>
<br />
<br />
{"foo":"bar"}

Technically, the API behaved consistently: it claimed to respond with HTML, and so it did. But in practice, this behavior renders the API mostly useless for consumers. Sure, I could search the response and parse only the JSON part. While finding clever hacks is fun, it's usually not sustainable in the long run. Not to mention that keeping error messages exposing your source files structure is a security risk. I reported the issue to the API provider (somehow surprised no one reported this issue before). They were swift to respond and improved their API based on my suggestions.

While I love to solve technical challenges as any other developer, sometimes it's really easier to talk to people. However, it frustrates me to think how much time is wasted by examining on poorly documented APIs with surprising behaviors. And while we should be advocating for better developer experience even for internal and partner APIs, we can also share our discoveries by abstracting APIs into reusable use cases.

Now to you: What was the weirdest API behavior you dealt with?

Cover photo credit: Lukas on Pexels.

“Look ma, no config file!” Introducing OneSDK 2.0

Jan Vlnas — Tue, 16 Aug 2022 13:08:00 +0000

OneSDK is a universal client for consuming API integrations. It is a core component of the Superface ecosystem — whether you pick an existing integration, or decide to build one yourself.

Today, we are excited to announce OneSDK v2.0.0, a new major version which simplifies the usage of Superface integrations and reduces the memory footprint. If you are using OneSDK in your application, pay attention to the breaking changes and check out the upgrade guide. But first, let’s dive into the new features.

Use OneSDK without configuration

The most significant change in this release is simplified use of integrations published in the Superface registry. Previously, you had to use the Superface CLI (@superfaceai/cli package) to install the integration profile and configure providers. This updated the super.json configuration file and saved .supr files into local project.

With OneSDK v2, these steps are not required anymore. To start using Superface integrations, pick one from the catalog, install the @superfaceai/one-sdk package, and use the provided example code.

As an example, let’s display a weather forecast with my favorite weather service, wttr.in.

First, install OneSDK with npm:

npm i @superfaceai/one-sdk

And paste the following code into a weather.js file:

const { SuperfaceClient } = require('@superfaceai/one-sdk');

// Change to your city!
const city = 'New York City, NY, USA';

const sdk = new SuperfaceClient();

async function showWeather() {
  const profile = await sdk.getProfile('weather/forecast-city@1.0.0');
  const result = await profile.getUseCase('GetWeatherForecastInCity').perform(
    {
      city,
    },
    {
      provider: 'wttr-in',
    }
  );

  console.log(result.unwrap());
}

showWeather();

When you run node weather.js, you should get a temperature forecast for the next three days:

[
  {
    averageTemperature: 28,
    date: '2022-08-11',
    maxTemperature: 34,
    minTemperature: 23,
  },
  {
    averageTemperature: 26,
    date: '2022-08-12',
    maxTemperature: 30,
    minTemperature: 22,
  },
  {
    averageTemperature: 25,
    date: '2022-08-13',
    maxTemperature: 31,
    minTemperature: 20,
  },
];

OneSDK v2 will fetch and cache profiles at runtime, which means super.json configuration is no longer necessary. If you already use OneSDK with super.json, you don’t need to change anything. The super.json file acts as a central place for locking profile versions and decoupling providers configuration from the code. It is also needed for using local profiles and maps (see breaking changes below).

Pass security values at runtime

While OneSDK doesn’t need a configuration file, many providers require an API token or another form of authentication. Previously, we used integration parameters for passing provider-specific values at runtime, such as OAuth access tokens. Now you can also use security values in the perform method. Pass in a security option with required security values; here is an example for sending an email with SendGrid:

const profile = await sdk.getProfile('communication/send-email@2.1.0');

// Use the profile
const result = await profile.getUseCase('SendEmail').perform(
  {
    from: 'no-reply@example.com',
    to: 'jane.doe@example.com',
    subject: 'Your order has been shipped!',
    text: 'Hello Jane, your recent order on Our Shop has been shipped.',
  },
  {
    provider: 'sendgrid',
    security: {
      bearer_token: {
        token: '<your token from sendgrid>',
      },
    },
  }
);

The property name (bearer_token in this example) of the security value is provider-specific. The code examples in the catalog will help you figure out what values need to be passed in.

Fewer dependencies, less memory usage

We want OneSDK to be usable everywhere you can run Node.js, regardless of resource constraints. Previous versions required a Comlink parser, which in turn depended on the TypeScript package at runtime. In OneSDK v2, we have removed the parser, resulting in a leaner package with a smaller memory footprint.

With a single profile and a single map, OneSDK v2.0.0 consumes 30 MB less memory than OneSDK v1.5.2 (benchmark code)

Since the parser is no longer included in OneSDK, local maps and profiles need to be compiled with Superface CLI. Read below about this breaking change.

Breaking changes

Change in a major version implies breaking changes, which is also the case for OneSDK v2. The most significant breaking change is the removal of the Comlink parser, which affects OneSDK users with local profile and map files.

We have also changed the default cache location and reduced the usage of the superface/ directory. These changes (hopefully) won’t break your application, but can help you clean up the previously generated files in your project.

Local profiles and maps require a compile step

If your project depends on local profiles or maps, OneSDK v2 won’t automatically parse them anymore, and throws an error during initialization. To see if you are affected, check the superface/super.json for "file" properties, for example, the following configuration depends on a local profile (my-profile.supr) and a map (my-profile.my-provider.suma):

{
  "profiles": {
    "my-profile": {
      "file": "./my-profile.supr",
      "providers": {
        "my-provider": {
          "file": "./my-profile.my-provider.suma"
        }
      }
    }
  }
}

OneSDK will look for compiled *.ast.json files to load them at runtime. Run npx @superfaceai/cli@latest compile in your project to compile the source profiles and maps. We recommend adding a compilation step to your existing build step or to run it at the application start. More details are available in the upgrade guide.

Clean up of `superface/` directory

With the use of super.json configuration becoming optional, and the removal of local .supr files, we took the first steps towards phasing out the superface/ directory.

The cache directory location changed from superface/.cache to node_modules/.cache/superface. superface/.cache directory can be removed.
superface/grid directory can be removed.

You can find more detail in the upgrade guide.

Tell us your feedback

We are eager to hear your feedback about our latest release. Join us on the Superface Discord server to tell us what you are building, or hit us up on Twitter. If you run into any issues, report them in the OneSDK repository. Or just reach out to us — we are happy to help!