DEV Community

loading...
Cover image for AMP CMS: API

AMP CMS: API

valeriavg profile image Valeria ・10 min read

Web Application Programming Interface (API) is an HTTP server accepting requests in a certain format and responding to them accordingly.

Because we're building API for use in conjunction with the AMP framework, our obvious choice is to build REST API.

REST API dictates that the server will perform an action based on HTTP Method (GET, POST, PATCH, PUT, DELETE) with an entity, provided in request URL, using request query parameters and body, if applicable. All server responses are also obligated to have a proper HTTP Response Code, indicating response success or error state (e.g. 200, 201, 400, 404, 405, 500)

The Beginning

I've started with creating a GitHub repository with nothing but a license and a readme file with a short description and a roadmap:

- [ ] API
  - [ ] Core functionality
  - [ ] Authorization
  - [ ] Permissions
  - [ ] Custom user fields
- [ ] Control Panel
  - [ ] Base layout
  - [ ] Scripts Editor
  - [ ] Styles Editor
  - [ ] Content/Markup Editor
  - [ ] File manager
- [ ] Page Analytics
- [ ] Deploy at least one project with AMP CMS
- [ ] Digital Ocean One-Click-Deploy
Enter fullscreen mode Exit fullscreen mode

I've made up a quick logo in Figma, using AMP Styleguide and AMP logos:

Alt Text

And now it is finally time to create the very first folder in the project: docs.

Documentation

As a creator, I want my creations to live and evolve. And to ensure it I highly recommend documentation-driven-development.

Writing documentation is also a good way to plan functionality and outline the solution before diving into actual development. And it's super boring to do when the project is ready.

GET: Read, List, DELETE: Delete

No request body is allowed.
Read, List, and Delete parameters are accepted via query/search parameters:

GET /api/item?id=1
GET /api/items?limit=10&offset=5
Enter fullscreen mode Exit fullscreen mode

POST: Create, PATCH: Update, PUT: Upsert

Allowed body: JSON String, URL Encoded Form Data, or Multipart Form Data.
The provided body is passed in the input parameter, along with query/search parameters and files if applicable.

Responses

API always responds with JSON.

Errors

If the response was successful, the returned status code will reflect that (200 or 201).
In case of user input errors API will respond with the following structure:

{
  "errors": [
    {
      "name": "title",
      "message": "Title is required"
    }
  ],
  "code": 400
}
Enter fullscreen mode Exit fullscreen mode

Note: Response status code will be also set the same code

Bootstrapping API

Finally, we can dive into development. Let's start with yarn init (or npm init if you prefer). We'll use semantic versioning, so this version will be 0.0.1.

Next, let's install the dependencies:

yarn add typescript ts-node 
yarn add @types/node @types/mocha @types/chai @types/supertest mocha chai ts-node-dev supertest --dev
Enter fullscreen mode Exit fullscreen mode

We'll use mocha, chai and supertest to test API and the rest comes along with TypeScript.

Add .gitignore before pushing anything to the repository:

node_modules
*.log
.env
Enter fullscreen mode Exit fullscreen mode

We don't want to save logs, local environment files, or node_modules in the repository. The reason for the latter is because some of the modules are built for the host platform (e.g. Mac) and may be running on a different one (e.g. Linux) and we'll be using only a fraction of the modules in production. Apart from the fact that this folder is deeper than the black hole, as you know.

We'll need tsconfig.json :

{
  "compilerOptions": {
    "target": "ESNext",
    "module": "commonjs",
    "allowJs": true,
    "noImplicitAny": false,
    "moduleResolution": "node",
    "esModuleInterop": true
  },
  "include": ["src/**/*.ts"],
  "exclude": ["node_modules"]
}
Enter fullscreen mode Exit fullscreen mode

We'll be compiling TypeScript for NodeJS, so there's no need to target old ES versions.

Let's add common scripts to our package.json:

{
...
"scripts":{
  "start": "ts-node src/index",
  "dev": "ts-node-dev src/index",
  "test": "mocha -r ts-node/register src/**/*.spec.ts"
}
...
}
Enter fullscreen mode Exit fullscreen mode

API Middleware

Let's create the following files structure:

src/
- api/
-- api/index.ts
-- api/api.spec.ts
Enter fullscreen mode Exit fullscreen mode

Internally, API resolvers will be somewhat similar to GraphQL resolvers, i.e. async functions, accepting request parameters, and request context:

{
  item: {
    GET: ({ id }, { items }) => {
      return { item: items.get(id) };
    },
    POST: ({ input }, { items }: typeof ctx) => {
      const id = "itm_" + items.size;
      const item = { id, ...input };
      return { item, code: 201 };
    },
    DELETE: () => ({
      success: true,
    }),
  },
  items: { GET: () => ({ items: [...items.entries()] }) }
}
Enter fullscreen mode Exit fullscreen mode

Resolvers should always return an object. Responses can have an explicit status code, otherwise, the server will respond with code 200.

Let's write an integration test for the API middleware. We'll skip it for now, but it's important to see where we're heading:

// api/api.spec.ts
import { describe, it } from "mocha";
import { expect } from "chai";
import request from "supertest";
import middleware from "./.";

const items = new Map<string, { id: string; title: string }>([
  ["itm_1", { id: "itm_1", title: "Item#1" }],
]);

const ctx = { items, log: console };

const resolvers = {
  item: {
    GET: ({ id }, { items }) => {
      return { item: items.get(id) };
    },
    POST: ({ input }, { items }: typeof ctx) => {
      const id = "itm_" + items.size;
      const item = { id, ...input };
      // We're not pushing to the items to keep tests clean
      return { item, code: 201 };
    },
    DELETE: () => ({
      success: true,
    }),
  },
  items: { GET: () => ({ items: [...items.entries()] }) }
};

const api = middleware(resolvers, ctx);

describe.skip("API Middleware Integration test", () => {
  it("GET /api/item", (done) => {
    request(api)
      .get("/api/item?id=itm_1")
      .set("Accept", "application/json")
      .expect(200)
      .expect("Content-Type", /json/)
      .end((err, res) => {
        if (err) return done(err);
        expect(res.body.item).to.have.property("id", "itm_1");
        done();
      });
  });
  it("GET /api/items", (done) => {
    request(api)
      .get("/api/items")
      .set("Accept", "application/json")
      .expect(200)
      .expect("Content-Type", /json/)
      .end((err, res) => {
        if (err) return done(err);
        expect(res.body).to.have.property("items");
        expect(res.body.items).to.have.length(1);
        done();
      });
  });
  it("POST /api/item", (done) => {
    request(api)
      .post("/api/item")
      .set("Accept", "application/json")
      .send({ title: "Item#2" })
      .expect(201)
      .expect("Content-Type", /json/)
      .end((err, res) => {
        if (err) return done(err);
        expect(res.body.item).to.have.property("title", "Item#2");
        done();
      });
  });
  it("DELETE /api/item", (done) => {
    request(api)
      .delete("/api/item?all=true")
      .set("Accept", "application/json")
      .expect(200)
      .expect("Content-Type", /json/)
      .end((err, res) => {
        if (err) return done(err);
        expect(res.body).to.have.property("success", true);
        done();
      });
  });
});

Enter fullscreen mode Exit fullscreen mode

The middleware function will accept two parameters: resolvers object and context. This time there are only two context elements: an in-memory map, serving as a data source, and a generic logging object, set to console.

The middleware function needs to return a function, that in its turn accepts request and response objects respectfully.

This middleware needs to route the request to the proper resolver, collect and pass request parameters as object properties, and return JSON response.

These are three things. That means we'll need three functions.

Let's add a stub in api/index.ts for now (to avoid typescript errors):

//api/index.ts
export default function api(...params: any): any {
  return (req, res) => {
    res.end();
  };
}

Enter fullscreen mode Exit fullscreen mode

Request routing

Let's start with routing. Create routing test in api/routeRequest.spec.ts:

import { describe, it } from "mocha";
import { expect } from "chai";
import routeRequest from "./routeRequest";

describe("routeRequest", () => {
  it("routes existing requests", () => {
    const GET = () => {};
    const DELETE = () => {};
    const POST = () => {};
    const PATCH = () => {};
    const getItems = () => {};

    const resolvers = {
      item: {
        POST,
        GET,
        PATCH,
        DELETE,
      },
      items: {
        GET: getItems,
      },
    };
    const makeUrl = (url: string) => new URL(url, "http://localhost");

    expect(routeRequest(makeUrl("/api/item?id=1"), "GET", resolvers)).to.equal(
      GET
    );

    expect(routeRequest(makeUrl("/api/item"), "POST", resolvers)).to.equal(
      POST
    );

    expect(
      routeRequest(makeUrl("/api/item?id=1"), "DELETE", resolvers)
    ).to.equal(DELETE);

    expect(
      routeRequest(makeUrl("/api/item?id=1"), "PATCH", resolvers)
    ).to.equal(PATCH);

    expect(
      routeRequest(makeUrl("/api/items?limit=3"), "GET", resolvers)
    ).to.equal(getItems);
  });
});
Enter fullscreen mode Exit fullscreen mode

This function we can implement right away in api/routeRequest.ts:

import { APIResolver, APIResolvers, HTTPMethod } from "../types";
import { HTTPMethodNotAllowed, HTTPNotFound } from "./errors";

export default function routeRequest(
  url: URL,
  method: HTTPMethod,
  resolvers: APIResolvers
): APIResolver {
  const path = url.pathname
    .replace(new RegExp(`^\/api\/`, "i"), "")
    .toLowerCase();

  if (!(path in resolvers)) throw new HTTPNotFound();
  if (!(method in resolvers[path]))
    throw new HTTPMethodNotAllowed(
      Object.keys(resolvers[path]) as HTTPMethod[]
    );
  return resolvers[path][method];
}
Enter fullscreen mode Exit fullscreen mode

You probably noticed types file:

//src/types.ts
export type HTTPMethod = "GET" | "POST" | "PATCH" | "PUT" | "DELETE";

export type MaybePromise<T> = Promise<T> | T;

export type LogFunction = (...args: any) => void;

export type APILogger = Record<"error" | "info" | "log" | "warn", LogFunction>;

export type APIContext = { log: APILogger };

export type APIResolver<P = any, C = any, R = any> = (
  params: P,
  context: APIContext & C
) => MaybePromise<R>;

export type APIResolvers = Record<
  string,
  Partial<Record<HTTPMethod, APIResolver>>
>;

export type APIError = { name: string; message: string };

export type APIErrorResponse = {
  errors: Array<APIError>;
  code?: number;
};
Enter fullscreen mode Exit fullscreen mode

And errors dictionary:

// src/api/errors.ts
import { HTTPMethod } from "../types";

export class HTTPError extends Error {
  constructor(public code: number, public message: string) {
    super(message);
    this.code = code;
  }
}

export class HTTPNotFound extends HTTPError {
  constructor(message?: string) {
    super(404, message ?? "Not Found");
  }
}

export class HTTPMethodNotAllowed extends HTTPError {
  constructor(methods: HTTPMethod[]) {
    super(405, `Allowed: ${methods.join(", ")}`);
  }
}

export class HTTPServerError extends HTTPError {
  constructor(message?: string) {
    super(500, message ?? "Internal Server Error");
  }
}
Enter fullscreen mode Exit fullscreen mode

Run yarn test and you'll see our first passing test! Yay!

Request parameters

Now let's write a function, that will collect parameters from each incoming request. It needs to retrieve and parse:

  • JSON body
  • query parameters
  • URL Encoded form data
  • Multipart form data

As usual, we'll start with the test:

import { describe, it } from "mocha";
import { expect } from "chai";
import { IncomingMessage } from "http";
import { Readable } from "stream";
import requestParams from "./requestParams";

const mockRequest = ({
  url,
  type,
  body,
  method,
}: {
  url?: string;
  type?: string;
  body?: string;
  method?: string;
}): IncomingMessage => {
  const stream = Readable.from([Buffer.from(body ?? "")]) as any;
  stream.headers = {
    "content-type": type,
    "content-length": body?.length,
  };
  stream.url = url;
  stream.method = method ?? "POST";
  return stream;
};

describe("requestParams", () => {
  it("parses JSON body", async () => {
    const req = mockRequest({
      type: "application/json",
      body: '{"success":true}',
    });
    expect(await requestParams(req)).to.deep.eq({
      input: { success: true },
      files: {},
    });
  });
  it("parses query params", async () => {
    const req = mockRequest({
      url: "/api/items?limit=10&offset=5",
      method: "GET",
    });
    expect(await requestParams(req)).to.deep.eq({ limit: "10", offset: "5" });
  });
  it("parses url encoded form", async () => {
    const req = mockRequest({
      url: "/api/item?id=1",
      type: "application/x-www-form-urlencoded",
      body: "title=Item&published=true",
    });
    expect(await requestParams(req)).to.deep.eq({
      id: "1",
      input: {
        title: "Item",
        published: "true",
      },
      files: {},
    });
  });
  it("parses multipart form data", async () => {
    const req = mockRequest({
      url: "/api/item?id=1",
      type: "multipart/form-data; boundary=12345",
      body:
        "--12345\r\n" +
        'Content-Disposition: form-data; name="name"\r\n\r\n' +
        "Example\r\n" +
        "--12345\r\n" +
        'Content-Disposition: form-data; name="file1"; filename="file1.txt"\r\n' +
        "Content-Type: text/plain\r\n\r\n" +
        "File Contents\r\n" +
        "--12345--",
    });
    const params = await requestParams(req);
    expect(params).to.have.property("id", "1");
    expect(params.input).to.have.property("name", "Example");
    expect(params.files.file1).to.have.property("name", "file1.txt");
  });
});

Enter fullscreen mode Exit fullscreen mode

For the body-parser implementation, we'll use formidable npm module:

yarn add formidable
yarn add @types/formidable --dev
Enter fullscreen mode Exit fullscreen mode

The api/requestParameters.ts code will look the following way:

import { IncomingMessage } from "http";
import { IncomingForm } from "formidable";

export default async function requestParams(req: IncomingMessage) {
  const params = {} as any;
  const url = new URL(req.url, "http://localhost");
  url.searchParams.forEach((value, key) => {
    params[key] = value;
  });

  if (["GET", "DELETE"].includes(req.method.toUpperCase())) return params;

  params.input = await new Promise((resolve, reject) => {
    new IncomingForm({ multiples: true } as any).parse(
      req,
      (err, fields, incomingFiles) => {
        if (err) return reject(err);
        params.files = incomingFiles;
        resolve(fields);
      }
    );
  });
  return params;
}
Enter fullscreen mode Exit fullscreen mode

Middleware

We've implemented most of the middleware functionality, so let's change describe.skip to describe at the beginning of the integration test and implement the middleware function:

// api/index.ts
import { APIContext, APIResolvers } from "../types";
import { HTTPNotFound } from "./errors";
import requestParams from "./requestParams";
import routeRequest from "./routeRequest";

export default function api(resolvers: APIResolvers, context: APIContext): any {
  return async (req, res) => {
    const log = context.log;
    try {
      res.setHeader("Content-Type", "application/json");
      const method = req.method?.toUpperCase();
      const url = new URL(req.url, "http://localhost");
      const resolver = routeRequest(url, method, resolvers);
      const params = await requestParams(req);
      const response = await resolver(params, context);

      const sendResponse = (code: number, response: object) => {
        res.statusCode = code;
        res.write(JSON.stringify(response));
        res.end();
      };

      if (!response) {
        throw new HTTPNotFound();
      }
      const code = "code" in response ? response.code : 200;
      return sendResponse(code, response);
    } catch (error) {
      res.statusCode = "code" in error ? error.code : 500;
      let message = error.message;
      if (res.statusCode >= 500) {
        log.error(error);
        message = "Internal Server Error";
      }
      res.write(
        JSON.stringify({
          error: message,
          code: res.statusCode,
        })
      );
      res.end();
    }
  };
}

Enter fullscreen mode Exit fullscreen mode

Spoiler alert: all tests will pass now.

API Modules

I'm going to split AMP CMS functionality into modules.
They'll be stored in src/modules.

We'll start with the health module, which will answer { status: "OK" } to GET /api/health:

// src/modules/health/index.ts
export const resolvers = {
  health: {
    GET: () => ({ status: "OK" }),
  },
};
Enter fullscreen mode Exit fullscreen mode

We'll need to add this module to API by adding the following code into barrel file src/modules/index.ts:

//src/modules/index.ts
import * as health from "./health";

export default health;
Enter fullscreen mode Exit fullscreen mode

Once we'll have more modules we'll adjust the code to merge them into one.

And, finally, we'll create src/index.ts:

// src/index.ts
import { Server } from "http";
import modules from "./modules";

import API from "./api";

const port = 8080;

const server = new Server((req, res) => {
  const api = API(modules.resolvers, { log: console });
  if (req.url?.startsWith("/api")) return api(req, res);
  res.statusCode = 404;
  res.write("Not Found");
  res.end();
});

server.listen(port, () => {
  console.log(`http://localhost:${port}/`);
});
Enter fullscreen mode Exit fullscreen mode

If you type yarn dev in the terminal and open http://localhost:8080/api/health you'll see:

{"status":"OK"}
Enter fullscreen mode Exit fullscreen mode

All the other requests will return 404 and text Not Found.

Adding code analyzer and continuous testing

Let's add two development tools that can help any repository stay tidy and avoid breaking changes.

The first tool is called CodeClimate. It's free of charge for open source projects. Here's the official tutorial on how to add your first repo.

When everything is set, navigate to "Repo Settings" -> "Test Coverage" and copy "Test reporter ID":

Alt Text

Your value should be different from mine.

Now we'll set up Github Actions to run our test, collect test coverage, and report to CodeClimate.

Create a .yaml file in .github/workflows directory with the following content:

name: AMP CMS CI
on: [push]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Use Node.js v14.5.0
        uses: actions/setup-node@v1
        with:
          node-version: v14.5.0
      - run: yarn
      - name: Test & Coverage
        uses: paambaati/codeclimate-action@v2.5.6
        env:
          CC_TEST_REPORTER_ID: <PASTE_YOUR_REPORTER_ID_HERE>
        with:
          coverageCommand: yarn coverage
          coverageLocations: "./coverage/lcov.info:lcov"

Enter fullscreen mode Exit fullscreen mode

We'll need to install two more moduled, namely nyc and its TypeScript preset to collect coverage:

yarn add nyc @istanbuljs/nyc-config-typescript --dev
Enter fullscreen mode Exit fullscreen mode

And a new script to package.json:

{
...
 "scripts": {
    "start": "ts-node src/index",
    "dev": "ts-node-dev src/index",
    "test": "mocha -r ts-node/register src/**/*.spec.ts",
    "coverage": "nyc --reporter=lcov --reporter=text-summary mocha -r ts-node/register src/**/*.spec.ts"
  }
...
}
Enter fullscreen mode Exit fullscreen mode

And .nycrc file:

{
  "extends": "@istanbuljs/nyc-config-typescript",
  "all":true,
  "exclude":["**/*.spec.ts"],
  "include":["src/**/*.ts"]
}
Enter fullscreen mode Exit fullscreen mode

Running yarn coverage should show something like this:
Alt Text

Let's add two newly created folders: coverage and .nyc_output to .gitignore.

Phew, good job! Let's mark it in the roadmap:

- [ ] API
  - [X] Core functionality
Enter fullscreen mode Exit fullscreen mode

Add, commit, push!
We can check Github Action is running and grab badges from CodeClimate for our repository:

Alt Text

I encourage you to fix the issues, code climate found on your own, otherwise, you can always peek at my commits :-)

Good job on making this far! See you soon in the next part of the series, where we'll be implementing authorization, users, and permissions.

Discussion (0)

pic
Editor guide