DEV Community: Suhas Chatekar

What makes a chatbot great?

Suhas Chatekar — Mon, 01 Jan 2018 20:38:59 +0000

This post was originally published on my blog.

Last month I attended the Big Data London conference. One of the sessions run by Microsoft had a slide (#7) that roughly read like

What makes a bot great?

not how much "AI" it has

Not how sophisticated the language model is

Not whether it uses voice

Not whether it used guided UI

It is the personality of the bot that makes it great

What does "personality of a bot" even mean?

Honestly, it is as complicated as the personality of us humans. There is no one way to define what a great personality is. It's a combination of a number of personality traits. And the same is true for bots. There are a number of traits that a great bot has. In this post, I will go over some of these traits that I have learned.

#1 - Give your bot some character

First of all, give your bot a name. When we built our first bot we did not do this. We ended up having everyone refer to it as just a chatbot. But then we learned that you can make the bot more lively simply by giving it a name. Now, when you are giving it a name, think about whether you want it to be male or a female character.

Do you want your bot to come across as a machine or as a human? This is very important and will go a long way in giving your bot a character. Generally, bots with machine-like characters will be mostly very static in their conversations. On the other hand, human-like bots will engage their users in lively conversations.

Once you have done that, think of a bot persona. A number of factors will drive the bot persona - demographics of the target customer base, business domain and the task at hand, to name a few. Bot persona will help you greatly when you are work out responses to different situations.

#2 - Watch the tone of voice

Thanks to Catherine Hornby for this tip.
After you have picked up a persona for your bot, it is essential to decide how that persona will respond to different conversations e.g. formal, informal, use of slang, colloquialism etc. When doing UX designs, we tend to put together style guides and colour pallets to help wider teams come up consistent UI design. The same can be done with bots. You can have a copywriter put together a conversation style guide for different bot persona. Anyone building a bot conversation can refer to this style guide and come up with consistent bot responses.

#3 - Be humorous

Humour is everywhere around us, so why not with bots? When we ran the first internal test of our bot, a significant proportion of conversations started with users trying to be humorous. Even when your bot is built around a non-humorous business problem, the moment users find out that they are talking to a bot, there will always be a few who would try to be naughty and start saying funny things to the bot. You may not want to go overboard with handling every possibility in the universe but having humorous responses to some funny statements from the users will make your bot stand out.

#4 - Do not be monotonous

This is the most important thing. Over time, your users will keep coming back to the bot and ask similar questions. It is easy to build a non-lively conversation by having the bot respond in a monotonous manner.
Imagine you welcome every time the user starts talking to the bot. If you welcome the user by saying "Welcome back Bob" every time, then soon, that will become very monotonous and less engaging for the user.

It does not take a lot to bring variation in bot responses if you are using tools like Watson Conversation or Microsoft Bot Framework. Most of us will ignore this during development as it's an overhead. But do not forget to add this in before you go live or even go out for a wider beta test. And do not add variations just for the welcome message but for every response that is coming back from the bot.

#5 - Handle small talk

Small talk is relatively straightforward to handle and very easy to fall through the cracks. Some bot services like Dialog Flow will handle small talk automatically for you. All you have to do it enable it. But for others, you need to build conversations around small talk. If you are sailing in this boat, consider the following

Greet the user with a welcome message at the beginning of the conversation and assume that user may reply with small talk like "thanks", "hello".
Always end the conversation with small talk like "bye", "see you soon" etc. and assume that user will respond with similar words
Messaging is inherently slow and full of latency so it's possible that user become impatient when they do not see a response from the bot in time. Some user may try to talk to the bot using a word like "Hi", "Still there?" while the bot is still working on their previous request. It's best to handle such utterances appropriately. and make sure that you remove monotony from these responses.

Small talk is a never-ending thing. There will never be enough of it. But if you look at it from your domain lens, you can come up with a reasonably small list of things that people can say that bot can reply in a friendly way. And for the rest, you can always have a number of variations of the same response (remember trait #3 - Do not be monotonous)

#6 - Handle out of context conversations to a reasonable degree

One of the best things about natural language interfaces is that users get the freedom to express in whatever way they want. But this also becomes one of the biggest pain points for us, bot developers. Now, we have to deal with users who change the conversation context in the middle of a conversation. Let's look at an example of a hotel finder bot

Bot: Welcome to hotel finder. Where are you looking to travel?
User: I am looking for a room for 2 in Dubai next week
Bot: Sure thing. Let me find the best room for you
Bot: What do you think of the following options?
...
User: how long before I can cancel a booking?

The conversation was in the middle of a context around hotel search. User asks a question about the cancellation. The best thing for the bot to do here is to answer the user's question on cancellation (if modelled to do so) and then go back to hotel search. Microsoft Bot Framework supports such use cases out of the box. With others, you will need to do some work yourself but this is definitely possible.

Handling out of context conversation is important because when the user makes a typo that results in bot not recognising what the user is saying, then you would want to present a friendly message to the user and then get back into the previous conversation context.

#7 - Use sentiment analysis tools to determine user's mood

The whole natural language understanding field is in its infancy still. The lack of maturity in the technology makes it difficult to build bots that can match a human's speaking abilities and wit. So there will be times when you will leave a customer unhappy, frustrated or angry. Use sentiment analysis tools on every user statement to understand user's mood and respond accordingly. Do not wait for an angry or frustrated customer to tell you that they want to speak to a human. Do not forget to ask a happy customer for their feedback.

#8 - Handoff to a human before the conversation becomes too frustrating for the user

Make it easy for human agents to take over a conversation from the bot. Make it easy for humans to switch from the bot to a human (e.g. when they type help). Make it easy for the bot to hand the conversation over to a human agent.

You may not want to bake all of the above into your bot from day 1. But at least give the users an option to switch from a bot to a human, to begin with. Analyse how your users are interacting with the bot and decide what is the right time to get the other two features.

#9 - Offer a help menu

Last but not least, offer a help menu in the form of a conversation. The simplest thing you can do is when the user types something like "help" or "I need help", you can present a message back to the user listing everything that the bot can help the user with. In this response, you can also describe how to talk to a human.

Again, Microsoft Bot Framework offers such feature out of the box. For other services, you will need to build this yourself.

Converting a JavaScript project to Typescript, one file at a time

Suhas Chatekar — Mon, 21 Aug 2017 19:04:18 +0000

Convert a JavaScript application to Typescript

JavaScript is good but, like me, if you come from statically typed languages then it becomes a bit annoying to deal with lack of types when the project grows big. Luckily there is Typescript but adding Typescript retrospectively may not be a very straight-forward job. Recently I successfully converted a sizable JavaScript project into Typescript, one file at a time. Converting one file at a time to Typescript is really powerful because then you can make the change incrementally without having to stop delivering the features that your product owner wants. I tried doing this different ways. Here, I will talk you through the one that worked.

This a long-ish read. I have divided the content into three main sections

Adding typescript config and webpack
Adding Type declaration files
Convert the code into typescript

The source code for this article is on GitHub. It is a very simple calculator application with just two files. I think that is enough files to demonstrate the approach. The initial javascript source is in master branch. the changed source code at the end of every section is in appropriate branches in the same repository.

Let's drive straight in then.

1. Adding typescript config and webpack

Any typescript code has to be transpiled down to JavaScript before it can be run. That is where webpack comes in handy. If you have never used Webpack before, then I recommend reading A detailed introduction to webpack before proceeding.

We start by installing webpack using the following command

npm install --save-dev webpack

Note that we are adding webpack as a development dependency. This is because it's only used to convert typescript code into javascript. Next we need a webpack configuration. Add a new file named webpack.config.js at the root of the project and the following content to it

var path = require('path');
var webpack = require('webpack');


module.exports = {  
  entry: { 
    index: "./index.ts"
  },
  target: 'node',
  module: {
    loaders: [
      { test: /\.ts(x?)$/, loader: 'ts-loader' },      
      { test: /\.json$/, loader: 'json-loader' }
    ]
  },
  plugins: [
    new webpack.DefinePlugin({'process.env.NODE_ENV': '"production"'})
    ],
  resolve: {
    extensions: ['.ts', '.js', '.json']
  },
  output: {
    libraryTarget: 'commonjs',
    path: path.join(__dirname, 'lib'),
    filename: '[name].js'
  },
};

Let's quickly go over the contents of this file. We have imported path module to make it easy to do some path manipulation in the output section of the config. We have also imported webpack to define a plugin in the plugins section. Let's not worry about this just yet.

The file is basically just exporting a JSON object which webpack uses as configuration. Webpack has a large number of configuration options but the ones we have configured here are minimal ones needed for a typescript project. Let's look into each of the configuration we have defined in this object

entry

This tells webpack where to begin transpiling. Webpack will start with the files specified in entry, converts them into JS (see module section next) and then goes through every module that these modules import till it has reached the end of the tree. We do not have to have a single entry point. We can provide any number of entry points we want here. We have specified index.ts as our entry point. This file does not exist yet. We will eventually convert our entry module index.js into index.ts.

target

Target tells webpack where you want to run the final Javascript code. This is important because the code that gets generated to be run on server side is different from the code that gets generated to be run in a browser. For this example we specify node which is for the code that gets run on the server side

module

This is where the most of the magic happens. We have specified on the loaders part of this object. Webpack uses different loaders to transpile files. In our case, we have a ts-loader to transpile any Typescript files and a json-loader which I have left there in case we add a json file later on. Loaders need to be installed separately and they come as their own NPM packages. For our config, we need to install ts-loader and json-loader usin the following command.

npm install --save-dev ts-loader json-loader

plugins

Let's ignore that for a moment

resolve

This is where you tell webpack which file extensions to scan during its transpilation process. We have added .ts and .js both as we want to convert one js file to ts at a time. This means, we will have a mix of js and ts files in our project and we want webpack to consider both

output

This is where we tell webpack how do we want the output of the transpilation to appear. We are saying that we want the output files to be named after the key name we used for the file in the entry section. We want the output files to be copied into a folder named lib under the current directory. And we want webpack to use commonjs module system.

Again, if this is the first time you are using webpack, then do not worry too much about the content of this file. This is a minimal config that just works for any server side code.

Next we need a Typescript config. Add a file named tsconfig.json to the project. Again, the minimal contents for this file are as below

{
    "compilerOptions": {
        "target": "es5",
        "module": "commonjs",
        "noImplicitAny": true,
        "lib": [
        "es5","es2015", "es6", "dom"
        ]
    }
}

This is telling the Typescript compiler that we want the resulting JS to be ES5 compliant and we want to use commonjs as our module system. We have also added a noImplicitAny which is set to true. This forces you to declare any variable of type any instead of leaving the type declaration out and compiler marking the variable as any. This helps to catch cases where we forget to declare type for a variable.

Next we need a way to invoke webpack. There are two ways you can do this. The webpack npm package that we installed earlier, you can install that globally and just run webpack from the console at the root directory of the project. Or you can add an NPM script in your package.json that uses the locally installed webpack version like below

"scripts": {
    "build": "node_modules/.bin/webpack --config webpack.config.js"
},

Note that I have padded a --config flag which is not really needed because webpack looks for a file named webpack.config.js by default. But if you prefer to name your webpack config file differently then make sure you pass the --config flag.

The source code at the end of this section is in add-webpack branch.

2. Add Type declaration files

We need to find the first module that we can safely convert to Typescript. This is usually the entry module of our project. In our example, that would be index.js. To use the full power of Typescript in our converted module, we need to have type declaration files for other modules that this module is dependent on.
There are two concepts around type declaration files that we need to understand. I am assuming that you know what type declaration files are, if not, I would recommend reading the official guidance on this topic

We need to explicitly install type declaration files for any external module. In our example, we have an external module called prompt-sync for which we will need to install type declaration files
For our own modules that we have not converted into Typescript yet, we need to write type declaration files ourselves as a stop-gap arrangement till the time we convert that module into Typescript

Installing type declaration files for external modules

Type declaration files for most NPM packages are already made available by the community. We can run the following command to install the Type declaration files for our prompt-sync package

npm install --save-dev @types/prompt-sync

If the type declaration file is available, it will get installed. If not, you will see an error. You will need to create the necessary type declaration files yourselves.

Creating type declaration files for own modules

Type declaration files for a module contain interface, function and type declarations for the bits that the module exports. They are declared in a file with extensiond.ts and named after the module name or index.d.ts. For instance, the type declaration file for the prompt-sync module that we just installed is named index.d.ts and you can find it in node_modules/@types/prompt-sync folder. That is one of the known location that typescript compiler searches during module resolution. You can read more about the module resolution process that typescript compiler follows in the Typescript Handbook. One of the strategies used by the compiler to resolve modules is to look for a type declaration file matching the module name at the same location as the imported module. For instance, if we import a module like below

import * as calc from './calculator'

then typescript compiler will look for a calculator.ts or calculator.d.ts file in the current directory. We can use this mechanism to put our existing calculator.js file behind a type declaration by creating a file calculator.d.ts like below

declare module calculator {
    export function add(a :number, b :number): number
    export function subtract(a :number, b :number): number
    export function multiply(a :number, b :number): number
    export function divide(a :number, b :number): number
}
export = calculator;

Notice that this is exposing the same methods as our calculator module but has annotated arguments and return values with a number type. This file needs to be placed next to calculator.js.

Creating Type declaration files for external modules

We do not have any external module in this example that does not have Type declaration files available. But if that were the case with you, you can combine the knowledge from the above two points. First you build your own type declaration file and name it index.d.ts. This can include only the methods/interfaces from the external module that you are using in your code. This type declaration file file needs to be kept under the folder node_modules/@types/{module_name}/

I have never personally tried this so cannot vouch for reliability but this is what community defined Type declaration files are doing under the hood.
The source code at the end of this section is in add-types branch.

3. Convert the entry module into TypeScript

Finally we are ready to convert our first module into TypeScript. There is not much really in this step. Rename index.js to index.ts and start rewriting the module in typescript. If you use the import syntax for bringing in the dependent modules then TypeScript compiler will look at the type declaration files of the target module and enforce type checking in addition to usual Javascript compiler checks. Here is how my converted index.ts file looks like

import * as p from "prompt-sync"
import * as calc from "./calculator"
let prompt = p();

function readInput() {
  console.log("Welcome to the calculator. Choose one of the following options");
  console.log("1. add");
  console.log("2. subtract");
  console.log("3. multiply");
  console.log("4. divide");
  console.log("5. exit");

  var option = prompt(">> ");

  if (option !== "5") {
    console.log("Enter the first number");
    let a = parseInt(prompt(">> "));

    console.log("Enter the second number");
    let b = parseInt(prompt(">> "));

    let c;
    switch(option){
      case "1": {
        c = calc.add(a, b);
        console.log(`a + b = ${c}`);
      }
      break;

      case "2": {
        c = calc.subtract(a, b);
        console.log(`a - b = ${c}`);
      }
      break;

      case "3": {
        c = calc.multiply(a, b);
        console.log(`a * b = ${c}`);
      }
      break;

      case "4": {
        c = calc.divide(a, b);
        console.log(`a / b = ${c}`);
      }
      break;
    }    

    readInput();
  }
}

readInput();

console.log("Thank you for using calculator. Good Bye");

Yeyy. We converted our first module from javascript to typescript. If you run npn run build at this point, you will notice the webpack successfully gives us a packaged bundle in lib/index.js that is ready to use.

The source code at the end of this section is in convert-entry-module branch.

4. Keep going

Converting the first javascript file is a big win. You have basic plumbing in place now to take on the bigger task. You may want to expand your webpack configuration to include other types of files you may have in your project, add production build steps like minification, uglification etc. At the same time, you also need to keep converting more and more files from javascript to typescript. The next logical step is to get rid of our own type declaration files by converting the javascript modules into typescript. Let's change the calculator module to get rid of calculator.d.ts. There are a number of ways, you can rewrite calculator module using typescript. The simplest is to just export the four methods in the module like below.

export function add(a: number, b: number): number {
  return a + b;
}

export function subtract(a: number, b: number): number {
  return a - b;
}

export function multiply(a: number, b: number): number {
  return a * b;
}
export function divide(a: number, b: number): number {
  return a / b;
}

Delete the calculator.d.ts file and re-run npm run build you would get your packaged bundle in lib/index.js.

That's it. We have converted everything in this project from javascript to typescript. The source code at the end of this section is in keep-going branch.

Return well-formed error responses from your REST APIs

Suhas Chatekar — Fri, 18 Aug 2017 14:00:58 +0000

REST recommends using standard HTTP status codes to indicate what has happened on the server in response to a request. Errors from 4xx and 5xx family are used to indicate that request is not processed and there is some error either on the client side or on the server side e.g. 401 Unauthorised is used to indicate that the request cannot be processed because it does not anything to prove the identity of the sender. Standard HTTP status codes are a good starting point to indicate error condition but they are not enough. Especially if you are developing APIs that are consumed by large number of developers or are open to access by anyone then you may want to return well formed error responses that provide more details around the error. Initial inspiration for this idea comes from Generate Structured Errors chapter of the GitBook HTTP API Design Guide. I personally recommend this book to anyone who wants practical advice on building REST APIs.
At Collinson Group we have started returning JSON similar to the one below in some 4xx/5xx responses

[{ 
  "errorCode": "BAD_FORMAT", 
  "field": "email", 
  "originalValue": "suhas.chatekar", 
  "mesage": "{email} is not in correct format",
  "helpUrl": "/help/BAD_FORMAT#email" 
  }]

The above JSON is returned in response body whenever we believe that the standard HTTP status is not enough for the calling application to know what has gone wrong. Let’s quickly see what purpose each of these fields fulfil and what values they may contain.

errorCode

As the name suggests this field holds an unique error code. We decided to use string based error codes as they are easy to read. People also use numbers for this. I personally feel that numbers are cryptic for this kind of information.

field

Name of the field which has this error. This is the same name that appeared in the incoming request.

originalValue

This field contains the original value from the request. This is optional in most cases. While the client can make use of this information if they need to, there is one situation where we found returning this information to be useful. We had a POST API that accepted an array of the same resources. If more than one resource fails the same validation on the same field then we end up returning same error response twice. How will the client know which resource was in error? Adding original value in the response helps track the client which resource in the request had an error.

message

This field contains a user-friendly message. If the consumer of the API is a front end application then it may choose to use this message directly to convey details of the error to the end user. There are two things to note around this field. One, the example above shows the message as tokenised with {email} as a token in the message. The API should replace this token with the original value from the request before returning the response. Two, this message should be localised in a language that client prefers. This assumes that you have got a mechanism in your API for your clients to specify a preferred language.

helpUrl

This field contains URL to a page that describes this particular error in more detail. The help page may answer the following questions (in addition to any other details you may want to include)

What causes this error? Examples will help here.
How to fix this error?
If you continue to get the error, how do you get help?

I call this mechanism discover-able documentation because developers of the client applications do not have to know where to find this documentation when they encounter an error, they automatically discover appropriate documentation instantly.

Error codes that weÂ use

The following list of error codes satisfies most common field level errors.

There are some domain-specific error codes we use that I have omitted. You can define as granular error codes as possible. Are you using well-formed error responses in your APIs? Do you find this approach useful?

How do you choose between an interface and abstract class?

Suhas Chatekar — Sat, 25 Feb 2017 18:37:36 +0000

I have been asking this question in interviews for some years now. I get to hear a lot of interesting answers from folks. All these answers have helped me make a great addition to my knowledge. If you google this topic you would find a lot of interesting articles. This article on MSDN offers a good discussion on the topic. To summarise the recommendations from there and few other articles I have come across, here are the reasons in favour of abstract classes

Use abstract class if you have a default implementation of some behaviour that child classes do not have to implement
Prefer abstract classes if your contract has a possibility of changing over time. So if you are using an abstract class and need to add a new method to your abstract class, you can happily add that without breaking any code using that class. The same is not true for interfaces.

And these are the reasons in favour of interfaces

Since multiple inheritance is not supported in C#, you cannot inherit your class from two abstract classes. An interface is your only option in such situations.
If there is no default or common behaviour among all the classes that are inheriting from abstract class then interface may be a better choice.

I personally do not believe that these give a complete picture.

What is wrong with this reasoning?

First, above recommendations are mostly around syntax a particular language supports and not around semantics (e.g. use interfaces if you need multiple inheritance). By semantics I mean, what definition of an interface fundamentally differentiates it from an abstract class?
Second, I feel the above criteria are too futuristic. By that I mean, they all depend on you knowing how your design is going to take shape in future. At times, I may have some idea of how my design is going to take shape in future but most of the times I do not have enough clarity to know in advance

Whether I am going to need to inherit from multiple abstract classes or not
Whether there is going to be a default implementation of some contract or not
Whether I would add a new method to a contract or define a new contract entirely in order to implement a change

So, if you do not know which way your software is going to go, there is no way you can base your decision of interface or abstract class on these reasons.

So how do we decide?

Lately, I have been using below heuristic to determine when to use interfaces/abstract classes and I feel quite excited about it as it works most of the time.

Interfaces represent capabilities and abstract classes represent type

in other words

Implementing interfaces represents can-do relationship and inheriting from (abstract) class represents an is-a relationship

To elaborate this point, let's consider following two classes

public class Camera 
{ 
  public void Shoot() 
  { 
    //Take a picture here 
  } 
} 

public class Gun 
{ 
  public void Shoot() 
  { 
    //Hit the target 
  } 
}

Both Camera and Gun can Shoot, that is their capability. But they are both not the same type of things, they are completely different. So an interface like below would make more sense here

public interface IShootable 
{ 
  void Shoot(); 
} 

public class Camera : IShootable 
{ 
  public void Shoot() 
  { 
    //Take a picture here 
  } 
} 

public class Gun : IShootable 
{ 
  public void Shoot() 
  { 
    //Hit the target 
  } 
}

If you have learned OO programming the same way I did, then you would remember the classic shape example as below

public abstract class Shape 
{ 
  void Draw(); 
} 

public class Rectangle : Shape 
{ 
  public void Draw() 
  { 
    //Draw a rectangle here 
  } 
} 

public class Circle : Shape 
{ 
  public void Draw() 
  { 
    //Draw a circle here 
  } 
}

Rectangle and Circle inheriting from Shape makes perfect sense here because Rectangle/Circle are a type of Shape.

In closing

As your software grows and new features start pouring in, old features keep changing, you would hit a point where you would say to yourself “Oh God, I should not have used an abstract class here”. Instead of trying to predict feature about who will reuse which code, focus on determining whether it is a capability that you are abstracting away or a common type. If you find that line of thinking difficult, then work out if it is a can-do relationship that you are trying to model or an is-a relationship.

Why you should use standard HTTP methods when designing REST APIs

Suhas Chatekar — Thu, 23 Feb 2017 21:25:00 +0000

One of the characteristics of a good REST API is that it uses the standard HTTP methods in a way they are supposed to be used. We hear this all the time and this is the most fundamental guideline of REST. As generally understood, we use the following HTTP methods while designing REST APIs

GET – For returning resources
POST – For creating a new resource
PUT – For updating a resource
PATCH – For updating a resource
DELETE – For deleting a resource

But beyond the basics, why is it like that? Isn't use of two methods, PUT and PATCH for an update operation confusing?

What do the specs say?

Let's first get the obvious out of the way. Let's see what the specs say

GET

The GET method requests transfer of a current selected representation for the target resource.GET is the primary mechanism of information retrieval

What this really means is that you should use GET for an API method that returns the latest version of the resource identified by the API URL.

POST

The POST method requests that the target resource process the representation enclosed in the request according to the resource's own specific semantics.
This is slightly confusing but that is because POST can be used to do more than one thingâ€Š–â€Šone of which is creating a new resource.

This is confusing because if the server can perform more than one operations in response to a request then how do consumers find out what happened exactly? The specification says that the server should make use of one of the 2xx family of status codes to indicate to the client what happened on the server.

PUT

The PUT method requests that the state of the target resource be created or replaced with the state defined by the representation enclosed in the request message payload.

So PUT can be used to update a resource as well as to create a new resource. That may sound strange at first, but let's keep things simple for now and trust we should use PUT for an API method that lets consumers update the resource identified by the API URL.

DELETE

The DELETE method requests that the origin server remove the association between the target resource and its current functionality.

Again, like POST, DELETE can be used for more than one operations that eventually result in something being removed. Let's trust we should use DELETE for an API method that let's consumers delete the resource identified by the API URL.

PATCH

The difference between the PUT and PATCH requests is reflected in the way the server processes the enclosed entity to modify the resource identified by the Request-URI. In a PUT request, the enclosed entity is considered to be a modified version of the resource stored on the origin server, and the client is requesting that the stored version be replaced. With PATCH, however, the enclosed entity contains a set of instructions describing how a resource currently residing on the origin server should be modified to produce a new version.

PATCH, like PUT, should be used for modifying a resource, however, the semantics of modification is performed as slightly different. We will get to this difference in the next section.

So our original understanding of what each of the HTTP verbs means is in line with what the specifications say what they mean. So far so good. Specification tells us in which situations we should we use each verb, but it's still just a specification. Our implementation can still disregard what the specification tells us to do and have the creation of new resources implemented behind a GET operation and deletions implemented a POST operation. However, there are two big benefits to respecting the specification.

Idempotence is not limited to mathematics

In mathematics, an idempotent operation is one which produces the same result every time it's performed on the same input. A counter is a good example of this. The operation to “set the counter to 6” is idempotent but the operation to “increment the counter by 1” is not. You see the difference? In the first case, no matter how many times you set the counter to 6, it's still set to 6 in the end. Whereas, in the second case, if the counter had an initial value of 4 and every time you increment it by 1, the counter gets set to different value.

Applied to REST, an idempotent operation is the one which does not change the target state of the server. On the other hand, a non-idempotent operation would always change the target state of the server. Notice that we are not talking about state change here but rather the target state of the server. HTTP specification states that GET, PUT and DELETE are idempotent operations. POST, and PATCH are non-idempotent operations. “Like earlier, this is just a specification. Why do you need to follow it?” you might ask. Well, there is a reason and a very good one.

An idempotent operation gives the consumer of the API a guarantee that the state of the server will be the same no matter how many times they call the API. This guarantee is very useful. Imagine that server executes the API method that consumer called successfully but there is an error while sending the response back to the consumer. The consumer ends up getting an error even though the operation has completed successfully on the server. What should the consumer do? Now, if the operation server performed was advertised as idempotent, then A consumer can send the request to execute the operation one more time. It can keep doing that as long as it sees a success response without causing the server to perform any unwanted state change. On the other hand, if the operation was not advertised as idempotent then the consumer would know that it need to check with the server before performing the operation again.

Creation of a new resource is one operation which changes the target state of the server every time it is run with the same inputs leading to a different target state after each execution. Same may apply to deletion of a resource depending on the implementation. Imagine what would happen if we put these operations behind a GET or a PUT verb? A consumer may end up executing these operations multiple times inadvertently affecting the target state of the server negatively. Returning value of a resource, on the other hand, has no impact on the target state of the server and hence should be implemented behind an idempotent verb GET.

There still is a difference between a PUT and a PATCH. Both are used to modify an existing resource. If you go back to what specification says about PUT you would notice that its similar to saying “set the counter to 6”. PUT is modifications where the consumer provides the target state for the complete resource and instructs the server to replace whatever value the resource has currently with the new value the consumer is sending. No matter how many times the consumer performs this operation, the end state of the resource is never going to change. PATCH, on the other hand, is similar to “increment the counter by 1”. PUT can be chunky for resources that are large in size and eats away a lot of bandwidth. PATCH was introduced to let consumers issue an instruction to the server that tells it how to change a part of the resource. This instruction if executed multiple times could result in an unwanted end state.

In the end, it's all down to the implementation but if your implementation respects the specification then the life of your API clients would be a lot easier. While we are talking about respecting the specification, there is another interesting characteristic that the specification says all HTTP servers and proxies should build when dealing with HTTP request. Your REST API that respects the specification can make good use of this.

Web is fast because of caching

The web is probably one of the heaviest users of cache. It not only makes loading of websites faster, it also works so reliably that not making use of it when it's at our disposal would be foolish. REST uses HTTP and so it should make use of its caching characteristics where possible. The HTTP cache specification is quite detailed about how it should be implemented by HTTP servers and proxies but I would like to pick two things from the specification that directly apply to REST.

An operation behind a GET endpoint does not change the target state of the server, therefore, the response of a GET endpoint can be cached resulting in further requests to the same endpoint being returned faster from the cache
An endpoint behind PUT, POST, PATCH and DELETE changes the target state of the server, therefore, a successful response out of any of these endpoints can be used to bust the previously cached responses.

I know that I am overly simplifying the caching specification of HTTP but this is one of the most elegant caching specification I have come across. And all it takes to take full advantage of this to respect the HTTP specification.

One last thing about caching that impacts the consumers of an API more than the developers of the API. An API that follows HTTP Cache specification is the most useful to a consumer as long as the consumer uses an HTTP client library that supports caching. I am sure there is at least one such library available for every leading programming platform. So encourage your consumers to use one of those libraries. Having said that, API developers should still make use of caching as HTTP and proxy servers will still cache the responses.

In closing

The Beauty of the HTTP API (and I am intentionally avoiding the word REST here) is that it provides a lot of flexibility to the API developer. Any flexibility is a good thing as long as it's not misused. I have seen many HTTP APIs that either does not make use of recommended HTTP methods or if they do, the developer has little knowledge of why he made those choices. I hope this post throws some light on why we should use recommended HTTP methods.