Communication in our day and age is fragmented. Some of the most popular mediums for communication have gone so far in the direction of catering to a very specific end user and creating a “secret club” that, you could argue, connecting is beginning to become increasingly difficult. We have Facebook Messenger and WhatsApp, Apple’s proprietary iMessage and Google’s Messages, and then there is also Slack, for like-minded communities and business use-cases, just to name a few.
At Stream, we think it is time to start unifying people again. There are efforts by platforms like Facebook to try to bring a few of the mediums, like text messaging and Facebook Messenger, into one, but the combinations are often awkward and have little traction 😬 (and, let’s be honest, who wants to give Facebook control over even more of our lives…?).
Stream’s Chat Messaging Service was built to be flexible through the use of webhooks, to let YOU decide how you want to bring people together and how your data should flow. With webhooks, a developer is capable of leveraging our messaging service to build just about any communication platform that they can dream up. So, naturally, I thought to myself, “What can I dream up”? 🤔
After analyzing my habits, I realized that I often miss important Slack messages simply because the push notifications don’t always get my attention (for example, if I have Slack open on my computer, but I’ve walked away from it, I sometimes miss important notifications). On the other hand, it’s common that I check my SMS messages frequently to ensure that I’m not missing out on the latest and greatest group conversation. 👯
That’s when it clicked.
💡 I wanted to build a bridge between a chat service and SMS. And, through the use of Stream Chat’s webhooks and Twilio’s SMS capabilities, the problem no longer seemed like a challenge, but, rather, a fun puzzle.
In this tutorial, I’ll breakdown how I went about building a two-way chat powered by Twilio and Stream Chat. And, to make it a little exciting, I’ll use quotes from Kanye West (courtesy of https://kanye.rest 😎) for the autoresponder bot that I decided to build into the project.
Here’s a sneak peek into what you’ll end up with at the end of the project (web interface on the left and my mobile SMS messenger on the right):
Prerequisites:
JavaScript experience
Node.js (ideally the latest version)
Yarn or npm
-
MongoDB locally or an account with MongoDB Atlas
If you want to skip this tutorial and play around with the service, dubbed “Chatty”, you can do so by visiting https://chatty-kathy.netlify.com on the web and sending an SMS to 702–820–5110 with the message “START”.
Table of Contents:
Create a free Stream Chat Trial
Create a Twilio SMS Account
Clone the Web Repo from GitHub
Setup MongoDB
Clone the Serverless Repo from GitHub
Ngrok to the rescue
Starting Our Serverless Lambda
Configuring the Twilio SMS Web
1. Create a Free Stream Chat Trial 💬
Stream Chat is generally a paid service; however, Stream offers a 14-day free trial with no credit card required. We’ll use this free trial to build out our application.
Head over to https://getstream.io/chat and click on the “Start Trial” button near the bottom (just above the UI Kit).
Then, follow the steps to provision your account. Once it’s provisioned, head to the dashboard at https://getstream.io/dashboard and create an application — any name will do, but feel free to have some fun and make it personal!
Once your application is created, click on the application name. The default view is for “Feeds”, so navigate over to the “Chat” section” by clicking on the link at the top.
Once you are within the chat section of your organization, collect the Key and Secret under the App Access Keys section — you’ll want to hold onto these as you’ll need them in later sections.
2. Create a Twilio SMS Account 📱
To keep this section short, I’m going to leave the Twilio account creation up to you. This link will point you in the right direction. Once you’ve provisioned your account, take note of your SID and Auth Token, both of which can be found on the Twilio dashboard.
A trial account is more than enough for this. There is no need to enter your credit card or purchase a number unless you would like to extend this tutorial.
3. Clone the Web Repo from GitHub ⌨
To get started with Chatty, let’s clone the web repo from GitHub. This will save us quite a few steps in terms of styling, setting up auth, views, etc. Make sure that you are in a working directory and run the following command:
$ git clone https://github.com/GetStream/stream-chat-chatty-web web
Once cloned, move into the web directory and run yarn:
$ cd web && yarn
Now, create your .env file and drop the following contents into your environment file (some variables are filled in but we’ll take care of the rest as we go):
REACT_APP_STREAM_KEY=YOUR_STREAM_KEY
REACT_APP_AUTH_ENDPOINT=YOUR_AUTH_ENDPOINT
REACT_APP_TWILIO_NUMBER=YOUR_TWILIO_NUMBER
REACT_APP_CHANNEL_TYPE=messaging
REACT_APP_CHANNEL_NAME=chatty-kathy
4. Setup MongoDB 🕹
You have two options for this step. If you’re running macOS, the easiest way to install and run MongoDB is to use the following command to install MongoDB with Homebrew:
$ brew install mongodb
Followed by this command to fire it up:
$ brew services start mongodb
When running, you can connect to a database server running locally on the default port:
mongodb://localhost
If you get stuck, MongoDB provides a great set of instructions on their website.
Alternatively, you can create an account with MongoDB Atlas. MongoDB Atlas will handle all of the necessary setup and infrastructure for free. You can sign up for MongoDB Atlas at https://atlas.mongodb.com.
5. Clone the Serverless Repo from GitHub 🖨
We’re going to use AWS Lambda for handling incoming webhooks from Stream and Twilio for both chat messages and SMS.
To do this, we’ll run an AWS Lambda locally using Serverless. To get started, clone the repo using the following command:
$ git clone git@github.com:GetStream/stream-chat-chatty-serverless.git serverless
Next, move into the serverless directory and run yarn to install dependencies:
$ cd serverless && yarn
Then, you’ll want to address some missing environment variables within your serverless.yml file. Within the serverless directory, open the serverless.yml file and edit the environment block of yaml accordingly:
environment:
DB_CONN: YOUR_MONGODB_CONNECTION_STRING
DB_NAME: CHATTY
DB_COL: users
TWILIO_SID: YOUR_TWILIO_SID
TWILIO_TOKEN: YOUR_TWILIO_TOKEN
TWILIO_NUMBER: YOUR_TWILIO_NUMBER
STREAM_KEY: YOUR_STREAM_KEY
STREAM_SECRET: YOUR_STREAM_SECRET
CHANNEL_TYPE: messaging
CHANNEL_NAME: chatty-kathy
6. Ngrok to the Rescue 🕵
If you’re unfamiliar with ngrok, you should take some time to look over their website at https://ngrok.com. It’s a truly awesome service that, in a nutshell, opens up your local server (which is behind a firewall) to the public internet.
For example, we’re going to start our Serverless Lambda function locally in the next section. Ngrok will allow us to bind to a port (our particular Lambda will run on port 8000) and kick back a publicly accessible URL. We’ll use this URL to allow incoming webhooks from Stream Chat and Twilio.
If you don’t have ngrok installed, you can download it here. Ngrok runs on multiple operating systems, so it’s okay if you’re not running macOS. Once installed, run the following command in your terminal to bind to port 8000:
$ ngrok http 8000
If all was successful, you should see the following screen with a publicly accessible URL (both HTTP and HTTPS):
Now that you’ve started ngrok, it will listen to any incoming connections to port 8000, regardless of whether or not the server is up and running. That said, let’s go ahead and move on so we can start our Lambda locally with Serverless offline.
7. Starting Our Serverless Lambda 🐑
With the repo cloned into the serverless directory and our dependencies installed, move into the serverless directory (note that you will need to do this in a new tab so that ngrok stays running and listening on port 8000) and run the command yarn start. You should see the output in your terminal that looks something like this:
Your Lambda is now up and running!
Let’s configure our webhooks in the next two sections…
8. Configuring the Stream Chat Webhook 🎣
Head back over to https://getstream.io/dashboard and click on the “Chat” tab. Scroll down to the “Chat Events” section and set the webhook toggle to “Active”.
For the Webhook URL, be sure to drop in your ngrok HTTPS URL with /chat tacked on to the end (for example, my ngrok URL for chat is https://47e41901.ngrok.io/chat. This will ensure that the correct handler picks up your incoming chat messages.
Click the “Save” button — top right.
Next, we’ll configure Twilio.
9. Configuring the Twilio SMS Webhook 📲
Because we didn’t configure a project, you’ll need to do that. Head back over to the Twilio dashboard and click on the “Programmable SMS” tab (second down on the left bank).
Next, click on “SMS” and then create a new project. Mine is called “Chatty”, so I already have one configured. There should be a big red “+” button that sends you in the right direction.
Run through the configuration steps and use Chatbot/Interactive 2-Way for your use-case. Then, under “Inbound Settings”, specify your ngrok URL with /sms tacked on the end (for example, my ngrok outbound HTTP POST request URL is https://47e41901.ngrok.io/sms.
Make sure that you check the box that says “PROCESS INBOUND MESSAGES” to enable webhooks.
That’s all! Let’s continue on…
10. Web Environment Configuration 🌎
Now that we have nearly everything we need for configuration in place, let’s go ahead and update our .env file for our web environment variables.
Open the .env file and drop in your Stream, Twilio, and auth route (e.g. https://47e41901.ngrok.io/auth) credentials.
Once complete, you can go ahead and start the app with the yarn start command!
If all went well, you should be greeted by a simple login screen where you can enter your username and phone number. Enter your information and hit “Start” and you will be shown a chat screen where you can chat away. Kathy, the bot, may even throw back some interesting quotes from Kanye West if you are lucky!
Breaking Down the Architecture 🗺
Now that we’ve gone through all of the steps to get this up and running, you’re probably wondering how the heck all of this works and fits together. The architecture is rather simple — everything flows bi-directionally making the architecture really simple to understand. Here’s a quick breakdown:
A user logs in with their username and phone number
The user data is sent to Stream and the AWS Lambda, for storage, returning a user token back to the web for use
All interactions on the web are sent to Stream chat directly, and then piped to Lambda via websocket where they are processed and routed
If there is an @ mention, the user is notified via Twitter on their mobile device
The user on the mobile device can then reply, sending it back through Twilio, then off to Lambda, and, finally, back through Stream Chat where it will make its way to the user’s web client
Web 💾
The web consists of two primary files — the first being Login.js (supported by several files) and the second being Chat.js, where all of the chat magic happens.
Login.js handles data collection (email and password). When the user hits the “Start” button, a POST is sent off to the Lambda where the user is stored in the MongoDB database. A user is then created in Stream Chat, and a Stream generated token is returned along with the user data (ID, name, etc.). Once the token is received by the callback, it is stored in sessionStorage for use within our chat.
import React, { Component } from "react";
import InputMask from "react-input-mask";
import mobile from "is-mobile";
import axios from "axios";
import "./App.css";
class Login extends Component {
constructor(props) {
super(props);
this.state = {
loading: false,
name: "",
number: ""
};
this.initStream = this.initStream.bind(this);
}
async initStream() {
await this.setState({
loading: true
});
const auth = await axios.post(process.env.REACT_APP_AUTH_ENDPOINT, {
name: this.state.name
.split(" ")
.join("_")
.toLowerCase(),
number: this.state.number
});
if (mobile()) {
return window.open(
`sms://${process.env.REACT_APP_TWILIO_NUMBER}?body=Hi, ${
auth.data.user.name
} here! How is everyone doing?`,
"_system"
);
}
sessionStorage.setItem("userData", JSON.stringify(auth.data.user));
sessionStorage.setItem("tokenData", auth.data.token);
await this.setState({
loading: false
});
this.props.history.push("/");
}
handleChange = e => {
this.setState({
[e.target.name]: e.target.value
});
};
render() {
return (
<div className="login-root">
<div className="login-card">
<h1>Chatty</h1>
<InputMask
type="text"
placeholder="Username"
name="name"
onChange={e => this.handleChange(e)}
/>
<br />
<InputMask
{...this.props}
type="tel"
placeholder="Phone Number"
name="number"
onChange={e => this.handleChange(e)}
mask="+1\ 999-999-9999"
maskChar=" "
/>
<br />
<button onClick={this.initStream}>Start</button>
</div>
</div>
);
}
}
export default Login;
Chat.js pulls the user and token data from sessionStorage, where it then starts a new chat for the user. I’m using our Stream Chat React Components to bring this to life, and, I must say, it’s an easy task.
import React, { Component } from "react";
import {
Chat,
Channel,
ChannelHeader,
Thread,
Window,
MessageList,
MessageInput
} from "stream-chat-react";
import { StreamChat } from "stream-chat";
import "./App.css";
import "stream-chat-react/dist/css/index.css";
class App extends Component {
constructor(props) {
super(props);
const { id, name, image } = JSON.parse(sessionStorage.getItem("userData"));
this.client = new StreamChat(process.env.REACT_APP_STREAM_KEY);
this.client.setUser(
{
id,
name,
image
},
sessionStorage.getItem("tokenData")
);
this.channel = this.client.channel(
process.env.REACT_APP_CHANNEL_TYPE,
process.env.REACT_APP_CHANNEL_NAME,
{
image: "https://i.imgur.com/LmW57kB.png",
name: "Kathy is Feeling Chatty About Kanye"
}
);
}
render() {
return (
<Chat client={this.client} theme={"messaging light"}>
<Channel channel={this.channel}>
<Window>
<ChannelHeader />
<MessageList />
<MessageInput />
</Window>
<Thread />
</Channel>
</Chat>
);
}
}
export default App;
The Lambda 🧞♂️
The Lambda consists of three primary functions as well as an additional helper function to store MongoDB connection state — all of which are stored in a single file called handler.js.
Connect establishes and stores the MongoDB connection state. This is helpful because Lambda caches for a bit of time, and it’s good practice to reuse a connection when possible.
async function connect(uri) {
// check if database connection is cached
if (cached && cached.serverConfig.isConnected()) {
return Promise.resolve(cached);
}
// database name
const dbName = process.env.DB_NAME;
// connect to database
return MongoClient.connect(uri, { useNewUrlParser: true }).then(client => {
// store in cache and return cached variable for re-use
cached = client.db(dbName);
return cached;
});
}
Auth takes in a name and email as POST parameters. From there, it establishes a database connection and creates a user in Stream Chat. Once a user is created, a token is generated and both the user object as well as the token are returned back to the client.
export const auth = async event => {
// extract params from body
const { name, number } = JSON.parse(event.body);
// establish database connection (cached)
const db = await connect(process.env.DB_CONN);
const phoneNumber = phone(number)[0];
// initialize stream chat
const stream = new StreamChat(
process.env.STREAM_KEY,
process.env.STREAM_SECRET
);
if (!name || !number) {
// respond with 200
return {
statusCode: 400,
headers: {
'Access-Control-Allow-Origin': '*',
},
body: JSON.stringify({ error: 'Missing name or number.' }),
};
}
try {
// create or update the user based on their phone number
const { value } = await db
.collection(process.env.DB_COL)
.findOneAndUpdate(
{
number: phoneNumber,
},
{
$setOnInsert: {
name,
number: phoneNumber,
active: true,
updated: new Date(),
},
},
{
upsert: true, // important so that it creates a user if they don't exist
returnOriginal: false, // important so that it always returns the data
}
);
// add index to phone number
await db
.collection(process.env.DB_COL)
.createIndex({ number: 1 }, { unique: true });
// setup user object for storage
const user = {
id: value._id.toString(),
name: value.name,
number: value.number,
role: 'user',
image: 'https://i.imgur.com/Y7reRnC.png',
};
// generate token and update users
const token = stream.createToken(user.id);
await stream.updateUsers([user]);
// respond with 200
return {
statusCode: 200,
headers: {
'Access-Control-Allow-Origin': '*',
},
body: JSON.stringify({ user, token }),
};
} catch (error) {
console.log(error);
return error;
}
};
SMS handles incoming webhooks from Twilio, processing the message and forwarding it over to the Stream Chat channel. The user can respond with “STOP” at any point in time to turn off the messages.
export const sms = async event => {
// extract body (querystring format coming from twilio)
const { From, Body } = qs.parse(event.body);
// establish database connection
const db = await connect(process.env.DB_CONN);
// initialize twilio client
const client = new twilio(process.env.TWILIO_SID, process.env.TWILIO_TOKEN);
// initialize stream client
const stream = new StreamChat(
process.env.STREAM_KEY,
process.env.STREAM_SECRET
);
// create the channel
const channel = stream.channel(
process.env.CHANNEL_TYPE,
process.env.CHANNEL_NAME
);
try {
// lookup the user based on their incoming phone number
const user = await db.collection(process.env.DB_COL).findOne({
number: From,
});
// only trigger response if the incoming message includes start
if (Body && Body.toLowerCase().includes('start')) {
await client.messages.create({
body: 'Get started at https://bit.ly/stream-chatty',
to: From,
from: process.env.TWILIO_NUMBER,
});
// respond with 200
return {
statusCode: 200,
headers: {
'Access-Control-Allow-Origin': '*',
},
body: JSON.stringify({ status: 'OK' }),
};
}
// deactivate user
if (Body && Body.toLowerCase().includes('stop')) {
// set user active status to false so they don't get any additional texts
await db
.collection(process.env.DB_COL)
.updateOne({ _id: From }, { $set: { active: false } });
// let the user know that they have been removed
await client.messages.create({
body: 'Sorry to see you go!', // message body for sms
to: From, // incoming twilio number
from: process.env.TWILIO_NUMBER, // twilio outbound phone number
});
// respond with 200
return {
statusCode: 200,
headers: {
'Access-Control-Allow-Origin': '*',
},
body: JSON.stringify({ status: 'OK' }),
};
}
// update acting user
await stream.updateUsers([
{
id: user._id,
name: user.name,
role: 'user',
},
]);
// send a message
await channel.sendMessage({
text: Body,
user: {
id: user._id,
name: user.name,
image: 'https://i.imgur.com/Y7reRnC.png',
},
number: user.number,
context: 'sms',
});
// respond with 200
return {
statusCode: 200,
headers: {
'Access-Control-Allow-Origin': '*',
},
body: JSON.stringify({ status: 'OK' }),
};
} catch (error) {
console.log(error);
return error;
}
};
Create has a little bit more logic going on within it. This particular function accepts webhooks from Stream Chat and sends a user an SMS anytime they are @ mentioned. The whole idea behind this is to keep the user updated on direct messages, but avoid noisy chat messages from coming through.
export const chat = async event => {
// extract the message body and setup the database
const data = JSON.parse(event.body);
// establish database connection (cached)
const db = await connect(process.env.DB_CONN);
// initialize twilio messages
const client = new twilio(process.env.TWILIO_SID, process.env.TWILIO_TOKEN);
// initialize stream chat
const stream = new StreamChat(
process.env.STREAM_KEY,
process.env.STREAM_SECRET
);
// create the channel
const channel = stream.channel(
process.env.CHANNEL_TYPE,
process.env.CHANNEL_NAME
);
try {
// only allow events that are not read, etc.
if (data.type !== 'messages.read' && data.message) {
const message = data.message;
// and only allow @ mentions (check if mentioned users array has mentions)
if (message.mentioned_users.length > 0) {
const mentioned = message.mentioned_users;
// loop through all of the messaged users
for (const mention in mentioned) {
// run a quick lookup against their user id
const user = await db
.collection(process.env.DB_COL)
.findOne({
_id: new ObjectID(mentioned[mention].id),
});
// only attempt to send a message if the user is active
if (user.active && message.user.id !== user._id) {
// send sms with twilio
await client.messages.create({
body: `Chat from @${data.user.name}:\n\n${
message.text
}`, // from user with message text on newline
to: user.number, // phone number from database
from: process.env.TWILIO_NUMBER, // twilio outbound phone number
});
}
}
}
if (data.user.id !== 'kathy') {
// send a random response
const random = await axios.get('https://api.kanye.rest');
// send a message
await channel.sendMessage({
user: {
id: 'kathy',
name: 'Chatty Kathy',
image: 'https://i.imgur.com/LmW57kB.png',
},
text: `@${data.user.name} Here's a Kayne quote for you – "${
random.data.quote
}"`,
mentioned_users: [data.user.id],
context: 'random',
});
}
}
// respond with 200
return {
statusCode: 200,
headers: {
'Access-Control-Allow-Origin': '*',
},
body: JSON.stringify({ status: 200 }),
};
} catch (error) {
console.log(error);
return error;
}
};
Final Thoughts 🤔
In this post, we’ve learned the inner workings of building a messaging service with Stream Chat and Twilio SMS. The frontend is built with React using the Stream Chat React Components, whereas the backend is entirely built in an AWS Lambda powered by Serverless.
If you’re looking to take this project to the next level, adding a few things such as MMS support would be a great idea. You could also learn how to launch Serverless on AWS, which we did not cover in this tutorial, as we hooked up the communication between services using ngrok.
I hope you found this tutorial helpful and encourage you to drop any thoughts or questions in the comments below.
If you’d like to learn more about Stream Chat, you’ll enjoy our API Tour at https://getstream.io/chat/get_started/. We also have various SDKs for many popular languages and frameworks, including iOS / Swift.
Happy coding! ✌
Top comments (2)
I'm assuming you mean Twilio here?
"If there is an @ mention, the user is notified via Twitter on their mobile device"
Just to reiterate, kanye.rest/ is pretty great. Haha.