DEV Community: Pooja Mistry

Visualizing a Globally Distributed Team Using Multiple APIs

Pooja Mistry — Thu, 26 May 2022 16:48:01 +0000

If the 2020 pandemic has taught us anything, remote-first work culture is the way of the future. As a new developer advocate at Postman, I’m excited to be working remotely for a globally distributed developer relations team supporting Postman’s community of more than 20 million users worldwide. With that said, I was inspired to build my first collection to highlight Postman’s global presence.

On Day 17 of 30 Days of Postman, I became intrigued with the Postman Visualizer feature. I knew it was going to be the heart of my project, so I set out to find a way to visualize my fantastic new Postman team members spanning the globe. In this blog post, I’ll show how I got started.

Working with an image in mind

There are Postmanauts living in all corners of the world, with my team comprising seven members that work across various time zones. A map was going to be the best kind of visualization to showcase our global diversity. After exploring the Postman API Network, I stumbled upon an Earthquake Map Visualization, which used D3.js libraries to map data against longitude and latitude coordinates.

Map of earthquakes around the world

I realized that I could visualize just about anything on this map as long as I had access to some longitude and latitude data.

Time to put this theory to the test. I built a sample POST request called Random Users using the Postman Echo API. The idea behind this request was to echo back some sample dummy data for ten users with random names, longitudes, and latitudes using Postman’s faker library.

let users = [];

for (var i=0; i<10; i++){
    let tempUsers = {
        name: pm.variables.replaceIn("{{$randomFullName}}"),
        latitude: pm.variables.replaceIn("{{$randomLatitude}}"),
        longitude: pm.variables.replaceIn("{{$randomLongitude}}")

    }
    users.push(tempUsers);
}
pm.collectionVariables.set('req_body', JSON.stringify(users));

In this pre-request script, I used dynamic variables to create objects for ten users with random names, latitudes, and longitudes. These objects are then pushed into an array and set into the request body using a collection variable and JSON Serialization. Once this request is sent, I can get random users:

Response that includes random users from Postman Echo Post Request

Now it’s time for the visualization. In the Test tab, I used the template from the Earthquake example and modified a couple of interesting points:

var response = pm.response.json().data;
let parsedData = [];

//data parsing
for (let users of response){
    let tempEntry = {};
    tempEntry.name = users.name;
    tempEntry.lat = users.latitude;
    tempEntry.long = users.longitude;
    tempEntry.circleSize = 6;
    tempEntry.color = "#F09D51";
    parsedData.push(tempEntry);
}
// visualize 
pm.visualizer.set(template, {
    data: parsedData,
    title: "Map of Random Users"
} );

The Echo API’s response data is being parsed for each of the users, name, latitude, longitude, and circle size and color for each data point. The pm.visualizer.set() method is then used to visualize the parsed data, and the map’s title is returned from echo API. Here is what we get:

Visualization of Random Users

Voilà, the theory worked! This proved that with some longitude and latitude coordinates, I could use this visualizer to map random users. This was going to be my template to visualize, but now it was time to work with some real user data.

Choosing the APIs

Getting data for real users can be tricky. I needed to find longitude and latitude data for my entire team, so I thought to use their location data from their Twitter profiles as a starting point.

The Twitter API is a really powerful API. To get a user’s data from a username, I specifically used the Twitter API user by username endpoint to retrieve location data for an individual username and the Twitter API User by Usernames to retrieve location data from multiple usernames. In Postman, the query params for user.fields is used to get all sorts of data from Twitter, such as:

Created_at,description,entities,id,location,name,pinned_tweet_id,profile_image_url,protected,url,username,verified,withheld

This request is forked from the Twitter API v2 Postman Collection, which allows anyone to access the wide plethora of Twitter endpoints with the right authorization from Twitter’s Developer Dashboard.

In my case, I used my Twitter handle, @poojamakes, to get my location data, which is Washington, D.C.

Twitter API V2 User by Username request

Once I could get a high-level location for each of my team members using their profiles, I needed a geocoding API that could give me their longitude and latitudes based on their location.

The PositionStack API was a perfect fit since it provided an accurate forward and reverse geocoding covering more than 2 billion places and addresses worldwide. To use this API, you will need to obtain an API key from PositionStack by clicking on Get Free API Key and selecting the free option for personal use.

In this scenario, I used the Forward Geocoding endpoint to make a forward geocoding request, using the API’s forward endpoint and specifying the query parameter with the location from Twitter.

API results:

Example of PositonStack Forward Geocoding endpoint

Now that I had the access to location using the Twitter API and was able to convert that location into longitude and latitude data using PositionStack API, I was well on my way to creating my map visualization.

I decided to get a little creative and also capture the weather data for each team member by using the Open Weather API using the Current Weather Data endpoint, which allows you to get current weather data for any location on Earth, including over 200,000 cities, by using geographical coordinates (lat, lon).

At this point I have all the pieces of the puzzle:

User’s Location data from Twitter User Look Up APIs
Longitude and latitude data from PositionStack API
Weather data from Open Weather API
Map visualization with Postman Echo

Now it’s time to build out a single workflow (or maybe many) to put all of these pieces together.

Building workflows

One thing I learned while working on this project was that there are so many different ways to build request workflows with Postman. In my case, I found three specific ways that I will be diving deeper into in my upcoming blog post. However, the common thread behind these workflows is the ability to create, set, and parse through data objects across each request using serialization. My colleague Ian Douglas wrote a wonderful blog post on this topic which talks about the idea of “serializing” data to store and convert complex data types when writing and reading from memory.

For this case, I am storing this data into object environment variables. Variables in general are necessary when working with multiple APIs and requests. In this scenario, I used an environment variable to set all of the Twitter usernames that I will be using in the Twitter API to get the location data and name for each user. I was then able to create an object with that data using pm.environment.set() and set that serialized data into a string, using JSON.stringify(), into another environment variable to be used in a query parameter for the PositionStack API to get each user’s longitude and latitude.

let response = pm.response.json();
// building users object 
let obj= { 
     name : response.data.name,
     location:response.data.location,
     latitude: null,
     longitude: null
}
pm.environment.set("user-object", JSON.stringify(obj));

Initially, this data is set as null, but as the data passes through another API and is deserialized and parsed using the JSON.parse(), more and more data can be added to each request.

var dataOut = pm.response.json().data[0];
let lat = dataOut.latitude;
let long = dataOut.longitude;

//add long and lat from object
var userObj = JSON.parse(pm.environment.get("user-object"));
userObj.latitude = lat
userObj.longitude = long

// set lat and long to main object
pm.environment.set("user-object", JSON.stringify(userObj));

We are getting the longitude and latitude data from the API’s response. Then the initial environment variable is parsed and the data is added to the object and finally set again to using pm.environment.set() to be used in another request.

Using these methods as a model for each scenario gives us the overall workflow:

Get user’s location from Twitter API.
Convert Location into longitude and latitude.
Use longitude and latitude to get weather data for each user based on their location.
Visualize all the data in the map.

Here’s a visual overview of everything in this workflow. The final POST request uses the Postman Echo API which replaces the random data with real data for each user from the Twitter, PositionStack, and Open Weather APIs:

API workflow

Create your own team visualizations

Overall, this collection was created to showcase the magic of Postman’s versatility. Just knowing the basics of working with variables, objects, serialization, and visualizations can take you a long way.

Keep an eye out for my upcoming blog post about the three ways to build Postman workflows, in which I’ll use this same example to dive deeper into working with individual Postman requests, sending requests from scripts, and building request workflows.

If you are interested in playing around with this collection and visualizing your own teammates across the globe, be sure to fork my collection.

The seven Postman developer relations teammates that are mapped are: @iandouglas736, @poojamakes, @DevRelSean, @arlemi, @jansche, @PetuniaGray, @DevRelKev

Give us all a follow or or use this collection to find other Twitter users in your network to visualize.

To get started with this collection:

Fork this collection into your own workspace.
Get Twitter bearer token and add bearer_token to current value in collection variable.
Get PositionStack API key and add api-access-key to current value in the collection variable.
Get Open Weather API key and add weather_api_key to current value in the collection variable.
Use collection runner to run through Individual Postman Requests and capture all objects as environment variables for each user.

Check out visualization in Postmanauts - Visualize request in all folders.

The post Visualizing a Globally Distributed Team Using Multiple APIs appeared first on Postman Blog.

Four Questions to Consider when Starting your API Testing Journey

Pooja Mistry — Mon, 10 Jan 2022 03:48:58 +0000

In today’s API economy, it has become more critical than ever for API-first companies to spend time strategizing on how to deliver the best products in the marketplace.

APIs are the connective tissue between applications, servers, and data. Companies now treat them more as a marketable product rather than just a technical interface. With that said, it can be a daunting task for both developers and testers alike to create a plan that ensures the highest quality. But have no fear! I like to think of this process as a journey. With that perspective in mind, here are four questions that would be useful to consider when testing your APIs.

Am I using the right tool ?

Before starting any journey, one of the most important aspects to consider is what tools you will need to make your trip a success? Perhaps you will need a car, possibly some compact luggage; however, when it comes to API testing, you will need a tool that allows you to send API requests and view the correct response effectively.

Postman is the ideal tool for the job. It is a collaborative API development platform that simplifies creating, using, and testing APIs with an intuitive user interface.

Send Requests with Postman

A few of the many advantages of the tool include :

Ease of Use - Postman users can begin their API journey quickly and effectively using Postman’s comprehensive platform and broad support for all possible HTTP methods and queries such as REST, SOAP, and GraphQL.
Robust testing capabilities - Postman supports multiple testing environments, test checkpoints, automated scripting, and built-in support for using variables for security. It also has support for keeping track of API responses which is critical for testers when it comes down to debugging, comparing versions, and managing inconsistencies.
Management and Collaboration - Postman Workspace allows users to manage versions, authentication protocols, cookies, and certificates. It is also dedicated to the real-time collaboration between teams with built-in version control, ensuring developers and testers can stay on the same page when testing version releases.

Once you use the right tool, you can consider the following three questions.

How do I start and what is my road map ?

When a new API is ready to test, the first test that you should consider is a smoke test. A smoke test is a small quick test that ensures that the basic functionality of the API is working.

This test includes :

Checking to see if API responds to first call
Testing API with small amounts of data to see if it responds with a payload in the correct schema
Testing API with proper authentication and seeing if it passes/fails to the valid authorization
Testing API with any other components that it’s supposed to interact with.

A smoke test is used to smoke out any apparent errors quickly spotted and resolved.

Once your smoke tests pass, it is time for you to create the correct road map. This road map can be defined by ensuring that you can hit all possible functionalities successfully. Usually, you can map out this strategy by considering HTTP methods and CRUD operations.

Some of the tests that can be included in your road map :

Functionality Tests - test of specific functions within the codebase
Validation Tests - tests to verify aspects of API its behavior, and efficiency
Sanity Tests - tests to verify if response makes correct sense
Integration Tests - tests to verify API can integrate with other components
Load Tests - tests to validate the performance under specific load requirements
Runtime Tests - tests related to monitoring, execution errors, resource leaks, or error detection
Security Tests - tests to verify authorization and validation of encryption methodologies to enable security from external threats

Once you have strategized which types of tests you will include in your road map, you can also consider making testing easier through automation. Automation can allow you to do regression testing, allowing you to run your entire test suite after a patch, upgrade, or bug fix. Postman is also an excellent tool for all things API automation; check out this blog: API Testing Tips from a Postman Professional to learn more.

What is the Happy Path and the Not -So-Happy Path ?

We may often think that if an API does what it is supposed to do, then it works perfectly. This is only the Happy Path piece of the puzzle when it comes to testing. The other piece is negative testing which is equally essential for ensuring quality.

By performing negative testing, you can see whether or not their API can deal with receiving incorrect or invalid data gracefully, such as with an error message instead of a hard crash or a complete stop.

This blog: Negative testing for more resilient APIs provides notable examples for negative testing, which I recommend you check out! Ideally, you would want to test for all API success and failure responses. Check out this comprehensive list to learn more.

As you go about your testing journey, you may wonder how you can validate your positive and negative responses. Postman has functionality for creating test scripts, which genuinely helps testers level up their validation game.

Do these tests closely mirror the real world ?

The final factor to consider is testing in environments that simulate exact conditions the API will encounter in production or when it is released. Doing so will ensure that the test results accurately reflect the API’s ability to correctly function and perform when subjected to its intended working environment.

However, there may come a time when specific components or other APIs may be missing or unavailable. Simulating a real-world environment may rely on mocking components in this scenario. Mocking components is helpful because you can replace missing parts and customize features to deliver the ideal responses needed to complete the testing procedure.

Learn more about mock servers here

Although API testing can be a daunting process, I hope that after reading this blog, these four questions help you get a head start on your API testing journey. If you are interested in learning more about Postman, check out the learning center! In the meantime, know that every big journey begins with small steps - API testing included.

Reflecting on Challenges, Changes and Growth in 2020

Pooja Mistry — Thu, 14 Jan 2021 21:31:13 +0000

2020 has been a year of immense challenges, changes, and growth. One of the biggest challenges and changes that we overcame as a developer advocacy team was pivoting from in-person events to 100 percent digital!

Although making this pivot felt apprehensive at first, we were surprised by the many different growth opportunities.

Our audience and our reach was now worldwide, whereas before, our geo-locations bounded us
There were countless opportunities for collaboration with different teams and subject matter experts
We were able to create more content that can turn into a series
Most importantly, our content is documented and recorded for our audience to reference

Most of our advocacy last year happened through our Crowdcast channel.

Anyone who follows us has access to hundreds of webinars and series on topics ranging from AI/ML, Data Science, Cloud Computing, Serverless, Web development, Java, Quantum and many more hosted and taught by IBM developer advocates from around the world.

As an IBM Developer advocate, content producer, and educator, here are some of ….

My 2020 Digital Event Highlights :

Women in Data Science Digital Conference

In March, I hosted the first-ever digital Women in Data Science Conference with six incredible speakers.
Topics included :

What do I do with this? : Making Data Science Actionable
AI technology + answering human-ai interaction research
Changing the Face of Medicine: How Data is Improving Medical Education
Visualizing shared musical experiences
Data and Risk Management Impacts

Watch Replay Here

Call for Code - Covid 19 Challenge

In April, we launched the Call for Code Covid-19 Challenge! This challenge was designed to help developers worldwide fight a global pandemic with the use of technology. Watch the replay or read this blog to learn more and fork our starter kit to start with the code!

AI/ML Master Class Series

In May, I hosted a series about building, training, testing, and deploying AI/ML models. Check out this blog to learn more about this series and find the links to replays and resources.

My colleague Jenna also re-vamped this series in December and introduced further insights! Check out these replays here :

Node-RED Master Class Series

In June, we launched the Node-Red series, which focused on building applications using low code programming methods. Check out this blog to learn more about this series and find the links to replays and resources.

Exploring Quantum Computing with Qiskit

In July, I was able to host Doug McClure, a Research Staff Member and Manager of Quantum System Deployment at IBM Research in Yorktown Heights, to introduce basic ideas in quantum computing, survey state of the art in building real quantum computers, and provide a hands-on introduction to the Qiskit programming language for building and running quantum algorithms.

Watch Replay Here

Building Applications with Watson - A Developers Guide to the Watson SDK

In August, I created a workshop on getting started with the Watson SDK. There are over ten different unique services on the IBM Cloud that are free for developers to use with the lite account. Services such as assistant, speech to text, text to speech, discovery, visual recognition, tone analyzer, natural language processing, and language translator, to name a few, provide unique insights for developers to use in their applications.

Watch this replay to learn more.
Check out this git repo to get started with the code.

All thing Open - Embrace Challenge

Throughout the summer, IBM launched the Emb(race) Call for Code for Racial Justice Challenge to combat systemic racism. Focusing on Police & Judicial Reform and Accountability, Diverse Representation, and Policy & Legislation Reform, we are formed open source projects to help combat discrimination and promote racial justice. We invited people across the globe to channel their frustrations about systemic racism tangibly and authentically by getting involved with these projects and making an impact in communities.

Check out our GitHub and submit pull requests and issues work with our opensource project around Police & Judicial Reform, Diverse Representation, Policy & Legislation Reform: https://github.com/topics/embrace-call-for-code

Watch our session replay at the All things Open Conference to learn more.

Tech Talks with Bee Travels

In October through November, I hosted a 5 part series called Tech Talks with Bee Travels with my colleagues Max, Mofi, and Nick. Bee Travels is a polyglot demo microservice application that demonstrates key capabilities of Kubernetes, OpenShift, Istio, Knative, and many other cloud-native technologies. This series aims to touch on topics such as

Check out the docs, workshop, and tech talks to this project here.

Data & AI Conference - Building Smarter Conversational AI

In November, I spoke at the Data and AI Conference about building better conversational AI. This talk was about building conversational AI in chatbots with Watson Assistant and how Natural Language Processing plays a role in having meaningful conversations. I also discussed some tips and tricks into making your Assistant work smarter for you.

Watch replay here

IBM & Infosys Present: Modernize with Containers: Migrate Legacy Java Applications

In December, I wrapped up the year with an Infosys partner event where we talked about Application Modernization and had a hands on lab where we modernized a legacy java application with the help of containers.

Check out this replay to learn more.
Check out this GIST for all the resources.

Conclusion

Although 2020 was challenging in so many ways, I am grateful that technology has allowed me to work on so many different ideas and collaborate with so many amazing people world wide!

I hope you find some of my highlights and resources interesting as you go through your technical journey. I plan to keep creating and sharing content and topics with you throughout 2021!

Aside from the content I have produced this year, we have hundreds of different webinars on numerous topics; all hosted on our Crowdcast channel made by IBM developer advocates from around the world!

So be sure to follow our channel to stay up to date and learn more!

Node-RED from the basics to beyond in June

Pooja Mistry — Mon, 29 Jun 2020 22:32:21 +0000

In the month of June my colleague John Walicki (@johnwalicki) and I decided to do a series on Node-RED. This series showed users how to work with Node-RED on IBM Cloud to create event-driven applications . In part one of this series John and I go over Node-RED essentials and we build a text analyzer application using Watson APIs . In part two we go over building UI dashboards with Node-RED and finally in part three we take all the concepts from part 1 and 2 to build an interactive end to end application.

If you missed our series check out these resources and the replay to each event :

Part 1 : Node-RED Essentials

Check out the full replay of event here : https://www.crowdcast.io/e/node-red-series

Slides for event : https://slides.com/poojamistry/node-red-series1

Workshop : https://github.com/pmmistry/Node-RED-Series/blob/master/Labs/lab_1.md

Part 2 : Node-RED Dashboard and UI Techniques

Check out the full replay of event here : https://www.crowdcast.io/e/node-red-series-2

Slides for event : https://slides.com/poojamistry/node-red-part-2

Workshop : https://github.com/pmmistry/Node-RED-Series/blob/master/Labs/lab_2.md

Part 3 : Node-RED End to End

Check out the full replay of event here : https://www.crowdcast.io/e/node-red-series-3

Slides for event : https://slides.com/poojamistry/node-red-series-part-3

Workshop : https://github.com/pmmistry/Node-RED-Series/blob/master/Labs/lab_3.md

I hope this blog helps you as you begin your Node-RED Journey ! Register for IBM Cloud to get started with Node-RED on IBM Cloud! Make sure you check out IBM Developer for more resources, tutorials, blogs, code patterns and content in the Node-RED Space!

Give me a follow if you like this post, or follow me on Twitter @poojamakes to see what other webinar series I will be teaching next! 🙂

Building, Training, Testing and Deploying ML Models in May

Pooja Mistry — Tue, 02 Jun 2020 23:37:46 +0000

In the month of May my colleague Jenna Ritten (@jritten) and I decided to do a series on building, training, testing and deploying AI/ML models! This series showed users how to use Watson Studio which is an interactive, collaborative, cloud-based environment where data scientists can use multiple tools to activate their insights.

The goal of this series was to show the various tools that can be used within Watson Studio such as Visual Recognition, Natural Language Classifier , Auto AI and Juypter Notebooks. With this series we wanted to empower new and experienced data scientists with the knowledge and ability to train and deploy models in an effective and collaborative way.

If you missed our series check out these resources and the replay to each event :

Part 1 : Building and Deploying Models w/ Visual Recognition and Natural Language Classifier

Check out the full replay of event here :
https://developer.ibm.com/events/ai-ml-masterclass-part-1-building-and-deploying-models-w-visual-recognition-and-natural-language-classifier/

Slides for event : https://slides.com/poojamistry/custommodels#/

Workshop :

Part 2 : Building and Deploying models with AutoAI

Check out the full replay of event here : https://developer.ibm.com/events/ai-ml-masterclass-part-2-building-and-deploying-models-with-autoai/

Slides for the event : http://ibm.biz/wsmasterclassautoai_slides

Workshop : https://github.com/IBM-Developer-Austin/autoai

Part 3 : Building and Deploying models with Jupyter Notebooks

Check out the full replay of event here : https://developer.ibm.com/events/ai-ml-masterclass-part-3-building-and-deploying-models-with-jupyter-notebooks/

Slides for the event : https://slides.com/poojamistry/jupyternotebooks#/

Workshop : ibm.biz/jupyterworkshop

Register for IBM Cloud to get started with Watson Studio! Make sure you check out IBM Developer for more resources, tutorials, blogs, code patterns and content in the AI/ML Space!

I hope this blog helps you as you begin your AI/ML journey ! Give me a follow if you like this post, or follow me on Twitter @poojamakes to see what other webinar series I will be teaching next!🙂

Build a dashboard without a single line of code

Pooja Mistry — Wed, 13 May 2020 17:24:24 +0000

With the astronomical amounts of data we have passing through the web and IoT devices at any given second, it is critical now more than ever, that we have ways to visualize it. If you google how to do so, you will probably see more than 67 million results, however, from my perspective one of the easiest and intuitive ways to visualize this kind of data is with Node-RED dashboards.

Node-Red is a browser-based programming tool that allows you to connect code blocks that accomplish tasks. It uses nodes and flows to easily write code that can connect to APIs, hardware, IoT devices, or online services. A node is a predefined code block and flow is a connection of nodes, usually an input, processing, and output node. Node-RED can be run locally on your computer, on a device such as Raspberry Pi, or even in the cloud.

In my blog post I’ve outlined the many different reasons why I think Node-RED is a great tool, however, in this blog post I want to highlight just one. The true beauty of Node-RED lies in its ease of use. Node-RED allows users from any programming background to build something worthwhile. It democratizes programming and the ability to create and manipulate data in more ways than any other program would. It uses flow-based programming to visualize the data movement instead of having to write a lot of code, therefore being incredibly popular when it comes to prototyping as well as controlling data flow from IoT devices. With that being said, there are also very easy ways to create aesthetically visual and comprehensive user interfaces without writing a single line of HTML, CSS, or JS. This is where the Node-RED dashboards come into play.

Before we dive deep into how we create these dashboards, let's talk a little bit more about why.

A few reasons why it would be beneficial to create a dashboard that displays data from either web applications or IoT devices could be and not limited to:

Processing inbound data coming into your web application or IoT device automatically. Inbound data may include timestamps, messages, location, weather as well as any other influences that could influence decisions.
Monitoring as well as controlling limits of interest.
Receiving alerts on desired information such as failures or performance.
Sharing and analyzing visually consolidate data with your peers.

A few things to think about when designing these dashboards could be and not limited to:

What type of components should the dashboard display? ( Should there be graphs, sliders, gauges, forms ) .
Where will the received information go and what will process it?
What protocols should I employ? ( should I add any alerts ? )
Why should I even consider making a dashboard ( refer back to the upper section on the why 🙂 )

After considering both the why and the what of creating a dashboard, Node-RED makes it fairly simple to consider how.

Let's take a look at this example: The CPU Dashboard
The CPU dashboard monitors and creates alerts for CPU usage. When the CPU usage is greater than 50% it will create an alert monitoring the user which core the alert came from. ( Note this is a simulation )

Full workshop available here

In this example we are displaying the output of a quad core processor and its percentage usage. This is a simulation to show how the data is being recorded in real time and sending alerts to the user.

Here is what the Node-RED flow of this dashboard looks like :

This flow starts off with a dark blue input node that passes on timestamps as the input property. This node is then connected to a light blue UI switch node which can be toggled on and off. When on the switch node can pass on time stamp data, and when off no time stamp data is passed on which results in nothing. Assuming that the switch is on, or in other words if switch is true , we need to connect the light blue UI switch node to the yellow switch funtion node to then grab the simulated data that is stored in the CPU usage node.

If switch is on then get data from CPU usage node (dark blue with chip like image). This node is gathering all the simulation data and sending separate messages for each core. When you look at the dashboard you see a line chart under the CPU utilization section which graphs the collection of the CPU data with each line representing a separate core across a time stamp of when the switch is turned on.

This is done at the flow level by connecting the UI node for chart (teal node with chart image ) and setting its properties as line graph with the CPU usage node. The green node called msg.payload takes the output of the CPU usage node and prints it in the node-RED debug panel which acts like the console.log.

The CPU usage node is then also connected to two switch nodes

When the value of CPU is greater than 50 it sends an alert . The template node properties (orange node ) connected to alert node (teal) are as so :

Alert: CPU {{topic}} is {{payload}}%
Break output value from main CPU usage node to 4 separate gauges (teal) to visualize the data coming in at each core

By this point you can get the idea that the data is pushed and manipulated along each node and its output is being visualized by each dashboard UI node . Imagine instead of a CPU usage node as the main data point you can have other nodes providing you an influx of data such as an IoT device or even tweets coming in from social media. You can use the same methodology to visualize data no matter what input.

Creating this type of program can get very complicated when dealing with front end and server-side code. I hope after following this flow, you feel that Node-RED provides an easier way to create and visualize data that in many other cases could require more time, effort and knowledge to do. Node-RED is a great tool for everyone with all backgrounds and I hope that this example inspires you to go out there and create on you own!

Note: This particular example uses no code, however, if you do want to make robust dashboards you might want to include some function nodes to manipulate data using javascript. The purpose of this article is to show that applications can be created with no code / low code using Node-RED

Check out my Node-RED dashboards Workshop to get step by step instructions on how to create this flow as well as 10 other examples!

To get started make sure you register for the IBM Cloud and create a Node-RED starter kit! This will have all you need to get up and running with Node-RED dashboards today!

I hope this blog helps you as you begin your next prototyping journey! Give me a follow if you like this post, or follow me on Twitter @poojamakes to see what classes I am teaching on Node-RED! 🙂 🏙️

Building solutions that fight back

Pooja Mistry — Mon, 13 Apr 2020 22:16:37 +0000

A few weeks ago, a colleague of mine asked me to clear out my calendar for the week to work with him on a team project. At the time, I agreed without really realizing how impactful this project was going to be. Little did I know, this project would lead to building out an entire starter kit for the Call for Code COVID-19 challenge, which is designed to help developers worldwide fight a global pandemic with the use of technology!

Take on the Challenge Today

I joined the 8 am Zoom call that Thursday morning to ideate with the team on what the starter kit was going to be. Since the Call for Code COVID-19 Challenge was going to kick off the following day (March 20th), we were told that we had only one week to publish the starter kit and that we should all work together to build a solution around Crisis Communication.

Crisis Communication Starter Kit Get the Code

At this point, I would say that the entire team was still getting used to the lifestyle change of working from our respective homes due to the pandemic. However, we were a group of determined individuals, and with the help of our project manager - John Walicki, to keep us on track, we got to work 9 am sharp the following morning.

John Walicki - rock star Project managerUpkar Lidder - excellent engineer and Node.js integration expertMofi Rahman - amazing engineer and webhook integration expertDonna Byron - brilliant COVID-19 and CDC FAQ expertMatt Price - incredible Watson Assistant expert Jill Amaya & Emily Mitchell - show-stopping editors and publishers and Myself - awesome engineer & Slack integration expert

The idea behind our starter kit was to provide a solution for communication and messaging during a time of crisis. In this pandemic, communications systems are often the first to be overwhelmed with people trying to find necessary information about testing, symptoms, community response, and other resources. We thought that the best way to tackle this issue was to build out a chatbot using Watson Assistant, IBM Cloud Functions, and Watson Discovery News to demonstrate how we can integrate this solution in various ways. These integrations include Node.js application, Slack integration, and a speech to text application using Node-RED. The goal was to inspire developers worldwide to take on this challenge and use our solution as a blueprint to get started.

Our solution ultimately came down to 3 main elements :

The Chatbot
The Webhook
The Integrations

The conversational element of the chatbot was built using Watson Assistant, which made it very easy to build out the question and answer piece of the puzzle. We used a trusted source from the CDC FAQ page to populate the intents and entities of our bot, and we made sure to document all of our steps to inspire others to build an even more robust Q&A.

The webhook element of the chatbot provided much more robust functionality and sophistication. It turned our static Q&A bot into a dynamic application! We integrated Watson Discovery News as a news data source to get the latest news articles on COVID-19, and we also integrated the COVID-19 API to get more information on how many people have been affected. We did this through the use of serverless IBM Cloud Functions, which helped us integrate these two different data sources seamlessly.

Once we were happy with the chatbot Q&A and the data source outputs through the webhook, we finally had to think about the final element, which was the various ways we can integrate. Overall we demonstrated three ways to integrate our COVID Crisis Communications Bot:

1. Slack

2. Node.js Application

3. Speech to Text App with Node-RED

There are perhaps endless ways to integrate our bot. However, as a team, we felt that these three ways demonstrate the power of IBM Cloud and can provide a solid starting point to help developers take on the Call for Code COVID-19 challenge.

In the end, we published this starter kit with the hopes of inspiring others! Our starter kit was one of 3 starter kits that were published as resources for the Call for Code COVID-19 challenge. The other starter kits: Remote Education and Community Cooperation, also provide innovative solutions to get started with.

Take on the Call for Code Covid-19 challenge today!
Check out our starter kit here. Feel free to fork and submit pull requests!

Also if you want to learn more from us we will be doing a webinar on April 16th and April 20th!

Detecting Objects with a Tello Drone

Pooja Mistry — Wed, 25 Sep 2019 17:15:25 +0000

Have you ever wanted to detect objects using live video streaming from a drone? If so, with this tutorial you now can!

// Detect dark theme var iframe = document.getElementById('tweet-1174800354560630790-693'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1174800354560630790&theme=dark" }

Let's Get Started

In this tutorial, we will be using Node-RED. Designed and built by IBM, Node-RED is a free open source logic engine that allows programmers of any level to interconnect IoT, cloud-based systems, web services, databases, API’s and more!

Step 1: Get started by installing Node-RED locally

You can check out this tutorial on how to install Node-RED on your local computer , or follow the commands below :

sudo npm install -g --unsafe-perm node-red

This should install Node-RED, once installed run this command :

node-red

This should point you to a Node-RED URL http://127.0.0.1:1880/ that you can put in your browser to see the Node-RED editor

Note: In later steps, we will be adjusting the settings.js file that comes when you install Node-RED

Node-RED editor in your browser

Step 2: Install FFMpeg on your local computer

FFMpeg is a very powerful command-line tool that is used for performing various conversion operations on audio and video files. This tool will be the bread and butter for video streaming! It is free to use and is available for Windows, Linux and Mac operating systems

If you have a Mac, the best way to install it is by using homebrew.

Run: brew install ffmpeg to install FFmpeg locally to your computer.

This will be critical to see streams that are flowing in from your local computer!

Step 3: Install the FFMpeg Node in Node-RED

There are two ways you can install this node

Do npm install node-red-contrib-ffmpeg in the same directory that node-RED (from step 1) was installed
Open your node-RED browser and find node-red-contrib-ffmpeg in manage pallet

Open hamburger menu on right hand side > manage palette

Click install and you should see the ffmpeg node in your palette

Step 4: Train a Model

Follow this tutorial to train an object detection model .

Once you finish this tutorial you should end up with a model_web folder.

Step 5: Copy `model_web` into Node-RED and adjust settings

Once you have a model_web directory with your trained object model you will need to copy that into the directory that you installed Node-RED

We will then adjust the settings.js file in your node-RED director

Open settings.js and search for and uncomment out the following

httpAdminRoot: '/editor', httpStatic: '<path to model web directory >',

I set the endpoint of httpAdminRoot to editor, so now when you run node-red command to see your node-RED editor you will now go to this url http://127.0.0.1:1880/editor

httpStatic will include the location of your model_web folder

Step 6: Configure your Tello Drone

Before you can use your Tello Drone you MUST activate it in the official Tello Drone app. Once your drone is activated, you can connect to it's WiFi Network TELLO-XXXXXX and send it commands via UDP

Step 7: Import Flow into Node-RED

Start Node-RED locally and Import the following flow :

To import this flow , copy the json in the GIST above to your clipboard.

On your Node-RED editor go to the hamburger menu on top right-hand side > Import > Clipboard

Paste the json that you copied from your clipboard and click Import

You should see the following flow on your editor :

Step 8: Adjust Nodes and Stream from Tello!

Double click on the FFmpeg node and make sure it is configured for Tello Drone. Also by default, the URL will be set to /stream . Note, this can be changed to whatever url endpoint you want

In this example our stream will be accessible at ws://:/stream

Using the stream
To render the video stream in the browser, we use a library called JSMpeg.

*If you changed the stream url endpoint to something else , make sure you configure this in the template node on line 58. In the default we have it set as /stream *

const url = `ws://${window.location.hostname}:${window.location.port}/<url_endpoint>

Turn Tello Wifi On
Make sure your Tello drone is charged and you have turned it on. You should be able to connect to your Tello's wifi.

Once connected click Command and then Stream on

Go to http://127.0.0.1:1880/dashboard and you should see video streaming from your tello! If you have trained an object detection model, you should also see your tello detect objects! In my case I trained a model to distinguish between thumbs up and thumbs down

If you want to stop stream make sure to click on the stopstream inject button in the node-RED editor

The End

That's all folks :). Thank you so much for reading!

Check out this repo if you're interested in video streaming without the object detection!

Feel free to reach out to me with any questions in the comments below, or connect with me on twitter @poojamakes

If you found this tutorial fun and helpful, it would mean a lot to me if you gave it some <3 and shared it to help others! Thank you again!

Why you should be using Node-RED right now!

Pooja Mistry — Tue, 02 Jul 2019 02:31:53 +0000

This month I have spent most of my time exploring, learning and teaching Node- RED, and I have come to the conclusion that this awesome technology is one that you should be using right now!

Let me explain. Have you ever wanted to rapidly prototype something, whether that be a building an IoT device, a web service, or a larger proof of concept for your enterprise clients? Have you ever found yourself in an endless google search on how to get started or trying to figure out where your project will run? Well, with Node-RED you can start building, prototyping, and sharing within minutes.

Designed and built by IBM, Node-RED is a free open source logic engine that allows programmers of any level to interconnect IoT, cloud-based systems, webservices, databases, API’s and more! Here are 3 reasons why it is so awesome to use!

1. Node-RED’s power lies in its ease of use.

Using Node-RED is as simple as wiring graphical nodes together and deploying to see the desired out comes. It uses a visual web interface, where blocks –called nodes – can receive and send messages to other nodes. This makes the whole integration process easier by abstracting a lot of repetitive and boilerplate code needed to make these simple things.

Anyone with logic or programming background can easily start working with Node-RED. There are countless nodes in the node pallet that can help you build something worthwhile. These nodes allow you to inject your own data or data from other sources (i.e. social media, IoT devices, APIs and databases). You can also use function nodes to write your own logic as well as build your own web services user interface.

Here’s an example with a simple Hello World:

In this example, an inject node is used to send a message object manually to a function node, in this case “Hello World” and the function node replaces the msg.payload “World” with “Everyone, I hope you enjoy Node Red” . Then, it passes the message forward to a debug node, which prints it on the debug tab.

This simple flow can be incredibly powerful because when you are able to build out a hello world application, you can easily build something much more intricate without having to learn how to use several APIs and libraries. All it takes is being able to drag, drop, wire, and deploy and you’re ready to go!

2.Node-RED allows developers to prototype, test, build and rebuild within minutes
The countless examples and nodes available on Node-RED make it easy for developers to test out their ideas in minutes.

Let’s say you want to build an application that tells you whether a tweet is positive or is negative.

You can easily hook up a twitter input node and output tweets into your debug panel like so:

Now, once you have the input of tweets you want to hook up sentiment analysis , since this will give you the sentiment of the tweet and whether it represented positively or negatively.

In this example we have hooked up inject nodes to test the sentiment analysis service with static messages to represent influx of tweets. We can easily add tests to our flows to ensure that our services are working the way we want them to.

Now let’s say that we are happy with the way this flow is working however it would be awesome to represent the data in a dashboard / UI. There are nodes for this!

In this example we went from having an idea of getting tweets as inputs to building out a UI dashboard that shows the sentiment score or positive or negative tweets. Compared to the time and energy it would take to write your own tweet input, a sentiment algorithm, and a robust user interface, we can easily hook up the necessary nodes and get a working prototype in significantly less time.

Of course, this flow will not be the end-all-be-all, but it will show you how these services work together. If you decide to go with a prototype for more production-level work, your flow in Node-RED will provide you with a dependable road map. In many cases Node-RED also has you covered from prototype to production when building a single new function, a collection of functionalities, or an entire application from browser to database.

The ability to tweak, adjust, build and rebuild on the fly is Node-RED’s greatest feature. Get ready to start impressing those around you with robust prototypes that can be turned into production quality applications in less time than you think!

3.Node-RED is incredibly flexible and is built on dependable technology

Node-RED is built on top of one of the most dependable and ubiquitous technology stacks out there – JavaScript. This makes it highly flexible and easy to work with on web browser, server side, IoT projects and more.

Since you will be working in this space , there is incredible support from npm for Node-RED and there are over 3000 ready made nodes to get you started building whatever you want.

And if you don’t find a component that fits your needs from the thousands that are freely available, you can write your own.

You can run Node-RED as an IBM cloud service and easily have a connection to cloudant data base and IBM Watson services or you can install Node-RED locally. You can also install Node-RED in docker. It is highly flexible and easy to run and get started with, so the sky is the limit.

There you have it! Node-RED is incredibly powerful and useful for anyone who wants to take their ideas from concept to creation! It’s easy to use, its ideal for prototyping, and it is incredibly dependable. So, what are you waiting for, start building with Node-Red now!

I hope this blog helps you as you begin your next prototyping journey! Check out my Twitter Analysis workshop to get hands on how to on building twitter analyzers with Node-RED.

Give me a follow if you like this post or follow me on Twitter @poojamakes