DEV Community: Margo McCabe

Check out what Harper is up to

Margo McCabe — Thu, 26 Feb 2026 22:37:22 +0000

Stephen Goldberg for Harper

Feb 25

I Built an AI Ops Assistant That Replaced $2,400/month in SaaS — In 3 Days, for Under $100/month

#showdev #vibecoding #ai #rag

Comments 2

9 min read

8 Ways to Improve Application Performance

Margo McCabe — Thu, 10 Aug 2023 18:09:20 +0000

Intro

Application performance is critical for delivering a fast and responsive user experience. Slow performance, or high latency, can lead to frustrated users and lost revenue for the organization.

From a high level, application latency refers to the delay between the user's request and the application's response. Latency can not only impact the overall user experience and decrease engagement, but it can be a costly and complicated problem as well. App performance also impacts overall efficiency. Slow applications can slow down business processes, leading to decreased productivity and increased costs. All of these factors can reduce a company’s ability to compete in the market. In today’s world, customers expect fast and responsive applications. Businesses that are unable to deliver on this expectation risk losing customers to competitors who offer a better user experience.
‍
By implementing strategies and technologies that enable fast response times, organizations can streamline processes, improve productivity, and reduce spend.

What causes application latency or performance issues?

Often we take advantage of just how much is happening behind the scenes to deliver lightning fast responses, but we do notice immediately when there is a lag of more than just a few milliseconds. So what's causing this latency? Slow app performance can be caused by a variety of factors, including:
‍

Network issues: Slow network connectivity or high latency can cause app latency. This is particularly true for mobile apps, which often rely on cellular data connections that can be slower and less reliable than Wi-Fi connections.
Server issues: If an app relies on a server or cloud-based infrastructure, slow server response times or server downtime can cause app latency.
Inefficient code: Inefficient code can lead to slow app performance. This can be caused by factors such as poor algorithms, excessive use of CPU or memory resources, or inefficient use of network resources.
Too much data: If an app is processing or transmitting too much data, it can cause latency. This can be caused by factors such as large files, excessive requests, or inefficient data processing.
Device issues: Older or less powerful devices may struggle to run some apps, leading to app latency.
External factors: External factors such as interference or environmental factors can also contribute to app latency. For example, if an app relies on GPS data, poor GPS signal or interference can cause latency. ‍

In his article on Geo Distributed Databases, Jacob Cohen talks about some of the latency-sensitive applications where high latency quickly leads to poor user experience. Examples include Gaming, IoT, Robotics, and Autonomous Vehicles. Other examples of companies that clearly need to prioritize fast performance to keep users happy are apps like Netflix, Google, Uber, and Instagram.

Here are 8 ways to reduce application latency:

As with any solution, the specific techniques and strategies that will work best will depend on the specific needs and requirements of your application or organization. Below are some of the most common ways to improve your app's performance and keep your users happy:
‍

Use a distributed database or caching system to store and retrieve data more efficiently.
Optimize the network infrastructure to improve data transfer speeds and reduce bottlenecks.
Use a content delivery network (CDN) to distribute static and dynamic content to users more quickly.
Implement data compression techniques to reduce the amount of data that needs to be transferred.
Use load balancing and other techniques to distribute workloads across multiple servers and reduce the amount of time that each request takes to complete.
Optimize the application code to improve the efficiency and performance of the application.
Use in-memory databases and other types of high-speed storage to improve data access times.
Monitor and analyze application performance metrics to identify and address bottlenecks and other performance issues.

‍
Here at HarperDB, we talk a lot about latency, as it's one of the major challenges that we solve for organizations across the globe. It’s easy to see that application performance is important for user experience, business efficiency, customer satisfaction, and competitiveness. It’s highly beneficial to take the time to address the root cause of the problem as soon as possible, because by prioritizing fast application performance, organizations can improve their bottom line and build a strong brand image.

HarperDB: Staking a Claim as the Only Globally Distributed Edge Data Platform

Margo McCabe — Thu, 02 Feb 2023 16:41:21 +0000

Like most companies in general and especially tech startups, HarperDB has overcome many challenges to get to where we are today– which is the most exciting place we’ve been as a company thus far. We’ve learned that we were a couple years too early to market, and that being too early can surprisingly be more difficult than being too late. Over the years, we have explored different solutions and industries to find the right product-market fit, which has been a major roadblock. We learned from and built upon these experiences, for example if we hadn’t ventured into IIoT, HarperDB would not be what it is today in terms of scale and distributed capability. Due to perseverance, we have now positioned HarperDB as the only Globally Distributed Edge Data Platform, providing unprecedented value to our customers through avenues such as Edge Computing, AI/ML, APIs, and third-party integrations.

Considered a lean startup, HarperDB has achieved much more than competitors with much less funding under our belt. With 100K total HarperDB deployments and 10K community members/subscribers, our team continues to expand the product to meet the dynamic needs of our innovative customers. 2022 was HarperDB’s highest year in sales yet, and we expanded our enterprise partnerships and customer base, now working with companies like Lumen, Akamai, Google, Verizon, and Equinix to bring an end-to-end solution to the market.

More Reason to Celebrate

HarperDB is thrilled to be announced as one of Built in Colorado’s Best Places to Work, ranked at number two on the Best 2023 Startups list. Staying true to our core values as a company, our leadership has ensured that this continues to be a safe, fair, and enjoyable place to work, even through stages of rapid growth.
HarperDB’s CEO, Stephen Goldberg, was just named the 2023 Bill Daniels Ethical Leader of the Year. Now in its eighth year, the award recognizes local leaders that demonstrate remarkable integrity and ethics in business, exemplifying the ethical principles practiced by cable television pioneer Bill Daniels.
We just released HarperDB 4.0, which integrates with open-source connectivity platform NATS to enable limitless data distribution. Learn more in this SiliconAngle article.

As we wrap up 2022 and gain momentum for 2023, I sat down with HarperDB’s CEO & Co-Founder, Stephen Goldberg, for a brief Q&A:

What are HarperDB’s current strategic initiatives?

We started the company with the idea of trying to make the easiest database in the world that is also massively scalable. As we evolve, we still want to solve that, but we can also provide a lot of value driving low latency use cases because HarperDB is uniquely positioned as a fully distributed platform that allows for quickly distributing data globally anywhere on earth. So for now we are focused on distributed capabilities as well as performance.

Looking ahead, we will be focused on improving HarperDB as a platform. The product is extremely performant, easy to use, and highly distributable as is, but it could still use work as a platform in making developers’ lives easier so that they can have a global application platform in one place. This ties into edge computing because right now you can have HarperDB anywhere in the world, but in the future, we want that to be an entirely seamless process so that you can deploy globally with the click of a button.

Why is HarperDB focused on Edge Computing and how is the edge helping organizations that we work with?

Originally we were focused on IIoT and IoT, which is actually different from edge computing. IoT is really about the things (devices, etc.) in the field. While HarperDB has the ability to continue playing a role in that, where we can add the most value is at the edge– a step back from IoT. We are taking what is also the cloud and making it more local, closer to the end user, and much more distributed.

We accidentally built the most distributed database in the world that is uniquely horizontally scalable, so we found that our technology is really meaningful in these use cases where it can be deployed anywhere.

Telecom companies for example have noticed that HarperDB can be deployed anywhere in their network and solve a lot of problems around edge computing. Interestingly, people think that 5G makes edge computing easier, but in a lot of ways, it can make it more difficult. A lot of companies are dealing with major congestion issues due to different pipes of data going in and out from the cloud and now to and from the edge. This complex congestion in the data pipeline causes high latency and high cost. Because HarperDB can make your data available at the edge, edge computing can become real. You can build your entire application on HarperDB, so now you really have everything you need, along with the ability to use 5G and other globally distributed infrastructure now available.

What projects are you most excited about right now?

We’ve talked a lot about one of our customers named Edison Interactive. They were struggling with high latency and lack of real-time decision making, and we were able to simplify the transition to a distributed architecture, ultimately reducing latency from 5 seconds down to 20 milliseconds or less. (You can learn more about this case study on our website or on AWS’s blog)

We’re currently exploring similar use cases in industries like gaming, media, manufacturing, oil and gas, etc. Some of this we can’t quite talk about yet, but big things are happening. Overall, I’m not as excited about one specific area as much as being excited about the convergence of all these things and all these ideas that we’ve been talking about for a while finally becoming real.

What are some of the adoption challenges that enterprises face with Edge Computing?

There are some challenges around organizations taking the patterns that they’re comfortable with in the cloud and applying that to the edge. Security is obviously one that can also become interesting. It’s really important to understand that the edge, while similar to cloud, can be quite different.

The biggest challenge with implementing an edge strategy is the interoperability of the different partnerships that you need in place.

AWS made cloud so easy because they have everything you need consolidated into one package. At the edge, there is no dominant player enabling you to put your whole stack in one place, so organizations must put different solutions and partnerships together to achieve their end goal. With HarperDB, one large benefit is that we can run anywhere, on anything, and it doesn’t matter who’s above or below us– you get the same experience anywhere.

Is there space for AI and Machine Learning in Edge Computing? How does it all tie together?

We’re really excited about AI and machine learning (ML) on the edge. We’re already working on some projects in this space, Kevin from our team built a machine learning use case with HarperDB using TensorFlow for a customer. Machine learning is awesome for understanding your data after the fact in the cloud, but at the edge, you can do real-time ML and personalized content experiences and personalized interactions. ML and AI can require huge amounts of compute and huge amounts of data, but if you distribute that, you can do really exciting things such as having human interaction-like levels of data at the edge. By distributing to the edge you reduce the barrier to entry, reduce latency and cost, etc.

How is HarperDB partnering with other organizations to make Edge Computing real?

We’re partnered with Verizon, we worked with them on the Edison use case. Companies like Akamai are helping us distribute our technology globally. HarperDB is also partnering with a lot of cloud providers to run on their edge offerings with Wavelength and MEC (Multi-Access Edge Computing). These partnerships are exciting because to deliver a globally distributed platform, it takes more than just one piece of the puzzle, so these companies allow us to deliver a robust solution to our customers.

Predictions related to Edge, Cloud, and IoT moving forward in the next 5-10 years?

All of these siloed areas are moving towards a more powerful convergence. Machine learning and AI is one discipline, you also have data science, 5G, and different edge offerings. Then there’s a different sector of edge computing without 5G, which is much more distributed using existing technologies like containerization, Kubernetes, and other fabric-level technologies, to create a highly distributed edge. You also have blockchain, and so on. So we have this fairly fragmented set of disciplines which are beginning to converge and overlap. It’s too siloed right now but there is a gravitational pull that’s moving them towards each other.

Once these disciplines become more overlapped, we’ll begin to see more incredible use cases coming to fruition and the future becomes real. You have robotics, real-world human interaction with true AI (not just ML), and personalized interaction. For example, imagine walking into a store or hotel room and the entire experience is tailored to you, autonomous vehicles, etc… All of these different moving parts coming together will make these ambitions real, and it’s exciting, to say the least.

20+ Best HarperDB Tutorials

Margo McCabe — Thu, 26 May 2022 22:56:52 +0000

HarperDB is quickly becoming the database of choice for developers and enterprise organizations alike. HarperDB is a data and application platform that is independent of hardware and network providers, completely agnostic of where it resides, and peer-to-peer. It can handle 20k writes/second and 120k reads/second for a single node, with global data replication in under 100ms. It’s easy to deploy in the cloud, on-prem, and/or out to the edge. By prioritizing performance and efficiency, HarperDB is reducing latency and complexity related to data management and app development.

Here’s an extensive list of HarperDB tutorials (in no particular order), which will be helpful for everyone from beginners to experienced HarperDB users.

1. How to use HarperDB Custom Functions to Build your Entire Backend, by Andrew Baisden

“Towards the end of 2021 HarperDB released version 3.1 which introduced a new feature for the platform. HarperDB grew from a distributed database to a distributed application development platform. Basically this means that you can now host your database and server API on the same platform with full CRUD functionality! It is a single solution that takes care of all of your backend requirements.”

Github repo

Tech Stack:

Frontend: HTML, CSS, TypeScript, React
Backend: Node.js, HarperDB

Learn how to:

Get started with HarperDB
Use HarperDB Custom Functions
Create the backend database and API on HarperDB
Create the frontend React Application

2. Using TensorFlowJS & HarperDB for Machine Learning, by Kevin Ashcraft

“This article will explain how to train and use a TensorFlowJS model to classify dog breeds with HarperDB Custom Functions, using the Stanford Dogs dataset and MobileNetV2 as a base for transfer learning.”

GitHub repo

Tech Stack: Node.js, TensorFlowJS, MobileNetV2

Learn how to:

Install training data and TensorFlowJS modules
Train and verify the model
Add and remove new training data
Train with GPU

3. Build a Full Stack Productivity Timer App with NextJS and HarperDB, By Danny Adams

“Building full stack applications can be tough. You have to think about frontend, APIs, databases, authentication - and how all of these things work together. So, in this article, I'll show you how to do all of those things using NextJS and HarperDB. We'll be building a full stack task timer app that includes JSON Web Token Authentication, fetching data using HarperDB's built-in API, and rendering the data with NextJS. We will also make use of NextJS's API.”

GitHub repo

Tech Stack: NextJS, TypeScript, TailwindCSS, React

Learn how to:

Set up HarperDB
Create a Layout Component to Wrap Every Page
Create Reusable Components
JSON Web Token
Create a useUser custom hook
Create the Add/Select Taskbar

4. Building a C# Blazor App with NoSQL and SQL Using HarperDB, by @towernter

“This tutorial describes how to use HarperDB Database and Blazor. For this tutorial, we are going to build a simple Blazor .Net Core app using HarperDB.”

Github repo

Tech Stack: C#/.NET, Blazor, Visual Studio

Learn how to: Create a basic C# Blazor app and how to use HarperDB SQL and NoSQL operations in C#.

5. Analyze Twitter’s Reaction to Taylor Swift with HarperDB, by Aakriti Sharma

"Creating systems that extract, analyze, predict, and forecast data seamlessly are the need of the hour. To do so, we require high performance database management systems that help with scaling and optimization of the data transaction process. Services like HarperDB Cloud help with managing huge databases on the cloud, making the process more scalable, accessible, and rapid, with a built-in HTTP API endpoint accessible in all major languages."

Learn how to: Extracting tweets that include the topic ‘Taylor Swift’ and perform a detailed analysis by exploiting natural language processing.

6. How I used HarperDB Custom Functions and Recharts to create Dashboard, by @atapas

“The custom functions from HarperDB let you create APIs on the go. Learn how to create a dashboard using Recharts and HarperDB custom functions. Custom functions are a fantastic inclusion to the stack… you can create data store, APIs from the same place without worrying about deployment or managing them.”

Github Repo

Tech Stack: Recharts, React, JavaScript

Learn how to: create APIs with the help of HarperDB custom function and visualization using a Reactjs based library called Recharts.

7. Migrating a legacy application from MongoDB to HarperDB, by Lucas Santos

“Lots of people deal with legacy applications in a daily basis, and some of the changes they have to make include migrations from a tech stack to another. The objective is to perform the full migration without touching a lot of code.”

(Bonus: there’s also a video demo of this)

Github repo

Learn how to:

Set up the database
Migrate the code
Testing
Custom Functions

8. Building Microservices using Spring Boot + HarperDB and Deploying it on AWS, by Apoorv Tyagi

“Learn how to use Spring Boot and HarperDB to create a microservice, and look at how to deploy the complete application on AWS Elastic Beanstalk. You will be building an Employee Leave Management System. This application will be responsible for tracking the detailed record of employees' leaves. You will also be implementing the functionality to add, edit, and cancel leaves.”

GitHub Repo

Tech Stack: Java, Spring Boot, AWS Elastic Beanstalk

Learn how to:

Set up HDB
Create a Spring Boot App
Design REST APIs
Test the Endpoints
Create a JAR File
Create an Elastic Beanstalk environment
Configure Spring Boot Through Environment Variables
Test the app in the cloud

9. VueJS App with HarperDB, by Milecia

“HarperDB gives you a way to interact with your database with a client-side app. The HTTP API lets you submit database queries in JSON format, which makes it easier for you to dynamically generate your queries.”

GitHub repo

Tech Stack: VueJS, JavaScript

Learn how to: Use HarperDB with Vue to create a simple dashboard a user could login to and interact with.

10. How To Use HarperDB Custom Functions With Your React App, by @tyaga001

“Last week, I got a chance to explore HarperDB - a fast, modern database that allows you to develop full-stack apps. Let's learn about HarperDB and how to build a React app using HarperDB Custom Functions!.”

GitHub repo

Tech Stack: React, JavaScript,

Learn how to: Build a to-do list React app using HarperDB Custom Functions.

11. CRUD REST API using Node.js, Express, HarperDB, Docker, by Francesco Ciulla

“This example is to show how simple it is to get started using HarperDB cloud instance, monitor the DB entries. The UI can also be used to add/remove/delete values.”

Github repo

Tech Stack: Node.js, Express, Docker

Learn how to: How to create a Dockerized Create, Read, Update, Delete Application, connected with a HarperDB Cloud instance.

12. Libshare - Curate & Share Your favorite JavaScript Libraries, by @adisreyaj

“Libshare helps you curate and share your favorite NPM libraries with the community. Showcase all the awesome libraries that you used in your side project and give them visibility. Made for the ♥ of open-source.”

Github repo

Tech stack: Angular, Node

Learn how to: Build an app using Angular as the front-end framework and HarperDB Custom Functions for the entire back-end.

Set up HDB *Create APIs
Use env variables in HarperDB Custom Functions
Deploy CF
Set up the UI

13. Create a serverless book library app with HarperDB and Gatsbyjs, by Tapas Adhikary

“Recently, I had an opportunity to try out a modern, easy-to-use, fast database, HarperDB. As I learned how to set up and use it, I could further explore building a serverless app using it.”

GitHub Repo

Tech stack: Gatsby.js, React.js, Node.js

Learn how to: Create a serverless app using HarperDB & Gatsbyjs. In this step-by-step tutorial, we will learn to create a book library app and deploy.

14. Build A REST API With HarperDB and FastifyJS, by Catalin Pit

“This article will teach you how to use Node.js, Fastify and HarperDB to build a course management system. This application will help you track the courses you are doing and the courses you plan to do.”

GitHub repo

Tech stack: Node.js, Fastify

Learn how to: Use these Node.js, Fastify and HarperDB together to build a REST API

15. Build a Python App with HarperDB, by @bhavaniravi

“While you are busy solving business problems, services like HarperDB will take care of managing, scaling, replicating, and optimizing your databases. HarperDB is a cloud service that exposes your data via an API endpoint, making it language agnostic. It supports all major languages like Python, Java, NodeJS, etc.”

GitHub repo

Tech stack: Python, Flask

Learn how to: Build the backend of a Book Journal app w/ Python

16. Building a collection of Tech Blogs using ReactJS and HarperDB, by Bonnie

“I want to show you how to create a dynamic web page with a collection of awesome blogs of your choice. We will use React for the frontend and HarperDB as our database in this project. HarperDB is where all the dynamic data displayed on the front end will be stored.”

Github repo

Tech stack: HTML, CSS, Javascript, React.js

Learn how to: Use HarperDB’s Custom Functions to create API endpoints that will help us to make an API request to a standalone API server inside HarperDB. The API request will get the dynamic blogs data stored in HarperDB, and then React will display it on the frontend.

17. Creating your First CRUD Restful API with Node.JS and Hoppscotch, by @unclebigbay

“If you're here and want to know how to build APIs, then you're most welcome, because, in this article, I'll show you the very importance of an API and how to create your first CRUD restful API using Node.JS, Express.js, and Hoppscotch.”

Github repo

Tech stack: HoppScotch.io, Node.js, Express

Learn how to: Build a backend API for the popular Todo List project, and at the end of this article, you'll have a basic understanding of how API works and how to build your own APIs.

18. HarperDB and WebSockets, By Ethan Arrowood

“WebSocket is a computer communications protocol, providing full-duplex communication channels over a single TCP connection. HarperDB clusters replicate data between instances using the bi-directional pub/sub model. Clients can subscribe to the same WebSocket channels HarperDB uses and access all of the updates of a given table.”

Github repo

Tech stack: WebSocket, Node.js, Docker

Learn how to: Create a Node.js project powered by WebSockets and HarperDB. The application will be a data entry CLI with an additional display script.

19. Query data from HarperDB with GraphQL using StepZen, by Roy Derks

“With HarperDB, you’ll get the best of both types of databases, as you can interact with the database via SQL and NoSQL through their REST API. They don’t (currently) support GraphQL out of the box, but you can use StepZen to get a GraphQL API that interacts with the REST API that HarperDB exposes.”

Github repo

Tech stack: Stepzen, GraphQL

Learn how to: Set up HarperDB and query its data not via REST but with GraphQL using StepZen. With just a few commands from the StepZen CLI you can transform the HarperDB REST API into GraphQL.

20. Build a Hacker News Clone using React and HarperDB, by @ubahthebuilder

“One of the most effective and fun ways to boost your web development skills is by cloning an existing web page or website. In this tutorial, we’ll be cloning the Hacker News home page, making use of React and HarperDB.”

Github repo

Tech stack: React, Node

Learn how to: Build a Hacker News Clone using React and HarperDB

21. INTRODUCING: The Anonymous Feedback Platform powered by the CHERN Stack, Unclebigbay

“The CHERN stack is a development tool that combines Clerk dev, HarperDB, Express.js, React.js, and Node.js.”

Frontend repo / Backend repo

Tech stack: React, Node, Netlify, Heroku, Clerk, Express, Flutterwave

Learn how to: Build the Anonymous Feedback Platform, to bridge the gap (confidence) between the person willing to give feedback and the receiver, using the CHERN stack.

22. Build a Note Keeping API with NodeJS, Express, and HarperDB, by Lucky Victory

“In the previous article, we built a quote generator api, but this time we are going to do something different, substantial, and much more interesting, we are going to build a Note Keeping API.”

Github Repo

Tech Stack: Node.js, Express, Javascript, VS Code

Learn how to:

Integrate API keys into an API
Validate the user-supplied API key
Generate a new API key for the user
Revoke and expire an API key
Authentication and Authorization, how to authorize a user to access only their content
Use middleware in ExpressJS

23. Building a Contact Manager with Python, Flask, and HarperDB, by Lordghostx

“By integrating HarperDB with Python and Flask, we built a contact manager with CRUD functionalities with minimal effort. We saw how to create cloud databases, manage them, and execute queries using HarperDB.”

Github repo

Tech stack: Python, Flask

Learn how to: Building a Contact Manager with Python, Flask, and HarperDB

24. Building An Inventory Tool Using Appsmith and HarperDB, by Dancia Jos

“This guide explains how to quickly build an Inventory Management tool to manage movie rentals by connecting a HarperDB backend to an Appsmith frontend. It also implements authentication using HarperDB’s role-based access control feature.”

(Bonus: there’s also a video demo of this)

Tech stack: Appsmith, HarperDB

Learn how to: Create an inventory tracking tool that helps small businesses sign up on a platform to view and manage products in their inventory using HarperDB and Appsmith.

25. Build a Dynamic REST API with Custom Functions, by Terra Roush

“Simply put, Custom Functions (CF) are serverless, highly customizable API endpoints that interact with our HarperDB Core operations. You could define a CF using SQL to return all dog data records where the owner’s name is “Geena Davis” and sort them by the dog’s name. You can create a CF that deletes a dog’s data record using a route parameter and NoSQL. You can also make a CF with validation that saves a dog’s record with required properties. Custom Functions are powered by Fastify and are extremely flexible.”

GitHub repo

Learn how to: Build a dynamic REST API with Custom Functions. “I’ll describe what I’ve built first, and then I’ll show how you can use it for your own devices. Instead of going one by one through the routes, I’ll highlight the POST for its simplicity and the PUT for its complexity.”

Bonus section:

If video tutorials are more your vibe, you can check out the livestreams and demos on our YouTube channel.
Check out all of the HarperDB-Hashnode Hackathon submissions for more project inspiration.
Looking for a full course instead of a tutorial? You’re in luck:

Full Stack Grow Udemy Course, by Safa Alai

“Building a scalable web app in the Cloud from the ground up using web components, Docker containers and HarperDB. This course then gives you everything you need to get going with a serious web app, that puts full control into your hands and enables you to build your app from a hobby up to a highly successful business.”

Learn how to: Program a full-stack web application and release it to the Cloud

Requirements: Basic Javascript, Basic HTML and CSS, Basic knowledge of SQL

Looking for something that you don’t see here? Have an idea for a HarperDB tutorial? Please let me know!

🎙️Select* Podcast: For Devs & Tech Leaders

Margo McCabe — Fri, 22 Apr 2022 15:35:39 +0000

We’re already almost halfway through season two of Select*, the podcast hosted by HarperDB, and it seems like a good time to do a quick recap. We have once again been able to get some super smart and interesting people on the show -- it’s so fun being able to chat with them! Below you’ll see a summary of this season so far, including a few paraphrased moments from the episodes that I really enjoyed. You can find all of our episodes on the Select* RSS feed, on the podcast landing page, on YouTube, or wherever you listen to your podcasts!

S02 E1: TensorFlow.js, Building Community, & Coding for the Greater Good w/ Bekah HW

We started season 2 of SELECT* with BekahHW, a Software Developer & Technical Community Builder. Bekah talks about how learning to code can be therapeutic, what it was like building the Virtual Coffee community, and more recently becoming a Technical Community Builder. We also touch on what is TensorFlow.js and why it's important, as well as some really cool use cases for Machine Learning.

After spending ten years as a college English instructor, @bekahhw pivoted to coding and graduated from the Flatiron School Software Engineering program in May of 2019. She has spent most of her tech career specializing in front-end development, but has also created the developer community Virtual Coffee, spoke on podcasts and at conferences, and continued to mom her four kids. She's excited to be joining her first DevRel team with Deepgram as their Technical Community Builder.

Show moments:

When talking about how you don’t have to be an expert on a subject in order to teach it, Bekah mentioned:

It can sometimes be hard to ask questions because we’re not used to it, but we’re all looking up stuff all of the time. It’s not gonna look the same all the time, tech changes too quickly, so ask those questions because you will learn much more and much more quickly if you’re willing to ask. There’s always something new to learn out there.

S02 E02: Life as a Front End Engineer at an Early Stage Startup w/ Annie Liew

Episode 2 features Annie Liew, also known as @anniebombanie, a Front-end Engineering Lead at Pastel, for a discussion on life at an early stage startup. In this episode we learn a bit about Annie's journey, as well as the following topics:

Tell us about the company you're at and their product
What's it like working at an early stage startup? Were you scared to take the plunge into the startup world?
Pros and cons of being on such a small team? Tips for folks in a similar position?
What are you focusing on at the moment, main goals for 2022?
What technologies / frameworks / tools are you currently learning or really excited about right now?

Annie comes from a multidisciplinary design & teaching background. She's currently the Front-end Engineering Lead at Pastel and is active in both the design and developer community. In her free time, she reads, travels and makes CSS art! You can see Annie's website here.

Show moments:

When talking about overcoming challenges related to being on a smaller team:

I was concerned about mentorship on a small team, which I do have with my CTO, but at Pastel me and my CTO are the only two devs. So I’m not able to have my decisions challenged all the time, and sometimes it feels like everything is up to me. What I’ve done is get involved with a lot of communities so that I can get an outside perspective. If you’re at a very small startup and want to hear different opinions, just ask an external community so you can bounce your ideas back and forth.

S02 E03: What Web3 Means for Developers w/ Pratham Prasoon

In this episode we chat with Pratham Prasoon, a Programmer, Student, Developer Advocate, and web3 super genius, about what is web3 and what it means for developers. Some additional topics covered include:

How do NFTs work?
What are potential concerns or downsides of web3 and NFTs?
What tools and technologies should people be learning who want to move towards web3?
What are you working on next / what side project are you excited about?

Pratham is 17 years old. Building in web3 and machine learning and sharing his journey.

Show moments:

I asked Pratham to define web3 in his point of view:

Typically when you’re in the web2 world, creators are putting their heart and soul into making content, basically working for the platform if you think about it. It’s the platform that benefits and makes all the money. The web3 world is about the creators and the users of the platform being the main ones benefiting from the system. It’s mostly about decentralization - meaning everyone has the power to participate in the network and be a part of the platform and they can get tangible benefits out of it.

(Side note: You can also read about what web3 and decentralization mean for data storage in my recent blog).

S02 E04: Why I'm Excited About Machine Learning & Python w/ Patrick Loeber

In this episode we chat with Patrick Loeber, a Software Engineer & YouTuber who is teaching the community about Python and Machine Learning. Topics covered include:

Why are you excited about machine learning, why is python your framework of focus?
Is there a difference between AI / ML / deep learning, how do they intersect?
Top tips / biggest mistakes you see for folks learning python? Challenges with machine learning?
Where do you see machine learning being used now and 10 years from now?
Favorite technologies / tools, things you're learning now?

Patrick (@pat_loeber) is a Software Engineer and Developer Advocate with a passion for Machine Learning and Data Science. He’s the founder of http://python-engineer.com and the corresponding YouTube channel where he creates programming content for over 100k subscribers.

Show moments:

How do AI / ML / deep learning intersect?

AI is like the big buzzword for everything. Machine learning is a sub-area of AI. By definition, Machine Learning is when the computer tries to learn something on its own based on explicit instructions. You do this by showing framing data to the computer and the computer tries to learn with this data. Deep learning is another sub-area of Machine Learning that involves neural networks (another buzzword) which are inspired from human brains where you connect neurons. Deep learning can figure out more complex stuff.

S02 E05 - From Classical Musician to Software Developer w/ Jessica Wilkins

In this episode we chat with Jessica Wilkins, a former classical musician who came across an awesome opportunity in software development through technical writing. Questions we covered include:

What do you do now, how did you get to where you are today?
How did you transition into tech, how did platforms like FreeCodeCamp help with the transition and create strong connections in the community?
What was it like coming from a non-technical background?
Were there parallels between being a classical musician and learning to code? How were you able to pull from previous knowledge to assist with the learning process?
Other tips for folks just starting out with their dev career?
What technologies and/or tools are you focused on right now or excited about?

Jessica (@codergirl1991)is a classical musician turned junior developer and technical writer. Prior to joining the tech industry, she spent her time running her own sheet music company (JDW Sheet Music) as well as performing and teaching in Los Angeles, CA. She now enjoys working as a developer and working with React, Node and Express. She is also a prolific technical writer for freeCodeCamp, and created the Black Excellence Music Project which is dedicated to black artists who have made significant contributions to the arts.

Show moments:

When talking about coming from a non-technical background:

I think the pandemic made everyone really think about what they’re doing, and a lot of people realized they wanted to move into a different career. We’re starting to see much more diverse career changers that are successful developers now working in the industry. If you’re self taught, you not only need to study the code part, but you also need to study how to get a job. A lot of people don’t know how to get that first job, and get the hiring manager’s attention. There's so many great resources out there like Danny Thompson's Linkedin series on Youtube, also the 100devs series on how to get a job. Not a lot of people spend that time to understand how the industry works which is especially important when you’re a career changer.

S02 E06: Decentralized Computing w/ a VP Systems Engineer, José Domingos

In this episode we chat with José Domingos, a VP of Solutions Engineering at Ori Industries, which is a unique edge orchestration and networking solution.

In your words, define decentralized computing. Why is it important? Why are companies focused on it? How does it tie into blockchain and web3?
How is Ori enabling / enhancing decentralized computing?
How are HarperDB and Ori working together?
What types of use cases are a fit for decentralization?
Are there any risks / downsides to decentralized computing? * What's the best process to move towards this methodology?
Looking ahead: what can we expect from Ori next? What are you excited about?

José has 20+ years experience in telecommunications and networking, designing and architecting very large scale monitoring systems, for global and in-country networks. At Ori Industries he is the VP Systems Engineering, responsible for Solution Integration. José has been working on Edge and Cloud Native technologies for the last 4 years and his passion is technology and science in general. José also loves the outdoors, trekking, swimming or cycling, whatever gets him out in nature.

Show moments:

Why is decentralized computing important:

Decentralized computing is not a new concept. I like to think it’s like a cycle. First we had highly centralized compute, then mainframe, then the PC which changed to more decentralized, then came cloud computing. The network cloudification was also in place. Public cloud was effectively centralized compute again. Now with this new cycle with edge computing, where the computing is closer to the consumers and producers of data, that’s where we are with decentralization. The challenge now is harnessing the ease of centralized computing but the distribution of that computing everywhere - how can we make that work as an industry.

That's a wrap for season two so far, stay tuned for more awesome episodes comin' your way. And before we go, let’s not forget the incredible guests we had throughout season one. Shoutout to a few of our most popular episodes from 2021:

Thanks for listening! If you have any questions, comments, or feedback, please comment below. Interested in nominating a guest or suggesting a topic? Let me know!

Virtual Coffee w/ HarperDB's EVP of Engineering ☕️🧑‍💻

Margo McCabe — Tue, 19 Apr 2022 14:57:43 +0000

While swag giveaways are always fun, this time we're doing a different kind of giveaway -- one that will (hopefully) provide a lot more value than a free T-shirt 😉.

We're offering a virtual coffee, also known as a video chat, with HarperDB's EVP of Engineering & Co-Founder, Kyle Bernhardy. This is a unique opportunity to ask any questions - whether it be related to entrepreneurship, programming, data, leading a team, journey into tech, etc. - or just a chance to sit and chat with a super cool and knowledgable dude. Kyle is an incredible resource, and a great contact to have in your network.

Entering this giveaway contest is easy! All you have to do is engage with our post on Twitter or Instagram, and the winner will be picked at random by the end of the week.

🕺🕺🕺

Who is Kyle?
Kyle Bernhardy has over 20 years of experience in the tech industry and has held leadership roles in product development, support, and consulting at companies ranging in size from startup to publicly traded. Kyle is the co-inventor of 2 patents, and he's an expert in cloud architecture as well as multiple programming languages.

Kyle was also recently on the Stack Overflow Podcast and the Code Story Podcast sharing a bit about the HarperDB founding story, and he was even on the Code Newbie Podcast talking about different database architectures and their use cases.

Excerpt from the Code Story podcast description:

Kyle Bernhardy had a less than traditional path to the tech world. He has been in the industry since the late 90’s, but it all started when he was growing up, attempting to program text adventures games on his Commodore 64. In High School, he started running and that became a major part of his life – so much so, he went to run for Penn State University. He settled into an exercise and sports science major, so he could stay close to the sport.

Post college – he didn’t know what to do with his degree. He worked in a few corporate fitness centers, but didn’t enjoy what he was doing and didn’t see a path for himself. During that time, he ended up getting a job in tech support. He grew in this role, in his knowledge of DB basics, SQL and supporting database reporting. Once this happened, he moved into the development world. Programming for Kyle is a creative outlet, but not for art… for problem solving. In 2010, he started working for a data center company and met his now co-founder. They would hang out and chat about what it would be like to build something together. While working for a large data aggregation company, he worked through a number of data availability, reliability, and infrastructure around massive data processing. Internally, he thought there must be a better way.

If you can't tell, I'm super excited about sharing this opportunity! Pop on over to Twitter if you'd like to have a virtual coffee with Kyle, and please leave a comment below if you have any questions.

Migrate an App from MongoDB to HarperDB Livestream

Margo McCabe — Wed, 16 Feb 2022 20:02:37 +0000

Don't Miss This Opportunity to Code with Lucas🚀

Lucas Santos, Senior Software Engineer and Google Dev Expert, demonstrated the migration process from a MongoDB database to a HarperDB instance using clients and SDKs. He refactored a legacy, layered app, showcasing necessary steps in the app like date conversion, UUID handling, and more. The core of this demo includes converting an application that's using MongoDB with an event sourcing architecture over to HarperDB, as well as details like passing operations to HarperDB, and using HarperDB's Custom Functions feature. (Lucas uses TypeScript for the demo and Custom Functions are written in Node.js)

This livestream is for developers and tech leaders of any skill level, and will help you strengthen your app development skills, as well as general database / data management knowledge.

Watch the recording or check out the written tutorial to code along.

Resources

Why migrate to HarperDB?

While MongoDB might be the most prolific NoSQL database around, there are certain cases that might be better served with another solution. Examples would be projects where you need both SQL & NoSQL, rapid app development, integration, edge computing, and distributed computing. Or, maybe it's just time to try something new! HarperDB is faster than other DB's in benchmarks, has a native REST API, supports SQL on JSON, and scales out of the box for concurrent operations.

Livestream Q&A:
Why switch to HarperDB?
Lucas Santos: I liked the way Harper doesn't only have search by hash, but you can actually use tokens to authenticate in the database so you can completely remove the authentication layer from your app and put it on Custom Functions in the database. You can just generate tokens inside the database without having to expose that functionality. The other thing is that you can use SQL and NoSQL... most of the time I use NoSQL but I've had a lot of opportunities where I really wish MongoDB could support SQL, so the best of both worlds is amazing.

Favorite thing about HarperDB?
Lucas Santos: One is the built-in functions, and the Custom Functions is really nice because you can migrate a whole app inside of it. So if you have a small app that you want to spin up without hassle, you can put everything in Harper and you'll have the API and the database in the same place, so it's just one thing to deal with.

Meet Lucas Santos! Software Engineer & Google Dev Expert

Lucas Santos (@khaosdoctor on Dev) works as a Software Engineer at Klarna. Technology lover and, since 2011, works with cloud-native applications. Loves to teach and learn, writer of "Kubernetes: Everything you need to know about container orchestration". Granted the title of Google Developer Expert, Microsoft MVP, and Docker Captain who has been talking about technology for over 4 years both nationally and internationally. Member & Contributor to the Node.js Foundation, KEDA Project, and the TypeScript language. Also is one of the core members of awesome communities such as NodeBR, JSSP, VueJs SP, Typescript BR, NodeSchool SP and ABCDev.

Web3 & Decentralization: What it Means for Data Storage

Margo McCabe — Wed, 02 Feb 2022 16:26:00 +0000

The recent explosion of web3 content, discussion, and debating seems to have happened overnight. Or, maybe we just recently started using the term to describe what we’ve been moving towards since the Internet was first conceptualized. Either way, there are already tons of great articles out there on web3 and blockchain technology. Instead, I wanted to dig a bit deeper into how web3 and decentralization are transforming my world: data storage.

In case you need a refresher, web3 is basically the decentralized web - a group of technologies that are decentralized and used to create decentralized applications. @dabit3 suggests that some of the characteristics enabled by web3 are decentralized web infrastructure ownership, native digital payments, self-sovereign identity, distributed infrastructure, and open backends. The pathway towards web3 is focused on becoming more independent. So, unlike web2 (which we’re in now), web3 means instead of a single company centralizing control, individual contributors own and control the underlying technology stack. In other words, web3 uses blockchains and distributed peer-to-peer networks instead of server-client relationships.

The Problem With Web2

@edatweets_, along with many knowledgeable folks in the tech community, pointed out that the main issue of web2 is that it’s highly centralized. Large companies own servers that provide messaging, searching, storing, etc., and they have complete control and ownership over the services they provide. This is spot on, and something we’ve been talking about quite a bit in the data management world. In my Hybrid Cloud blog last year, I mentioned that large cloud providers are not highly distributed and tend to have large data centers for specific regions. These centralized data centers may have been the right solution at the time they were implemented, but the users are no longer trusting their information, and web3 is a way to bring back independence. My colleague Jacob Cohen explained how this centralization also causes massive latency issues due to offloading processing from an app to an external server. Some of this latency is because in web2, the external server is often a monolithic database residing in a single cloud region.

You most likely know by now that the answer to some of these web2 data storage challenges is decentralization. Distributed computing is certainly used in web2, but it’s mainly centralized and owned by the same company. But how do we achieve distributed data storage? In his geo-distributed databases blog, Jacob mentions that you need distributed data centers and a database technology that can be distributed. The expansion of this distributed storage ecosystem allows users to manage their own data so that they no longer have to use centralized platforms. For example, you can create your own blog and geo-host it instead of relying on a big business forum that can censor your content.

Web3 Data Storage

In a web3 tech stack, Nader mentions that you might use peer-to-peer solutions similarly to how you would use a database in a traditional tech stack, but instead they are replicated across n number of nodes on a decentralized network, and therefore more reliable. This is music to my ears! It’s exactly what companies like HarperDB are doing to enable decentralized data storage.

The question here is: is it really possible for a database to be entirely decentralized? Blockchain technology is not the same as a database, it stores some data but it’s mainly just transaction metadata. So if you need to store or transport any type of data, which you probably do, you will most likely also need a database. While blockchain may enable you to achieve full trustless decentralization, we still need data storage off-chain for certain projects. This means that you won’t be purely decentralized if you need database-style storage, but this trade-off might be worth it in the end.

HarperDB: a Web3 Database?

I think of web3 and decentralization on a spectrum - it’s not just one or the other, but you can take incremental steps on the path towards your end goal. For now, the data solution might be to at least use a database that is independent of hardware and network providers, completely agnostic of where it resides, and peer-to-peer. HarperDB, a decentralized peer-to-peer database, is certainly on the list of options for web3 databases today.

With HarperDB’s flexible deployment options, users can avoid vendor lock-in and run the database anywhere. It can be deployed on any combination of cloud providers, data centers, and/or edge devices, and anything in between. Additionally, all of those HarperDB nodes can communicate with each other (assuming some sort of network connectivity) via clustering and replication. HarperDB’s powerful single endpoint REST API provides an intuitive interface everywhere it is deployed, ultimately simplifying workload and reducing application complexity.

True Edge Persistence

HarperDB is the same codebase, with a small footprint, regardless of where it is installed. You can connect multiple HarperDB instances together in a cluster where you can set up bi-directional replication at a table level (pub-sub model). HarperDB’s globally distributed peer-to-peer read/write consistency and API distribution capabilities shift application workloads directly to the edge, independent of the cloud. You can shift out to the edge by way of the HarperDB stack to enable edge decisioning without having to refactor your application. This eliminates gaps between data collection and cloud, enables real-time data sync, reduces latency, and improves user experience.

Looking Ahead

The team at HarperDB has been focused on decentralization since inception, continuously evolving the technology to meet the data management needs of our innovative users. More recently, we have partnered with large network and infrastructure providers to offer end-to-end solutions like Hybrid-Cloud, Edge Computing, Machine Learning, and Real-Time Data Sync. The goal is to continue to make developers’ lives easier, while facilitating autonomy and freeing companies from lock-in.

We have come a long way in the world of data management, and maybe the next step will be integration between off-chain data solutions with blockchain protocols. Like any technology innovation, while there are many reasons to get excited about web3, there are also some potential drawbacks to consider. This is why it’s always essential to take a step back to ensure that the technologies and methodologies being implemented are the best fit for your specific product or use case, instead of just jumping on the bandwagon of the next hottest thing. What do you think? Are we moving full steam ahead towards a web3 world? What will data storage look like in 5, 10, 20 years? Share your thoughts below.

HarperDB in the Real World

Margo McCabe — Thu, 14 Oct 2021 15:11:00 +0000

By now you may be familiar with HarperDB and how we’re different from other databases out there, but you might be wondering how folks are currently using HarperDB in real-world applications. Or, if you already have a database of choice that’s working well, you may be wondering why you would take the plunge to try something new. Fair enough! To provide some clarity, I’d like to share a few examples of how HarperDB is currently being used in production.

Quotes from HarperDB Users

"I’m planning to break out a Monolith and use HarperDB as my datasource for the Front End part of the Platform solution I’m refactoring. One of the main reasons for considering HarperDB, as a Solutions Architect, the amount of time that is required from DEV to PROD to setup is close to none and not to mention that it is easy to adopt it and start working towards a deliverable from Day 1. All of my PoC’s, for Front End, Middleware or BackEnd, use a local HarperDB instance to start off."
-Rick Limon, Framework Science, Mexico

“I found out about HarperDB in a Tweet which mentioned how using Flutter with HarperDB was the ultimate combo and believe me they were not telling lies. I have used HarperDB for several projects now which has turned out great. I am currently using HarperDB to remake the Domino's pizza app for android and I can say for a fact everything has been running smooth as expected. I chose HarperDB because of their great response time and how easy it is to integrate due to the cloud functions, and on no occasion have I been let down or has anyone I referred given back a report contrary to what I told them HarperDB does. For that reason I stand with HarperDB”
-Collins, Nigeria

"We started looking at all the different databases that are scalable like Couchbase and a bunch of others, but we ended up focusing on HarperDB because of the flexibility... HarperDB scales beautifully; a containerized version of HarperDB tied to persistent storage allows me to scale HarperDB to meet my performance goals. The workload it can perform is amazing, and the ability to actually scale horizontally is amazing as well because it’s not typical for database engines.”
-Ron Lewis, Lumen Technologies, USA

“I am a front end developer, currently the maintainer of an open source project... as a real time stream framework for geo-distributed system, we decide to use HarperDB as our database in scenarios of AI / IoT / Virtual Office.”
-Yona

“I plan on using HarperDB for my side project (A web application for SCD NGOs). This will be used for the following sections: blog section of application, gallery section, other similar pages as well.”
-Emmanuel Kolawole, Estonia

“We are trying to improve our data processes, and I am excited to use HarperDB to store our data and plug it to Tableau dashboards!”
-James

“Here are a few things from my journey with HarperDB:

I came to know about it from the Hashnode hackathon, I made an application where people send birthday wishes to a person and all of them are curated in a single place, for example this these are some of the wishes my friend received, they are in a regional language (Tamil).
Then I also made another application for good luck, I used to feel worried before job interviews, so I added a few quotes in the database, and decided to make this application, people can get some confidence boost here.
Since the fetch operation is really easy in HarperDB, this is my go-to platform for storing access tokens for api's which should not be displayed in public, tokens that are refreshed with each session. HarperDB is really convenient, in my workplace we use Azure SQL mostly, but I use HarperDB for all the prototyping and testing.
I liked Firebase too, but learning curve of HarperDB is smaller compared to Firebase, which is really cool.”

-Deepak Rawat, India

“I am loving my experience with HarperDB so far, I am working on a number of GPS related projects as well as some PWA’s that are going to be utilizing HarperDB.”
-Jason, USA

Case Studies

In addition to quotes from devs in the community, I wanted to share a couple case studies where you can read about HarperDB projects in greater depth.

US Army: Accelerating Tactical Edge with HarperDB

HarperDB has partnered with Command Cyber Solutions to provide the Army with an innovative solution that will enhance the Army’s tactical edge and data orchestration capabilities. Command Cyber will leverage HarperDB to develop an edge-based analytics platform that can capture facial recognition data and distribute it to other HarperDB nodes for real-time alerting and detection.

HarperDB will be deployed on the extreme edge, a mobile command center, and a server to not only collect and orchestrate data throughout the nodes, but also propagate rules engines and AI models from the command center to the edge to update logic and thresholds on the fly. This will allow the Army to adjust the priority level of the faces detected from the command center in case a high threat is found. In addition, priority levels of faces detected will be automatically upgraded when they are captured in high sensitivity areas.

If interested, you can view and download the one-pager here to learn more about the Army case study.

High Speed Data Monitoring with Casne Engineering

HarperDB and Casne Engineering partnered to develop high performance data acquisition and in line analytics capabilities that enable organizations to capture sub-second data streams from industrial control systems like PLC’s and SCADA systems. The solution then filters out the massive amounts of irrelevant data at the edge, and forwards only the pertinent or anomaly data for use in OT historians and machine learning applications.

This high frequency in-line analytics approach allows operators to gain actual real-time insights and detect anomalies that would normally be missed using traditional low frequency data collection techniques. HarperDB’s data management solution enables organizations to efficiently collect, process, and analyze data streams in conjunction with other unstructured data including machine vision, geospatial, and Internet data feeds.

HarperDB and Casne recently completed a project for a client in the utilities / energy sector, where they were able to capture high resolution data to predict and prevent equipment failures and downtime with a single data management solution from plant to cloud.

If interested, you can view and download the one-pager here to learn more about this case study.

You can read about other industries that greatly benefit from high-performing, low-latency, geo-distributed databases like HarperDB in this article as well.

Developer Examples / Featured Projects

Lastly, I thought it might be helpful to share a few tutorials and other resources where you can follow along to build cool things with HarperDB:

Featured projects on our website
All Hashnode-HarperDB Hackathon project submissions
Livestream / code along events
A few specific tutorials:

So, what do you think, is there anything in here that piques your interest or at least helps to provide some clarity as to why and how folks are using HarperDB? What are your thoughts?

If you’re currently working with HarperDB, we’d love to hear from you! Share a bit about your project or experience in the comments below.

Build an App Using HarperDB's New Custom Functions 🚀

Margo McCabe — Tue, 07 Sep 2021 15:41:54 +0000

Don't miss this opportunity to see HarperDB 3.1 in action!

Code along with HarperDB's VP of Product, Jaxon Repp, in this livestream to code along and build your own app, while utilizing new (and super exciting) functionality with the HarperDB 3.1 release! Jaxon will build out an API and UI that utilizes HarperDB and our new Custom Functions, which enable developers to code their own custom functions, utilizing core HarperDB methods to define a custom API endpoint, resulting in a simplified tech stack and improved performance. Custom Functions are written in Node.js and the UI will be in React.

Watch livestream here

To Code Along You Will Need:

This event is for developers and tech leaders of any skill level, and will help you strengthen your app development skills, as well as general database / data management knowledge.

Meet Jaxon Repp, VP of Product at HarperDB

Jaxon has 25 years of experience architecting, designing, and developing enterprise software. He has served as founder of three technology startups, run Technology, Product, and Solution Architecture at several others, and consulted with multiple Fortune 500 companies on IoT and Digital Transformation initiatives. A partially-reformed developer, he understands what it’s like to wrestle with technology instead of benefiting from it, and believes passionately that if the Jetsons never had an episode where a config file error brought down the food-o-matic, it surely should not be a problem now.

HarperDB: More Than a Database

Margo McCabe — Thu, 12 Aug 2021 15:05:00 +0000

I recently had a very interesting conversation on our podcast with Ron Lewis, the Director of Innovation and Engineering at Lumen Technologies. Ron brought up the notion that HarperDB is more than just a database, and for certain users or projects, HarperDB is not serving as a database at all. How can this be possible?

Database, Explained

Well, what really is a database? Wikipedia states that, “In computing, a database is an organized collection of data stored and accessed electronically from a computer system.” Another site simply states that “A database is a systematic collection of data. They support electronic storage and manipulation of data. Databases make data management easy.”

So at its core, yes, HarperDB is certainly a database and can fulfill this functionality (after all, that’s what the DB stands for). But it can do so much more. For example, there are many cases where organizations keep their existing database system(s) in place, and use HarperDB to extend their current functionality or for a different capability altogether. Especially when it comes to solving complex enterprise data management challenges, the answer rarely (if ever) comes down to this database vs. that database. There’s much more to it. There are many different moving parts related to capturing the right data, getting data to where it needs to be in a timely manner, analyzing and acting on that data, etc. This is really where HarperDB shines.

HarperDB: A Runway for Launching Industry 4.0 Technology

Ron mentioned in our podcast discussion:

“The reason I see HarperDB as a disruptive technology is because you often call HarperDB a database, but it’s not really a database. It’s maybe what some people call a data mesh or data fabric... I see HarperDB more as a data surface, especially with Functions. The whole idea is to be able to converge and contextualize data to support decision making.”

The new Custom Functions Ron refers to will enable users to define their own API endpoints within HarperDB, ultimately expanding HarperDB from a distributed database to a distributed application development platform with integrated persistence. So, now we’re thinking of HarperDB as a data mesh, fabric, or surface instead of a database. That’s a lot of buzzwords! Let’s take a step back.

When I asked Ron what initially drew him and his team to HarperDB, he provided some great insight. Ron mentioned that they were working on a project for the Department of Defense (DoD) that required moving, contextualizing, and converging of data, and they needed something super fast and intuitive. They were essentially looking for something easy to use and easy to deploy that’s also flexible and scalable. Once HarperDB and Ron connected, he discovered that HarperDB could be deployed on devices as small as microprocessors like a Raspberry Pi or Tinker Board, all the way up to large scale servers, cloud machines, or supercomputers. This piqued his interest, as he needed the ability to do large scale analytics and move the data between devices in a simple manner.

At a basic level, we quickly realized that Ron and the HarperDB team were asking the same questions:

When we look at how much data has to move, how much is being created on an hourly basis from OT data onsite etc., how do we manage, transport, and take advantage of all of that data?
How do we get the data to where it needs to be in the most efficient manner possible?

Extended Functionality

Ron said that with HarperDB, he and his team could “define the data movement and do all these crazy cool things.” As they were looking at different military adaptations they were able to take data that is running integrated into a controller environment (OT) and expose that data without needing to have a human machine interface (HMI). They could securely move that OT data into the cloud, into a highly scalable enterprise analytics domain powered by HarperDB in the cloud on compute nodes.

There are many use cases similar to this, where HarperDB can provide a holistic solution that makes data sync and management easy. In the defense space, HarperDB’s bidirectional data movement enables the collection and movement of data and logic in real time, shifting decision making throughout the network as needed. Gaming and media industries benefit from HarperDB’s high performance and low latency, with clear implications for both the organization and the end-user. Retail and ticketing can recognize and block bad bots in real time with HarperDB’s global replication and edge persistence. The list goes on! You can read more about industries that benefit from a high performing, low latency, geo-distributed database here.

Why HarperDB?

Ron explained, “We started looking at all the different databases that are scalable like Couchbase and a bunch of others, but we ended up focusing on HarperDB because of the flexibility. Then, Stephen came up with the idea of Functions because a lot of what we did required us to put an API proxy in front of the data engine. He said, how about I make your life simpler? It’s just amazing how HarperDB checks all the boxes.”

Ron continued, “If you think about how databases communicate and the different models, I love the way HarperDB does it through the native integration of all of these components. No matter what it’s running on or where, HarperDB is disruptive because I’m able to move the different types of data, and different types of assets like functionality, from place to place seamlessly without having to worry about the interoperability of different data engines, nor do I have to worry about the size and scale. Databases are not typically designed as persistent vs. non-persistent, they tend to be scaled vertically instead of horizontally. HarperDB scales beautifully; a containerized version of HarperDB tied to persistent storage allows me to scale HarperDB to meet my performance goals. The workload it can perform is amazing, and the ability to actually scale horizontally is amazing as well because it’s not typical for database engines.”

Therefore, HarperDB is a unique solution for complex enterprise data challenges because the database engine is small and flexible enough to run on a microcontroller running on an onboard system, that can also be extended to edge bare metal or some kind of edge compute environment for higher fidelity analysis, and can also be moved to the cloud -- all at the speed of the Internet. HarperDB can scale vertically and horizontally while meeting performance needs. It really is more than just a database.

To sum it up, Ron stated:

“From a data driven ecosystem, HarperDB is paving the path forward moving from mesh to fabric to actual data surface and providing that contextualization of data right out of the database engine, which will be key to a fundamental shift in application behavior.”

Predictions for the Future

To wrap up our conversation, I asked Ron about the future of technology. He mentioned a few key things:

The move from cloud to edge is almost certain.
The nature of applications will change to take on a more distributed nature, along the lines of distributed functionality with edge workloads managed and deployed from some type of cloud orchestrator.
As we look at the nature of apps changing, data will be more contextualized from a database engine or persistence layer perspective rather than an application or business layer, and HarperDB is leading the charge on that.

There you have it folks. The future is all about data. There is a constant need for organizations to have their data where they need it, with the ability to orchestrate data where it’s both being created and consumed. If you’re not evaluating your data and how you’re handling your data assets, where will you be in 1, 5, or 10 years from now?

Improving Mobile App Performance with a Powerful Database

Margo McCabe — Thu, 22 Jul 2021 14:49:00 +0000

You are probably aware that there are seemingly endless options to consider when it comes to selecting a database and other technologies for your mobile app. With so many choices, it can be difficult and confusing to determine what really matters when it comes to your tech stack. A little while back, I wrote an extensive article on the different database architectures and use cases available to provide guidance on picking the right technology for the right project. While that is still an accurate and solid resource, this article takes a deeper dive into considerations for improving the performance of specifically mobile apps.

Mobile vs. Web Apps

First and foremost, perhaps we should take a quick look at the difference between mobile and web apps. Mobile apps live and run on a mobile device itself, whereas web apps are accessed through a web browser and will adapt to whichever device you’re viewing them on. Native mobile apps are built for a specific platform, such as iOS for Apple or Android for, well, pretty much everything else. They are downloaded and installed through an app store and have access to system resources, such as GPS and the camera function. Web apps, however, are not native to a particular system, and do not need to be downloaded or installed. Due to their responsive nature, they may look and function a lot like mobile apps, which is where some confusion arises.

To dig a bit deeper, mobile apps that are created for a targeted platform using the platform-specific SDK are categorized into Native Mobile Apps. Whereas Hybrid Mobile Apps are developed on platforms that provide code that is compatible with all available operating systems. Lastly, you may have heard of Progressive Web Apps (PWA’s), which many people claim is the future. With PWA’s, “The focus is towards creating web applications that have a look and feel exactly like native apps, without users having to download and install any software.”

Mobile App Performance Challenges

While many companies may have incredible technology otherwise, their mobile apps often struggle with performance, latency, and/or connectivity issues. This can be due to numerous factors. Perhaps the organization did not put a lot of time or resources into building their mobile apps from the ground up, and therefore they do not stand on a solid foundation. Or maybe they are battling with latency issues due to centralized databases and cloud/vendor lock-in. Speaking of the database - is it equipped to handle the right amount of users and frequent updates? Does it store data offline or handle complex queries? These are all things to consider when building a new mobile app from scratch, or adding new features to an existing app, etc. Bear in mind that it’s not impossible (and actually with some technologies it’s quite easy) to migrate from one database to another if needed.

Database Considerations

In my database architectures and use cases blog, I mentioned that when selecting a database, it’s important to consider your data type / structure, data volume, consistency, write & read frequency, hosting, cost, security, and integration constraints. While these certainly stand true, there are a few additional things you may want to consider when it comes to mobile apps:

Support for multiple mobile app platforms
Scalability
Data synchronization
Multi layer data model considerations
Network connectivity
Pushing new app updates and database changes
Resolving data conflicts between devices

Clearly there is a lot to consider here, and it can be quite overwhelming to figure out what type of database technology might be the best fit. For starters, some databases fit nicely into one category when it comes to data structure / type of functionality. Other technologies take more of a hybrid approach, enabling cross-functionality or combining the functionality of different tools into one. So, unless you have a very specific project or limited long term goals, it might be best to go with a more flexible hybrid technology that encompasses more functionality into one package to reduce the number of systems needed.

So, Which Database Should You Use?

Here I’ll try to provide a fair (while potentially still a little biased) explanation as to why HarperDB is the best option for improving the performance of your mobile app. From a high level, HarperDB is a modern hybrid database that combines the functionality of some of the best tools on the market into one, so it really covers a lot of bases. As a distributed database, it can be installed anywhere while presenting a single interface across a range of clouds, with backend ability to keep data synchronized everywhere. HarperDB is read and write optimized, handling upwards of 100K requests per second per node.

Of course there are other great options out there as well. For example, this article published several years ago lists different database options based on functionality needed. HarperDB basically works everywhere in that table, and provides all of the functionality mentioned.

Instead of trying to compare 300 different database options on the market, it might simplify your decision to do comparisons of this vs. that. This will enable you to take a deeper look at your needs and compare things like performance and cost. For example, in benchmark tests, HarperDB is 37 times faster than MongoDB and tends to be more cost effective. It also supports SQL on JSON. Cockroach is a better fit for fintech use cases, whereas HarperDB may be a better fit for industries like gaming, ticketing, military, and media. You can’t lock out HarperDB’s database globally, and our clustering methodology relies on eventual consistency which allows for more efficient replication than more structured options. Relational databases like MySQL can be significantly more resource intensive, requiring more maintenance and strict data structure. Whereas HarperDB can run on machines of all vertical scale from a Raspberry Pi to a supercomputer, requiring little to no maintenance, and with a dynamic schema allowing for easy data ingestion. This HarperDB vs. MongoDB vs. PostgreSQL comparison may be helpful as well.

Are you facing challenges around app latency? Or worried about being truly geo-distributed? With HarperDB, you can simply spin up more nodes to scale horizontally, putting HarperDB in various regions closer to your end users, which will reduce latency and improve app performance, all while accessing data in real time. By distributing APIs and data storage, and shifting application logic to the edge, you can remove bottlenecks and reduce infrastructure and cost. Globally replicating data at the speed of the Internet, HarperDB reduces app latency, improves performance and accessibility, and reduces overall complexity when it comes to data management. Lastly, HarperDB is about to release Custom Functions, which will enable developers to code their own custom API endpoints, utilizing core HarperDB methods, which results in a simplified tech stack and improved performance.

Mobile apps are constantly evolving, and you need a flexible database that can adjust on the fly without affecting performance. Even the most advanced and impressive technologies in the world will be brought down by a poor foundation or outdated database. Staying up to speed (literally) with modern technology is the best way to remain afloat in this highly competitive market. Therefore, the selection when it comes to your tech stack should not be taken lightly.

DEV Community: Margo McCabe

Check out what Harper is up to

I Built an AI Ops Assistant That Replaced $2,400/month in SaaS — In 3 Days, for Under $100/month

8 Ways to Improve Application Performance

Intro

What causes application latency or performance issues?

Here are 8 ways to reduce application latency:

HarperDB: Staking a Claim as the Only Globally Distributed Edge Data Platform

More Reason to Celebrate

As we wrap up 2022 and gain momentum for 2023, I sat down with HarperDB’s CEO & Co-Founder, Stephen Goldberg, for a brief Q&A:

What are HarperDB’s current strategic initiatives?

Why is HarperDB focused on Edge Computing and how is the edge helping organizations that we work with?

What projects are you most excited about right now?

What are some of the adoption challenges that enterprises face with Edge Computing?

Is there space for AI and Machine Learning in Edge Computing? How does it all tie together?

How is HarperDB partnering with other organizations to make Edge Computing real?

Predictions related to Edge, Cloud, and IoT moving forward in the next 5-10 years?

20+ Best HarperDB Tutorials

1. How to use HarperDB Custom Functions to Build your Entire Backend, by Andrew Baisden

2. Using TensorFlowJS & HarperDB for Machine Learning, by Kevin Ashcraft

3. Build a Full Stack Productivity Timer App with NextJS and HarperDB, By Danny Adams

4. Building a C# Blazor App with NoSQL and SQL Using HarperDB, by @towernter

5. Analyze Twitter’s Reaction to Taylor Swift with HarperDB, by Aakriti Sharma

6. How I used HarperDB Custom Functions and Recharts to create Dashboard, by @atapas

7. Migrating a legacy application from MongoDB to HarperDB, by Lucas Santos

8. Building Microservices using Spring Boot + HarperDB and Deploying it on AWS, by Apoorv Tyagi

9. VueJS App with HarperDB, by Milecia

10. How To Use HarperDB Custom Functions With Your React App, by @tyaga001

11. CRUD REST API using Node.js, Express, HarperDB, Docker, by Francesco Ciulla

12. Libshare - Curate & Share Your favorite JavaScript Libraries, by @adisreyaj

13. Create a serverless book library app with HarperDB and Gatsbyjs, by Tapas Adhikary

14. Build A REST API With HarperDB and FastifyJS, by Catalin Pit

15. Build a Python App with HarperDB, by @bhavaniravi

16. Building a collection of Tech Blogs using ReactJS and HarperDB, by Bonnie

17. Creating your First CRUD Restful API with Node.JS and Hoppscotch, by @unclebigbay

18. HarperDB and WebSockets, By Ethan Arrowood

19. Query data from HarperDB with GraphQL using StepZen, by Roy Derks

20. Build a Hacker News Clone using React and HarperDB, by @ubahthebuilder

21. INTRODUCING: The Anonymous Feedback Platform powered by the CHERN Stack, Unclebigbay

22. Build a Note Keeping API with NodeJS, Express, and HarperDB, by Lucky Victory

23. Building a Contact Manager with Python, Flask, and HarperDB, by Lordghostx

24. Building An Inventory Tool Using Appsmith and HarperDB, by Dancia Jos

25. Build a Dynamic REST API with Custom Functions, by Terra Roush

Bonus section:

🎙️Select* Podcast: For Devs & Tech Leaders

S02 E1: TensorFlow.js, Building Community, & Coding for the Greater Good w/ Bekah HW

S02 E02: Life as a Front End Engineer at an Early Stage Startup w/ Annie Liew

S02 E03: What Web3 Means for Developers w/ Pratham Prasoon

S02 E04: Why I'm Excited About Machine Learning & Python w/ Patrick Loeber

S02 E05 - From Classical Musician to Software Developer w/ Jessica Wilkins

S02 E06: Decentralized Computing w/ a VP Systems Engineer, José Domingos

Virtual Coffee w/ HarperDB's EVP of Engineering ☕️🧑‍💻

Migrate an App from MongoDB to HarperDB Livestream

Don't Miss This Opportunity to Code with Lucas🚀

Watch the recording or check out the written tutorial to code along.

Why migrate to HarperDB?

Meet Lucas Santos! Software Engineer & Google Dev Expert

Web3 & Decentralization: What it Means for Data Storage

The Problem With Web2

Web3 Data Storage

HarperDB: a Web3 Database?

True Edge Persistence

Looking Ahead

HarperDB in the Real World

Quotes from HarperDB Users

Case Studies

US Army: Accelerating Tactical Edge with HarperDB

High Speed Data Monitoring with Casne Engineering

Developer Examples / Featured Projects

Build an App Using HarperDB's New Custom Functions 🚀

Don't miss this opportunity to see HarperDB 3.1 in action!

Watch livestream here

More on HarperDB Custom Functions:

Meet Jaxon Repp, VP of Product at HarperDB

HarperDB: More Than a Database

Database, Explained

HarperDB: A Runway for Launching Industry 4.0 Technology

Extended Functionality

Why HarperDB?

Predictions for the Future