DEV Community: Reshuffle

ReshuffleDB - the DB that's always there for you

Amir Shevat — Thu, 14 Nov 2019 21:29:32 +0000

One of the most fundamental aspects of fullstack development is working with databases. Even though no two web applications are the same, the need for remote data is nearly ubiquitous. You might assume that due to the massive advancements in cloud computing over the last 10 years provisioning a DB in 2019 would be fairly painless. But it's not.

At Reshuffle we understand that nearly all real world applications need data. This is why any application deployed to the Reshuffle platform automagically has a database provisioned alongside it. That way, the database you always end up needing is already there when you need it.

Building an app on Reshuffle is 100% free and comes with it's own DB. Here is our shiny new documentation, please let us know what you think!

Reshuffle DB docs
Reshuffle homepage
Our Discord

The Magic of the Short Dev Cycle

Avner Braverman — Fri, 25 Oct 2019 19:19:29 +0000

From kernel drivers to modern web apps: The faster you build, the more you deliver.

As developers, we know that shorter dev cycles lead to better productivity and align us better with business goals. Back in 2010, I was working on a Linux kernel driver and experienced first hand the magical impact of a shorter dev cycle. I'd like to share this story here, and show how you can enjoy the same magic in your web dev workflow.
Almost ten years ago I was building an enterprise storage product at IBM. The very day I joined, I inherited the kernel driver for the box's fibre channel card. That high-speed connection was the primary read and write interface to our storage device.
That driver originated with the card manufacturer itself. It later went through an open source phase and then through a couple of iterations with developers inside our group. When it found its way to me, the driver consisted of 37,000 lines of code! And it had a bug.
It was one of those nasty bugs. The kind that just happens every once in a while and nobody knows why. To make things worse, that particular driver bug generated a kernel panic (the Linux equivalent of the Blue Screen of Death) and caused one of the internal components to restart. This was a customer visible event with the side effect of slowing the entire storage device down for a few hours. Hardly a desired situation.
Several of our best kernel experts looked through the driver code, and couldn’t find anything wrong with it. The code seemed fine. The data seemed fine. The card seemed fine as well. No one could put their finger on any underlying issue, yet the bug kept rearing its ugly head. Something had to be done.

Getting the driver out of the kernel

One day we realized that we really didn't care so much about the bug. We did, however, care a lot about its side effects. A kernel panic is a major incident, whereas a driver failure is a minor event we could easily live with.
We came up with a plan to convert this kernel driver into a regular user application. Application failures cannot cause kernel panics and so the side effects of the bug would be eliminated.
Over the course of two weeks, we built a micro version of the driver to control the fibre channel card from inside the kernel. We then moved the bulk of the driver's code into a regular application, where it ran on top of a thin shim we built to emulate the kernel environment.
The results were incredible. Immediately, our storage system became much more stable, with zero impact on performance. The nasty bug was still there, but instead of crashing an entire machine, it would now crash a single application. No more panics. No more restarts.
Surprisingly, the biggest benefit was something we never anticipated. Building a kernel driver takes time. The development cycle for kernel software on our system was 3-4 minutes. Every change made to the code took 3-4 minutes to test. That's a very long time, and you tend to wander off to check your email, browse the web or make a nice hot cup of tea while you wait.
With the driver now running as a regular application, it took mere 3-4 *seconds* to test code changes. This was life-changing for us. Three seconds is a short enough period and so you don't lose focus. You are waiting synchronously to see how your changes affect your code. You are in the zone.
This unexpected side benefit triggered an amazing coding frenzy. Over the course of these two months, I rewrote every line of code in that driver. I could easily run small and quick experiments, and ended up deleting 30,000 lines of unneeded logic. At the end of those two months, the driver consisted of only 7000 lines of code. I also ended up eliminating that original bug, but at that point (even though we didn't care so much anymore). The experience opened my eyes to the magic of the short development cycle.

Webpack

A few years later, I started to dabble in web development. I have always been fascinated by user experience, and the web was the new front. Building for the web was an immediate delight. You code, save, reload, and voila - changes appear in the browser right in front of your eyes.
Webpack has taken this a step forward. It includes a local development server which continuously watches your source files. It also injects a small piece of JavaScript code into your pages, that continuously monitors the server for updates. Now, every time you save your code, the server detects the change and tells the page to reload. If you use auto-save, you are basically able to see your app changes as you type. Amazing.

This is the best development experience I have ever had. This kind of immediacy helps you find and fix problems faster. It removes all mental clutter and lets you focus on what you're building. It unleashes creativity like nothing I have ever seen before.

So what about the backend?

Unfortunately, only frontend development provides this kind of experience today. Backend code typically takes minutes to update. Frontend developers who depend on independent backend teams may find themselves waiting for days until backend changes come through.
We think this is unacceptable. At Reshuffle, we enable React developers to build full-stack applications, by extending the magic of Webpack to the cloud. The idea is simple: you add backend logic directly to your frontend projects. This backend logic consists of simple JavaScript functions, that your frontend code can call like any other function. Simple, right?
Behind the scenes, Reshuffle seamlessly deploys those functions to the backend runtime, be it a local development server or a fully scalable, production-grade cloud environment. We replace the original functions with stubs that generate HTTP requests to the server, taking care of encoding, serialization and errors in the process.
The result is a fast, seamless coding experience. You can focus on your app instead of boilerplate code, deployment or managing the cloud. You get that short, immediate dev cycle for your frontend and your backend. Your newfound productivity lets you focus on being creative.
Try it out today at reshuffle.com.

A Product Story

Ryland G — Thu, 01 Aug 2019 14:17:24 +0000

Cover Credit

Setup

Seven months ago, the idea for a “FaaS webpack integration” emerged at a team brainstorming session. As with most great ideas, it was initially misunderstood, miscommunicated and therefore, dismissed.

Three months ago, it became apparent that some changes were needed. We had a few great customers with some specific use cases, but in order to continue growing, we needed something more. We still believed strongly in our core product value (scalable and easy to use cloud compute), and therefore weren’t looking to create something entirely new. Instead, we wanted to take the product we loved and shift it, so more developers could love it too.

Over the following month, all energy was dedicated towards understanding our options. We revisited past ideas, came up with some new ones and did as much market research as possible. The outcome: a narrow list of 4-5 ideas for repurposing our existing product to broaden developer appeal. On paper, a few of the possibilities seemed to have real potential, but the excitement wasn’t there. As a business, not choosing a potentially lucrative path due to lack of internal excitement is one of the hardest choices to make, but almost always the right one. Great products are equal parts understanding your users, solid engineering, and passion. It’s possible to have a successful product without passion, but not a truly great one.

Two months ago, a reimagining of the webpack idea popped into my head, and I started to feel the excitement. After spending some serious time researching the front-end ecosystem, it became clear that there was still no adequate solution for full-stack development. There were a myriad of offerings, but none of them seemed to address the crux of the problem.

// Detect dark theme var iframe = document.getElementById('tweet-1123232047441633282-927'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1123232047441633282&theme=dark" }

Product Definition

Using the webpack plugin as a basis, I began imagining a platform that would enable front-end developers to focus only on the things that bring them value. Armed with this idea and the approval of Avner (our CEO), I quickly began work on a demo version of the system with the help of Vladimir (one of our senior engineers).

Two days, a few thousand lines of code and too many cups of coffee later, the demo system was working. As an engineer, I can proudly say it was one of the most hacked together, janky pieces of software that I’d ever been a part of, but it worked. I was also quite nervous, I felt very passionate about the idea and was worried others might not like my interpretation of it. Avner was the first to try the demo, he instantly dispelled any worries I had when he said “Where has this been all my life”.

As the rest of the team heard about the idea, excitement started to grow again. Not wanting to lose momentum, I began pushing for an initial “demo release”. The demo release would only include a video and a blog post, showcasing this new product. It’s always important to validate your assumptions and beliefs, before investing dev resources in a full fledged product effort. The demo proposal was quickly approved. So I began working on a product document, that I could use to align with the rest of the team. During this time, Roey (another senior engineers) had played with the demo system, and started making some feature additions. Within a day he released a highly improved version of the original demo system. Working closely with him, I was able to quickly iterate on an initial proposal that could be shared with the team.

After collecting more feedback from the team, I set to work preparing for the release. It was Monday, and I had agreed to Friday for the release date. There was now a deadline looming. I needed to make the videos, get the demo system and code presentable enough for the video and prepare the material that I planned to release on social platforms. The first thing I handled, was setting up accounts for social sharing, those can take days to become usable (for example reddit). Next priority, was getting the code presentable, as this was a blocker for the video and other content. This process took a few iterations, which is a challenging feat, considering our 10 hour time gap. Everyone pushed extra hard and stayed in constant communication, allowing us to avoid basic misunderstandings. During the periods of downtime, I made the necessary modifications to the demo system so it would be presentable for the video.

The Final Push

It was Wednesday by the time the code had reached an agreeable state. I now had two days to publish the blog post and make the accompanying video. After a 14 hour push on Wednesday, I was able to finish the initial draft of the blog post and video. For our team in Israel, weekend begins on Friday, making “Israeli Thursday” the “American Friday”. This meant, it was the last real opportunity to get feedback from the team. Unfortunately, when you do 14 hours of work without communicating, there’s a high chance of misalignment. So when I woke up Thursday morning, it was to a daunting number of comments, suggestions and fixes on my post and video. It was obvious that my document and video didn’t meet the team’s expectations. So instead of trying to “put a bandaid on a broken leg", I decided to start from scratch, using the feedback I had received as a guide. After another seriously long day, V2 of the blog post and video were complete. At this point I was feeling very stressed. Release was set for the next day, and both components of the release (blog post and video), were entirely unreviewed.

I woke up the next day, fearing the worst. My phone was full of notifications from Google docs. I immediately jumped on the computer, and was incredibly relieved to see that this time I've hit home. While there were considerable comments, they were much less fundamental compared to the previous day. All of the actionable feedback could easily be completed within a few hours. This was still far from optimal, as it’s never a good feeling to be working on a release the day of release. It was also Friday in the states, which is deep into the Israeli weekend. Fortunately, Michael (our CTO) sacrificed his weekend and supported me in every way he possibly could. This allowed me to quickly address the feedback on the post and video, making them release ready.

As I went to publish the post on our blog, disaster struck. The npm package ecstatic which our blog depends on, had been unpublished entirely from npm and removed from github, just 1 hour before. I first attempted to look for an older version, but the author had been incredibly diligent about wiping everything ecstatic related. Luckily, Michael had a cached version of the blog on his computer. Using him as a proxy, I was able to make the final formatting changes and corrections, allowing me to release the blog and video.

Gathering Feedback

It was time for the hard part, getting people interested enough to give feedback. There’s absolutely no easy way to go about this, and usually brute force is the most viable option. I knew that if I limited the marketing effort to a single platform, we would receive nowhere near enough feedback to reach a reliable conclusion about the product. To increase our chances of success, I opted for a breadth approach. This meant posting on countless subreddits, Slack, Discord and Gitter channels, Spectrum chat, Twitter, YCombinator, Dev.to and many many more. Even then, things were difficult. Getting people to give feedback is much harder than getting them to retweet, like or upvote, the conversion rate is incredibly low.

But it seemed my efforts were not in vain. After a few hours, our post on /r/reactjs hit the front page. The comments we received were incredibly insightful, but there were only a handful. This isn’t surprising considering we only had a video and blog post, meaning that potential users had “nothing to try”. In fact, the number one criticism we received across all platforms was, “it’s hard for me to judge it without trying it”.

Once it became clear that the reddit posts were not generating enough feedback, I shifted efforts elsewhere. I knew that the only reliable way to get more feedback was by starting 1:1 discussions on a live chat platform. So I began posting in every Discord, Slack and Gitter channel related to our space, immediately responding to anyone who showed interest. This is an incredibly uncomfortable undertaking for many people (myself included), but resulted in the most (and highest quality) feedback by a large margin. It’s also very time consuming, so the rest of my day was spent strictly DMing potential users.

By the end of the day, I had managed to collect a substantial amount of feedback. My hundreds of conversations allowed me to produce a list of 40-50 tangible criticisms and observations, which is exactly what I set out to collect.

Takeaways

There are definitely some learnings from the experience, I’ve shared those below

Communication is critical. Without daily syncs and continuous alignment checks, the release would have never been possible. Teams that are constantly communicating are usually constantly productive. This is especially important with a remote team spread over different time zones.

Prepare in advance. Because I didn’t fully prepare for the social marketing effort, a few channels weren’t available to us. Remember, each platform has its own requirements and etiquette for posting. You definitely want to get those things underway far before release day.

It’s never too early for feedback. We didn’t have a usable product, demo or even solidified APIs and we were still able to get the validation we needed. The top priority for any new product should always be validation.

Initiate dialogue. It’s always uncomfortable to start a conversation with a stranger. Most times, if you’re willing to initiate, people are more than happy to give their opinions. The majority of the positive interactions I had, stemmed from me cold starting a conversation.

Overall, it was an incredibly valuable and defining experience for us. We gained the confidence needed to feel comfortable moving forward with a new product.

Updated (July 31 2019): Over the last few months we've been hard at work to provide something that will change developers lives. Binaris is now ShiftJS. If you've ever thought that full stack development isn't quite as easy as it should be, ShiftJS is for you. We have a killer team, and with the addition of Amir Shevat (previously VP Product Twitch/Amazon) as CPO, we are now full fledged ninja assassins. Join us!

We are now in private beta. If you’d like to give ShiftJS a try, sign up for the waitlist here.

https://shiftjs.com/

For those interested in seeing the original blog post and video in question.

https://blog.binaris.com/cloudless/