DEV Community: Daniel Kolb

Vote for what you want to learn! YouTube Channel for Shopify Developers!

Daniel Kolb — Mon, 24 Jan 2022 13:20:49 +0000

Hi all!

I’ll start a YouTube channel about code/design/product management shortly. For the first few months, I’ll focus on Coding with Shopify.

Currently I’m in the process of picking the right topics. I will include Tutorials, Live Code streams and “XXX in 100 seconds” videos.

I’ve already prepared a couple of ideas here and would like you to VOTE them and also add more to the list. This can be done anonymously, so please don’t break it.

Here you go: https://daniel-kolb-youtube-ideas.nolt.io/top .

Thanks to everyone who want to participate!

NoSQL Database Design for E-Commerce Apps in 2021

Daniel Kolb — Sat, 09 Jan 2021 19:41:59 +0000

You're building things for the e-commerce industry? Then you should join my newsletter: https://shophackers.news/

Working with a lot of data like products, orders, categories, users and payments is a very important topic when building e-commerce applications. In this post you'll learn the very basics of structuring your noSQL schema so it's fast and scalable for e-commerce scenarios.

Why you should consider using a noSQL database in 2021

NoSQL databases like MongoDB are still pretty popular amongst modern app development projects and evolved a lot in the last time. The community released many decent packages that help you working with noSQL databases on a very scalable level - like schema generators and battle-proofed packages for combining it with JS frontends or graphQL APIs.
Besides that, especially MongoDB did an awesome job with providing a very good cloud solution (MongoDB Atlas: https://www.mongodb.com/cloud/atlas) that perfectly fits in a modern web tech stack.

Are noSQL databases the right choice for an e-commerce application?

In case you've dropped the idea of using noSQL databases in the last years because you've heard of them not being the right choice for any apps that need to manage a lot of complex & transactional data, you should give them a shot again. Things changed quite a lot. Actually, Reaction Commerce wrote a decent post about that more than 3 years ago and the technology improved even more since then: https://blog.reactioncommerce.com/why-nosql-databases-are-perfect-for-ecommerce/.

To summarise: With MongoDB (and any other noSQL database) you can build safe, reliable, scalable, cloud-based databases that are very fun and easy to code with.

What you should consider when you create schemas for e-commerce apps

When you build e-commerce experiences you typically have to cater for:

up to thousands of products
products that have CMS-like content like images and descriptions
products that are available in up to thousands of variants...
variants that are defined by many different attributes
probably thousands of orders...
orders might have multiple products (and their variants)
orders might have different states that might change multiple times a day
orders might be assigned to customers
customers might be users with a login name, password, address, etc.
there's usually also quite a lot of CMS content like blog posts, marketing landing pages, etc.
and much more

As you can see, you should make sure that your database works for a huge amount of data sets that are connected to each other.

Embedding Data: Work with data fast & easy

When you work with MongoDB, you create JSON-based entries (called "documents") in your database and group them in so-called "collections".

For example, the collection "products" contains multiple documents while each document contains the data for a product like this one:

// a document in the "products" collection
{
    "_id" : ObjectId("5e451f4cd249baf1d045a778"),
    "title" : "Hackathon T-Shirt",
    "price" : 3.99,
    "currency" : "USD",
    "description" : "My Awesome Shirt",
    "sku": "DEV1337",
    "createdAt" : ISODate("2021-01-02T10:05:00.610Z"),
    "stock" : 12,
    "sizes": ["xs","s","m","l","xl","xxl"],
    "colors": ["red","green","blue"],
    "vendor" : "DEVSHIRTS",
    "vendorSlug": "devshirts",
    "vendorDescription": "DEVSHIRTS is a fashion label that sells T-Shirts for devs."
}

Query embedded data

In this example I've embedded all data for this product in its document. By embedding the data directly to the object I can get the product with all of its properties with a single query like in this example:

const product = db.products.findOne({"sku":"DEV1337"}).fetch();
const productTitle = product.title; // "Hackathon T-Shirt"
const vendor = product.vendor; // "DEVSHIRTS"
const amountSizes = product.sizes.length // 6

Super clean and easy to write, isn't it? However, you should keep in mind that query for multiple products will return the product with all of its properties which can become up to 16MB.

Modify embedded data

Let's say one vendor has 1000+ products and you decide to change a vendor's description. Now all 1000+ products have to be changed with the same operation to change it "vendorDescription" that needs to be changed.

db.products.updateMany({vendor: "DEVSHIRTS"}, {$set: {vendorDescription: "DEVSHIRT's new description"}});

Such operations are easy to code but might need a lot of processing power when they need to be done multiple times per minute or per second in a huge database. One solution for this is referencing the data instead of embedding it into the object.

Referencing Data: Work with distributed data sets

You probably know this way of working with data from regular SQL-databases like MySQL. Instead of adding all data to the same document, you just add a reference ID to another entry in another collection like I do here for the vendor:

//a document in the "products" collection:
{
    "_id" : ObjectId("5e451f4cd249baf1d045a778"),
    "title" : "Hackathon T-Shirt",
    "price" : 3.99,
    "currency" : "USD",
    "description" : "My Awesome Shirt",
    "sku": "DEV1337",
    "createdAt" : ISODate("2021-01-02T10:05:00.610Z"),
    "stock" : 12,
    "sizes": ["xs","s","m","l","xl","xxl"],
    "colors": ["red","green","blue"],
    "vendor" : "23ae11d117baf1d127c99efd334"
}

//a document in the "vendors" collection:
{
    "_id" : ObjectId("23ae11d117baf1d127c99efd334"),
    "title" : "DEVSHIRTS",
    "slug": "devshirts",
    "description": "DEVSHIRTS is a fashion label that sells T-Shirts for devs."
}

Query referenced data

When you query referenced data, you need to write the first query to get the referenced id and a second query to get the object you're looking for:

// Want to get the vendor description of a product...
const product = db.products.findOne({"sku":"DEV1337"}).fetch();
const vendorId = product.vendor; // "23ae11d117baf1d127c99efd334"

const vendor = db.vendors.findOne({"_id":vendorId}).fetch();
const vendorDescription = vendor.description

There's also a way to do this on a database level with the $lookup functionality: https://kb.objectrocket.com/mongo-db/how-to-use-the-lookup-function-in-mongodb-1277

Modify referenced data

This is a big upside when working with referenced data instead of embedded data: You need to update way fewer documents, probably only one. For example, updating the description of a vendor can be done by a very performant single operation now:

db.products.update({vendor: "DEVSHIRTS"}, {$set: {vendorDescription: "DEVSHIRT's new description"}});

Even if you have thousands of products from this vendor in your database, MongoDB will only need to update one single entry.

When to use embedded vs. referenced data

To decide whether you should embed or reference your data is one of the most important aspects when building database schemas.

Referencing data might often look like a "clean" way but when you start to create as many references as possible the amount of code & queries you'll need to write will increase tremendously - think of my product example above with references to sizes, colors, currencies, etc..
Besides that, your database will get bombed with queries and operations that might be needed when you'd just embed the data.
On the other hand, too much embedded data could mean that you query and work with objects that are way bigger than actually needed, which also slows down your app.

Only reference when needed

From my experience, it's best practice to embed as much data as possible and only reference other documents if it's really needed and makes sense for your specific application.

For example, if you only need to attach a few variants with unique attributes to a product, there's no need of creating a "variants" collection for that. On the other hand, if your variants are basically their own products with a lot of attributes (like title, images, sku, prices, etc.) and even get shared across multiple products, it's a better idea to put those in their own "variants" collection so you can query and modify them independently from the products.

Tip: data models can come in handy

Oftentimes it also helps to select the schema based on the data model. While embedded data is usually fine for "One-To-One" and "One-To-Few" relations, referenced data shines for "One-To-Many" and especially for "One-To-VeryMany" relations.

What's next

I hope I could give you a basic understanding of embedded & referenced data and you have an idea about structuring e-commerce data now.
If you want to learn more, MongoDB has quite nice tutorials, guides, and presentations you can have a look at:
https://docs.mongodb.com/manual/applications/data-models/
https://www.mongodb.com/blog/post/performance-best-practices-mongodb-data-modeling-and-memory-sizing
https://www.mongodb.com/presentations/schema-design-best-practices-series-modelling-relationships

Shopify for Frontend Engineers | #1 Dev Env, ThemeKit, Liquid

Daniel Kolb — Sat, 02 Jan 2021 17:12:22 +0000

Shopify gained a lot of traction in the last months - probably because E-Commerce is in very high demand amidst a pandemic that forces many retail stores to shut down. That's also why Shopify developers are in high demand right now.

I've been working with Shopify for several years now and introduced many developers that are used to other CMS like Wordpress, Magento, Prestashop etc. to Shopify. From my experience, many developers have troubles getting into the concepts of working with a SaaS-CMS, which applies to Shopify and results in pretty unique ways to build the thing you want to create.

For example, Shopify doesn't offer a real, full-fledged possibility to work with a Shopify store's files locally and there's also a lot of black box magic happening on the Shopify servers that you just have to deal with - e.g. automatic generated, captcha-enabled forms that get connected to the Shopify servers by default.

Besides that, many developers have problems grasping very Shopify specific E-Commerce concepts - like the idea of so called "collections" instead of ordinary product categories.

However, in this and my upcoming blog posts I'll focus on...

Setup a development environment using a development store and ThemeKit
Modifying popular Shopify themes
Give you guidance on how you can craft your very own, 100% custom built Shopify frontends
How you can build headless storefronts for Shopify

So let's start with the most important concepts. They're pretty basic but if you don't know about them you'll waste hours or days bumming you head against a wall because a lot of things might work differently than you'd expect.

Tool #1: Partner Dashboard & Development Stores

Since you can't install Shopify locally (like you would with let's say Wordpress), you need to work with an online development instance of Shopify. Yep: it isn't possible to work with Shopify without an active internet connection.

To create a development store visit https://partners.shopify.com/signup and create a Shopify Partners account. It's for free and allows you to create an unlimited amount of development stores to code with. Development stores are free as well. Creating development stores is super straight forward and is probably not a major hurdle for you if you know how to code. That being said, if you face problems creating a development store have a look at this super detailed how-to written by Shopify: https://help.shopify.com/en/partners/dashboard/managing-stores/development-stores

Tool #2: ThemeKit - a remote control for your development store

Since Shopify is 100% SaaS based, you aren't able to create a local Shopify environment on your machine. Instead, you must work with your (live online) development store by changing the files on the files on the Shopify servers. To help you with that, Shopify created a command line tool called "ThemeKit": https://shopify.github.io/themekit/ .

Using homebrew you can install it as easy as

brew tap shopify/shopify
brew install themekit

After you've created a private app in your development store to get the API credentials you need for ThemeKit (as it's described in the "Getting Started" guide here: https://shopify.github.io/themekit/), you just have to type

theme download

to download your theme.

After that you can execute

theme watch

to let ThemeKit monitor your local files and push changed files to the Shopify servers.

To wrap up, what ThemeKit essentially does is:

downloading the theme files from your development store to your local machine
whenever you do a change to a local file, it gets automatically uploaded via SSH to the Shopify servers and connected to your development store

The second point is super, super important and I saw many devs not grasping this concept even after they've worked with Shopify for several weeks. To make it clear: ThemeKit exclusively works as kind of a remote-control for your Shopify store. It does so by uploading your locally changed file to the Shopify servers and replace whatever is on the Shopify servers with your local file.

That means you should really cater for a version control system like git(hub) to keep track of your local changes - Shopify won't help you with that.

Tool #3: Liquid - Shopify's templating language

You probably at least heard of Smarty, Handlebars, JSX and all of that different templating languages. Shopify just provides their own, pretty slick templating language called "Liquid": https://shopify.github.io/liquid/ . Liquid is actually open source and even used by other projects like the static site generator Jekyll.
However, when you're working with Shopify themes you'll build page templates with Liquid, meaning instead of let's say hard-coding a product title for a product page, you use something like

{{product.title}}

to dynamically print the title of the current accessed product, while many products share the same page template file.

Important: Liquid gets compiled to html/css on the Shopify servers when a user wants to access a store page (well, they've caching in place but leave that for now). This means that whatever liquid magic you do in your code, only the Shopify servers know about it and compile your liquid code to pure html/css files that get sent to the client.

On that note: If you write JavaScript for the client, keep in mind that any liquid variables will get compiled before the frontend (including the script) gets sent to the user. Meaning that any liquid variable in your JavaScript will get compiled (most probably) to a string you can use, but you won't be able to assign any new values to liquid variables using JavaScript.

Shopify offers a "cheat sheet" that helps you to find the variables you might be looking for: https://www.shopify.com/partners/shopify-cheat-sheet .

Tool #4: HTML/CSS/JS and an IDE of your choice

Well, nothing new here. Besides Liquid and ThemeKit you can use the tools you probably know and love for your frontend projects: HTML as markup, CSS for styling and JS to make things fancy & interactive. In regards to the IDE: if you use Visual Code watch out for liquid extensions that help you writing Shopify theme code.

But how can I use React/Vue/Angular/?

Unfortunately there aren't very straight-forward ways to use these frameworks for Shopify frontends. The most common way to do so is by using Shopify's Storefront API (https://shopify.dev/docs/storefront-api) and build a custom headless frontend. This comes with some restrictions like not being able to use Shopify's native Customizer (a drag&drop editor for merchants) and therefore is probably not what you're looking for. However, I'll make a tutorial on this too, so I have you covered.

Fun fact: A lot of Shopify themes use jQuery for the javascript part. You might love or hate it, but probably hate it.

Let's start coding!

Okay, at this point you should have understood the concepts of ThemeKit, Liquid and have a development store you're keen to modify. In the next tutorial we'll start doing changes to the current installed theme, create new elements and learn how to code for Shopify's Customizer so merchants are able to modify your elements using a drag&drop editor.

Shopify for Developers — the Good, the Bad and the Ugly

Daniel Kolb — Mon, 24 Aug 2020 11:23:31 +0000

I guess it was in 2015 when I discovered Shopify for the first time. Back then, it had a very sketchy reputation in my area (Europe) for being the most popular shop system for very small merchants and drop shippers because one can get started in minutes and all the complicated stuff like payments, nice looking designs and hosting are included out of the box. Sounds good? Not for everybody.

Why you may have heard bad things about Shopify

In case you don’t know yet: drop shippers are merchants who don’t actually purchase, store and ship products using their own warehouses and employees. Let alone producing the goods. Instead, drop shippers just buy their products whenever an order comes in (usually on Chinese Alibaba or similar) and enter their customer’s address as the shipping address. Merchants usually do this if they don’t have the budget to create & store their own products which is usually a lot more profitable. There’s a very big community and consulting businesses for drop shippers. While most drop shippers don’t actually make much money, there are also some unicorns who can make a decent living from that.

However, using the most popular shop software for that kind of bedroom entrepreneurs might feel exciting when you’re just starting out because it sounds like it’s really foolproof.
But in contrast, imagine you’re the E-Commerce Manager of a multi million dollar company. You have to decide whether you should tell your boss that your company should run on a service mainly used by those newish entrepreneurs or should use a well known and established brand like Magento instead. In this scenario, Shopify will actually appear rather amateurish and like a bad decision to you.

So it came that the idle rumor of Shopify not being sophisticated enough for any businesses besides bedroom entrepreneurs quickly spread and established.

Today, Shopify proved that it can handle enterprise sized business processes. In many cases even better than most other shop systems on the market thanks to its modern tech stack and user-centric design decisions. It’s becoming more and more popular all over the globe and is the engine that runs more than 1 million shops today. Many big companies joined the plattform — like gymshark, TESLA, Hasbro, Heinz and more. Shopify even introduced Shopify Plus, which is a service exclusively for high volume merchants, but runs the same core like the regular Shopify.

What makes Shopify different?

The biggest difference of Shopify compared to other solutions like Magento or Prestashop is that Shopify is a SaaS — a Software as a Service. Shopify themselves talk about Shopify not as a software, but rather a “platform”. If you register on Shopify’s website as a merchant, you’ll get only a subdomain (which you can connect to your own domain later on) and therefore a spot on their server farms that run all Shopify shops. You don’t need to rent and maintain your own server infrastructure — this is all completely handled by Shopify. Good thing is: As long as Google Cloud is online, Shopify stores are online. Anice fact for traffic spikes on Black Friday.

“Well, SaaS means that I won’t be able to modify the system” you might think and indeed, changing Shopify’s core isn’t possible at all. However, changing the code of a system is usually considered very bad practice and is usually not needed. That’s why you normally build plugins for Wordpress/Magento/Prestashop rather than changing the code of the system’s core.
Same story in regards to Shopify: You can build plugins instead of changing the core. These plugins are called “apps” in the Shopify ecosystem and integrate with Shopify solely by using their APIs.

Shopify’s APIs

When you’re in the platform business, your APIs are your bread and butter topics since they’re the only way to enable others to add features to your platform. Sounds like a reasoner, right? That’s why Shopify really cares much about their APIs. They were one of the first E-Commerce companies that made GraphQL a first class citizen in their ecosystem while still maintaining their old REST APIs. Besides that, they have an extensive documentation, many tutorials, GraphQL learning kits, blog posts and even CLI tools and npm packages to support working with their APIs. They have a big website for developers that you can find here.

Using Shopify’s APIs you can do CRUD operations to products, collections/categories, customers, pages, frontends, orders and even communicate with their own apps like Ping (Chatbot), Kit (Marketing Automation) or the Shopify POS App (selling with Shopify in brick and mortar stores). Sounds pretty sophisticated, right? Yep, the possibilities are exciting and Shopify usually adds new API features on a monthly or even weekly basis.

Shopify’s App Tech Stack is for everyone!

Sometimes it feels a bit complicated to use APIs instead of calling system methods like you would usually do with your plugin in Wordpress or Magento, because you have to take care of API limits and payload structure and so on. However, good thing is: You don’t need to adopt Shopify’s system tech stack — I have no idea what it is but I read that they coded a lot in Ruby. And even better: You don’t need to use PHP — ever. Well, of course you could, but you don’t want to, do you?

Instead, you can use whatever you like: You’re a Python, Java or Ruby dev and love to use Angular and Vue on the frontend?
Build a restaurant / search / review / whatever app for Shopify!

You like Node+React? No problemo, amigo!
Shopify loves you and provides a ton of npm packages, CLI tools and even a whole component library.
Downside: You have to take care of hosting your app and also do the whole authentication process yourself (passing hmac back and forth and more funny things). However, there’s Heroku and there’s an auth-packages from Shopify, so not really an issue that will hold you back from getting started.

Shiny Shopify: How to create and modify Themes

Up until here, I just spoke about creating apps which is the path you’ll choose when you need to do massive changes to Shopify, do backend-related work or want to earn decent money by selling your app in the Shopify App Store .
If you just want to do frontend modifications by creating a new theme or change the current theme of a Shopify store, you don’t need to use Shopify’s APIs at all.
Instead, you can connect your IDE via theme kit and edit the frontend code right away in VS Code/Atom/Brackets/Sublime Text/Notepad++. Alternatively, Shopify even provides their own code editor which is embedded in the Shopify Admin. Shopify uses the template language Liquid which will look and feel very familiar to you if you ever worked with Smarty, JSX or handlebars before. It allows you to assign variables and do basic stuff like looping over products and getting images in different sizes.

How to get started: Become a Shopify partner

As of today, Shopify offers no certification process, so all you need to do is to register here to create a free partner account. With your partner account, you can create development stores which allow you to build things in a Shopify shop for free — forever. Besides that, Shopify provides you a partner academy which is nice to understand most of the basic concepts. They won’t show you many details and you’ll get stuck for sure — that’s why I’m here to help you.

I’m going to release several tutorials, best-practice posts and also business related content about Shopify in the upcoming weeks here on Dev.to. Thanks! Cheers!