DEV Community: Sendcloud

Our meetup on releasing frequently without disrupting users, and writing better git commits

N.J. van Alphen — Thu, 31 Mar 2022 12:43:56 +0000

March 9 another Sendcloud Engineering meetup took place. What follows is a summary of the virtual, lunchtime session with Nicole van Alphen, Product Lead at Sendcloud, and Tom de Bruijn, Software Developer at AppSignal, community organizer, and open source maintainer.

Shipping changes to business critical applications

The different teams at Sendcloud iterate very quickly, and on some days release up to 15 times an hour. But, where does releasing to the user fit in? What's an appropriate time to release to the user?

Customer feedback following a lot of small releases with changes to the frontend was one of frustration, confusion, some customers mentioned their workflow got disrupted. Users lack the time and focus to adopt new features instantly. While you're shipping new features, customers are trying to get their shipments fulfilled.

Collecting feedback during the development cycle is challenging. And, related, releasing new features with confidence can be a struggle.

Nicole: "We are not the user of the software we build. Keeping the user in mind is paramount. For instance, a picker in the warehouse can be a seasonal worker who doesn't have the time to be onboarded onto your platform. It needs to work for them right away."

Sendcloud releases software to different types of users in the following phases:

Data management
Internal users
BETA community
- Our ~500 Beta users (or: power users) have been instrumental to help figure out whether we are working on the right things.
All users
New users
Default
Sunset

Sendcloud found low risk ways to release new features with 'try out' and 'rollback' banners. Nicole: "And when customers decide to switch back, we ask them to give feedback, but it's entirely optional."

With her colleague Caroline Overgoor (UX Manager at Sendcloud), Nicole wrote the blog post “Releasing the release strategy for SaaS software development teams” on this topic.

Automating opinions about Git

We can find a wide range of not-useful commits in the wild:

"WIP"
"Fix bug" (which bug?)
"closes #123" (involves extra step to find the issue tracker)

... the above messages don't tell future you, or your colleagues an awful lot. Instead, commits should tell a story. Commits should have a subject and a message. Commits should be small, should explain what problem they're solving, why this is the right solution (and maybe what alternatives were considered). Bonus points for examples included, and links to resources.

Why? Commits are documentation. They're not THE documentation, but they are documentation and they are read during review, and later, during debugging.

Tom wrote down all his opinions about what makes a good commit, and gave a talk about it at a conference. The feedback was positive, everybody agreed they wanted to write better commits, but then no-one actually did, or does.

It seemed like the only path forward was to enforce good commits. When Tom couldn't find a tool to do the job for him he decided to write a linter himself, using Rust.

The formatting rules the linter he developed checks for:

Actual changes (yes, you can push empty commits)
Helpful branch names
The maximum width of the commit's subject is 50 characters or less
A commit message being there, and is max 72 characters
Punctuation like an exclamation mark in the subject line (Tom is not a fan)
Using the imperative mood for the subject
Ticket numbers in subjects (those should go in the message instead)
fixup or merge commits, to then suggest rebasing before pushing or merging

Lintje (which means "little ribbon" in Dutch, "lint" in Dutch is a ribbon) is open source and able to be run in your CI, and Tom is accepting pull requests.

Watch the recording for the full length talks, and Q&A.

https://youtube.com/watch?v=Td3AOrEytxM

Our next meetup is planned for April 14. RSVP on meetup.com.

The process of decoupling our front-end

Ana Madalina Filote — Tue, 01 Mar 2022 13:20:51 +0000

As a company grows, it starts facing a lot of new challenges, one of them being scalability: the codebase becomes harder to maintain and new features are more difficult to integrate without introducing technical debt. The natural outcome of this is finding ways to decouple parts of the platform and make them work in a more independent way.

Over three years ago, we started migrating our codebase from AngularJS to Vue.js but we couldn’t fully remove all the dependencies of Angular, given the way the architecture was made. We further decided to split our frontend and backend services into two separate repositories.

Separation of concerns

The first step in this process was to have a release plan to keep track of all the steps needed when migrating sections of the frontend (e.g. open a merge request in the new repository with the migrated module, open a merge request in the old repository to release the module under a beta flag, etc) and create consistency in the work done by the developers. We also had a migration spreadsheet to keep track of all the migrated sections.

To make sure that we weren’t introducing any bugs and that the components were working correctly, a migration checklist was created as guidelines. A few examples of what we needed to take into account when migrating was to fix translations (in the old repository we were using Django templates, while in the new one we were using vue-i18n), fix routing, fix imports and many other dependencies that we needed to keep in mind.

What were the benefits

For the users - there was a clear drop in loading times as we fully removed the support for Angular.js.
For the developers - as a result of having two separate repositories, there is no dependency between frontend and backend when running the pipelines, which leads to faster deployments. We also no longer need to support Angular.js related back-porting.
For the company - it allows faster times to deploy features and bug fixes.

What we learned

Communication is very important. The fact that we informed the Product Owner and the Stakeholders about the importance and the benefits of the migration contributed a lot to making this process move faster.

We also made sure to inform the stakeholders about each migration release that we had, asking them to keep an eye on any bugs that might come up, and also keeping them in the loop with the progress.
One of the things that we could have done better is to divide some of the migrated modules into smaller ones. For example, the last migrated module that we had was also the biggest and most important one (it included the main view that was used by the majority of the users). Because we decided not to break it into smaller pieces (as it was quite difficult to do that), it took a lot more effort and testing.

Planning your own migration?

To summarise what we’ve done during the split:

Have a document with the release plan to keep track of all the steps needed for the migration
Have a document with the migration checklist
Have an Excel sheet for keeping track of the migrated components and discuss about the progress and blockers at every chapter meeting
Inform the Product Owner and the Stakeholders about the necessity and the impact of the migrations so that you can include the tasks related to the migrations in the sprint
Have a lot of patience :)

Building for the (next) Web

Arjen de Ruiter — Mon, 31 Jan 2022 07:05:52 +0000

January 13 the second Sendcloud Engineering meetup took place. We noticed we have a lot of knowledge and experience internally that we would like to share with a broader audience. At the same time we want to learn from others in our industry. As such, we started this meetup series.

A summary of last week's meetup focused on (the workings of) the web.

In The next web, Hidde de Vries (@hdv on Twitter), a front-end web developer and accessibility specialist, and ex W3C/WAI, ex Mozilla, talked about how the web has developed and how it can evolve further still.

Hidde takes us back to the year 1989, where a proposal by Tim, Berners-Lee at CERN is deemed "vague, but exciting...". It of course is the foundation of the World Wide Web. In 1990 the pillars that underpin the Web, HTML, URI (URL), and HTTP were formulated. It was decided that the Web should be free (of fees and permissions). Tim Berners-Lee later said that had the internet been proprietary, it would have never taken off like it did.

In 1994 the W3C was founded with the aim to "realize the full potential of the Web". But have we realized its full potential? W3C is the authority on the subject of Web Standards, so that tools that use the Web work in the same or a similar way, that they are interoperable.

Web 2.0 was coined in 1999 by a UX consultant and was different from the "screenfuls of text and graphics", but "the ether of interaction, where everything is connected". Essentially Web 2.0 was the same tech (HTTP), used in more places, and in more exciting ways. Further popularized in 2004 as "Web as a platform". The new buzz words became user generated content, software as a service, folksonomy, open APIs, widgets... What did Tim Berners-Lee think of this "new" Web? "Web 2.0 is, of course, a piece of jargon, nobody even knows what it means."

The concepts evangelists believe make up Web3 - a term coined in 2014 by the inventor of a blockchain technology - are decentralized, transparent, blockchain, immutable, creators, and scarcity. But, is Web3 really decentralized? A lot of "Web3" APIs, wallets, exchanges are central entities and/or run on central services like Google or AWS. Is it transparent to folks who lack deep technical knowledge? Is it immutable when exchanges often are able to freeze assets or do "hard resets" when they are hacked?

Is it really going to protect creators when anyone can create NFTs, even of what's not theirs? Royalties are not defined in ERC-721. And when you buy an NFT, the unique hash is the only thing that's scarce...

But the question Hidde really urges us to ask ourselves: is Web3 hyped by anyone who does not have a vested interest? Can Web3 benefit society?
Some of us just want to access information securely, accessibly and with minimal privacy impact.

Hidde sees plenty more exciting futures for the web. One that has better styles, easier payments (like W3C's proposed recommendation), easy and safe authentication. A web with more art, more personal sites. In any case, the next Web is not set in stone. There are many challenges and plans beyond those posed by the Web3 crowd.

In UX and DX in 2022, Rhian van Esch, Senior Frontend developer & architect at Sendcloud, explores how to achieve the best user experience without sacrificing developer experience.

What then makes good user experience, and what factors can be affected by the development tools we chose? Google uses Web Vitals as "quality signals that are essential to delivering a great user experience on the web". Rhian focuses on the metrics Largest Contentful Paint, which measures loading performance of a web page, and First Input Delay, which measures how quickly your page becomes interactive.

If your page loads a large JavaScript bundle, the browser needs to download it, parse it, and execute it. If you've built a single-page application, it's common that you need the bundle - or possibly multiple bundles - to be downloaded, parsed and executed before anything appears on the page. This impacts your Largest Contentful Paint metric.

For the First Input Delay metric, JavaScript again plays a role: if your JavaScript takes a long time to execute, you can end up locking up the main thread of the browser while waiting for execution to complete. Users can click on things all they like, but nothing will happen.

A good frontend developer experience is being able to build a UI that is highly interactive, that data is loaded without having to navigate away from the current page, and any navigation should be extremely fast. If you think that sounds like a mobile application... well, you're not wrong.

What else do we want? Component-driven development and state management. Enter the JavaScript frameworks. React, Vue.js, Angular, will allow you to build app-like experiences. Although popular, well-supported frameworks allow us to work faster and more effectively, they come at the before-mentioned cost.

If you're hell-bent on using JavaScript, modern frameworks encourage code-splitting, where you don't send the user all the JavaScript for the entire application, but maybe only the initial view. We can also ask the browser to cache these files aggressively.

Server-side rendering (SSR) renders the entire page on the server initially (rather than rendering it on the client using JavaScript) and then uses a technique called "hydration" to add back the interactivity using JavaScript. This leads to a very fast initial render that isn't blocked by JavaScript. The downside is that, all things considered, you are sending the user everything twice.

Of course, not everything needs to be a SPA. Ask yourself if you genuinely expect deep engagement. You'll still want to build a fast site - because who wants a slow website? - and use components. What are your options?

Rhian suggests checking out Astro. By default, everything compiles to plain HTML with no interactivity. Then you have to specify which components are interactive. You can have them become interactive at runtime, when the browser viewport is a certain size, or first when they are scrolled into view.

Qwik is another tool that aims to solve the performance problem. Like Astro, Qwik hydrates the components that need it. It can even hydrate these components out of order, and asynchronously - it gives very fine-grained control over this hydration.

Partial hydration has been around for longer than you might think. Marko is a tool built by eBay which had the concept of partially hydrating the page as early as 2014.

Keeping in mind that performance is a vital part of user experience, Rhian encourages everyone to submit their developer tooling to due diligence.

Watch the recording for the full length talks, and Q&A.

Our next meetup is planned for March 9th. RSVP on meetup.com.

APIs first at the first Sendcloud Engineering Meetup

Evgeny Sharypin — Thu, 23 Dec 2021 10:59:11 +0000

On December 16th, the first Sendcloud meetup took place. We noticed we have a lot of knowledge and experience internally, that we would like to share with a broader audience. At the same time we want to learn from others in our industry. As such, we started this meetup series, and in this blog I'll try and summarize what happened.

Rather just watch a video?

In API First: increasing quality while reducing costs, Alejandro Lopez Ariza, QA Automation Engineer and Postman Supernova, states that APIs are the nexus of modern software projects.

As an industry we've been shifting steps left in the software development cycle that originally were treated as an afterthought. We tackle things like testing early, and continuously, because we know that the costs associated with fixing a bug that's already in production are exponentially higher than finding it during the design phase.

The traditional way of implementing new features posed several problems. Functional requirements influenced by UI constraints, or a lacking multi-device strategy, just to name a few.

When we instead treat our APIs as first class citizens, and prioritize their development, we:

Remove dependencies between teams
Increase productivity
Reduce costs
Increase quality
Provide a better developer experience
Improve the maintainability of our APIs

Alejandro recommends using tools like Swagger or Postman to automatically create documentation (but to treat it as a starting point), and to use API mocking to unblock development and reduce dependency on third parties.

In Carrier SDK: Sendcloud's Story of Switching to Services, Evgeny Sharypin, Software Development Manager at Sendcloud, shares how the company transitioned from a monolith to a services system.

In early 2020, Sendcloud expected a huge growth in our user base. We knew that our architecture wouldn't stand the challenge. The solution? The Carrier SDK: a collection of APIs and packages. Today there are more than 25 carrier integrations built using this SDK, and by mid 2022, we plan to move all of our existing integrations to be operating within this distributed fleet of services. By 2025 we expect the amount of integrations to be in the hundreds.

The Carrier Blueprint is a reusable Python package that serves as a base for all our carrier services and enables us and our integrators to develop new integrations rapidly, while keeping the services’ codebase lean, flexible and easy to maintain.

The 'vertical siloing' of our services (carriers) was one of the crucial prerequisites for this project to succeed, as it allowed us to design an HTTP API that would replace the existing Python API, without redefining or refactoring the interface much.

The second aspect of carrier integration that played in our favour was that by keeping the state inside our monolith, our services are almost completely stateless. Stateless services don’t compete for shared resources and are easy to scale.

A selection of Evgeny's great parting advice:

Define milestones upfront, but avoid analysis paralysis
Make a disposable prototype first, and actually dispose of it afterwards
When building a service ecosystem for integrators, provide a template that allows them to focus on the business logic
Use known paradigms, such as 12-factor app
Think about observability early on

Watch the recording for more advice, and Q&A.

Our next meetup is planned for January 13. RSVP on meetup.com.
Collapse

The big break-up: an independent order fulfillment service

João Pedro Battistella Nadas — Tue, 14 Dec 2021 16:01:20 +0000

Together, João Nadas, Raymond Penners, Kostas Petrakis, Joris van Rooij, Marlin Cremers, and Yunier Rojas make up the Infrastructure team. Although “infrastructure” is a bit of a misnomer, since the team isn't really focused on what is commonly thought of as infra. They’re all Backend Developers, João is the Product Owner. Raymond has been with Sendcloud since the development team was just 5 people, back in 2016.

This bunch took on the momentous task of extracting a large chunk from the monolith - essentially splitting it in half - to live on its own. No easy feat, as they soon discovered.

João summarizes: the Sendcloud platform is essentially everything we have, including background tasks neatly, and tightly, coupled in. And for a small team, moving fast, a monolith works. However, with 70 developers working across different teams at Sendcloud today, you hit a breaking point.

The scope was clear: keep a bunch of functionality in the monolith, but take the orders fulfillment service out.

The scope

In Python you can do imports from all directions, which may cause cyclic dependencies. When connecting everything is so native to a language, it’s extra painful to break it up. And we wanted a true split, not just putting code in 2 repositories. We wanted to really decouple services. You can only move fast when you’re bigger, when you don't have too many inter-dependencies.

Why split out the orders fulfillment service, you ask? It’s a natural split, coherent. Coincidentally it makes up about half of the service Sendcloud provides. The infrastructure team was brought to life for the purpose of creating the infrastructure for the two services to talk to each other.

How did we get here?

Less than 1.5 years ago we had 3 teams, today we’re at 9 product teams and 2 platform teams. Merging code needs queues and communication. Moving towards a microservices architecture improves velocity. It allows teams to make optimizations without depending on libraries.

The database for the orders is under heavy operation at any given time. Having that processing at the same time the rest of the platform needs to interact with the database, you start hitting limits.

To further stress how decoupling was a necessity: our test suites took very long to run. A deploy to a branch easily would take 30/45 minutes. Before changes show up in production we were at 1.5 hours, which simply is unacceptable for a hotfix.

But the ordeal was something like a car (Sendcloud), which is being transformed while it’s driving, with feature teams adding functionality. And then the Infrastructure team swoopes in, splits it in half, making it functionally 2 cars, without ever stopping the car. Continuing the metaphor, other services were built in the meantime, causing whole other cars to join the fleet, orbiting the original car closely, again sharing resources.

The groundwork

In late 2018 we already split up the database, but we stuck with the monolith. It wasn’t the right time to split up the codebase. Late 2019 some initial work was done, but more as a side-activity, inspired by how our pipelines started to hurt.

We created an orders directory, put all related code in there and out of the monolith, and tried to make it work again. In Python it's quite easy to make a big bowl of spaghetti code work. To make sure we weren’t taking shortcuts we tracked these cyclic imports (“forbidden imports”) and corrected those one by one by putting in place APIs. Whenever a developer created a new feature that introduced new forbidden imports, the pipeline failed on purpose.

We had to build some missing infra bits, prerequisites, and non-functionals. Stuff that isn’t vital to the business and easily gets pushed down on the backlog. Which is exactly what had happened before. From a management perspective, the big split did not get the priority it should have had. We worked on this while also shipping other code and projects. For example: at this time the DevOps team changed from Terraform to Kubernetes, we needed to decide what we wanted to go with. We went with Kubernetes, but even though everything was already dockerized, it still added work to our backlog.

Crunch time!

We set ourselves an artificial target date to push the project, including 5 working days to get it deployed: the 5th of November. Sendcloud handles hundreds of thousands of orders every day, we can’t afford downtime. That’s why we need to deploy services live. We do so today by serving traffic to both EC2 and Kubernetes, split for gradual rollout.

Like the rest of the world, we were all working from home, but because this was all hands on deck we came together in the office for these few days. Even the CEO checked in on us and encouraged us to not play so safe! To motivate ourselves further, we adopted the V for Vendetta theme: remember remember the 5th of November!

We did a soft release to our Pre-prod environment the week before, to check that everything works. We went so far as to DDoS our test endpoint by watch -n .01 curl test_endpoint, as well as run http load tests on our test environment. Once we went live, legacy got all the traffic at first, then when we put the ingress load balancing to work, traffic gradually switched to the other service.

We set up heavy monitoring in advance to track what happened while we made changes. Our monitoring stack is Datadog and Sentry. The orders service includes background tasks, event consumers, public API endpoints, and XHR endpoints for the frontend, that all needed to be moved. During the whole procedure we were in close contact with support, we could rollback when customers would complain.

We didn’t strictly make the November 5 deadline, so we directed all traffic back to the monolith for the weekend, since we still supported that setup in parallel. Monday is the busiest day of the week for Sendcloud. It’s when our customers collect weekend orders, so we couldn't just get cracking at 8am after the weekend.

The following week, the new service deployed with flying colours in 2 days. Zero issues were recorded that were split related. What remained was fixing non-functionals that were not production service related.

Coincidentally, on that Friday, something was deployed that broke one of our other services, and everybody was pointing the finger at us. People were spooked of issues the split could cause, must have been the time of the year*. Even with a small delay, we pushed before the busiest time of the year (Black Friday / Cyber Weekend), or else we’d have to wait until 2022.

To celebrate the achievement, everyone in the team received a nice bottle with a thematic label (featured in the cover of this post), with the graph that measured their progress during a large chunk of the split.

What’s next?

We advocate heavily for standalone services that are only very loosely coupled. While we were working on the orders fulfillment service split we already started splitting up more services. The Account service split only took 3 sprints (6 weeks). With the “big breakup” on track, we felt confident about separating out more, smaller services.

Everything went well on Cyber weekend - in fact, Sendcloud hit its record high in terms of number of shipments in a day!

Now with this milestone reached, we will focus on creating the building blocks for making our architecture more horizontally scalable. More on that soon.

Looking to join a company that tackles interesting engineering challenges like the one described above? Sendcloud is hiring! Also check out our events on meetup.com. We look forward to meeting you - virtually - soon!

*Halloween got our colleagues jumpy it seems.

Conventional Comments at Sendcloud: better feedback

Rob van Gansewinkel — Fri, 10 Dec 2021 13:59:25 +0000

Rob van Gansewinkel, Senior Front-end Developer at Sendcloud

We’re all familiar with the problem: it is easy to misinterpret written feedback. Written feedback lacks information that’s encapsulated in (non-)verbal communication, like tone of voice. Also, in written feedback missing the intent behind a comment is commonplace, when you’re not being explicit (enough).

About 1.5 years ago, a colleague within the frontend chapter shared a link to the Conventional Comments project. After review, this seemed like a possible solution for what we experienced, even more so while working from home.

Conventional Comments can be used in any kind of review or feedback process, including (but certainly not limited to) code reviews. It adds intent and importance to the comment, promising shorter lead times in a review/feedback process.

Using Conventional Comments

The standardized format contains a mandatory “label” and “subject”, as well as an optional “decoration” and “discussion”:

Format:

<label>([decoration]): <subject>
[discussion]

The different label types include:

praise: highlight something sincerely positive
nitpick: small, trivial, but necessary changes
suggestion: propose improvements to the current subject
issue: highlight specific problems
question: to address a potential concern (relevancy uncertain)
thought: represent an idea that popped up from reviewing
chore: simple tasks that must be done before the subject can be accepted

Example

Imagine you looked at a colleague’s draft for a customer newsletter, and you think they took some liberties describing a product feature. Before hammering away on your keyboard, consider the following:

Bad:

This is not worded correctly.

Better:

suggestion: This is not worded correctly.

Or:

nitpick (non-blocking): This is not worded correctly.

Best:

suggestion: This is not worded correctly.

Can we change this to match the wording of the marketing page?

You’ll find that the third iteration helps you as the reviewer to provide clear & actionable feedback, limits the chance of startling the receiver, and results in shorter lead times in the overall feedback process.

Check out the project’s documentation for more guidance.

Feedback at Sendcloud

With some other enthusiasts in the Frontend chapter we decided to give it a go. While at first it may feel artificial - and you’ll find yourself editing messages you already sent to apply the standard - it helps you focus on the content of your message. It helps guide your writing.

Adoption was a gradual process within the frontend discipline but it is fully embraced by now. I used a Chrome extension for a while to remind myself to use the standard, but eventually it became so natural that I didn't need it anymore. Writing the Conventional Comments way quickly became a habit. You almost prefix your questions in one-to-one, non-digital communication as well!

Half a year ago we introduced the idea to our colleagues within the backend discipline, and they’re gradually adopting it. The UX team started 2 months ago, and they are enthusiastic users. I presented the project at the November 4 Uxify meetup, which we hosted at our Eindhoven office.

We noticed that it’s easier to ask for reviews when everyone works in the same format. Adopting Conventional Comments has benefitted cross-discipline, and cross-team collaboration at Sendcloud. I would recommend anyone to try it out for themselves!

praise: Thanks for reading this article!

How to conduct UX research in a team of two

Bianca Gorini — Thu, 09 Dec 2021 08:10:06 +0000

When it comes to starting the development of a new feature, the UX team is usually called to gather data and provide coherent flows that will impact the user journey positively and, hopefully, provide higher value in the user lifetime. When the UX department, or better the design and research department, consists of two (amazing) women against 25+ developers in an agile environment, the expectations are much higher. Let’s say that delegating is not an option. Our tasks don’t just revolve around managing the user experience — we are also in charge of conducting in-depth desk and field research and managing communication with stakeholders. In a typical sprint we can be found doing anything from:

The list can often be longer if we include DesignOps and user-centered advocacy, but I’m not here to bore you!

To be honest, designing features 360 degrees is a great opportunity for a UX designer and I feel extremely lucky to be given so much trust, but it can be exhausting to be in a one-man show. If you are also a UXer in a developer’s world, I would like to provide you with some insight on how we structured our UX process at Sendcloud. Feel free to leave comments and to advise us on better practices to follow, we will be forever grateful.
Today, I would like to talk about how we gather enough knowledge to start working and ideating on a new project or iteration. I won’t tell you how to do it, because I’m not even sure we know what we are doing (we know, I’m joking), but I will make sure to share enough details on how we do it and why.

Learning the business: Trend research & competitor analysis

Our job entails designing a B2B2C solution that involves shipping and logistics for eCommerce throughout Europe, and I can tell you — it ain’t easy. In general, the topics are usually very complex and involve a vast net of stakeholders. Let’s give an example: our internal and external stakeholders would like us to build something related to delivery methods in a shop checkout. This could be anything, but it needs to involve this specific step of the buyer and the seller’s journey. When the brief is so open, we have the possibility to take a step back from solutions and concentrate on understanding trends and general knowledge regarding that specific topic.
Generally, I like to embrace the unknown and find reliable material to study. Most of the time, consultancy companies (especially the big 5) create annual or even quarterly whitepapers about different innovations and trends in services. Those reports are not scholarly papers, but they usually cover a vast amount of topics and fields. Moreover, they are written by the big boys.

They can often afford to conduct surveys with a large sample of people, and if you are lucky enough, they probably asked some valuable questions for your target audience.

Remember to read some of those and to make sure they are recent studies: never trust one source and old news.
Usually, these reports help me to understand what is the general business value and, sometimes, to retrieve some initial user insights and needs. Plus, they usually are an easy way to access quantitative data, which is most useful when you need to pitch the idea to the entire company.
What is also interesting is to look at what your competitors are doing. You have probably heard it a thousand times over, but competitor analysis still remains one of my favorite things to do. Now, I’m not asking you to complete a full-blown in-depth analysis (why not leave that to marketing?), rather gain insight into what others are doing in the field. This builds an understanding of what is necessary to have and what features can make the difference in the user journey and workflow, especially for a B2B solution.

Talk to your colleagues: stakeholder syncs

If in the previous section we aim to understand the basics of the project, here we try to gather enough support in the company to help us with gathering more users and leads insights. Generally, in a tech company, the product owner is expected to have recurring meetings with sales and marketing, to establish priorities and shape the future of the product. However, their forte is not gathering quantitative data, and UX remains detached from the conversation.
Where you might see missed communication opportunities is in Sales, who speak to clients on a daily basis. Generally speaking, this actually collects many valuable insights for the UX process, which unfortunately get lost in platforms such as Hubspot. Recently, my team (Caroline and I) decided to start having our own sync with sales and marketing. These sessions provide the opportunity to speak about missing features, and why those are so important for the user. We also learn how to read between the lines, and we start understanding and seeing patterns in user behavior and needs. Moreover, the constant contact that those departments have with our users and our future users, make them the perfect detectives.

There are a few rules to remember: keep it short, come prepared with what you want to know more about, make sure to set action points to show your colleagues that their input will be actively used.

If we are lacking insights on a new possible feature or on the adoption of an existing one, we get them to ask some more specific questions to the people they will talk to. In this way, we can gather some data quickly that otherwise would have taken us much more time to collect. This allows us to see what features we need to get more sales, and thus a mutually beneficial department relationship is formed!

One tip, if you are starting to ask other departments to gather user insights, make sure to be specific in your request and make sure they are aware of the context you are researching on. If it’s not clear to them, the data gathered could be biased or too general.

Let numbers talk: data & analytics

At Sendcloud our design decisions are data-driven rather than basing them solely on our own ideals. As designers, we understand that the end-users experience has a learning curve, and when we assume what they can understand, we don’t address why or how they understand the product. Analytics can help not just when designing new features but through-out the iterative process.
How do we design with data in mind and as an iterative process? By continuously running A/B testing. This helps us understand how new features or changes can improve the user experience. Sometimes, even the most minor changes can have a substantial impact. But this is not the only way to gather data, I will also discuss how you can ask users directly through interviews and surveys.

“Don’t change something If It’s not broken”

But how do you know when you should change a product or service? Our process involves beta testing before we complete a roll-out, which means we release a feature to a small group of users, allowing us to gage If the user will actually use a feature from their feedback. This provides us with data to improve our UX before the feature is rolled out.

It’s also important to plan for tracking feature usage while refining a user story. We can only define a feature “successful” if we have the means to monitor its usage. Talk to your team and decide which events need to be tracked. We usually combine backend events with frontend ones, and we make sure to aggregate them in an analytics tool for easy reporting. We are currently using Mixpanel, which provides clear developers documentation.

Lean UX: feedback repository & NPS scores

Did you ever have sales, support, or other departments providing you with user comments? Did you ever think “I need to remember this”? At Sendcloud, designers are often swamped with user feedback (and it’s honestly great). One day, tired of losing precious information through Slack or Emails, we came up with the idea of creating a user feedback repository. Essentially, we created a place where we could gather and tag user feedback incoming from different sources, from colleagues to NPS scores. To make sure we would not have to do too much manual work, we created a # feedback channel on Slack and asked our colleagues to share customers’ feedback directly in that channel. This did not only stimulate many people to contribute to the cause of making our platform more user-centered but also created more transparency on where the comments were coming from and from which target audience. Finally, all the comments are gathered in an application called EnjoyHQ, that not only provides an integration with Slack but also integrates with NPS tools and Hubspot. The EnjoyHQ team wrote an article about our experience using this tool, you can read it here.

Ask the user directly: customer visits, interviews, and surveys

Taking the time to visit our users has allowed us to reveal our product’s unmet needs and a chance to build empathy with our users. You can gauge a lot from visiting a person in their own setting, mostly, how do they naturally go about their tasks? Perhaps they develop workarounds which they may not have mentioned before. As designers, these naturalistic observation sessions provide us with more reliable data. They give us an understanding of how the user interacts with Sendcloud in real life. Alongside this, they are more likely to find out our user’s pain-points and personal benefits compared to a lab setting.
We have also been able to travel to our users, giving us an opportunity to further research ethnographic aspects of our international user experience.

Often our clients in Benelux will experience very different pain-points to those located in the south of Europe. It’s important for us to take note of cultural differences for the heuristic aspect of our design since we can’t presume to understand the nuances of a foreign culture.

When we return to our office in Eindhoven, we share our findings with the rest of the product teams, whether it’s an issue with how the product functions or it’s localization, field studies can return almost any kind of information. They can also assist you in building the groundwork that will help you design more focused studies later. Personally, I find that interviews work really well in an agile environment since they provide the needed information quickly and efficiently.

We use qualitative surveys to ask for comments, feedback, suggestions, and other kinds of responses that aren’t as easily classified. With surveying, we can use a smaller sample number and still achieve rich data. The same as we apply an iterative design process to our design; we apply to the surveying method by testing and applying open-ended feedback. This way we ensure that we have obtained high-quality information and can be applied to a specific feature or a process. In turn, this data can be used to inform personas, journey maps, feature ideas, and workflow ideas.

Final words from us

As Sendcloud grows, we are continuously challenged to change the way in which we work to meet our user’s needs. That is why we want to keep up-to-date with what is happening in the design world. Writing about our process along with attending conferences allows us the opportunity to see what other UX designers are doing. We are always open to conversations around the UX process, If you have any questions or something to share, please leave a comment below!

Interested in joining our awesome team? We are hiring! Check out our job page.

Many thanks to Corlynne O’Sullivan for helping me writing this post.

From Majestic Monolith to Decoupled Service Architecture

Tim Drijvers — Thu, 09 Dec 2021 07:28:49 +0000

In a set of upcoming blog articles, I will take you through some of the design decisions we have taken and the challenges we’re facing in the current and future architecture of our platform. This article outlines the main reasons why we’re moving away from our monolith application setup.

At Sendcloud we run a shipping platform built using technologies like Python, Django, our main database is Postgresql, Redis for caching, and Dramatiq for background tasks.

For many years we embraced the monolith pattern at Sendcloud. For a small start-up with a team of < 20 developers, this was a great way to develop an application. We carefully designed our monolith-first strategy. The simplicity of our stack allowed for fast iterations, trying to get a product-market fit at a rapid pace. The majority of our platform’s code is kept in a single repository, while different domains are divided into Django apps making sure developers didn’t step on each other’s toes. For a long time, a developer could keep the entire code-base in his head.

After all these years we’ve decided to move away from our monolith setup, breaking our platform into smaller services.

Our main drivers to do so are:
Decrease complexity and cognitive load for developers,
Reduce deployment times and risks,
Allow for different scaling and robustness requirements per service,
Introduction of new technologies.

Our codebase has grown quite big, although well documented and with extensive test coverage, it’s hard to comprehend the entire code base. Especially for newcomers to our team, the project is getting overwhelmingly big. It’s hard to oversee the impact of changes you’re making or to keep up with changes your colleagues are making. Moving to smaller services allows our teams to move faster and keep an overview of the services they are working on. Implementing strict contracts (APIs), allows teams to efficiently work on these services in parallel.

Each deployment is a potential risk to disrupt our service. We have an extensive continuous integration and deployment setup, currently making 10–15 deployments per day mostly containing a wide range of code changes. Each time, however, we build and deploy our entire platform, even for small changes. With a growing team, the new releases will contain more changes, increasing the risk of deployments and making it harder to relate an error in production to a merge request. All in all, this is causing a waste of valuable computational resources, unnecessary long time to deploy, and this is slowly creating a bottleneck since more developers want to join the merging train or spend more time on triage of errors.

Sendcloud is also rapidly expanding internationally (we launched our product in Spain and Italy this year and will do so in the UK later this year as well), and our customers are active at extended hours. This makes maintenance without impacting our customers harder. Having separate services, databases, and infrastructure for various purposes will allow us to perform maintenance on these services with reduced customer impact. If these separate services are still tightly coupled, the impact of such maintenance will easily cascade through the platform. To improve this robustness it’s key that these services are loosely coupled.

Once the APIs are clearly defined between connecting services, new technologies can be introduced in our stack as well. Sendcloud highly depends on external services, where we make a lot of API calls to these services. Python isn’t necessarily the most efficient tool to handle a lot of blocking IO operations in parallel. Having strictly defined contracts, allows teams to experiment with new technologies, in our case we’re gradually introducing more Go into our stack.

In upcoming articles, more details will be shared on technical decisions we’ve taken. A good read is Sam Newman’s book — Monolith to Microservices, which outlines various patterns on moving from a monolith to a microservices architecture.