DEV Community: Ollie Bannister

StudySpy: Building the new PublicApi v2

Ollie Bannister — Sun, 15 Sep 2024 14:47:15 +0000

How I built StudySpy's Public API from scratch: A solo developer's journey through tech choices, security challenges, and creating real value. Learn key strategies for your own API projects.

This year, I embarked on an exciting journey to build a new Public API system for StudySpy, one of my clients. The goal was to create a system that would allow StudySpy to sell access to their valuable dataset to external parties. This project wasn't just about opening up new revenue streams; it was about innovation and expanding the value StudySpy could offer to the New Zealand education sector.

APIs play a crucial role in modern software systems, facilitating the flow of essential data across various components and platforms. By enabling third parties to build products on top of StudySpy's infrastructure, we were set to significantly increase the platform's value and reach.

But our vision extended beyond external use. We aimed to create an API that would also serve as an internal standard for data access across the platform. This dual-purpose approach promised to streamline operations and set a new benchmark for efficiency within StudySpy.

Building the Infrastructure

Starting such a project can be daunting, but I've learned that the act of beginning is often the most crucial milestone. Once you have something tangible, it becomes much easier to iterate and improve. However, before diving into coding, I knew that designing the right infrastructure from the start would be half the battle. The old adage "measure twice, cut once" came to mind.

My focus was on three key areas:

Technologies: Choosing the right tech stack
Structure: Ensuring maintainability through proper project organization
Security: Implementing robust measures to protect the main platform

Technologies

Given that the main StudySpy platform runs on .NET 5/6+, I decided to leverage this technology as the foundation for the new API system. This decision immediately descoped a significant amount of effort – a critical consideration for a solo developer.

I also wanted to incorporate Swagger with it'OpenAPI specification, tools I'd had positive experiences with in the past. These two key technologies formed the base upon which I built the rest of the stack.

The final tech stack included:

.NET
Swagger + OpenAPI specification
Microsoft SQL Server
Docker
Microsoft entity framework

Project Structure

For project structure, I opted to create a new area within the existing solution rather than a separate .sln file. This approach maintains a cohesive view of the entire codebase, which is particularly beneficial for smaller teams.

I started by defining the routes for the initial version of the API:

/api/v2/campuses
/api/v2/campuses/[id]
/api/v2/courses
/api/v2/courses/[id]
/api/v2/intakes
/api/v2/intakes/[id]
/api/v2/providers
/api/v2/providers/[id]

Based on these routes, I created the following directory structure:

PublicApi/
├── Controllers/
│   ├── CampusesController.cs
│   ├── CoursesController.cs
│   ├── IntakesController.cs
│   ├── ProviderController.cs
│   └── ScholarshipController.cs
├── Models/
│   ├── Campus.cs
│   ├── Course.cs
│   ├── Intake.cs
│   ├── Provider.cs
│   └── Scholarship.cs
└── Services/
    (initially empty, to be populated as needed)

I began by defining the models, which shaped the API's response payloads. With these in place, I could then set up basic GET endpoints for both list and individual resource retrieval, initially returning dummy data.

Here's a basic example of one of the domain-driven endpoints:

PublicApi/Models/Campus.cs

using System.Collections.Generic;

namespace StudySpy.Web.Areas.PublicApi.Models
{
    public class Course
    {
        public string Name { get; set; }
        public string SsId { get; set; }
        public string SsProviderId { get; set; }
        public string LevelOfStudy { get; set; }
        public List<string> AreasOfStudy { get; set; }
        public List<string> SubjectsOfStudy { get; set; }
        public string Details { get; set; }
        public string EntryRequirements { get; set; }
        public string Duration { get; set; }
        public bool Active { get; set; }
        public long LastUpdated { get; set; }
        public bool InternationalFlag { get; set; }
    }
}

PublicApi/Controllers/CoursesController.cs

using Microsoft.AspNetCore.Mvc;
using System.Collections.Generic;
using StudySpy.Web.Areas.PublicApi.Models;

namespace StudySpy.Web.Areas.PublicApi.Controllers
{
    [Area("PublicApi")]
    [ApiController]
    [Route("api/v2/[controller]")]
    [ApiExplorerSettings(GroupName = "PublicApi")]
    public class CoursesController : ControllerBase
    {
        [HttpGet]
        public ActionResult<IEnumerable<Course>> GetCourses()
        {
            // Dummy data for illustration
            var courses = new List<Course>
            {
                new Course 
                { 
                    Name = "Computer Science",
                    SsId = "CS101",
                    SsProviderId = "UNIV001",
                    LevelOfStudy = "Undergraduate",
                    AreasOfStudy = new List<string> { "Technology", "Mathematics" },
                    SubjectsOfStudy = new List<string> { "Programming", "Algorithms", "Data Structures" },
                    Details = "A comprehensive course covering the fundamentals of computer science.",
                    EntryRequirements = "High school diploma with strong mathematics background",
                    Duration = "4 years",
                    Active = true,
                    LastUpdated = 1630444800, // Unix timestamp for September 1, 2021
                    InternationalFlag = true
                },
                new Course 
                { 
                    Name = "Business Administration",
                    SsId = "BA201",
                    SsProviderId = "UNIV001",
                    LevelOfStudy = "Graduate",
                    AreasOfStudy = new List<string> { "Business", "Management" },
                    SubjectsOfStudy = new List<string> { "Finance", "Marketing", "Operations" },
                    Details = "An MBA program designed for aspiring business leaders.",
                    EntryRequirements = "Bachelor's degree and 2 years of work experience",
                    Duration = "2 years",
                    Active = true,
                    LastUpdated = 1641024000, // Unix timestamp for January 1, 2022
                    InternationalFlag = true
                }
            };

            return Ok(courses);
        }

        [HttpGet("{id}")]
        public ActionResult<Course> GetCourse(string id)
        {
            // Dummy data for illustration
            var course = new Course
            {
                Name = "Data Science",
                SsId = id,
                SsProviderId = "UNIV001",
                LevelOfStudy = "Graduate",
                AreasOfStudy = new List<string> { "Technology", "Statistics" },
                SubjectsOfStudy = new List<string> { "Machine Learning", "Big Data", "Statistical Analysis" },
                Details = "An advanced course in data science and analytics.",
                EntryRequirements = "Bachelor's degree in a quantitative field",
                Duration = "2 years",
                Active = true,
                LastUpdated = 1651363200, // Unix timestamp for May 1, 2022
                InternationalFlag = true
            };

            return Ok(course);
        }
    }
}

This setup allowed me to perform my first test using a Bruno client, and voila! I received my first response. Now we're getting somehwere 🎉

Security

Before shipping anything, implementing security measures was crucial. I focused on two main concerns:

Unauthorized access: I implemented a rough-cut API key authentication strategy and added our new API pages to the robots.txt file to prevent search engine indexing. This would mean no-one would really stumble across our API accidentally, and if they did, they wouldn't be able to access our system without a valid API key.
Protection against DoS: To mitigate the risk of database read operation overload (intentional or unintentional), I implemented API key rate limiting in combination with IP rate limits. I set a sensible limit of 120 requests per minute (2 requests per second) to maintain a reasonable SLA while protecting the system from accidental request floods.

With these security measures in place, I was able to release the initial implementation of the new API routes to a production environment and take them for a spin.

Making it a Product

While I had a basic API in production, transforming it into a valuable product required several additional steps:

Adding Business Logic

We needed to get the routes wired up so they could start returning something valuable to the user. I used Microsoft's Entity Framework ORM to pull records from the SQL database and map them to the API response payloads. This process involved creating data access layers and implementing the necessary business logic in each controller.

Creating Services for Reusable Logic

To promote code reuse and maintain a clean separation of concerns, I abstracted common business logic into services. By the end of the project, I had created several utility services:

Services/
├── AuthorityFormatter.cs
├── DateFormatCalculator.cs
├── DeliveryCalculator.cs
├── DurationCalculator.cs
├── HtmlHelper.cs
├── MacronRemover.cs
├── ProviderTypeFormatter.cs
├── RegionMapper.cs
├── StreetAddressFormatter.cs
├── StringFormatter.cs
└── SubjectTaxonomyMapper.cs

These provided me with a way I could share standarised business logic across various endpoints.

Implementing Pagination

Pagination was crucial for allowing third parties to navigate through records at the API level efficiently. I created a pagination model and service:

PublicApi/Models/Pagination.cs

using System.Collections.Generic;

namespace StudySpy.Web.Areas.PublicApi.Models
{
    public class ApiPaginatedResponse<T>
    {
        public List<T> Items { get; set; }
        public Pagination Pagination { get; set; }
    }

    public class Pagination
    {
        public int Skip { get; set; }
        public int Limit { get; set; }
        public int Count { get; set; }
        public string NextPage { get; set; }
        public int TotalCount { get; set; }
    }
}

I then wired this object up to all API response payloads:

var response = new ApiPaginatedResponse<Campus>
{
    Items = CoursesPayload,
    Pagination = pagination
};

return Ok(response);

Implementing Correct HTTP Response Codes

An often overlooked but crucial aspect of API design is the proper use of HTTP response codes. These codes provide immediate feedback to API consumers (and developers) about the status of their requests, making the API more intuitive and easier to work with.

I made sure to implement a range of appropriate status codes in our API responses:

200 OK: For successful GET, PUT, or PATCH requests
400 Bad Request: When the request is malformed or contains invalid parameters
401 Unauthorized: When authentication is required but not provided or is invalid
403 Forbidden: When the authenticated user doesn't have permission to access the requested resource
404 Not Found: When the requested resource doesn't exist
429 Too Many Requests: When the client has sent too many requests in a given amount of time (rate limiting)
500 Internal Server Error: For unexpected server errors

By consistently using these status codes across all endpoints, we ensure that our API communicates clearly with its consumers, delivering the best developer experience for uses of our API.

Customizing the UI

To enhance the user experience and bring our brand to the API, I customized the Swagger UI by replacing the generic branding with StudySpy's logo and applying clean, consistent styling.

I also added example response payloads for various endpoints:

Swagger/
└───Examples
    ├───CampusExamples
    │       CampusListResponseExample.cs
    │       CampusResponseExample.cs
    │
    ├───CourseExamples
    │       CourseListResponseExample.cs
    │       CourseResponseExample.cs
    │
    ├───IntakeExamplesse | Format-Tree                                                     [History]
    │       IntakeListResponseExample.cs
    │       IntakeResponseExample.cs
    │
    ├───ProviderExamples
    │       ProviderListResponseExample.cs
    │       ProviderResponseExample.cs
    │
    ├───ScholarshipExamples
    │       ScholarshipListResponseExample.cs
    │       ScholarshipResponseExample.cs
    │
    └───StatusCodes
            400ResponseExample.cs
            401ResponseExample.cs
            404ResponseExample.cs
            429ResponseExample.cs

Provisioning API Keys

Before sharing the new Swagger documentation, I generated and distributed API keys to the third parties who would be using the new API. This process involved creating a secure system for generating, storing, and managing these keys.

You don’t need to over-think this. We assigned API keys to a singleton and added code comments to identify the associated users. This simple system allowed for easy management and potential future revocation of access if needed.

Lessons Learned

Building this project was an exciting challenge that taught me several valuable lessons:

Start small and iterate: Beginning with a minimal viable product and improving it continuously proved to be an effective strategy.
Define first, then build: Having a clear understanding of the desired responses greatly simplified the process of writing business logic.
Don't stress the small stuff: Getting the service into users' hands quickly for feedback is crucial, even if it's not 100% polished.
Leverage existing resources: Choosing to use languages and frameworks already in play resulted in a more cohesive solution and easier future maintenance.

Future Horizons

The development of this API has not only expanded StudySpy's offerings but has also opened up new possibilities for innovation in the education sector for New Zealand. As more organizations recognize the power of APIs in driving growth and fostering ecosystems, projects like this will become increasingly vital.

Whether you're considering building an API for your own project or looking to leverage existing APIs in your organization, remember that the journey of a thousand miles begins with a single step. Start small, focus on delivering value, and don't be afraid to iterate and improve as you go.

If you’ve made it this far and are interested in viewing the swagger documentation on the live site, you can view it here:
https://studyspy.ac.nz/api/v2/documentation

Like you, I'm always interested in learning more. If you have any feedback or questions about this project or API development in general, feel free to reach out to me on Twitter.

JS Toolbox 2024: Essential Picks for Modern Developers Series Overview

Ollie Bannister — Mon, 04 Mar 2024 04:07:46 +0000

Introducing JS Toolbox 2024

Staying ahead of the curve in JavaScript development requires keeping on top of the ever-evolving landscape of tools and technologies. As we head into 2024, the sprawling world of JavaScript development tools will continue to transform, offering more refined, efficient, and user-friendly options.

This series, JS Toolbox 2024, is your one-stop for a comprehensive overview of the latest and most impactful tools in the JavaScript ecosystem.

Across the series, we'll look at various tools, including runtime environments, package managers, frameworks, static site generators, bundlers, and test frameworks. This series will help you effectively choose the best available tools, analysing their functionalities, strengths, weaknesses, and how they fit into the modern JavaScript development process.

Whether you're a seasoned developer or just starting out, this series will give you the necessary knowledge to select the right tools for your projects in 2024. Here’s what we’ll be getting into in this series.

Series overview:

1. Runtime Environments and Package Management

In the first instalment, we explore runtime environments, focusing on Node, Deno and Bun. We'll gain insights into their histories, performance metrics, community support, and ease of use.

The segment on package management tools compares npm, yarn, and pnpm, highlighting their performance and security features. We provide tips for choosing the most suitable package manager for your project.

2. Frameworks and Static Site Generators

The next post in the series provides a thorough comparison of popular frameworks like React, Vue, Angular, and Svelte, focusing on their unique features and suitability for different project types.

We also take a look into static site generators, covering Astro, Nuxt, Hugo, Gatsby, and Jekyll. We take a detailed look into their usability, performance, and community support.

3. Bundlers and Test Frameworks

In part 3 We jump into the world of bundlers, comparing webpack, esbuild, vite, and parcel 2. This section aims to guide developers through each bundler, focusing on their performance, compatibility, and ease of use.

The test frameworks section provides an in-depth look at MochaJS, Jest, Jasmine, Puppeteer, Selenium, and Playwright. We review each framework's ease of use, community support, and overall robustness, supplemented with example unit tests.

What you'll take away

JS Toolbox 2024 provides an informed view of the current JavaScript development tools landscape. Through personal insights and industry observations, I’ve aimed to guide developers through a selection of the myriad tools available, helping you make educated decisions that meet your project needs and personal preferences. Be sure to check out the entire series for insights that could shape your development workflow and tool choices in 2024 and beyond.

Like you, I’m always curious and looking to learn. If I've overlooked a noteworthy tool or if you have any feedback to share, reach out on Twitter or LinkedIn.

JS Toolbox 2024: Bundlers and Test Frameworks

Ollie Bannister — Mon, 04 Mar 2024 04:07:28 +0000

JavaScript is bigger than ever, and the ecosystem is nothing short of overwhelming. In this JS toolbox 2024 series, we’ve selected and analyzed the most noteworthy JS tools, so that you don’t have to.

In part 1 of this series, we explored the foundations of any JavaScript project: Runtime environments and package management. In part 2, we focused on JavaScript frameworks and static site generators. Now, in part 3, we turn our attention to the tools that refine and optimize the development process: Bundlers and test frameworks.

In this post:

Bundlers
- Webpack
- EsBuild
- Vite
- Parcel 2
Test frameworks
- MochaJS
- Jest
- Jasmine
- Puppeteer
- Selenium
- Playwright
What to choose?

Bundlers

In modern web development, bundlers play a pivotal role in getting your project production-ready. They are essential tools for combining multiple files and assets into streamlined, optimized bundles, effectively managing complex dependencies, and enhancing application performance.

This section delves into the world of bundlers, focusing on popular tools such as webpack, esbuild, vite, and parcel 2.

An overview of bundler popularity trends.

In 2023, Webpack has maintained its position as the most popular project bundler, but Vite is rapidly gaining ground and could soon challenge for the top spot. Meanwhile, Parcel and esbuild have also experienced steady growth, making the project bundler category even more competitive.

Webpack

Webpack is a powerful and widely-used module bundler for JavaScript applications. It’s known for its flexibility and extensive plugin system, making it a popular tool in complex web development projects.

It also benefits from being a long-standing player in the JavaScript ecosystem. Over the years, Webpack has adapted to the evolving demands of web development, establishing itself as a reliable and robust tool.

Advantages

Extensive customization: Webpack offers a highly configurable environment, allowing developers to tailor the bundling process to their specific needs through a vast array of plugins and loaders.
Strong community and ecosystem: With a large community backing it, Webpack benefits from a rich ecosystem of plugins, extensive documentation, and community support.
Optimized bundle handling: It excels in handling various assets and dependencies, offering features like code splitting and lazy loading to optimize application performance.

Disadvantages

Steep learning curve: The complexity and vastness of Webpack’s configuration options can be overwhelming for beginners or simpler projects.
Configuration overhead: Setting up and maintaining a Webpack configuration can become cumbersome, especially in larger projects with more specific requirements.

EsBuild

EsBuild is a relatively new, blazing-fast JavaScript bundler and minifier. It stands out for its high performance, significantly speeding up the build process in development pipelines.

Advantages

Exceptional speed: EsBuild is known for its incredibly fast build times, often orders of magnitude quicker than traditional bundlers, which greatly enhances developer productivity.
Simplicity and ease of use: With a straightforward setup and minimal configuration, EsBuild is accessible and easy to integrate into development workflows.
Efficient bundling and minification: It efficiently handles the tasks of bundling and minification, reducing the size of output files and improving application load times.

Disadvantages

Limited plugin ecosystem: Being newer in the market, EsBuild has a smaller range of plugins and integrations compared to more established bundlers like Webpack.
Less feature-rich: While its simplicity is a strength, EsBuild may lack some of the advanced features and customizability found in other, more mature bundling tools.

Vite

Vite is a modern front-end build tool that significantly improves the development experience with its fast cold server start and hot module replacement. It leverages native ES modules and is optimized for speed and efficiency.

Advantages

Rapid development cycle: Vite offers a fast server start and quick updates with its hot module replacement, streamlining the development process.
Optimized for modern browsers: Utilizing native ES modules, Vite is designed for modern browsers, reducing the need for extensive transpiling and bundling.
Ease of use: With sensible defaults and minimal configuration, Vite simplifies setup and usage, making it accessible for new developers and efficient for experienced ones.

Disadvantages

Dependency on modern browser features: Vite’s reliance on native ES modules means it may not be the best fit for projects needing to support older browsers.
Growing ecosystem: While rapidly evolving, Vite's ecosystem and range of plugins are still in the growth phase and may not yet match the breadth of more established tools like Webpack.

Parcel 2

Parcel 2 emphasizes a zero-configuration approach to bundling web applications. It's a powerful tool that offers a hassle-free developer experience, focusing on simplicity and speed.

Advantages

Zero configuration required: Parcel 2 stands out for its no-config setup, allowing developers to get started quickly without the need for intricate configuration.
Fast build times: It offers rapid build and rebuild times, enhancing productivity and streamlining the development process.
Out-of-the-box features: Parcel 2 comes with many built-in features like code splitting, module replacement, and support for various languages and file types.

Disadvantages

Less granular control: The zero-configuration approach, while convenient, can limit developers who need more control over the bundling process.
Smaller community: Compared to tools like Webpack, Parcel 2 has a smaller community, which might mean fewer resources and plugins available to developers.

Test frameworks

Test frameworks are crucial for ensuring the quality and reliability of JavaScript applications, particularly through comprehensive code coverage. These frameworks provide the structured environment needed for writing, executing, and managing tests, playing a pivotal role in validating code functionality and detecting bugs. As JavaScript is adopted in both front-end and back-end project stacks, the robustness of testing frameworks becomes more important, helping developers maintain high standards of code integrity and application performance.

This section will introduce you to a variety of JavaScript test frameworks, each with unique attributes tailored to different testing requirements.

An overview of Framework popularity trends.

Over the past year, the JavaScript testing framework landscape has remained relatively solid. No major disruptor technologies have proven to challenge Puppeteer for the top spot, although the rapid growth of Playwright gives us something to keep our eyes on through 2024.

These technologies aren’t directly comparable however due to the varying niches that they fill. In this section, I will cover each technology to give you the context to help you make the right decision when choosing the testing framework for your project.

MochaJS

MochaJS is a feature-rich JavaScript test framework running on Node.js, making it a popular choice for both backend and frontend testing. It's known for its flexibility and straightforward syntax, allowing for asynchronous testing and the use of any assertion library.

Example test case syntax:

describe('Array', function() {
  describe('#indexOf()', function() {
    it('should return -1 when the value is not present', function() {
      assert.equal([1, 2, 3].indexOf(4), -1);
    });
  });
});

Advantages

Flexibility and compatibility: MochaJS is highly flexible, allowing developers to choose their preferred assertion libraries, mock/spy libraries, and more.
Rich feature set: It offers a wide range of features like asynchronous testing, test coverage reports, and hooks for setting up conditions before running tests.
Good documentation and community support: MochaJS has comprehensive documentation and strong community support, making it easier for new users to get started and find solutions to common problems.

Disadvantages

Global variables: Mocha exposes global variables, which can be an issue for developers preferring a more modular approach.
Slower than some rivals: While Mocha is versatile, it can be slow compared to some newer testing frameworks, particularly in large test suites.

Jest

Jest, created and maintained by Facebook, is a JavaScript testing framework with a focus on simplicity. It is often recognized for its zero-configuration setup and seamless integration, especially with React applications, making it a favored choice for developers in both small-scale projects and large enterprise applications.

Example test case syntax:

test('adds 1 + 2 to equal 3', () => {
  expect(1 + 2).toBe(3);
});

Advantages

Zero-configuration: Jest is designed for out-of-the-box usability with minimal setup, offering a smooth testing experience right from the start.
Built-in mocking and assertions: It comes with powerful mocking capabilities and a rich set of assertion APIs, eliminating the need for additional libraries.
Snapshot testing and coverage: Jest provides snapshot testing, which is great for catching unexpected changes, and built-in coverage reports to keep track of your test suite's effectiveness.

Disadvantages

Performance in large projects: While generally fast, Jest can exhibit slower test execution times in very large projects or complex test suites.
Configuration complexity in advanced scenarios: Despite its zero-config appeal, advanced use cases might require navigating through more complex configurations and setups.

Jasmine

Jasmine is a behavior-driven development framework for testing JavaScript code. It's known for its clean, straightforward syntax and does not rely on any other JavaScript frameworks or a Document Object Model (DOM). Jasmine is often praised for its simplicity and ease of use in both frontend and backend testing environments.

Example test case syntax:

describe("A suite", function() {
  it("contains spec with an expectation", function() {
    expect(true).toBe(true);
  });
});

Advantages

Clean, readable syntax: Jasmine's syntax is designed to be easy to read and write, making it accessible for developers of all skill levels.
Independent of other technologies: It doesn't depend on any external software or browser DOM, making it suitable for testing any JavaScript application.
Out-of-the-box functionality: Jasmine comes with everything you need for testing your JavaScript code, including test doubles and asynchronous testing support.

Disadvantages

Limited mocking capabilities: Jasmine's mocking capabilities, while present, are not as extensive or straightforward as some other testing frameworks.
Slower performance with large test suites: In scenarios involving large test suites, Jasmine can sometimes be slower compared to other modern testing frameworks.

Puppeteer

Puppeteer is a Node library that provides a high-level API to control headless Chrome or Chromium. It's primarily used for browser automation, making it a powerful tool for end-to-end testing of web applications, taking screenshots, and generating pre-rendered content from web pages.

Example test case syntax:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://example.com');
  // other actions...
  await browser.close();
})();

Advantages

Browser Automation: Puppeteer excels in automating and testing browser environments, offering precise control over Chrome or Chromium.
Rich Feature Set for Testing: It supports a wide array of testing needs, from capturing screenshots and generating PDFs to testing modern web applications.
High-Level API: Puppeteer's API abstracts away complexities, making it relatively easy to write scripts for browser-based testing.

Disadvantages

Limited browser support: Puppeteer primarily works with Chrome and Chromium, which might not be suitable for cross-browser testing needs.
Higher resource usage: Being a full browser automation tool, it can be more resource-intensive compared to some lightweight testing frameworks, potentially leading to slower performance in resource-constrained environments.

Selenium

Selenium is an extensively used open-source automation framework for web applications. It allows for cross-browser testing by automating browser actions, making it a staple tool for end-to-end testing in diverse web development environments.

Example test case syntax:

WebDriver driver = new ChromeDriver();
driver.get("http://www.example.com");
WebElement element = driver.findElement(By.name("q"));
element.sendKeys("Hello Selenium WebDriver!");
element.submit();

Advantages

Cross-browser compatibility: Selenium supports testing across a wide range of browsers, including Chrome, Firefox, Safari, and Internet Explorer, making it ideal for cross-browser testing.
Support for multiple programming languages: It offers the flexibility to write tests in several programming languages, such as Java, Python, C#, and JavaScript.
Strong community and ecosystem: Being one of the oldest and most established testing frameworks, Selenium has a large community, extensive resources, and integration support.

Disadvantages

Complex setup and learning curve: Setting up Selenium can be complex, and it requires a fair amount of learning, particularly for those new to testing automation.
Performance issues: Selenium tests can sometimes be slower compared to some modern testing frameworks, especially when running complex test suites or tests across multiple browsers.

Playwright

Playwright is a relatively new but powerful node library for browser automation. Developed by Microsoft, it enables cross-browser web automation that is ever-reliable for end-to-end testing. Playwright is compatible with all modern web browsers, including Chrome, Firefox, and Safari, and offers capabilities for both headless and non-headless browser testing.

Example test case syntax:

const { chromium } = require('playwright');

(async () => {
  const browser = await chromium.launch();
  const page = await browser.newPage();
  await page.goto('https://example.com');
  // other actions...
  await browser.close();
})();

Advantages

Cross-browser support: Playwright supports automated testing across multiple browsers and platforms, enhancing its utility in diverse development scenarios.
Modern and robust API: It offers a modern, feature-rich API that allows for advanced browser automation, including support for single-page applications.
Fast and reliable: Playwright tests are generally faster and more reliable than those performed with older tools, offering improved stability and performance.

Disadvantages

Relatively new in the market: As a newer tool, Playwright is still building its community and ecosystem, which can currently lack the extensiveness of more established frameworks.
Learning curve: While its API is user-friendly, there is still a learning curve, especially for developers new to browser automation and end-to-end testing.

What to choose?

Choosing the right tools for your JavaScript project, is a crucial decision that can significantly impact your development workflow and the quality of your final product. Each tool comes with its own set of strengths and drawbacks.

For bundlers, the decision might hinge on factors like ease of configuration, build speed, and ecosystem support. In test frameworks, the decision could depend on the nature of your testing needs, ranging from unit and integration testing to end-to-end browser automation.

Your choice should align with the specific needs of your project, the skill set of your team, and the nature of your development environment. It's about finding the right tool that not only addresses your current requirements but also adapts to the evolving demands of your project. A comprehensive understanding of each tool's capabilities and limitations is key to making an informed decision that enhances both your development process and the overall quality of your JavaScript applications. Remember, the right set of tools can empower your development team, streamline your processes, and lead to the creation of a robust, efficient, and maintainable codebase.

Like you, I’m always curious and looking to learn. If I've overlooked a noteworthy tool or if you have any feedback to share, reach out on Twitter or LinkedIn.

JS Toolbox 2024: Frameworks and static site generators

Ollie Bannister — Mon, 04 Mar 2024 04:06:52 +0000

In 2024, JavaScript is bigger than ever. The ecosystem is just as huge, and almost impossible to keep track of, so I’ve had a go at picking out 2024’s most essential JS tools for you.

In part 1 of this series, we reviewed runtimes and package managers, the foundational building blocks of your software project. So in part 2, we're analyzing the tools which form the walls and roof that give your software project its structure: frameworks and static site generators.

For this installment of JS Toolbox 2024, we explore various frameworks & generators available in the JavaScript & TypeScript ecosystem, analyzing their strengths, weaknesses, and ideal use cases.

You can also check out Part 3, covering bundlers and test frameworks, here.

In this post:

Frameworks
- React
- Vue
- Angular
- Svelte
Static Site/Server side Generators
- Astro
- Nuxt
- Hugo
- Gatsby
- Jekyll
What to choose?

Frameworks

JavaScript frameworks are an essential part of modern web development, providing tried and tested tools for building scalable, interactive web applications. Many modern companies use frameworks as a standard part of their tooling, so understanding the current state of frameworks is crucial for projects old and new.

In this section, we look at some of the most prominent JavaScript frameworks that shaped web development in 2023 and speculate on what 2024 will have in store.

An overview of framework popularity trends.

In 2023, React and Vue.js have continued to lead the pack. We’ve seen some consistent growth from both Angular and Svelte; however, given Svelte’s trajectory I can see the gap closing on its competitors in 2024. Let’s get into what’s good and not-so-good about these leading frameworks.

React

Developed by Facebook, React rapidly gained popularity as a framework for developing dynamic user interfaces for high-traffic web pages. Known for its efficiency, React utilizes a virtual DOM, which simplifies integration with various applications. This approach enables fast and smooth user experience, making it a favored choice for modern web development.

React components are typically written using JSX, which allows you to write HTML-like syntax in your JavaScript.

import React from 'react';

function Greeting({ name }) {
  return <h1>Hello, {name}!</h1>;
}

export default Greeting;

Advantages

Component-based architecture: React's design is centered around reusable components, allowing developers to build encapsulated components that manage their own state. This architecture makes it easier to manage and maintain large applications, as well as to reuse code, leading to more efficient development processes.
Virtual DOM for enhanced performance: React employs a virtual DOM (Document Object Model) that optimizes rendering and improves application performance. This means React can update changes in an application's UI quickly and efficiently, leading to a smoother and more responsive user experience, especially for complex applications with dynamic content.
Strong community and ecosystem: React has a large and active community, backed by Facebook's support. This vast ecosystem provides a wealth of resources, including libraries, tools, and third-party extensions. The strong community support also ensures regular updates, a wealth of learning resources, and a wide range of solutions for various development challenges.

Disadvantages

Steep learning curve for beginners: Despite its popularity, React can be challenging for beginners to grasp. Its component-based architecture, JSX syntax, and overall ecosystem can be overwhelming initially. Newcomers to React often need time to become comfortable with concepts like state management, lifecycle methods, and the use of props.
Rapid pace of development: The fast evolution of React and its ecosystem can be a double-edged sword. While regular updates introduce new features and improvements, they also mean developers need to continuously adapt and learn. This rapid change can lead to challenges in keeping projects up-to-date and may require frequent refactoring or rewriting of existing code to align with the latest best practices.

Vue

Vue.js, often simply referred to as Vue, has established itself as a highly adaptable and progressive JavaScript framework, primarily used for building user interfaces and single-page applications. Its gentle learning curve and incrementally adoptable architecture have made it a popular choice among developers, particularly for its ease of integration into existing projects.

Vue components are often structured with a template, script, and style section. This example uses the Vue 3 Composition API:

<template>
  <h1>Hello, {{ name }}!</h1>
</template>

<script>
import { ref } from 'vue';

export default {
  setup() {
    const name = ref('Vue User');
    return { name };
  }
};
</script>

<style>
h1 {
  color: blue;
}
</style>

Advantages

Intuitive and easy to learn: Vue's straightforward and clear documentation makes it accessible, especially for those new to JavaScript frameworks. Its template syntax and use of plain JavaScript make it easier to get started and become productive quickly.
Reactive and composable: Vue's core feature is its reactivity system, which ensures efficient updates and rendering. The framework's focus on component composition allows for building complex interfaces while keeping the codebase organized and maintainable.
Versatile ecosystem: Vue offers a rich ecosystem with solutions like Vuex for state management and Vue Router for routing, providing developers with a comprehensive toolkit for modern web development. The framework is also compatible with a variety of other libraries and tools, making it versatile for different project needs.

Disadvantages

Less market share and community size: Compared to giants like React, Vue has a smaller community and market share. While growing, this can sometimes mean fewer resources, third-party libraries, and extensions than more widely adopted frameworks.
Risk of over-flexibility: Vue's flexibility, while a strength, can also be a drawback. The freedom in structuring applications can lead to inconsistencies in large projects or among different developers, potentially affecting maintainability and scalability.

Angular

Angular, maintained by Google, is a robust and feature-rich framework designed for building scalable and sophisticated single-page applications. It's widely recognized for its ability to create efficient, large-scale enterprise-grade web applications, offering a comprehensive solution with a powerful suite of tools.

Angular components combine a TypeScript class with an HTML template, typically decorated with metadata.

import { Component } from '@angular/core';

@Component({
  selector: 'app-greeting',
  template: `<h1>Hello, {{ name }}!</h1>`,
  styleUrls: ['./greeting.component.css']
})

export class GreetingComponent {
  name = 'Angular User';
}

Advantages

Full-fledged framework: Angular is a complete framework that includes everything needed for front-end development, from a powerful templating engine to integrated state management, making it a one-stop solution for complex applications.
Advanced dependency injection: Angular's advanced dependency injection system is a key feature that facilitates greater efficiency in building and managing applications. It simplifies the process of supplying components with the data and services they need, leading to more modular, flexible, and easily testable code.
Consistent code structure: Angular enforces a consistent code structure and best practices through its opinionated nature. This uniformity is advantageous for team collaboration and when scaling up the application.

Disadvantages

Steep learning curve: Angular's extensive set of features and functionalities, while powerful, often presents a challenging learning curve, particularly for beginners or those accustomed to more straightforward frameworks.
Performance overhead: While powerful, Angular's complexity can lead to performance overhead in smaller projects. The framework is better suited for larger applications where the full feature set can be effectively utilized.

Svelte

Svelte is an innovative component-based framework that shifts much of the work to compile time, offering a unique approach to building interactive web interfaces. Unlike traditional frameworks that apply updates to the DOM in response to state changes, Svelte compiles applications to highly efficient vanilla JavaScript at build time, resulting in faster and more responsive applications.

Svelte components are written with HTML, JavaScript, and CSS in a single file, with reactive state updates.

<script>
  let name = 'Svelte User';
</script>

<h1>Hello, {name}!</h1>

<style>
  h1 {
    color: green;
  }
</style>

Advantages

Compile-time innovation: Svelte's unique compile-time approach leads to less boilerplate code and higher performance, as it generates optimized JavaScript code that directly updates the DOM.
Ease of learning and use: With a syntax closer to vanilla JavaScript and HTML, Svelte is straightforward to learn, particularly for those with basic JavaScript knowledge. This simplicity also makes it easier to start building applications quickly.
Minimal runtime overhead: Svelte applications do not carry the weight of a framework's runtime, resulting in smaller bundle sizes and faster load times, which is particularly advantageous for performance-sensitive applications.

Disadvantages

Smaller community and ecosystem: Svelte’s community, while active and growing, is smaller than those of more established frameworks like React or Angular. This can mean fewer resources, libraries, and community support compared to its larger counterparts.
Less mainstream adoption: Being relatively new and different, Svelte has not seen as widespread adoption in the industry as some other frameworks. This might pose challenges in finding experienced developers or integrating with certain existing technologies and tools.

Static site/Server-side generators

Static site generators have become a cornerstone of modern web development, offering a streamlined approach to building fast and secure websites. These tools pre-render pages at build time, creating static HTML files which results in blazing-fast load times and improved performance. With the growing need for speed, security, and scalability in web development, static site generators are increasingly becoming a popular choice for both developers and businesses.

In this section, we explore some of the leading static site generators that defined the 2023 web development landscape, and the ones we should keep on the radar for 2024.

An overview of Framework popularity trends.

It’s been an exciting year in the static site/server-side generation landscape, most notably for Astro, a relatively new contender that is taking the sector by storm. Hugo has maintained its lead, sitting over 15k stars ahead of Gatsby. Nuxt’s steady growth has enabled it to win popularity over Jekyll.

The popularity of these static site generators in 2023 sets the stage for further growth and innovation in 2024, as developers continue to prioritize speed, security, and scalability in web development on the server.

Astro

Astro is a modern static site generator that allows developers to build fast, content-focused websites using a component-based architecture. It stands out for its ability to deliver lightning-fast performance by only sending essential JavaScript, leveraging partial hydration.

Advantages

Optimized performance: Astro optimizes loading performance by serving mostly static HTML and hydrating components with JavaScript only as needed, making sites faster and more efficient.
Flexible component architecture: It supports various frontend frameworks within the same project, allowing developers to use the best tool for each specific component.
Enhanced developer experience: Astro offers a streamlined developer experience with intuitive syntax, fast build times, and a focus on delivering high-quality content.

Disadvantages

Relatively new in the market: Being a newer entrant in the static site generator ecosystem, Astro might not have as extensive a community or as many resources as more established tools.
Limited plugins and integrations: As it is still growing, Astro's ecosystem of plugins and integrations may be less developed compared to other static site generators, potentially limiting its out-of-the-box capabilities.

Nuxt

Nuxt.js is a powerful Vue.js framework that simplifies the development of universal or single-page Vue apps. Known for its versatility, Nuxt provides a streamlined development experience with its convention-over-configuration approach, offering a robust platform for both static site generation and server-side rendering.

Advantages

Automatic code splitting: Nuxt.js automatically splits code into manageable bundles, improving load times and performance, especially for complex applications with many components.
Easy to use with Vue.js: Leveraging Vue.js, Nuxt offers an enjoyable development experience with a familiar, component-based architecture and an intuitive setup, ideal for both newcomers and experienced Vue developers.
Server-side rendering and static site generation: Nuxt's flexibility allows developers to choose between generating a static site or using server-side rendering, making it adaptable to various project requirements.

Disadvantages

Learning curve for non-Vue developers: While Nuxt simplifies Vue.js development, those unfamiliar with Vue might face a learning curve in understanding its ecosystem and architecture.
Overhead for simple projects: For smaller, less complex projects, the extensive features and structure of Nuxt.js could introduce unnecessary overhead, making simpler static site generators a more efficient choice.

Hugo

Hugo is a fast and flexible static site generator, renowned for its incredible speed in building static websites and blogs. It stands out for its efficiency and ease of use, making it a popular choice for developers looking to quickly deploy high-performance sites.

Advantages

Exceptional build speed: Hugo's most notable feature is its blazing-fast build time, efficiently handling even large sites with thousands of pages.
Ease of use and setup: Hugo is simple to set up and operate, with a straightforward structure that's easy to understand, making it accessible for both beginners and experienced developers.
Robust content management: It offers robust content organization, with a strong emphasis on content management that allows for a high degree of customization and flexibility in content presentation.

Disadvantages

Limited plugin ecosystem: Compared to other static site generators, Hugo has a more limited range of plugins and themes, which might restrict some customization options.
Steep learning curve for advanced usage: While basic usage of Hugo is straightforward, leveraging its more advanced features requires a steeper learning curve, particularly for those not familiar with GoLang, the language in which Hugo is written.

Gatsby

Gatsby is a modern static site generator based on React, known for creating fast and optimized web applications. It integrates seamlessly with various data sources, making it a popular choice for building complex, data-driven sites with rich interactive features.

Advantages

Performance optimization: Gatsby automatically optimizes site performance, implementing best practices like code splitting and lazy loading images, leading to fast-loading, efficient websites.
Rich data integration: Thanks to its powerful data layer, Gatsby can pull data from multiple sources (CMSs, APIs, databases, etc.), allowing for flexible and dynamic site generation.
Robust plugin ecosystem: Gatsby boasts a large collection of plugins, enabling developers to easily add functionality and integrate with various tools and services.

Disadvantages

Complexity for beginners: Due to its reliance on React and GraphQL, Gatsby can be complex for beginners or for those unfamiliar with these technologies.
Build times: For very large sites, Gatsby's build times can be longer than some other static site generators, which might impact development and deployment efficiency.

Jekyll

Jekyll is one of the earliest and most established static site generators, widely used for creating simple, blog-aware websites. Built in Ruby, it integrates seamlessly with GitHub Pages, making it a preferred choice for personal, project, or documentation sites.

Advantages

Simple and easy to use: Jekyll's straightforward setup and minimalistic approach make it ideal for beginners or for projects that require simple static sites without the need for complex features.
Strong integration with GitHub Pages: Jekyll offers seamless integration with GitHub Pages, providing an easy pathway to deploy and host websites directly from GitHub repositories.
Rich theme ecosystem: It has a robust ecosystem of themes, allowing for quick and easy customization of websites without extensive development.

Disadvantages

Limited to simple use cases: Jekyll is less suited for complex applications as it lacks the advanced features and flexibility offered by newer static site generators.
Ruby dependency: The need to have Ruby installed can be a barrier for those not familiar with the Ruby environment, potentially limiting its accessibility to a wider audience.

What to choose?

Each tool, with its unique features and drawbacks, caters to different requirements and preferences. Understanding these is crucial in making an informed decision that aligns with the specific needs and goals of your project.

Whether it's the blazing-fast builds of Hugo, the robust data integration of Gatsby, the simplicity of Jekyll, the React-based architecture of Gatsby, or the sheer efficiency of Astro and Nuxt, each offers distinct advantages that are pivotal for certain types of projects. However, it's equally important to weigh these against their potential drawbacks, whether it’s complexity, build times, ecosystem limitations, or specific technology dependencies.

In the end, the choice depends on a balance of factors: the complexity of your project, the scalability you need, your team's familiarity with the underlying technologies, and the specific features you require. By carefully assessing each tool's capabilities and how they match your project's requirements, you can choose a tool that not only meets your current needs but also supports the future growth and evolution of your project.

Like you, I’m always curious and looking to learn. If I've overlooked a noteworthy tool or if you have any feedback to share, reach out on Twitter or LinkedIn.

JS Toolbox 2024: Runtime environments & package management

Ollie Bannister — Mon, 04 Mar 2024 04:06:24 +0000

Just as any durable structure needs a solid foundation, successful JavaScript projects rely heavily on starting with the right tools.

This post, the first in our JS Toolbox 2024 series, explores the core pillars of the JavaScript & TypeScript ecosystem: Runtime environments, package management, and development servers.

See part two, covering frameworks and static site generators, here.

In this post:

Runtime environments
- Node.js
- Deno
- Bun
Comparing JS runtimes
- Installation
- Performance, stability, and security
- Community
Package managers
- NPM
- Yarn
- pnpM
- Bun
What to choose

Runtime environments

In JavaScript development, runtimes are the engines that drive advanced, server-centric projects beyond the limitations of a user's browser. This independence is pivotal in modern web development, allowing for more sophisticated and versatile applications. The JavaScript runtime market is more dynamic than ever, with several contenders competing for the top spot. Node.js, the long-established leader in this space, now faces formidable competition from Deno and Bun.

Deno is the brainchild of Ryan Dahl, the original creator of Node.js. It represents a significant step forward in runtime technology, emphasizing security through fine-grained access controls and modern capabilities like native TypeScript support.

Bun has burst onto the scene, releasing version 1.0 in September 2023. Bun sets itself apart with exceptional speed, challenging the performance standards established by its predecessors. Bun's rapid execution capabilities, enabled by just-in-time (JIT) execution, make it a powerful alternative in the runtime environment space.

An overview of framework popularity trends.

The popularity of Node.js has continued to grow over 2023, and I anticipate this will continue into 2024. There has been a slight downtrend in growth trajectory, which I’d guess is due to the other tooling growing in market share.

Deno has seen substantial growth over 2023. If the current trend continues I anticipate Deno to overtake Node.js in popularity in 2024, though it’s worth mentioning that star-based popularity doesn’t reflect usage in the field. Without a doubt, Node.js will retain its position as the lead environment for production systems throughout 2024.

Bun has seen the largest growth in this category over the past year. I anticipate that Bun will find a steady foothold and continue its ascent, following the release of version 1.0. It’s early days for this new player, but comparing early stage growth to others in the category, it’s shaping up to be a high performer.

Node.js

Node.js, acclaimed as the leading web technology by StackOverflow developers, has been a significant player in the web development world since its inception in 2009. It revolutionized web development by enabling JavaScript for server-side scripting, thus allowing for the creation of complex, backend-driven applications.

Advantages

Asynchronous and event-driven: Node.js operates on an asynchronous, event-driven architecture, making it efficient for scalable network applications. This model allows Node.js to handle multiple operations concurrently without blocking the main thread.
Rich ecosystem: With a diverse and extensive range of tools, resources, and libraries available, Node.js offers developers an incredibly rich ecosystem, supporting a wide array of development needs.
Optimized for performance: Node.js is known for its low-latency handling of HTTP requests, which is optimal for web frameworks. It efficiently utilizes system resources, allowing for load balancing and the use of multiple cores through child processes and its cluster module.

Disadvantages

Learning curve for asynchronous programming: The non-blocking, asynchronous nature of Node.js can be challenging for developers accustomed to linear programming paradigms, leading to a steep learning curve.
Callback hell: While manageable, Node.js can lead to complex nested callbacks – often referred to as "callback hell" – which can make code difficult to read and maintain. However, this can be mitigated with modern features like async/await.

Deno

Deno represents a step forward in JavaScript and TypeScript runtimes, leveraging Google’s V8 engine and built in Rust for enhanced security and performance. Conceived by Ryan Dahl, the original creator of Node.js, Deno is positioned as a more secure and modern alternative, addressing some of the core issues found in Node.js, particularly around security.

Advantages

Enhanced security: Deno's secure-by-default approach requires explicit permissions for file, network, and environment access, reducing the risks associated with an all-access runtime.
Native TypeScript support: It offers first-class support for TypeScript and TSX, allowing developers to use TypeScript out of the box without additional transpiling steps.
Single executable compilation: Deno can compile entire applications into a single, self-contained executable, simplifying deployment and distribution processes.

Disadvantages

Young ecosystem: Being relatively new compared to Node.js, Deno’s ecosystem is still growing, which may temporarily limit the availability of third-party modules and tools.
Adoption barrier: For teams and projects deeply integrated with Node.js, transitioning to Deno can represent a significant change, posing challenges in terms of adoption and migration.

Bun

Bun emerges as a promising new contender in the JavaScript runtime space, positioning itself as a faster and more efficient alternative to Node.js. Developed using Zig and powered by JavaScriptCore, Bun is designed to deliver significantly quicker startup times and lower memory usage, making it an attractive option for modern web development.

Currently, Bun provides a limited, experimental native build for Windows with full support for Linux and macOS. Hopefully, over 2024 we see full support for Windows released.

Advantages

High performance: Bun's main draw is its performance, offering faster execution and lower resource usage compared to traditional runtimes, making it particularly suitable for high-efficiency requirements.
Integrated development tools: It comes with an integrated suite of tools, including a test runner, script runner, and a Node.js-compatible package manager, all optimized for speed and compatibility with Node.js projects.
Evolving ecosystem: Bun is continuously evolving, with a focus on enhancing Node.js compatibility and broadening its integration with various frameworks, signaling its potential as a versatile and adaptable solution for diverse development needs.

Disadvantages

Relative newness in the market: As a newer player, Bun's ecosystem is not as mature as Node.js, which might pose limitations in terms of available libraries and community support.
Compatibility challenges: While efforts are being made to improve compatibility with Node.js, there may still be challenges and growing pains in integrating Bun into existing Node.js-based projects or workflows.

Comparing JavaScript runtimes

Installation

Each JavaScript runtime has its unique installation process. Here's a brief overview of how to install Node.js, Deno, and Bun:

Node.js

Download: Visit the Node.js website and download the installer suitable for your operating system.
Run installer: Execute the downloaded file and follow the installation prompts. This process will install both Node.js and npm.
Verify installation: Open a terminal or command prompt and type node -v and npm -v to check the installed versions of Node.js and npm, respectively.

Managing different versions of Node.js has historically been a challenge for developers. To address this issue, tools like NVM (Node Version Manager) and NVM Windows have been developed, greatly simplifying the process of installing and switching between various Node.js versions.

Deno

Shell Command: You can install Deno using a simple shell command.
- Windows: irm https://deno.land/install.ps1 | iex
- Linux/macOS: curl -fsSL https://deno.land/x/install/install.sh | sh
Alternative methods: Other methods like downloading a binary from the Deno releases page are also available.
Verify installation: To ensure Deno is installed correctly, type deno --version in your terminal.

Bun

Shell Command: Similar to Deno, Bun can be installed using a shell command. For instance, on macOS, Linux, and WSL use the command curl https://bun.sh/install | bash.
Alternative methods: For detailed instructions or alternative methods, check the Bun installation guide.
Verify installation: After installation, run bun --version in your terminal to verify that Bun is correctly installed.

Performance, stability and security

In evaluating JavaScript runtimes, performance, stability, and security are the key factors to consider. Mayank Choubey's benchmark studies provide insightful comparisons among Node.js, Deno, and Bun:

I’d recommend giving the post a read if you’re interested in the specifics. Otherwise, I’ll do my best to summarize the results below.

Node.js

Historically, Node.js has been known for its efficient handling of asynchronous operations and has set a standard in server-side JavaScript performance.
In the benchmark, Node.js displayed solid performance, reflective of its maturity and optimization over the years. However, it didn't lead the pack in terms of raw speed.
As Node.js has been around for a long time and proven its reliability, it wins the category of stability.

Deno

Deno, being a relatively newer runtime, has shown promising improvements in performance, particularly in the context of security and TypeScript support.
The benchmark results for Deno were competitive, showcasing its capability to handle server requests efficiently, though it still trails slightly behind in raw processing speed compared to Bun.
Given its emphasis on security features like explicit permissions for file, network, and environment access, Deno excels in the category of security.

Bun

Bun made a significant impression with its performance in this benchmark. It leverages Zig and JavaScriptCore, which contributes to its faster startup times and lower memory usage.
In the "Hello World" server test, Bun outperformed both Node.js and Deno in terms of request handling speed, showcasing its potential as a high-performance JavaScript runtime.
With its significant speed improvements, Bun leads in the category of performance.

These results suggest that while Node.js remains a reliable and robust choice for many applications, Deno and Bun are catching up, offering competitive and sometimes superior performance metrics.

Bun, in particular, demonstrates remarkable speed, which could be a game-changer for performance-critical applications. However, it's important to consider other factors such as stability, community support, and feature completeness when choosing a runtime for your project.

Community

The community surrounding a JavaScript runtime is vital for its growth and evolution. It shapes development, provides support, and drives innovation. Let's briefly examine the community dynamics for Node.js, Deno, and Bun:

Node.js: Node.js has one of the largest, most diverse communities in software development, enriched by a wide array of libraries, tools, and resources. Its community actively contributes to its core and modules, bolstered by global events and forums for learning and networking.

Deno: Deno's community is rapidly growing, drawing developers with its modern and security-centric features. It's characterized by active involvement in the runtime’s development and a strong online presence, particularly on platforms like GitHub and Discord.

Bun: Although newer, Bun’s community is dynamic and quickly expanding. Early adopters are actively engaged in its development and performance enhancement, with lively discussions and feedback exchanges on online platforms.

Each of these communities, from Node.js’s well-established network to the emerging groups around Deno and Bun, plays a crucial role in the adoption and development of these runtimes. For developers, understanding the nuances of these communities can be key to leveraging the full potential of a chosen runtime.

Package managers

If you’ve ever worked on the front end of a modern web application or if you're a full-stack node engineer, you’ve likely used a package manager at some point. The package manager is responsible for managing the dependencies of your project, such as libraries, frameworks, and utilities.

NPM is the default package manager that comes pre-installed with Node.js. Yarn and PNPM compete to take NPMs spot as the package management tool of choice for developers working in the JavaScript ecosystem.

An overview of Framework popularity trends.

NPM

Node Package Manager or NPM for short, is the default and most dominant package manager for JavaScript projects. It comes pre-installed with Node.js, providing developers with immediate access to the npm registry, allowing them to install, share, and manage package dependencies right from the start of their project.

It was created in 2009 by Isaac Schlueter as a way to share and reuse code for Node.js projects. Since then, it has grown to become a huge repository of packages that can be used for both front-end and back-end development.

NPM consists of two main components:

NPM CLI (Command Line Interface): This tool is used by developers to install, update, and manage packages (libraries or modules) in their JavaScript projects. It interacts with npm’s online repository, allowing developers to add external packages to their projects easily.
NPM registry: An extensive online database of public and private JavaScript packages, the npm Registry is where developers can publish their packages, making them accessible to the wider JavaScript community. It's known for its vast collection of libraries, frameworks, and tools, contributing to the versatility and functionality of JavaScript projects.

This star graph doesn’t capture much in terms of the overall popularity of NPM CLI given that this tool comes pre-installed with Node.js. Knowing this, it’s worth also reviewing the overall download count of these packages.

NPM currently has 56,205,118,637 weekly downloads.

Woah, 56.2B! It’s safe to say NPM isn’t going anywhere. From the graphs we can see the steady incline in the overall popularity of this tool through 2023. I predict this growth will continue through 2024.

Yarn

Yarn is a well-established open-source package manager created in 2016 by Facebook, Google, Exponent, and Tilde. It was designed to address some of the issues and limitations of NPM, such as speed, correctness, security, and developer experience.

To improve these areas, Yarn incorporates a range of innovative features. These include workspaces for managing multiple packages within a single repository, offline caching for faster installs, parallel installations for improved speed, a hardened mode for enhanced security, and interactive commands for a more intuitive user interface. These features collectively contribute to Yarn’s robustness and efficiency.

It features a command-line interface that closely resembles NPM's, but with several enhancements and differences. It utilizes the same package.json file as NPM for defining project dependencies. Additionally, Yarn introduces the yarn.lock file, which precisely locks down the versions of dependencies, ensuring consistent installs across environments. Like NPM, Yarn also creates a node_modules folder where it installs and organizes the packages for your project.

Yarn currently has 4,396,069 weekly downloads.

Given that Yarn and pnpM require manual installs this does mean the download counts are un-comparable with NPM but it still gives us a glance at the overall trends.

In 2023, Yarn appears to have lost some of it’s growth trajectory but still remains the most popular alternative to NPM for package management.

pnpM

Performant NPM or pnpM for short, is another alternative package manager for JavaScript that was created in 2016 by Zoltan Kochan. It was designed to be faster, lighter, and more secure than both NPM and Yarn.

It excels in saving disk space and speeding up the installation process. Unlike npm, where each project stores separate copies of dependencies, pnpm stores them in a content-addressable store. This approach means if multiple projects use the same dependency, they share a single stored copy, significantly reducing disk usage. When updating dependencies, pnpm only adds changed files instead of duplicating the entire package.

The installation process in pnpm is streamlined into three stages: resolving dependencies, calculating the directory structure, and linking dependencies, making it faster than traditional methods. pnpm also creates a unique node_modules directory using symlinks for direct dependencies only, avoiding unnecessary access to indirect dependencies. This approach ensures a cleaner dependency structure, while still offering a traditional flat structure option through its node-linker setting for those who prefer it.

pnpM currently has 8,016,757 weekly downloads.

pnpM's popularity surged in 2023, and I foresee this upward trend extending into 2024, as an increasing number of developers recognize its resource efficiency and streamlined project setup.

Bun

As Bun comes with an npm-compatible package manager, I felt it was worth mentioning here.

I've covered Bun in the Runtime Environments section above.

What to choose?

Choosing the right tool for your project in 2024 depends on a variety of factors including your project's specific requirements, your team's familiarity with the technology, and the particular strengths of each tool. In the dynamic world of JavaScript development, having a clear understanding of these factors is crucial for making an informed decision.

For those prioritizing stability and a proven track record, Node.js remains a top recommendation. It's well-established, supported by a vast ecosystem, and continues to be a reliable choice for a wide range of applications. Node.js's maturity makes it a safe bet, especially for projects where long-term viability and extensive community support are essential.

On the other hand, if you're inclined towards experimenting with the latest advancements in the field and are operating in a Linux-based environment, Bun presents an exciting opportunity. It stands out for its impressive performance and is ideal for those looking to leverage the bleeding edge of JavaScript runtime technology. Bun’s rapid execution capabilities make it a compelling option for performance-driven projects.

When it comes to package management, pnpm is an excellent choice. Its efficient handling of dependencies and disk space makes it ideal for developers managing multiple projects or large dependencies. With its growing popularity and focus on performance, pnpM is well-suited for modern JavaScript development.

JavaScript tools in 2024 offer a massive range catered to different needs and preferences. Whether you opt for the stability of Node.js, the cutting-edge performance of Bun, or the efficient dependency management of pnpM, each tool brings unique strengths to the table. Carefully consider your project’s requirements and team’s expertise to make the best choice for your development journey in 2024.

Like you, I’m always curious and looking to learn. If I've overlooked a noteworthy tool or if you have any feedback to share, reach out on Twitter or LinkedIn.

The art of shipping and monitoring software with speed and confidence

Ollie Bannister — Fri, 12 Jun 2020 00:55:48 +0000

Software teams are under increasing pressure to ship code faster than ever before, but without the right workflow and tools in place, this can introduce unnecessary risk and headache. We wanted to share how to configure deployments, identify issues, and track performance gains using tools and process to get the best results and enable you to ship software with speed and confidence.

The tools we will be using in today’s example include Jenkins, Octopus, and Raygun. These are all available on a free trial so you can follow along.

(We partnered with Octopus to deliver a live demo of the whole process - you can watch the recording here).

Application lifecycle management (ALM)

The power of using these tools in conjunction with one another brings confidence to your deployments, giving you the time to focus on the things that matter… delivering great software to your customers.

Before we jump into the technicals, let’s take a quick look at the Application Lifecycle Management (ALM) process and see where Octopus and Raygun fit into this picture.

Octopus lives inside the Release, Deploy, and Operate stages of this lifecycle, giving you the ability to reliably release software in a scalable manner. Once your website or application reaches the operation stage Raygun is able to continue this flow right through to the Monitor and Planning stages.

Having the ability to collect metrics specific to each deployment helps us detect issues early, diagnose the cause of the problem and make a plan to resolve these issues by providing software teams with the information required and giving them confidence knowing no stone is left unturned.

Tooling

Here’s a brief overview of some of the tools that we will be using today to ensure we’re all up to speed.

Jenkins

Jenkins is a free, open source automation server that helps automate the non-human part of the software development lifecycle. This includes continuous integration and facilitating technical aspects of continuous delivery.

Jenkins is highly customizable with plugins to support almost any build pipeline. It supports version control tools and can execute a wide range of scripts.

Octopus

Octopus is an automated deployment and release management tool for managing releases, automating complex application deployments, and automating routine and emergency operations tasks.

Octopus offers a large range of build server integrations along with a vast variety of deployment target options. It can perform custom steps making it flexible enough to fit any deployment pipeline.

You can see a Sample Octopus application on samples.octopus.app/.

Raygun

Raygun is a software monitoring suite that gives you visibility into how users are really experiencing your software. It’s designed to provide you with actionable data you need to find and fix software issues and performance problems for your end-users.

Raygun has providers for the most popular programming languages and frameworks, and offers a large range of integrations so you can tailor your monitoring to suit your workflow.

Deploying, monitoring, and identifying a problematic deployment

Now that we’ve covered off the tooling we are going to be using, let jump into the fun stuff!

In this section, I’ll be introducing some issues into my demo website, running a build, deploying this out to our webserver then covering how we can detect and identify the problem quickly.

Imagine this flow as introducing an unintended issue to your codebase and overlooking the problem. Even with the best processes in place, we can still introduce new errors into production. Perhaps you don’t have sufficient tests or maybe a method or utility was poorly named, whatever the case I’m sure you’ve come across something similar before.

Healthy site

First things first, let’s take a look at our healthy demo website. On this site, we have a customer signup form. We are able to successfully create a new user and we don’t exhibit any issues inside our developer console.

This will serve as the base of our website state, now let’s go ahead and introduce some problems.

Adding bugs

On this site, I’ve added an errors.js script and set enableErrors = true; - this will serve as our chaos monkey, causing errors to be thrown when the user fills out the signup form. This is to emulate us accidentally introducing bugs into production.

Using git I’ve committed these changes and pushed up to our git repository, this will be picked up by our CI/CD pipeline in the next phase.

Jenkins

After our changes have been pushed up, Jenkins will automatically trigger a build for us. This is done using Jenkins “Poll SCM” feature and linking this to our Github repository. This will perform a check every 60 seconds, looking for any changes and if these are found it will kick off our initial job “01_Raygun_CI”.

The purpose of this job is to take our master branch, install any dependencies, run our task runner (Grunt) then generate a NuGet package for us to deploy using the generated distribution files.

Once the “01_Raygun_CI” job has been completed this will then trigger downstream projects that have been set up to deploy the built package out to a dev environment “02_Raygun_DeployToDev”, test environment “03_Raygun_DeployToTest” and then onto our production environment “04_Raygun_DeployToProd”.

Octopus

When Jenkins triggers our 03_Raygun_DeployToTest and 04_Raygun_DeployToProd jobs this will cause two deployments to be registered inside octopus under the specific build number. These kick-off a post-build action that will then deploy this package out to our Azure server.

For each deployment we have a 3 step process:

Step 1: Manual intervention

The manual intervention step allows us to implement an additional check before the deployment goes live. This is perfect in situations where we want to approve any changes before Octopus continues with the deployment.

In our demo flow today, we have configured this step to only run when we’re deploying out to our production environment. I have approved the deployment to emulate what might happen when a problem is overlooked.

Step 2: Deploy website

Our deployment step is what will upload our files to our webserver. Depending on the type of deployment (test or production) this will change the deployment target respectively. This makes maintenance easy as we only have to define a single process for across multiple deployment options.

Step 3: Register deployment

Finally, once our site has been approved and deployed we want to notify Raygun that a deployment has gone out and provide any associated metadata to do with the deployment.

To achieve this we just need to execute a simple PowerShell script that will fill our payload with some Octopus variables; like build number, user information and any deployment comments then post this off to our deployments endpoint, effectively timestamping our deployment.

More information on how to set this up can be found in our deployment documentation and you can see and install the Raygun register deployment Octopus community step in your Octopus instance.

Encounter problems

Now that the errors we introduced have been released out to production, we can revisit our demo website and try completing the signup process again.

As we filled out the form we triggered three different errors when filling and submitting the form, however, we were still able to proceed with the signup. This gave us the impression that everything was working as intended. This makes identifying issues with your software a difficult task without appropriate monitoring in place.

Identifying issues and bad deployments using Raygun

Making use of a monitoring tool such as Raygun makes it easy to detect issues early, diagnose the cause of the problem and come up with a plan to resolve these issues.

Looking at the Raygun Crash Reporting dashboard we can see that there was a deployment that correlated in a spike in errors.

We can drill into the specific deployment and see information such as new errors caused since the deployment, any regressions (issues that have reappeared) and also any issues that are still occurring. Making use of our Slack integration and email alerts ensures you can pick up on problems quickly and resolve them before they start impacting customers and costing you revenue.

Our latest build version introduced 3 new errors.

In addition to this, by making use of our top-level filters feature in conjunction with Real User Monitoring we can also gain visibility into specific performance metrics associated with each build version.

Comparing two versions against each other can also highlight any positive or negative impacts our changes have had on our website.

Our latest build version made things slower.

Resolving issues

Now that we’ve identified that our latest deployment wasn’t desirable, we can jump in and fix the problems. To do this I’m simply going to revert my previous change setting enableErrors = false;, committing and pushing up my changes and letting the build pipeline work it’s magic.

We’re fairly certain we resolved the errors captured by Raygun but often at times, it can be hard to be 100% sure. This is where our “Resolved in version” status comes in handy. Resolving these errors and setting the build version to match our latest deployment gives us peace of mind that if these errors arise after this has gone out we will be notified once again.

Rolling forwards vs Rolling back

One approach within Octopus to fix any bad deployments is to re-deploy the previous version. So, for instance, if you deployed version 2020.3.16, which caused issues, you could re-deploy the previous version and be back stable while you figure out what the problem is with your new release so you can then roll forward to a new version with the required features.

Rolling back doesn't scale to systems that have a data layer, which is what almost all applications have in one way or another. In this approach, you probably have about 1-2 minutes during which you can rollback, and not many bugs, if any, are caught in Production in that time frame, and if you're past this point, it is the point of no return for that release. Your only option is to roll forward with a new version that contains the bug fix, which is affecting your Production systems as you're looking at a considerable amount of data loss.

One approach to help mitigate these issues is blue-green deployments. Blue-green deployments are a pattern whereby we reduce downtime during production deployments by having two production environments ("blue" and "green"). You could deploy different releases to different environments, and test this before switching them over at the Load Balancer level.

Blue/Green is not without its issues. With Blue/Green, you need to keep the data layer consistent for both releases and requires a lot of discipline from database developers. You can use Turquoise Database Development to stick to predefined development rules that will help keep your data layer consistent and flexible enough to deal with different releases.

The real issue here is that you haven't tested or monitored enough. The recommendation is to test and monitor more in your Dev, Test & Pre-Production to catch any of these bugs before they become bugs affecting mission-critical applications. The issue is that data layers are almost impossible to roll back to unless in a situation where there is a system-wide failure and it becomes a necessity.

The one key take away from this section is to ensure that you have a fully automated CI/CD process with monitoring to allow you to get fast feedback on your application and rollback or roll forward based on your and your applications requirement.

Wrapping up

Setting up the correct tooling to build, deploy, and monitor your website plays a key role in ensuring your users are left feeling confident in your service. Hopefully today I’ve demonstrated that using Jenkins, Octopus and Raygun in conjunction with one another can make this process easy for you.

Resources mentioned

Live recording

Link

Blog posts

Inside look at Raygun’s CI/CD workflow

Documentation

Octopus documentation

Raygun documentation

Videos

Octopus
Raygun

Tooling

Free Starter edition for Octopus Server

14-day free Raygun Platform trial

Previous webinar recordings and upcoming webinars

Octopus
Raygun

Samples

Confidently ship code using Continuous Integration (CI) and Continuous Delivery (CD)

Ollie Bannister — Fri, 12 Jun 2020 00:18:57 +0000

We all want to ship code on a daily basis.

As developers, we want the faster feedback cycle that comes with continuous delivery, and we know it's better for our team and product in the long run. So why do so many development teams still have such long release cycles?

While long release cycles are tried and true and can be seen as more stable, teams sacrifice much more than faster feedback cycles.

Break the habit of slow releases

When a company is in its infancy, it needs to establish many processes, one of them being the release cycle that it wishes to implement. Although this process can mature over time and can be improved upon, more often than not, it usually remains the same as new updates are released. Years can go by, and release cycles stay the same.

Historically longer release cycles have been the status quo—teams that follow waterfall methodology will be familiar with this. But, with the rise of Agile practices, more software teams are breaking away from longer cycles and replacing them with shorter, incremental alternatives.

Longer release cycles do have their strengths; they are well-tested, predictable, and stable. But these benefits come at a cost. With longer release cycles, teams are able to release large, press-worthy updates that can leave big impressions on customers, potential customers and competitors. However, many startups are able to out-gun their larger competitors by adopting shorter release cycles, which enables them to move rapidly, react to change, and focus on innovation.

By choosing not to adapt and leaving your longer release cycles to stagnate, you are giving your competition the opportunity to get the upper hand.

It’s time to break the habit using continuous delivery.

Benefits of continuous delivery

Continuous delivery comes with many benefits for all areas of your organization; as a software developer I have listed my top five that make the biggest impact on a daily basis below:

1. Shorter issue resolution times

When writing software, there’s always a risk that the new feature or piece of functionality you’ve created will introduce bugs to your application or website. Shorter release cycles, make it much easier for you to react and fix problems when they arise.

2. Focus on innovation

Shorter release cycles also enable you to release new features to customers as soon as they are ready. Depending on your method of delivery, this means that you can go from the conceptual stage of an idea to delivery within a matter of weeks.

3. Market responsiveness

Market conditions change constantly. Whether you’ve just discovered a new product is losing money, or that more customers are visiting your site from smartphones than laptops, it’s much easier to make a quick change if you are already practicing continuous delivery.

4. Faster feedback

Releasing new features early and often means you get more frequent feedback, giving you the ability to iterate and learn from your customers. This ensures you’re prioritizing the right kind of work that will make your customers happy.

5. Developer QOL

Continuous delivery means the responsibility of deployment is spread widely around the organization. Releasing smaller, incremental deployments to your software helps alleviate the stress that can come from one or two people having to deploy a massive piece of work.

Best of all, a successful release becomes a shared success amongst stakeholders—one you can all celebrate together.

Here at Raygun, we celebrated the release of our new Application Performance Monitoring tool with a Pizza party! It was a great opportunity to congratulate everyone involved in the project and, like, who doesn’t love Pizza?! 🍕

Raygun’s CI/CD workflow

Creating a CI/CD pipeline requires the development, test, and operations teams to work together to create a cohesive release process—and the key is automation.

At Raygun, we use a range of different tools and processes to ensure a smooth deployment and minimize impact while also being able to move rapidly and deploy often.

By using this process, we have been able to make over 365 releases in the past year to a range of different areas of our product. That’s one deployment every day! The below graphic shows our workflow.

Where Raygun's process starts

Issue and project tracking

Firstly, before we can start deploying anything, we need to know what pieces of work we are going to be completing. Having an issue tracking tool makes it easy to create, manage and prioritize items of work to give you a complete overview of the current roadmap and what's coming up.

At Raygun we use Atlassian's Jira tool — this provides the level of granularity that we require and makes splitting up pieces of work a breeze.

There are other great solutions out there, so it’s worth checking them out for yourself and choosing one that best suits your needs.

Possible alternatives (there are many others):

Development

Once we know what we are going to be working on, we then need an appropriate IDE for the work to be completed. Depending on what languages & frameworks your project uses the requirements needed from the IDE can differ.

The choice of IDE is down to developer preference, but I choose to use Rider when working in-app and VS Code when working on other areas of our product.

Version control

When working in a team, having good source control is a must. This enables teams to view historical changes made to the codebase, identifies who made those changes and enables developers the ability to work in parallel on the same files.

Git has become the major player in the version control space. As for hosting, there are many version control hosting solutions out there, but we use GitHub.

When we’re ready for our work to be merged and deployed we push up our code to GitHub and create a pull request. This then gets peer reviewed and once approved, merged.

Possible alternatives:

Build management

Build management is the first part of the CI/CD pipeline where we can introduce automation. In this step of the process, we can set up triggers to initiate a build when someone pushes code up to the GitHub repository. You can customize your build script to execute the required tasks to build your application and run a variety of integration tests on the updated build.

Once the build has completed and all the tests have finished running, we will now have a passed or failed build. If desired you can then have it automatically post into a notification service like Slack.

Running this every time someone pushes code brings two main benefits:

It’s easy to know if you have unintentionally impacted another part of the application causing unexpected or broken behavior.
You always have a build ready for deployment to either test or release without needing to do anything.

We use TeamCity as our build management server and write our tests using NUnit and Jest.

Possible alternatives to TeamCity:

Alternatives for testing tools depends on your technology stack and requirements.

Deployment

Gone are the days of manual deployment (well, to a degree). Automating your deployment process can severely decrease the complexity of making a release and ensures that certain processes are followed precisely. By using a tool such as Octopus Deploy, we are able to create releases at the click of a button.

We are able to log information around each release, view the dashboard of what releases have been deployed to which environments and historical deployment information for each project you have set up.

The best part is—if you needed to roll back to a previous stable release for whatever reason it's as simple as choosing one of the releases from the history and hitting deploy.

It’s not all machines, we want humans involved too.

We intentionally remove some of the automation from this part of the process so that we can mitigate some of the risks that come with the continuous delivery model.

We’ve automated everything up to the point of hitting the deploy button, this is because we want a human to be in control of this. By requiring the developer responsible for the changes to deploy out their work brings many benefits to the deployment process.

Because they are more involved in the process, they get to see their changes go live and get to be the first one to test it out in beta or production environments.

Possible alternatives to Octopus:

Alerts

After deployment, it’s often hard to gain visibility on any issues your users come across. Using an error monitoring tool such as Raygun you can pick up on these problems in real-time and fix them before they impact any further users.

Implementing this with our Slack integration you can be assured that bad deployments are caught early and resolved.

We have many other great integrations for a variety of different tools that I will cover in the next section.

Resolution

Now that the update has been shipped we can now close off the work item by going back into our issue management tool and updating the status to resolved.

It’s worth notifying stakeholders that this has been completed before looking at the backlog and starting on the next piece of work.

Success

Now that the work has been released it’s worth celebrating a little!

Announcing new features to customers via the what’s new section or product newsletters is a great place to start. You can then move on to doing things like NPS surveys and talking to customers through messaging tools such as Intercom.

Internal celebrations are also a great way to educate your colleagues about new features in your product. Here at Raygun, every week we get the time to demo things we have released to the rest of the team and congratulate one another.

Feedback

Feedback helps to close the loop on your CI/CD pipeline, and although you might not be involved with the customer feedback side, your product team will be listening and informing the product direction. While collecting feedback using tools like Intercom and AskNicely for NPS—we also use our knowledge that only 1% of customers will report issues, which is why we monitor everything for problems using crash reporting.

Integrations

If you’re looking at setting up your CD/CI pipeline, and you’re interested in adding an error tracking tool like Raygun, we have a range of integrations for a variety of different tools so Raygun can fit seamlessly into most tool stacks.

Our most popular integrations include:

Team collaboration

Slack
Campfire
Flowdock

For sending error notification messages to notify team members

Version control

GitHub
Bitbucket
GitLab

Connect Raygun error groups to issues, resolve Raygun errors from closing issues and view lines of code that caused certain errors to occur inside Raygun.

Issue and project tracking

Jira
Azure DevOps
Trello
Zendesk
FogBugz
Lighthouse
Assembla
YouTrack

Create new issues from inside Raygun and link errors groups to work items inside your issue tracking tool.

Service bus

Azure service bus
Amazon SQS

Configure your own custom service pipeline with Raygun integration.

Incident response

PagerDuty
VictorOps

On-call support for Raygun error rate monitoring.

But that’s not all, you can check our full list of integrations here.

Wrap up

Continuous integration and continuous deployment is modern development practice, and it’s worth the effort to implement. Raygun not only helps strengthen the CI/CD process, it also provides your team with visibility into software errors anywhere in the SDLC, including production. See how it works here.

Common front-end mistakes that cause layout problems

Ollie Bannister — Thu, 11 Jun 2020 22:50:57 +0000

Successful online services like marketing and e-commerce websites rely on exceptional customer experiences. As a front-end developer, it’s your job to create flawless layouts, so your customer can complete necessary actions (like submit a sign-up form or purchase a Christmas gift). As you probably know, layout issues cause the mistrust of your website, so you’ll want to fix them up ASAP.

These are some of the most common front-end mistakes that cause layout problems on websites, using examples from our website, Raygun.com. (In a test environment, of course!)

1. Stylesheet 404

When experiencing rendering issues, checking to see if the stylesheet is even loading is the first things I look for. By opening your developer tools in the browser and taking a look at the console, you will quickly be able to see if there has been an error.

My developer tools are showing an error on the homepage

In the example above, without the stylesheet loading, the browser doesn’t know how the page should be displayed. Rendering issues usually result in a long list of elements running down the left of the page with nothing but default styles being applied:

The result: the browser doesn’t know how to apply the styles creating a long list down the left of the page

2. Style specificity

Sometimes we generate more general styles that will be applied to multiple elements. General styles work well in some cases. However, if you don’t fully understand the order of precedence, styles can be accidentally applied to unintended elements.

Here, general styles have been applied to unintended elements

Not being specific with your styles can cause things to look broken, and if the selector is broad enough, it can even break styles across the whole site.

Layout issues in the header make it impossible to navigate around the website

3. Missing media queries

Media queries are used to manipulate element styles depending on a set of conditions. The most common example of this would be the (max-width: ) or (min-width: ) condition to change the way the page is rendered on different screen sizes.

If you are missing a media query or don’t have any to begin with this can make your webpage look broken on screens different to that it was created on.

Below I compare a desktop view of the Raygun homepage compared to what it would look like on mobile without any media queries to change the styles.

Desktop:

Without any media queries to change the styles, the desktop version looks fine

Mobile:

Mobile, however, is a different story

4. Incorrect stack order

When building more complex components such as a navigation you often need to overlap various elements to achieve the desired effect. These elements are layered by applying a z-index property to them to instruct the browser where elements sit in the hierarchy.

If the z-index property for an element is wrong, it can easily make a page feel broken causing users to leave your website.

Below I’ve created an example of what can happen if the z-index property is set incorrectly.

Here, the z-index is set incorrectly and is causing stacking

5. Incorrect field validation

Forms are often used to collect data from your users and are often crucial steps in the user journey. Areas like the signup page or checkout often need to validate the user input to make sure that it matches what’s expected. This validation is often a good thing. However, if the validation rules are incorrect it can lead to users getting unnecessary error messages and can even block users from completing the signup or order. We talk more about how this can affect your business here.

Here, my inputs are correct, but I can’t submit the form – frustrating!

Without the data matching the validation rules, users are usually blocked from submitting the form. There’s now a high chance that the user is going to abandon ship and leave your website.

6. Browser incompatibility

Choosing which browsers to support is often a challenge. In the ideal world, you would have support for all browsers and versions. But, in doing this, you would also have to make a trade-off with which browser features you can use around your website. If your users are using an old version of IE, for example, and you don’t take this into account when creating the site, things could look broken causing people to leave. Here’s what happened when we applied it to Raygun’s homepage.

In IE9, the Raygun homepage looks broken

Understanding your users is critical in choosing which browsers to support. Luckily if you use a tool like Raygun, you can get insight into both the browser and version. This makes choosing which browsers to support much easier.

Real User Monitoring shows Browser performance

7. Cache busting

Caching assets on your website is always a good idea to help improve load times, and this includes your styles. Don’t forget to update the CSS cache buster after updating your styles, or you will cause many display issues. (Unless you have already automated the process.)

This is because the browser thinks it already has the file in its cache so doesn’t need to re-fetch it. Depending on the updates you made to your styles, this can have a big impact on your website.

An easy way to add a cache buster to style links is to add ?_=1512598593150 to the href on your tag, swapping out the UNIX epoch value with the current one.

8. Native resolution

When using responsive values on images, you need to make sure that you also set the max width and height of the image. This way, the image will never surpass the native resolution. If you stretch an image past its original size, the best case scenario is that you will start to see pixelation. Worst case scenario, you could break it’s aspect ratio depending on the styles applied.

SURPASSING AN IMAGES NATIVE RESOLUTION CAN CAUSE PIXELATION, WHICH IS EASILY MISSED

Surpassing native resolution is probably one of the most common front-end mistakes because it doesn’t look too bad and is easily missed, but fix it up, and it will make your website look professional and polished.

Avoid these front-end mistakes for happier customers

These are just a few of the common front-end mistakes I’ve come across, but hopefully, they provided a good start to detecting and solving them in your applications. Do you have a common issue that you wanted to share? Let me know in the comments!

Thanks for reading! – Ollie

Sass tutorial: A first look at writing more maintainable styles

Ollie Bannister — Thu, 11 Jun 2020 21:22:15 +0000

Learning how to use HTML, CSS and the basics of JavaScript can be fairly simple. Almost anyone can produce a small, functional website providing you know how to use Google and have some patience.

However, if you bump the scale of that website up about 10x, add in another developer and create multiple variants – each with their own unique functionality and styles – your code is going to become a hassle to maintain and new hires are going to have a nightmare trying to figure out your codebase.

Writing more maintainable code is just the first step. Get the ultimate guide to JavaScript error monitoring here.

Using a CSS preprocessor like Sass (Syntactically awesome style sheets) effectively will help modularize your CSS code making it more maintainable, understandable and reusable. Sass consists of two syntaxes, the original syntax called “the indented syntax” and the newer “SCSS”. The examples I give in this post use the SCSS syntax as this is my personal preference, but you can use either. In this Sass tutorial, we’ll walk through the setup process and what you need to know for writing more maintainable and reusable code.

Setting up Sass in your new project

Firstly, before we dive into writing any Sass, make sure you have checked out the Sass documentation on installation and the Sass basics section to get the Sass compiler working on your system.

Once you have Sass successfully set up for your project, we can now start setting up the file structure for our stylesheet partials and master files.

I like to create a sass/ directory to hold all our precompiled code and a css/ directory to hold all the output. For easier maintenance on large projects, I would recommend creating extra directories within the sass/ directory to hold different types of partials. As a basic example, I will create the following folder structure:

As you can see in my example colors, components, layout, and typography have been separated into their own directories. This enables us and anyone else digging around in our codebase to easily find files later on.

Sass variables and style modularization

Adding Sass to your project brings many advantages. One of the biggest is the ability to set and use variables. When it comes to writing maintainable styles the use of variables is a no-brainer. To create a variable we just need to add a $ before the variable name then just assign the variable a value.

As an example, I’ll create a _colors.scss stylesheet partial within our sass/colors/ directory that will hold all our color values. Note that I prefixed the partial with an underscore. This lets Sass know that the file is only a partial file and that it should not be generated into a CSS file:

// _colors.scss

$white: #FFFFFF;
$black: #000000;

$red: #C62828;
$blue: #3b88ad;
$green: #2E7D32;
$yellow: #F9A825;
$grey: #424242;
// ...

Now that we have the _colors partial we are going to need to import it to the master screen.scss file found in the sass/ directory. Sass offers the ability to reference other stylesheet partials using the @import keyword, so, using this, I’ll import the _colors.scss into screen.scss like so:

// screen.scss

// Colors
@import "colors/_colors.scss";

Now any style partial added to screen.scss will have visibility of the color variables. I’ll go ahead and create a navigation partial inside of components/ , add it to screen.scss and use a color variable as one of the properties:

// screen.scss

// Colors
@import "colors/_colors.scss";

// Components
@import "components/_navigation.scss";

// _navigation.scss

.navigation {
  position: fixed;
  top: 0;
  width: 100%;
  height: 50px;
  color: $white;
  background: $grey;
  box-shadow: 0 0 5px rgba($grey, 0.8);
  text-align: center;
  line-height: 50px;

  ul {
    display: inline-flex;
    margin: 0;

    li {
      list-style: none;
      padding: 0 50px;
      height: 100%;

      &:hover {
        background: rgba($blue, 0.5);
      }
    }
  }
}

As you can see .navigation now has a background color of $grey and a text color of $white. This will match the value we assigned in the _colors.scss partial compiling to background: #424242; color: #FFFFFF; after running it through the Sass precompiler:

Now we have this all set up, say we wanted to change the color of $blue to a new color, all we need to change is the value in _colors.scss and recompile the Sass. The new value will be assigned to all $blue references:

Mixins and control directives

When it comes to writing maintainable styles it’s always a good idea to keep your code as DRY as possible. Utilizing Sass mixins is a great way to minimize the amount of code you write and helps make it easier for others to read.

Mixins are essentially blocks of code that let you group together style declarations that we can then reuse throughout our site. Mixins can be used in various ways. I would recommend checking out the docs for a more comprehensive look at these.

Another great feature of Sass is the ability to generate styles by using control directives. These include @if, @for, @each and @while. If you have used another programming language in the past you may already be familiar with these but they are basically a way to generate multiple styles based on a given expression.

Let’s create some font-size classes using the @each control directive…

Firstly we need to create a _font-sizes.scss partial and import it to screen.scss

// screen.scss

// Colors
@import "colors/_colors.scss";

// Font sizes
@import "typography/_font-sizes.scss";

// Components
@import "components/_navigation.scss";

Now specify all the font sizes we want to use, as a variable in _font-sizes.scss. We can then use this variable to create classes for each value using @each.

// _font-sizes.scss

$font-sizes: 8px 10px 12px 14px 18px 24px;

@each $font-size in $font-sizes {
  .font-size-#{$font-size} {
    font-size: $font-size;
  }
}

After the Sass has been run through the precompiler you will see that we now have a class for each value in your output file.

/* style.css */

.font-size-8px {
  font-size: 8px;
}

.font-size-10px {
  font-size: 10px;
}

.font-size-12px {
  font-size: 12px;
}

.font-size-14px {
  font-size: 14px;
}

.font-size-18px {
  font-size: 18px;
}

.font-size-24px {
  font-size: 24px;
}

Now that our font sizes are handled by the $font-sizes variable to add, delete or update a value we just need to edit the variable and Sass will update all the styles for us.

@each is great when you have a set of values you wish to create styles from. However when dealing with a range of values @for is better suited.

To demonstrate the @for control directive let’s create a simple grid layout. I have already created a _grid.Scss partial in the layout directory and included it in screen.scss. Now let’s add a $grid-columns variable, this will be equal to the number of columns you want the grid to have. Using @for in conjunction with $grid-columns will generate all the grid classes that we need.

// _grid.scss

$grid-columns: 12; // Will be a 12 column grid

.grid-column { // Default grid-column styles
  position: relative;
  float: left;
  width: 100%;
}

@mixin grid-classes($column) {
  $width: (100% / $grid-columns) * $column;

  .grid-column-#{$column} {
    width: $width;
  }

  .grid-column-offest-#{$column} {
    margin-left: $width;
  }
}

@for $column from 0 through $grid-columns {
  @include grid-classes($column);
}

As you can see from the example we have 12 grid columns, some default styles for .grid-column class, a mixin to create the grid-classes and the @for control directive. The @for loop will run from 0 through to whatever you set your $grid-columns variable as calling the grid-classes() mixin each time with the $column number being passed into it.

Then the mixin will create the .grid-column-{number} and also .grid-column-offset-{number} classes for us. After running this code through the precompiler this is what it produced:

/* style.css */

.grid-column {
  position: relative;
  float: left;
  width: 100%;
}

.grid-column-0 {
  width: 0%;
}

.grid-column-offest-0 {
  margin-left: 0%;
}

.grid-column-1 {
  width: 8.33333%;
}

.grid-column-offest-1 {
  margin-left: 8.33333%;
}

.grid-column-2 {
  width: 16.66667%;
}

.grid-column-offest-2 {
  margin-left: 16.66667%;
}

.grid-column-3 {
  width: 25%;
}

.grid-column-offest-3 {
  margin-left: 25%;
}
/* ...... */

We can now use these classes in our markup and will work as expected:

As we have automated the generation of our custom grid, say we wanted to change from 12 columns to 24. We can easily just update the $grid-column value and recompile our SASS code and everything else is done for us!

Wrapping up this Sass tutorial

Writing maintainable styles can easily be accomplished by using Sass, however, this is not the only language out there to do this. Other languages like LESS and Stylus offer similar features and I would recommend checking them out to find the preprocessor that works for you. If you want to see a full list of all CSS preprocessors Slant’s page has a list of all options with reviews on each one.

Thanks for reading, and hope you enjoyed this Sass tutorial on writing more maintainable and reusable code.

(Source code is available on GitHub.)