DEV Community: Allan MacGregor 🇨🇦

Advanced Dependency Injection in Elixir with Rewire

Allan MacGregor 🇨🇦 — Tue, 25 Jun 2024 12:00:00 +0000

In our last post, we explored how Dependency Injection (DI) is a powerful design pattern that can improve our ExUnit tests.

In this article, we will dive deeper into the topic of DI in Elixir, focusing on the Rewire library for Elixir projects.

We will cover Rewire's core concepts, how to get started with it, and practical examples. We will also see how to use Rewire alongside Mox.

Let's get started!

Introduction to Rewire

One of the challenges we faced in our previous article was the lack of a structured way to define and inject dependencies into our modules. We had to manually define our mocks for testing.

This is where Rewire and Mox come into play:

Rewire provides a more structured and flexible way to implement DI in Elixir projects.
Mox is a library that allows us to define mocks for our tests.

Combining these two tools can significantly improve the testability and modularity of our Elixir applications.

Let's get started by setting up a sample project that leverages both libraries.

Why Use Rewire and Mox for Elixir?

To recap part one of the series, we discussed the benefits of DI for testability and modularity. We saw how we can use pass-in dependencies via function parameters:

We have the EmailScanner module that relies on a SpamFilterService to check if an email is spam or not:

defmodule EmailScanner do
  def scan_email(spam_filter_service, email) do
    spam_filter_service.check_spam(email)
  end
end

We have the SpamFilterService module that implements the spam checking logic:

defmodule SpamFilterService do
  def check_spam(email_content) do
    String.contains?(email_content, "spam")
  end
end

We also have a MockSpamFilterService module that implements the SpamFilterService behaviour for testing purposes:

defmodule MockSpamFilterService do
  def check_spam(_email), do: false
end

Finally, we have a test that uses the MockSpamFilterService to test the EmailScanner module:

defmodule EmailScannerTest do
  use ExUnit.Case

  test "scan_email with non-spam email returns false" do
    non_spam_email = %Email{content: "Hello, world!"}
    assert false == EmailScanner.scan_email(MockSpamFilterService, non_spam_email)
  end
end

In Elixir, modules are stateless, so the primary way to pass dependencies to a module is via function parameters. While Elixir modules can have attributes, these are used for compile-time information and metadata, not for holding runtime state.

Take the EmailScanner module, for example. We have to pass the SpamFilterService as a parameter to the scan_email function. This is unnecessary, as the only reason to have this function parameter is to make the module testable.

Additionally, it creates a few problems with code readability and navigation:

Because the module is expecting SpamFilterService as a parameter, we can't easily see what the module depends on.
The compiler can't catch issues with the module implementation, because we can pass any module that implements the SpamFilterService behaviour.

This approach might work well for small projects, but as our project grows, we might find ourselves repeating the same pattern over and over again. With Rewire, we don't have to worry about these issues. We can just focus on writing clean and maintainable code while keeping any testing concerns, mocks, and stubs in our test files.

Getting Started with Rewire and Mox in Your Elixir Project

Let's now dive into using Rewire and Mox in practice.

Step 1: Create a New Elixir Project

Before incorporating Rewire and Mox, create a new Elixir project:

mix new email_scanner

This command generates a new Elixir project named email_scanner, including a supervision tree structure.

Step 2: Add Dependencies

To use Rewire and Mox, you need to add them to your project's dependencies. Update your mix.exs file as follows:

defp deps do
  [
    {:rewire, "~> 1.0", only: :test},
    {:mox, "~> 1.0", only: :test}
  ]
end

After updating the dependencies, run mix deps.get in your terminal to fetch and install them.

Next, let's define our two primary modules:

defmodule EmailScanner do
  def filter_email(email) do
    email
    |> mark_as_important()
    |> SpamFilterService.check_spam()
  end

  defp mark_as_important(email) do
    important_senders = ["boss@example.com", "hr@example.com"]

    updated_email =
      if Enum.any?(important_senders, fn sender -> sender == email.sender end) do
        %{email | important: true}
      else
        email
      end

    updated_email
  end
end

We are making our code example a bit more realistic. The filter_email function marks emails from important senders as important and checks if the email is spam using the SpamFilterService module.

Next, we'll define the SpamFilterService module:

defmodule SpamFilterService do
  def check_spam(email_content) do
    String.contains?(email_content, "spam")
  end
end

Let's create a basic test for the EmailScanner module:

defmodule EmailScannerTest do
  use ExUnit.Case

  describe "filter_email/2" do
    test "marks email as important from specific sender and checks for spam" do
      important_sender_email = %{sender: "boss@example.com", content: "Please review the attached report.", important: false}
      non_important_sender_email = %{sender: "random@example.com", content: "Check out these deals!", important: false}

      # Filtering emails sent from the important sender
      assert %{important: true, is_spam: false} = EmailScanner.filter_email(important_sender_email)

      # Filtering emails sent from a non-important sender
      assert %{important: false, is_spam: false} = EmailScanner.filter_email(non_important_sender_email)
    end
  end
end

In the above code, the EmailScanner module relies on the SpamFilterService module to check if an email is spam or not. However, we can't test the EmailScanner module without also testing the SpamFilterService module, which is not ideal.

We need to mock the SpamFilterService module so that we can test the EmailScanner module in isolation.

Step 3: Configuring Mox

Mox requires a bit of setup in your test configuration. Open or create a test/test_helper.exs file and add the following line to define a mock based on a protocol or behaviour your project uses:

ExUnit.start()
Mox.defmock(EmailScanner.SpamFilterServiceMock, for: EmailScanner.SpamFilterService)

Mox makes it easy for us to generate mocks based on behaviours or protocols, which is essential for testing modules that rely on these abstractions.

Once our mock is defined, we can use it in our tests instead of the real implementation. With Rewire, we can inject these mocks into our modules without relying on function parameters.

Core Concepts of Rewire

Rewire simplifies the DI process in Elixir by providing a macro-based approach to define and inject dependencies. It fits seamlessly within Elixir’s ecosystem, promoting clean and maintainable code.

Dependency Injection with Rewire in the `EmailScanner` Module

Let’s implement the EmailScanner module, which relies on a SpamFilterService to check if an email is spam or not. Using Rewire, we can easily inject this dependency. Take a look at the following code:

defmodule EmailScanner do
  def filter_email(email) do
    email
    |> mark_as_important()
    |> SpamFilterService.check_spam()
  end

  defp mark_as_important(email) do
    important_senders = ["boss@example.com", "hr@example.com"]

    updated_email =
      if Enum.any?(important_senders, fn sender -> sender == email.sender end) do
        %{email | important: true}
      else
        email
      end

    updated_email
  end
end

Mocking with Mox for Testing

To test the EmailScanner filter function, we can use Mox to mock the SpamFilterService module:

defmodule EmailScannerTest do
  use ExUnit.Case

  import Rewire
  import Mox

  rewire EmailScanner, SpamFilterService: SpamFilterServiceMock

  # Ensure mocks are verified after each test
  setup :verify_on_exit!

  describe "filter_email/2" do
    test "marks email as important from specific sender and checks for spam" do
      important_sender_email = %{sender: "boss@example.com", content: "Please review the attached report.", important: false}
      non_important_sender_email = %{sender: "random@example.com", content: "Check out these deals!", important: false}

      # Stub the SpamFilter service to return false for all emails
      stub(SpamFilterServiceMock, :check_spam, fn _email -> :false end)

      # Filtering emails sent from the important sender
      assert %{important: true, is_spam: false} = EmailScanner.filter_email(important_sender_email)

      # Filtering emails sent from a non-important sender
      assert %{important: false, is_spam: false} = EmailScanner.filter_email(non_important_sender_email)
    end
  end
end

Let's break down what is happening in the above test:

rewire EmailScanner, SpamFilterService: SpamFilterServiceMock: This line uses Rewire to replace the SpamFilterService dependency in the EmailScanner module with SpamFilterServiceMock for the scope of this test module. It effectively changes the behavior of EmailScanner to use the mock service instead of its real dependency.
setup :verify_on_exit!: A setup callback that ensures all expectations on mocks (defined using Mox) are met by the end of each test, or else the test fails. This is crucial for verifying that the mocked functions are called as expected.
Then, we define a test case that:
- Creates two email maps, one from an "important" sender and one from a "non-important" sender.
- Uses stub to define the behavior of the SpamFilterServiceMock, so check_spam/1 always returns false, simulating a scenario where no email is considered spam.
- Calls filter_email/2 on both emails, expecting the function to correctly identify and mark the important email and to correctly interact with the spam filter (mocked to always return false for spam checks).

Under the hood, Rewire is doing a couple of interesting things. First, it's important to understand the philosophy behind Rewire and the approach the author decided to take. rewire works by using macros to create a copy of the module. So, for every test, Rewire creates a new module with the specified stubs.

Creating a copy of each module instead of overriding the original module allows us to run tests in parallel without any side effects.

Things to Consider When Using Rewire and Mox

When using Rewire and Mox in your Elixir projects, consider the following:

Asynchronous Testing Compatibility: Rewire fully supports asynchronous testing with async: true. Unlike global overrides used by tools like Meck, Rewire creates a separate module copy for each test. This ensures that tests can run in parallel without interfering with each other.
Integration with Mox: Rewire complements Mox perfectly by focusing on dependency injection without dictating the source of the mock module. This synergy allows for efficient and seamless integration between the two, making them an excellent pair for Elixir testing.
Impact on Test Speed: Rewire might slightly slow down your tests, although the effect is typically minimal. Comprehensive performance data from large codebases is still pending.
Test Coverage Accuracy: Yes, test coverage is accurately reported with Rewire, ensuring that you can trust your test coverage metrics.
Compatibility with Stateful Processes: Rewire works well with stateful processes, provided that these processes are started after their module has been rewired. For processes started beforehand (like a Phoenix controller), Rewire may not be effective since rewiring can no longer be applied. It's recommended to use Rewire primarily for unit tests where this limitation doesn't apply.
Erlang Module Rewiring: Rewire cannot directly rewire Erlang modules. However, it allows for Erlang module references to be replaced within Elixir modules, offering a workaround for this limitation.
Handling Nested Modules: Rewire will only replace dependencies within the specifically rewired module. Surrounding or nested modules will remain unaffected, maintaining references to the original modules. For complete control, you may need to rewire these modules individually.
Formatter Configuration for Rewire: To prevent mix format from adding parentheses around Rewire, update your .formatter.exs file with import_deps: [:rewire]. This ensures that Rewire syntax is correctly formatted without unnecessary parentheses.

And that's it!

Wrapping Up

In this post, we've explored how Rewire and Mox can help with dependency injection in Elixir.

Stephan Behnke, the creator of Rewire, was motivated by a desire for a more elegant solution to dependency injection in Elixir, especially for unit testing. I believe he succeeded in providing a great tool for the Elixir community.

That said, Rewire is not a silver bullet and it might not be the right tool for every project. It is important to evaluate Rewire alongside tools like Meck and make a decision based on your project and team's needs.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Using Dependency Injection in Elixir

Allan MacGregor 🇨🇦 — Tue, 04 Jun 2024 12:18:21 +0000

While controversial in functional programming, dependency injection can be a useful pattern in Elixir for managing dependencies and improving testability.

In this, the first part of a two-part series, we will cover the basic concepts, core principles, and types of dependency injection. We'll explore its benefits in terms of modularity, testability, and maintainability.

Then, we will look into a specific scenario where dependency injection can be beneficial, in this case, testing.

Let's first explain what dependency injection is.

What Is Dependency Injection?

Dependency Injection (DI) is a software design pattern that involves supplying an external dependency to a component rather than allowing the component to create the dependency itself. This pattern is a form of Inversion of Control (IoC), where control over the dependencies is inverted from the component to an external entity.

The main goal of DI is to reduce coupling between components, making our system more modular, flexible to changes, and easier to test.

These are the core concepts of dependency injection:

Dependency: An entity that another entity depends on to function properly.
Injector: The mechanism that injects dependencies into a component.
Client: The component that depends on the provided dependencies to operate.
Service: The dependency that the client component uses.

Types of Dependency Injection

Constructor Injection: The dependencies are provided through the component's constructor.
Setter Injection: The dependencies are provided through setter methods or properties.
Interface Injection: The dependency provides an injector method that will inject the dependency into any client passed to it.

Advantages of Dependency Injection

Reduced Coupling: By decoupling components from their dependencies, systems become more modular, allowing for easier maintenance and scalability.
Increased Flexibility: Changing or updating dependencies does not require changes to the dependent components, making the system more adaptable.
Improved Testability: Dependencies can be easily mocked or stubbed in tests, allowing for more isolated and reliable testing.

How Dependency Injection Works

There are four steps you should take to leverage dependency injection in your program or service:

Define the Service Interfaces: These interfaces represent the abstract contracts that services must fulfill.
Implement the Services: Concrete implementations of the service interfaces are developed.
Configure the Injector: The injector is configured to know which service implementations to inject into which clients.
Inject Dependencies: When a client is instantiated, the injector supplies it with the required service implementations based on the configuration.

Dependency Injection Diagram

The following diagram illustrates the basic concept of dependency injection:

The Client requires a service interface to perform its function.
The Dependency Injector decides which implementation of the service interface (Service A or Service B) to inject into the client at runtime.
Service A and Service B are different implementations of the same service interface. The injector injects one of these into the client based on the configuration or conditions.

This pattern allows for high flexibility and decoupling of components within software applications, facilitating easier management, testing, and evolution of the application code.

How Can Dependency Injection Be Applied in Elixir?

As we mentioned earlier, dependency injection is a pattern that is more commonly associated with object-oriented programming languages. Functional programming languages like Elixir offer a different set of tools and idioms for managing dependencies and state. However, the principles of DI can still be applied in Elixir to achieve similar benefits.

In Elixir, the emphasis on explicit over implicit dependency management aligns well with DI principles. For testing purposes, DI allows developers to easily replace real implementations with mocks or stubs, facilitating isolated unit tests that are not dependent on external services or state. This approach enhances test reliability and execution speed, as tests become less brittle and more focused on the functionality being tested.

Practical Application of Dependency Injection in Elixir for Testing

let's look at how we can use dependency injection to inject mocks and configure dependencies in Elixir.

Injecting Mocks

One common application of DI in Elixir testing involves injecting mock modules or functions that simulate the behavior of real dependencies. This technique is particularly useful when dealing with external services like databases or APIs.

defmodule MyApp.MyModule do
  def fetch_data(dataSource) do
    dataSource.query()
  end
end

defmodule MyApp.MyModuleTest do
  use ExUnit.Case

  test "fetch_data returns expected result" do
    mockDataSource = %{
      query: fn -> {:ok, "mocked data"} end
    }

    assert MyApp.MyModule.fetch_data(mockDataSource) == {:ok, "mocked data"}
  end
end

In this example, MyApp.MyModule.fetch_data/1 depends on a dataSource that responds to a query function. During tests, a mock dataSource is injected, allowing the test to run independently of any external data sources.

Configurable Dependencies

Another DI strategy involves using application configuration to define dependencies, which can then be overridden in the test environment.

# config/config.exs
config :my_app, data_service: MyApp.DataService

# config/test.exs
config :my_app, data_service: MyApp.MockDataService

In your application code, you would fetch the dependency from the application configuration:

defmodule MyApp.MyModule do
  def fetch_data do
    dataSource = Application.get_env(:my_app, :data_service)
    dataSource.query()
  end
end

This simple example shows how DI can be achieved in Elixir by configuring dependencies at runtime, allowing for easy substitution of real implementations with mocks or stubs during testing.

Next, let's review a more practical example that uses DI to inject a mock service into a module for testing.

Testing with Dependency Injection

In this example, we will work on EmailScanner, a module that scans emails for spam. We will use a SpamFilterService to check if emails are spam and dependency injection to inject a mock SpamFilterService for testing.

Start by creating a new Elixir project:

mix new email_scanner

Now let's move on to implementation and testing.

Implementation with `ExUnit`

First, create the EmailScanner module. This module will depend on a SpamFilterService to check if emails are spam. In this case, the SpamFilterService will be injected as a dependency, making it easy to swap with a mock during testing.

defmodule EmailScanner do
  def scan_email(spam_filter_service, email) do
    spam_filter_service.check_spam(email)
  end
end

Testing `EmailScanner` with `ExUnit`

Now, let's write a test for the EmailScanner module using ExUnit. We'll create a mock SpamFilterService to inject during tests:

defmodule MockSpamFilterService do
  def check_spam(_email), do: false
end

In this mock, the check_spam/1 function always returns false, simulating a non-spam email. Next, let's create a test case that makes use of our new mock:

defmodule EmailScannerTest do
  use ExUnit.Case

  test "scan_email with non-spam email returns false" do
    non_spam_email = %Email{content: "Hello, world!"}
    assert false == EmailScanner.scan_email(MockSpamFilterService, non_spam_email)
  end
end

This test injects MockSpamFilterService into EmailScanner, isolating the test from the real spam filtering logic and focusing solely on the EmailScanner's behavior.

By doing this, we can decouple the EmailScanner module from the SpamFilterService, making it easier to test and maintain.

Now that we've taken a look at using ExUnit and testing, let's turn to some common dependency injection mistakes to avoid and best practices.

Common Dependency Injection Pitfalls and Best Practices

First, we'll touch on some pitfalls:

Over-Reliance on Mocks: While DI makes it easy to replace real implementations with mocks, overusing mocks can lead to fragile tests that are overly focused on implementation details rather than behavior.
Complex Dependency Graphs: Introducing DI without careful planning can lead to a tangled web of dependencies that are hard to manage and understand.
Ignoring the Complexity of Configuration: DI often requires some form of configuration to wire up dependencies. This configuration can become complex and unwieldy if not managed properly.

To help you avoid these pitfalls, here are some best practices to follow:

Define Clear Interfaces: Ensure that your dependencies have clearly defined interfaces.
Use Configuration Wisely: Be mindful of the complexity that configuration can introduce.
Leverage Elixir’s Capabilities: Take advantage of Elixir’s features, such as module attributes and configuration files, to manage your dependencies effectively.
Test with Real Implementations When Possible: While mocking is useful, also test with real implementations to ensure that your system works as expected in real-world scenarios.

That's it for this part of the series!

Wrapping Up and What's Next

In this article, we have covered the basic concepts of dependency injection, its application in Elixir, and how it can be leveraged for testing. We have also discussed common pitfalls to avoid and best practices to follow when implementing DI in Elixir.

In the next and final part of this series, we'll look specifically at advanced dependency injection in Elixir using Rewire.

Until then, happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Powerful Caching in Elixir with Cachex

Allan MacGregor 🇨🇦 — Tue, 19 Mar 2024 13:24:27 +0000

Developers often initially look to the Elixir language and stack because it's known for being able to handle massive amounts of concurrent requests and scale easily. This makes Elixir a great choice for building highly performant applications.

However, sometimes operations are computationally expensive and can slow down your application. This is where caching comes in.

In this article, we'll explore how Cachex, a powerful library tailored for Elixir, can help you add caching to your application and improve its performance.

Understanding Caching and Its Importance in Elixir

Caching is the art of storing data temporarily to reduce redundancy and improve access times. The primary benefits of caching include faster data retrieval, reduced load on primary data sources, and enhanced application responsiveness. However, caching is not without its drawbacks. Over-reliance can lead to stale data, and improper cache management can result in increased complexity.

Caching can be added to reduce bottlenecks and improve performance in Elixir: for example, when dealing with external systems, databases, or computationally expensive operations. Caching can also be used to store frequently accessed data in memory, such as user sessions or application state.

Now let's take a quick look at some benefits and disadvantages of caching.

Benefits of Caching

Caching comes with several benefits, including:

Improved performance: Caching can significantly reduce data retrieval times, making applications more responsive.
Reduced load on primary data sources: By serving data from a cache, there's less strain on primary data sources like databases, reducing the risk of them becoming a bottleneck.
Cost savings: Reducing the number of calls to external services or databases can lead to cost savings, especially if those calls are billable.
Enhanced user experience: Faster response times lead to a smoother user experience.
Scalability: Caching can help applications handle more users simultaneously by reducing the need for resource-intensive operations.
Reduced network traffic: Serving data from the cache can reduce the amount of data that needs to be transmitted over a network.
Offline access: In some scenarios, caching allows users to access certain pieces of data even when offline.

Drawbacks of Caching

Even though there are a lot of benefits to caching, there are some drawbacks you should be mindful of:

Stale data: Cached data can become outdated, leading to users receiving old or incorrect information.
Increased complexity: Implementing caching introduces another layer of complexity to system architecture and can complicate debugging.
Memory usage: Caching, especially when done extensively, can consume a significant amount of memory.
Cache invalidation: Deciding when and how to invalidate or refresh the cache can be challenging.
Cache warm-up: After a cache clear or system restart, the cache might be "cold" and it can take time to "warm up" to an optimal state.
Potential for cache thrashing: Rapidly adding and evicting items can lead to cache thrashing, where the cache doesn't provide its benefits effectively.
Maintenance overhead: Over time, your caching strategy might need adjustments, leading to additional maintenance work.

It's worth noting that while caching offers numerous advantages, it's essential to implement it judiciously, considering the specific needs and characteristics of each application.

Caching Options in Elixir

In the world of Elixir, many different options are available for caching. It is important to understand that each library has its own set of pros and cons. As developers, we need to consider the specific scenario where we intend to use caching and choose the right tool for the job.

Some of the options available to use are:

Cachex: A powerful caching library tailored for Elixir. It offers features like Time-to-Live (TTL), fallbacks, and locking mechanisms.
ConCache: A concurrent caching library for Elixir. It's built on top of Erlang Term Storage (ETS) and provides features like TTL and cache partitioning.
Nebulex: A flexible and highly configurable caching library. It supports different caching strategies and has built-in support for distributed caching.
Erlang Term Storage (ETS): While not exclusively a caching solution, ETS is an in-memory store that's often used for caching in Elixir applications. It's a core feature of the Erlang runtime system.
Mnesia: A distributed database management system that comes with Erlang/OTP. It can be used for caching, especially in distributed Elixir applications.
Redis via Redix: While Redis is not an Elixir-specific solution, it's a popular choice for caching in many applications. The Redix library allows Elixir applications to interact with Redis easily.
Least Recently Used (LRU) caches: There are several Elixir libraries, like lru_cache, that implement the LRU caching algorithm.

For the rest of this article, we'll focus on Cachex, because it's a robust caching solution that's easy to use and offers a wide range of features. It's also well-documented and has a vibrant community around it.

Introduction to Cachex for Elixir

Cachex offers a suite of features that make it an indispensable tool for Elixir developers. From simple key-value storage to advanced features like TTL settings and fallback mechanisms, Cachex provides a comprehensive caching solution.

Among the many features that make Cachex a powerful tool for caching in Elixir, the following stand out:

Performance: Cachex is designed for high performance, ensuring rapid data retrieval and insertion. This is crucial for applications that require real-time responsiveness.
Concurrency support: Elixir is known for its concurrency capabilities, and Cachex is built to handle concurrent operations seamlessly. It ensures that multiple processes can interact with the cache without causing data inconsistencies.
Advanced features, including TTL (allowing developers to specify how long an item should remain in the cache, to ensure that data doesn't become stale), fallbacks (mechanisms to compute values if they're missing from the cache, so that an application can still function even if a cache miss occurs), and locking mechanisms (ensuring data integrity during operations like cache updates).
Flexibility: Cachex is not just a simple key-value store. It supports complex data structures, making it versatile for a range of applications.
Distributed caching: While Cachex primarily operates as a local cache, it can be combined with other tools and libraries to support distributed caching scenarios, making it suitable for clustered Elixir applications.
Comprehensive documentation: Cachex comes with extensive documentation, making it easier for developers to get started and harness its full potential.
Integration with Telemetry: Cachex integrates with the Telemetry library, allowing developers to gather metrics and monitor cache performance in real-time.
Cache eviction strategies: Cachex supports various cache eviction strategies, such as LRU, ensuring that the cache remains efficient even as it fills up.
Fault tolerance: Built with Elixir's fault-tolerant nature in mind, failures are isolated and don't bring down the entire application.

Adding Caching to Your Elixir Application with Cachex

To get started with Cachex, we'll need to add it as a dependency to our project.

To install Cachex in an Elixir application, follow these steps:

Add cachex as a dependency: Open your mix.exs file and add :cachex to the list of dependencies:

defp deps do
 [
   {:cachex, "~> 3.3"} # The version number might change over time, so always check for the latest version.
   ]
end

Fetch and compile dependencies:

Run the following command in your terminal:

mix deps.get

This will fetch and compile the Cachex library along with any of its dependencies.

Start Cachex:

Before using Cachex in your application, you need to start it. You can start a new cache instance with the following command:

{:ok, _} = Cachex.start_link(:my_cache)

Here, :my_cache is the name of the cache. You can choose any name that suits your application.

Calling Cachex.start_link/1 will start a new cache instance with the default configuration. If you want to customize the configuration, you can pass in a map with the desired configuration options.

For example, if you want to set the cache size to 1000 items, you can do so by passing in the :size option:

{:ok, _} = Cachex.start_link(:my_cache, %{size: 1000})

It's important to note that Cachex is built on top of GenServer, so you can use the same techniques to start and supervise it.

Add to application supervision tree: If you want the cache to be supervised and automatically restart in case of failures, you can add it to your application's supervision tree. This ensures that the cache is always available throughout the lifecycle of your application.

In your application's supervisor, you can add:

children = [
 {Cachex, name: :my_cache}
]

This will start the Cachex cache when your application starts and supervise it.

Note: There are some scenarios where you might not want to start Cachex automatically. For example, if you're using Cachex in a Phoenix application, you might want to start it only when the application is running in production. In such cases, you can start Cachex manually in your application's supervision tree.

That's it! You've successfully installed Cachex in your Elixir application. You can now use its various functions to cache data, retrieve cached data, and manage your cache. Always refer to the official Cachex documentation for more detailed information and best practices.

Working with Phoenix

A common use case for Cachex is to cache data in a Phoenix application. Phoenix doesn't have a built-in caching mechanism, so developers often turn to third-party libraries like Cachex.

Before we can use Cachex in a Phoenix application, we need to add it as a dependency and initialize it. We can do this by following the steps outlined in the previous section.

Once Cachex is installed, we can use it in our Phoenix application. Let's look at a few examples of using Cachex in a Phoenix application.

Caching Database Queries

One common use case in Phoenix applications is to cache the results of database queries to reduce database load and speed up data retrieval. After fetching data from the database, you can store it in the Cachex cache:

def get_user(id) do
  case Cachex.get(:my_cache, id) do
    {:ok, nil} ->
      user = Repo.get(User, id)
      Cachex.put(:my_cache, id, user)
      user
    {:ok, user} -> user
  end
end

The function get_user/1 retrieves a user by their id. It first checks if the user is available in the cache, and if not, it fetches them from a repository (likely a database) and then caches the result for future calls. Let's break down what's happening here:

Function definition: The function get_user/1 is defined to accept a single argument, id, which identifies a user.
Cache lookup: The function starts by trying to retrieve the user from a cache named :my_cache using the provided id as the cache key. Cachex.get(:my_cache, id) attempts to get the value associated with the key id from :my_cache.
Handling cache results: The result of the cache lookup is pattern-matched using a case statement to determine the next steps.
- Cache miss {:ok, nil}: If the cache returns {:ok, nil}, it indicates a cache miss, meaning the user is not present in the cache.
- The function then fetches the user from a repository using Repo.get(User, id). This is likely a call to a database to retrieve the user data.
- Once the user data is fetched, it's stored in the cache using Cachex.put(:my_cache, id, user). This ensures that subsequent calls for the same user id will find the user in the cache and won't need to hit the database.
- Finally, the fetched user data is returned as the result of the function.
- Cache hit {:ok, user}: If the cache returns {:ok, user} where the user is not nil, this is a cache hit. The user data has been found in the cache. The function simply returns the cached user data without making any database calls.

Caching Views

Sometimes, we might need to cache rendered views to improve performance. For example, if we have a view that's expensive to render, we can cache it to reduce the load on the server and speed up data retrieval.

There are many reasons why views might be expensive to render. For example, they might contain complex logic or require multiple database calls. In such cases, caching can be a useful technique to improve performance.

We can use a plug to cache views in Phoenix. Here's an example:

Create a plug that checks if the view is cached and sends the cached content if it is. Otherwise, it continues with the pipeline and caches the response later.

defmodule MyApp.CacheViewPlug do
  import Plug.Conn

  def init(opts), do: opts

  def call(conn, _opts) do
    cache_key = "view_cache:#{conn.request_path}"

    case Cachex.get(:my_cache, cache_key) do
      {:ok, nil} ->
        # Cache miss, continue with the pipeline and cache the response later
        conn
      {:ok, cached_content} ->
        # Cache hit, send the cached content and halt the pipeline
        conn
        |> put_resp_content_type("text/html")
        |> send_resp(200, cached_content)
        |> halt()
    end
  end
end

Let's break down what's happening here:

Module definition: the defmodule MyApp.CacheViewPlug do: line defines a new module named MyApp.CacheViewPlug.
Importing Plug.Conn: import Plug.Conn: imports functions from the Plug.Conn module, which provides utilities for working with connection structs in the context of a Plug.
Initialization function: def init(opts), do: opts: is the initialization function required by the Plug behavior. It takes an opts argument (options) and simply returns it unchanged. In many Plugs, this function sets default options or validates the provided options. However, in this case, it's a simple pass-through.
Call function: def call(conn, _opts) do: is the main Plug function that gets executed for every request. It takes two arguments: conn (the connection struct) and _opts (the options, which are ignored in this case).
Generating the cache key: cache_key = "view_cache:#{conn.request_path}": constructs a cache key based on the request path. The cache key is prefixed with "view_cache:" to namespace or differentiate it from other potential cache keys.
Checking the cache: The case Cachex.get(:my_cache, cache_key) do statement checks the cache (:my_cache) for content associated with the generated cache_key.
Handling cache results:
- {:ok, nil} ->: This pattern matches a cache miss. If the cache doesn't have content for the given key, it returns {:ok, nil}. In the event of a cache miss, the function simply returns the unchanged conn, allowing the request to continue through the Plug pipeline. The intention is that the response will be cached later, presumably by another part of the application.
- {:ok, cached_content} ->: This pattern matches a cache hit. If the cache has content for the given key, it returns {:ok, cached_content}. In this case, the function sends the cached content as the response, sets the response content type to "text/html", and then halts the Plug pipeline to prevent further processing.

Now, add the plug to your pipeline in router.ex:

pipeline :browser do
  ...
  plug MyApp.CacheViewPlug
  ...
end

Finally, cache the view after rendering it:

def some_action(conn, _params) do
  content = render(conn, "some_template.html")
  cache_key = "view_cache:#{conn.request_path}"
  Cachex.put(:my_cache, cache_key, content)
  send_resp(conn, 200, content)
end

Caching API Responses

Another common Cachex use case for Phoenix is to cache API responses. This can be useful to reduce the load on external services and speed up data retrieval. Similar to caching views, we can use a plug to cache API responses in Phoenix.

Create a plug that checks if the response is cached and sends the cached content if it is. Otherwise, it continues with the pipeline and caches the response later.

defmodule MyApp.CacheAPIPlug do
  import Plug.Conn

  def init(opts), do: opts

  def call(conn, _opts) do
    # Create a cache key based on the request path and query parameters
    cache_key = "api_cache:#{conn.request_path}?#{conn.query_string}"

    case Cachex.get(:my_cache, cache_key) do
      {:ok, nil} ->
        # Cache miss, continue with the pipeline and cache the response later
        conn
      {:ok, {status, headers, body}} ->
        # Cache hit, send the cached response and halt the pipeline
        conn
        |> put_status(status)
        |> put_resp_header("content-type", List.keyfind(headers, "content-type", 0, {"content-type", "application/json"}))
        |> send_resp(status, body)
        |> halt()
    end
  end
end

Add the plug to your pipeline in router.ex:

pipeline :api do
  ...
  plug MyApp.CacheAPIPlug
  ...
end

Cache the response after sending it:

def some_api_action(conn, _params) do
  # Process the request and generate the response
  response = %{data: "Some API response data"}

  # Convert the response to JSON
  json_response = Jason.encode!(response)

  # Cache the response
  cache_key = "api_cache:#{conn.request_path}?#{conn.query_string}"
  cache_value = {200, [{"content-type", "application/json"}], json_response}
  Cachex.put(:my_cache, cache_key, cache_value)

  # Send the response
  json(conn, response)
end

Checking for Stale Cache Data

One of the main challenges with caching is ensuring that the data in the cache is up-to-date. If data becomes stale, it can lead to incorrect results and a poor user experience. Therefore, it's crucial to check for stale data and refresh the cache when necessary.

Cachex provides built-in mechanisms to handle stale data, primarily through its Time-to-Live (TTL) and fallback features.

TTL

TTL allows you to specify how long an item should remain in the cache. Once the TTL expires, an item is automatically removed from the cache. This ensures that you don't serve stale data older than a specified age.

# Store data with a TTL of 3600 seconds (1 hour)
Cachex.put(:my_cache, "key", "value", ttl: 3600)

When you attempt to retrieve this key after its TTL has expired, it will return as if the key does not exist in the cache.

Fallbacks

Cachex's fallback mechanism is a powerful feature that allows you to execute a function when a cache miss occurs. This can be especially useful for handling stale data. If data is not in the cache (either because it was never cached or because it was evicted due to TTL expiration), the fallback function can fetch fresh data.

fallback_fn = fn key ->
 # Fetch fresh data for the given key
 {:commit, fetch_fresh_data(key)}
end

# Attempt to get data from cache, use fallback if cache miss occurs
Cachex.fetch(:my_cache, "key", fallback: fallback_fn)

The :commit tuple ensures that fetched data is stored back into the cache.

By combining TTL and fallbacks, Cachex provides a robust mechanism to ensure that stale data is not served and that fresh data can be fetched and cached automatically when needed.

One thing to note is the difference between Cachex.get and Cache.fetch. The Cachex.get function returns the value associated with the given key, or {:ok, nil} if the key is not found in the cache. The Cachex.fetch function returns the value associated with the given key, or executes the fallback function if the key is not found in the cache. While both functions can be used to retrieve data from the cache, they have different behaviors when the key is not found. Cachex.fetch offers more advanced mechanisms to handle cache misses.

It's important to understand that the TTL and fallback features are not mutually exclusive, and can be used together to provide a more robust caching solution and a better user experience.

Wrapping Up

In this post, we've seen that caching is a powerful technique to significantly improve your Elixir application's performance. However, we've also explored how caching is not without its drawbacks.

Cachex, as representative of the vibrant Elixir ecosystem, showcases how community-driven tools can address complex problems with elegance and efficiency. But remember, Cachex is just the tip of the iceberg. The Elixir community is teeming with innovative libraries and frameworks, each solving unique challenges and pushing the boundaries of what's possible.

As you continue your journey with Elixir, I encourage you to explore the many tools and libraries available and discover how they can help you build better applications.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

How To Use Zig for Elixir NIFs

Allan MacGregor 🇨🇦 — Tue, 31 Oct 2023 14:44:38 +0000

Elixir excels at building scalable and maintainable applications. However, sometimes Elixir is not the best language to tackle specific tasks, and it can fall short in some areas, like direct system interaction.

Fortunately, Elixir offers NIFs (Native Implemented Functions): a path for integrating with other languages to improve these gaps. NIFs in Elixir act as an interface to call functions written in native languages like C or Rust, enabling developers to optimize performance-critical sections of their code.

NIFs can be written in many languages like Rust, Python, and Zig. For this tutorial, we will use Zig, due to its simplicity, speed, and safety.

Let's get going!

What We'll Cover

We'll cover the following topics in this tutorial:

Setting up Zig for Elixir NIF development.
Understanding NIFs in Elixir.
Writing a simple NIF in Zig.
Integrating the NIF with an Elixir application.

What Is Zig?

Zig is a general-purpose programming language designed for robustness, optimality, and maintainability.

It is known to provide excellent debugging and error-handling capabilities, making it suitable for various applications (including systems programming, embedded systems, and performance-critical applications).

Setting up Zig for Elixir Development

Let's now look at how we can set up Zig for Elixir NIF development.

Installing Zig

To get started with Zig, download the latest version of the Zig compiler from the official Zig website. Follow the installation instructions for your specific operating system.

Configuring Zig for Elixir NIF Development

To configure Zig for Elixir NIF development, you'll need to include the necessary libraries and dependencies.

Elixir NIFs rely on the Erlang NIF API provided by the erl_nif.h header file. Ensure you have the Erlang development package installed, and include the appropriate path to erl_nif.h in your build script or Zig build file.

Just add the following line to your src/main.zig file:

const erl = @cImport({
    @cInclude("erl_nif.h");
});

Importing this library will allow us to use the Erlang NIF API in our Zig code. It contains the native functions that allow our Zig code to interact with the Erlang VM. Later in this tutorial, we will see how to use this library to create our NIF.

Verifying the Setup

To verify that Zig is correctly set up for Elixir NIF development, we will create a simple "Hello, World!" project in Zig.

Go ahead and run the following commands:

mkdir hello-world
cd hello-world
zig init-exe

If zig has been installed correctly, you should see the following output:

info: Created build.zig
info: Created src/main.zig
info: Next, try `zig build --help` or `zig build run`

Next, we'll run zig build run on our terminal to ensure we can compile and execute a Zig program. If successful, you should see the following output:

All your codebase are belong to us.
Run `zig build test` to run the tests.

Now that we are able to compile and run Zig code, we can move forward.

Understanding NIFs in Elixir

Why Use NIFs in Elixir?

NIFs can be called from Elixir code, providing a performance boost for computationally intensive tasks. While Elixir excels at concurrency and fault tolerance, there might be better choices for performance-sensitive tasks.

By writing NIFs in a lower-level language like Zig, you can leverage the language's speed and efficiency without sacrificing Elixir's maintainability and concurrency features.

Advantages of Using Zig for NIF Development

Zig offers several advantages for NIF development:

Performance: Zig compiles to efficient native code, providing significant performance improvements over interpreted languages like Elixir.
Safety: Zig has strong static typing, compile-time checks, and a focus on error handling, reducing the likelihood of runtime errors.
Simplicity: Zig's straightforward syntax and semantics make it easy to learn and use, especially for developers familiar with C or C++. Zig's standard library is also very comprehensive, providing a wide range of functionality for common tasks.

Writing our First NIF

Let's start by setting up our Elixir project. Run the following command:

mix new example_nif
cd example_nif

This will create a basic Elixir project called ExampleNif. Next, let's initialize the Zig project inside our Elixir project with the following command:

zig init-lib

This will initialize a library inside our project that will create the following files:

build.zig - which contains the code for compiling and building our zig library.
src/main.zig - this is the main code of our library and will contain the logic that we will implement.

Writing a Simple NIF in Zig

Our first step when working with Zig is to update the build.zig file with the following code:

const std = @import("std");
const Pkg = std.build.Pkg;

pub fn build(b: *std.build.Builder) void {
    const mode = b.standardReleaseOptions();

    const nif_step = b.step("example_lib", "Compiles erlang library");
    const example_lib = b.addSharedLibrary("example_nif", "./src/main.zig", .unversioned);
    example_lib.setBuildMode(mode);
    example_lib.setOutputDir("build");
    example_lib.addIncludePath("/usr/lib/erlang/usr/include/");

    example_lib.install();
    example_lib.linkLibC();
    nif_step.dependOn(&example_lib.step);
}

This will add the Erlang header path to the build environment and also link with the C library.

Next, we will update the src/main.zig code. For this example, we'll create a simple NIF that takes two integers and returns their sum. This function will be written in Zig and called from Elixir code.

const std = @import("std");
const erl = @cImport({
    @cInclude("erl_nif.h");
});

export fn nif_add(
    env: ?*erl.ErlNifEnv,
    argc: c_int,
    argv: [*c]const erl.ERL_NIF_TERM,
) erl.ERL_NIF_TERM {
    var a: i32 = undefined;
    var b: i32 = undefined;

    if ((argc != 2))
    {
        return erl.enif_make_badarg(env);
    }

    _ = erl.enif_get_int(env, argv[0], &a);
    _ = erl.enif_get_int(env, argv[1], &b);

    const result = a + b;
    return erl.enif_make_int(env, result);
}

const func_count = 1;

var funcs = [func_count]erl.ErlNifFunc{
    erl.ErlNifFunc{
        .name = "nif_add",
        .arity = 2,
        .fptr = nif_add,
        .flags = 0,
    },
};

var entry = erl.ErlNifEntry{
    .major = erl.ERL_NIF_MAJOR_VERSION,
    .minor = erl.ERL_NIF_MINOR_VERSION,
    .name = "Elixir.ExampleNif",
    .num_of_funcs = func_count,
    .funcs = &funcs,
    .load = null,
    .reload = null,
    .upgrade = null,
    .unload = null,
    .vm_variant = "beam.vanilla",
    .options = 1,
    .sizeof_ErlNifResourceTypeInit = @sizeOf(erl.ErlNifResourceTypeInit),
    .min_erts = "erts-10.4",
};

export fn nif_init() *erl.ErlNifEntry {
    return &entry;
}

While this code may look intimidating, it's actually quite simple. Let's break it down:

We are first importing the erl_nif.h header file, which contains the definitions for the Erlang NIF API and the std library.
Next, we define our first and only library function, nim_add, which takes three arguments:
- env is a pointer to an ErlNifEnv struct, which contains the environment for the NIF. This struct is used to allocate memory, create Erlang terms, and more.
- argc is the number of arguments passed to the NIF.
- argv is an array of Erlang terms (the arguments passed to the NIF).
- The function code itself is a very simple and naive implementation that adds two integers.
After we have defined all our functions, we define a constant func_count (the number of functions in the NIF).
We then define an array of ErlNifFunc structs, containing the name, arity, and function pointer for each function in the NIF.
Finally, we define an ErlNifEntry struct, which contains the NIF's version, name, and functions. This struct is used to register the NIF with the Erlang VM.

It is important to highlight that the num_of_funcs field in the ErlNifEntry struct must match the number of functions in the funcs array. The name field must also match the module name and function in the NIF_MOD_FUNCS of the build.zig file.

Compiling the Zig Code Into a NIF

To compile the Zig code into a NIF, use the zig build-lib command, specifying the appropriate target and output file. For example, on a Linux system:

zig build example_lib

This command will produce a shared library file: libexample_nif.so in the build/ directory.

Integrating the NIF with Elixir

To use the NIF in Elixir, we need to follow several steps:

Load the shared library containing the NIF.
Define a fallback function to handle cases where the NIF is not loaded.
Call the NIF function from our Elixir code.

Loading the Shared Library

When using NIFs, Elixir needs to load the shared library containing the native code. To do this, we use the :erlang.load_nif/2 function, which takes two arguments:

The relative path to the shared library (.so on Linux, .dll on Windows, or .dylib on macOS).
An optional integer value (usually 0) can be used for versioning purposes.

In our Elixir module, we'll define a load_nif/0 private function that handles loading the shared library. We'll also use the @on_load module attribute to specify that load_nif/0 should be called when the module is loaded.

Defining a Fallback Function

It's important to provide a fallback function that will be called if the NIF fails to load. This function should have the same name and arity as the NIF function and return an error tuple such as {:error, reason} or call the :erlang.nif_error/1 BIF (Built-In Function) with an appropriate error reason.

In our example, we define an add/2 fallback function that calls :erlang.nif_error(:nif_not_loaded).

Calling the NIF Function from Elixir Code

Once the shared library is loaded and the NIF function is linked to the Elixir module, you can call the NIF function just like any other Elixir function. In our example, we call ExampleNif.add/2 to perform the addition using the NIF we wrote in Zig.

By following these steps, you can seamlessly integrate NIFs written in Zig into your Elixir applications, taking advantage of both the performance benefits of Zig and the maintainability and concurrency features of Elixir.

Writing Elixir Code to Use the NIF

Create a new Elixir module called ExampleNif:

defmodule ExampleNif do
  @on_load :load_nif

  def load_nif do
    :erlang.load_nif('./build/libexample_nif', 0)
  end

  def nif_add(_a, _b), do: :erlang.nif_error(:nif_not_loaded)

end

Let's review this. We:

Create an Elixir module called ExampleNif.
Specify that the load_nif/0 function should be called when the module is loaded.
Define a private function, load_nif/0, that loads the shared library containing the NIF.
Define a fallback function (nif_add/2) that will be called if the NIF fails to load.

Note: :erlang.load_nif loads and links the Zig compiled library to the Elixir module; and it also loads this function before the definition of our nif_add/2 fallback function. This is why we need to define the fallback function nif_add/2 with the same arity and name as the NIF function.

Because of Elixir's pattern-matching capabilities, the fallback function will handle calls to nif_add/2 if the NIF is not loaded, and if the NIF is loaded, the NIF function will handle the call as defined in load_nif/0.

Verifying That the NIF Works

To verify that the NIF is working as expected, create a simple test program in Elixir:

defmodule ExampleNifTest do
  use ExUnit.Case

  test "add/2 NIF" do
    assert ExampleNif.nif_add(1, 2) == 3
    assert ExampleNif.nif_add(-5, 10) == 5
  end
end

Run the test suite using mix test. If the tests pass, it indicates that the NIF is functioning correctly, and our addition is done through our Zig compiled library rather than on Elixir code directly.

Considerations

It is amazing that we can use code from another language through NIF. However, there are a few things you might want to be aware of, starting with the parameters we initially declared on our Zig library:

export fn nif_add(
    env: ?*erl.ErlNifEnv,
    argc: c_int,
    argv: [*c]const erl.ERL_NIF_TERM,
) erl.ERL_NIF_TERM {
    var a: i32 = undefined;
    var b: i32 = undefined;
...
    _ = erl.enif_get_int(env, argv[0], &a);
    _ = erl.enif_get_int(env, argv[1], &b);

We are parsing the first two arguments and casting them into an integer. What happens if we pass a float value to this function? Try it out:

iex(3)> ExampleNif.nif_add(-100.0,-11)
-1431655777

We get junk values, as there are no type checks or logic to handle float values in our Zig function. These can cause unexpected bugs and issues that developers need to be more vigilant to catch.

Wrapping Up

In this post, we set up Zig for Elixir NIF development, wrote a simple NIF in Zig, and integrated the NIF with an Elixir application.

By leveraging the strengths of both Elixir and Zig, developers can create efficient, maintainable, and performant applications.

That said, NIFs should be used with caution and deliberately. Ensure you add the necessary tests and failsafes.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

How To Instrument Your Elixir Application with AppSignal

Allan MacGregor 🇨🇦 — Tue, 31 Jan 2023 12:35:59 +0000

Instrumentation is an essential part of monitoring and operating an application, especially for apps heavily used in production. Even in today's everchanging technology landscape, visibility and observability still challenge developers and system administrators.

Metrics and logging are essential for monitoring and operating an application. Metrics measure an application's performance and system health, while logging records system health and application state. Instrumentation allows us to collect metrics and log events about an application's state and system health.

Thanks to instrumentation, you can gain great insights into your Elixir application.

In this tutorial, we will cover the importance of instrumentation, go over the basic concepts, and show you how to instrument your Elixir application with AppSignal.

Let's get started!

Why Instrument Your Elixir Application?

Application performance monitoring (APM) is the process of monitoring and keeping track of an application's performance. Many monitoring tools offer APM; in this article, we will focus on AppSignal.

Out of the box, AppSignal will handle error tracking, performance monitoring, and host metrics. However, to get the most out of AppSignal, we need to instrument our application to collect metrics and log events of specific interest to us.

With custom instrumentation, you can mark specific parts of an application to collect metrics and data, gaining insight into the deep inner workings of the application.

For example, we can use instrumentation to collect metrics about:

a specific function's performance
the number of times a particular function is called
the performance of a specific database query
the number of times a particular database query is called

With this information, developers can identify bottlenecks and refactor code to improve an application's performance.

Normally APM tools will not go deep into the layers of an application to collect metrics. Instead, they will collect metrics at the application level, such as the number of requests per second, the number of errors, and the average response time.

But as we've discussed, custom instrumentation allows developers to manually go deep into an application's layers to collect metrics and log particular events.

Application performance monitoring and custom instrumentation are essential for monitoring and operating an application and should be considered complementary tools.

Pre-requisites

To follow along with this article, you will need to have the following installed:

Elixir 1.10 or higher
PostgreSQL 12 or higher
Docker and Docker Compose v2 or higher

Start by cloning the RealWorld Phoenix application:

git clone https://github.com/tamanugi/realworld-phoenix.git

To get started, we need to install the dependencies and set up the database:

# Start Database
$ docker-compose up -d

# Install dependencies
$ mix deps.get

# Create and migrate your database
$ mix ecto.setup

# Start Phoenix endpoint with `mix phx.server`
$ mix phx.server

Validate that everything works by visiting localhost:4000/api/articles from your browser. You should be able to see the following JSON response:

{
    "articles": [
        {
            "author": {
                "bio": null,
                "following": null,
                "image": null,
                "username": "username"
            },
            "body": "It takes a Jacobian",
            "createdAt": "2022-11-06T17:43:38.000Z",
            "description": "Ever wonder how?",
            "favorited": null,
            "favoritesCount": 0,
            "slug": "how-to-train-your-dragon-1",
            "tagList": [
                "dragons",
                "training"
            ],
            "title": "How to train your dragon 1",
            "updatedAt": "2022-11-06T17:43:38.000Z"
        },
...
    ],
    "articlesCount": 10
}

Next, sign up for a free trial of AppSignal.

Once you have an AppSignal account, attach the AppSignal Elixir package to your application by adding the following to your mix.exs file:

# mix.exs
def deps do
  [
      {:appsignal_phoenix, "~> 2.0"}
  ]
end

Make sure you install the dependency by running mix deps.get.

Next, configure AppSignal by running the following mix command:

mix appsignal.install YOUR_PUSH_API_KEY

Note: YOUR_PUSH_API_KEY is the Push API Key for your AppSignal account. You can find your Push API Key in your AppSignal account under Settings > API Keys.

Follow the automated installation instructions and make sure you use the defaults. Then take one more step to capture the Phoenix HTTP requests. Open the endpoint.ex file and add Appsignal.Phoenix to the list of modules:

  # lib/realworld_phoenix_web/endpoint.ex
defmodule AppsignalPhoenixExampleWeb.Endpoint do
  use Phoenix.Endpoint, otp_app: :appsignal_phoenix_example
  use Appsignal.Phoenix

  # ...
end

Start the application again with mix phx.server, and make a couple of requests to the application by visiting localhost:4000/api/articles from your browser.

If the setup is successful, you should be able to see the requests in your AppSignal dashboard:

Here, we can see:

the number of requests per second
the number of errors
the average response time

As well as the default metrics, we can also see some custom metrics. Let's add custom metrics in the next section.

Custom Instrumentation for Your Elixir App

Now that we have AppSignal set up, let's implement some custom instrumentation. For this example, we will instrument the lib/realworld_phoenix_web/controllers/article_controller.ex/index/0 function to collect metrics on the number of times the function is called and the average response time.

Key Instrumentation Concepts

Before we proceed any further, let's quickly go over some key concepts:

Instrumentation is the process of adding observability code to our application. This code will collect metrics and log events of specific interest to us.
Trace: The record of the paths taken by a single request through the application. Tracing makes debugging less daunting by breaking down a request as the application processes it.
Span: Spans are the building blocks of traces, representing a single unit of work or operation within a system.

There are two ways to instrument our application. The first uses helper functions, and the second uses function decorators. We'll review both approaches, starting with the helper function approach.

Instrumentation Using Helper Functions for Your Elixir App

Using decorators to add custom instrumentation is relatively easy. However, there are cases where we might need more flexibility with instrumentation — for example, when we want to trace specific parts of a function.

Using the index/2 function as an example, we can use helper functions to add custom instrumentation. Open the lib/realworld_phoenix_web/controllers/article_controller.ex file and update the code to look like this:

# lib/realworld_phoenix_web/controllers/article_controller.ex
def index(conn, params) do
  # Add a delay to the function
  slow()

  keywords =  Appsignal.instrument("keywords_map", fn ->
    for {key, val} <- params, do: {String.to_atom(key), val}
  end)

  keywords = Appsignal.instrument("keywords", fn ->

      case Guardian.Plug.current_resource(conn) do
          nil -> keywords
          user -> keywords |> Keyword.put(:user, user)
      end
  end)

  articles = Appsignal.instrument("articles", fn ->
      Articles.list_articles(keywords)
      |> Articles.article_preload()
  end)

  render(conn, "index.json", articles: articles)
end

Using the Appsignal.instrument/2 function, we can wrap specific parts of the code and add events to the span. This allows us to break down the transaction into smaller parts and see where an application is spending most of its time.

Appsignal.instrument/2 allows us to instrument a function, taking three parameters:

name of the function
category (this can be left empty)
the function we are trying to instrument itself

Instrumenting Your Elixir App with the Function Decorator

The main advantage of function decorators is that we can instrument our application without modifying the code. For this app, we can use Appsignal.Instrumentation.Decorators.

Open the lib/realworld_phoenix_web/controllers/article_controller.ex file and add the following to the top of the file:

# lib/realworld_phoenix_web/controllers/article_controller.ex
use Appsignal.Instrumentation.Decorators

Also, to make this example more interesting, let's add a delay to the index/0 function. This will allow us to see the impact of the delay on the response time:

# lib/realworld_phoenix_web/controllers/article_controller.ex
def index(conn, params) do
  # Add a delay to the function
  slow()
  keywords = for {key, val} <- params, do: {String.to_atom(key), val}

  keywords =
    case Guardian.Plug.current_resource(conn) do
      nil -> keywords
      user -> keywords |> Keyword.put(:user, user)
    end

  articles =
    Articles.list_articles(keywords)
    |> Articles.article_preload()

  render(conn, "index.json", articles: articles)
end

# Decorate this function to add custom instrumentation
@decorate transaction_event()
defp slow do
  Enum.random(1000..6000)
  |>:timer.sleep()
end

Now that we have added the decorator, let's make a couple of requests to our application. You can do this by visiting localhost:4000/api/articles from your browser.

After refreshing the articles a couple of times, you should be able to see the following in your AppSignal dashboard:

We can see that the index/0 function was called and is reporting a mean of 5.1 seconds. If we click on the dashboard, we can check out the transaction details:

Here, there's information about how much time the transaction spent calling the database, parsing the route, etc. Because we added custom instrumentation, we know that the slow/0 function is the bottleneck.

Helper Vs. Decorator Functions

When it comes to choosing between decorators and helper functions, it really depends on your use case. Both approaches have their advantages and disadvantages.

Decorators are great for adding instrumentation to a function without having to modify the code. However, we can't add instrumentation to specific parts of the function and that might limit the amount of information we can gather. Yet, in cases where functions are fairly simple, decorators are a great way to add instrumentation without impacting on code.

Helper functions, on the other hand, allow us to add instrumentation to specific parts of a function. With decorators, we can only add instrumentation to an entire function. But with helpers, we can add another layer of granularity to dig deeper into potential bottlenecks in our application. Going back to our example above, we were able to break down the index/0 function into smaller parts and see where the application was spending most of its time.

However, helper functions require us to modify the code, which might not be ideal in some cases and can add complexity.

Most importantly, consider:

your use case
the amount of information you want to gather
how much granularity you need in your instrumentation.

In most cases, helper functions will be the best approach, as they can add a significant amount of granularity to your trace.

Wrapping Up

In this article, we went over the basics of adding instrumentation to an Elixir application. We learned how instrumentation can help us uncover bottlenecks and improve an application's performance. We also saw how AppSignal can help us aggregate and visualize the data we collect.

With AppSignal, we have a single tool that allows us to collect different data types, and explore the data to look for bottlenecks, issues, and potential correlations between different parts of a system.

Instrumentation is a powerful tool that will help you understand the inner workings of your application. You will be armed with the knowledge to resolve bottlenecks and catch potential issues before they become a problem.

Happy instrumenting!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Benchmark Your Elixir App's Performance with Benchee

Allan MacGregor 🇨🇦 — Tue, 13 Sep 2022 11:52:50 +0000

At some point, every software engineer will find themselves in a situation where they need to benchmark system performance and test the limits of what a given system can handle. This is a common problem in software engineering, and even more so in the applications that are well suited for Elixir.

Finding bottlenecks early on in an application can save a lot of time, money, and effort in the long run, and give developers confidence in the upper limit of a system.

In this post, we will introduce a tool called Benchee to benchmark parts of an Elixir application. We will also show you how to integrate Benchee with your automated test suite.

By the end of the article, you'll understand Benchee's full functionality and capabilities, and will be able to use it to measure your application's performance.

Let's get going!

Why Do I Need to Benchmark My Elixir Application?

Benchmarking, in a nutshell, is the process of measuring the performance of a system under specific loads or conditions. As part of the benchmarking process, you will be able to identify potential bottlenecks in your system and areas to improve.

For example, we can use benchmarking tools to answer questions like:

Can a system handle ten times the load of normal traffic?
Can the system run on a smaller infrastructure to handle the same load?
How long does it take to process 10, 100, 1,000, or 10,000 requests? Does the processing time scale linearly with the number of requests?

Just answering these questions alone can help your team avoid costly mistakes and proactively identify areas for improvement, avoiding downtime and unhappy users.

Benchmarking doesn't have to be expensive or time-consuming; it can be simple to get the right tools in place and make them part of an application's natural development life cycle.

What Is Benchee?

This is where Benchee comes in. Benchee is a tool that you can use to benchmark parts of an Elixir application. It is versatile and extensible, with more than a few plugins to enhance its functionality.

Prerequisites

Elixir Environment

To follow along, you will need to locally install Elixir and Phoenix. The easiest way to do so is to follow the official Elixir instructions, which will give you a couple of options for:

Local installation on Linux, Windows, and macOS
Dockerized versions of Elixir
Package manager version setups

I recommend a local installation for the best results.

Setting Up Our Elixir Application

For this article's purposes, we will set up a simple Elixir application that can calculate the Fibonacci sequence.

Start by creating a new application with mix new fibonacci_benchmarking:

As output, you will see the following:

* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/fibonacci_benchmarking.ex
* creating test
* creating test/test_helper.exs
* creating test/fibonacci_benchmarking_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd fibonacci_benchmarking
    mix test

Run "mix help" for more commands.

Next, in your favorite editor, add the following code to the lib/fibonacci_benchmarking.ex file:

defmodule FibonacciBenchmarking do
  def list(number), do: Enum.map(0..number, &fibonacci/1)

  def fibonacci(0), do: 0
  def fibonacci(1), do: 1
  def fibonacci(n), do: fibonacci(0, 1, n-2)

  def fibonacci(_, prv, -1), do: prv
  def fibonacci(prvprv, prv, n) do
      next = prv + prvprv
      fibonacci(prv, next, n-1)
  end
end

Note: The original code can be found in rosettacode.org.

Go to the fibonacci_benchmarking directory and run the following commands:

mix deps.get
iex -S mix

And once inside the elixir shell, you can run this:

iex(1)> FibonacciBenchmarking.list(10)
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

If you see the above output, you have successfully set up your application, and are ready to proceed with Benchee.

Implement Benchmarking on an Elixir Application

First, we will need to install Benchee. Start by adding the following to your mix.exs file:

defp deps do
  [
    {:benchee, "~> 1.0", only: :dev}
  ]
end

Next, run the following command:

mix deps.get

We can validate that Benchee is installed by running the Elixir shell and the following snippet:

Benchee.run(%{
    "10_seq" => fn -> FibonacciBenchmarking.list(10) end
})

Benchee might take a second or two to warm up, but on completion, you should see the following output:

Operating System: Linux
CPU Information: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
Number of Available Cores: 8
Available memory: 62.76 GB
Elixir 1.13.3
Erlang 24.2.2

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 0 ns
reduction time: 0 ns
parallel: 1
inputs: none specified
Estimated total run time: 7 s

Benchmarking 10_seq ...

Name             ips        average  deviation         median         99th %
10_seq      973.04 K        1.03 μs  ±2735.15%        0.77 μs        1.60 μs

The code above makes a call to FibonacciBenchmarking.list(10), and Benchee measures the time it takes to execute the function.

Let's take a moment to understand the output of Benchee. By default, Benchee will output the following information:

ips stands for iterations per second. This number represents how many times a given function can be executed in a second. Higher is better.
average is the average time it takes to execute the function. Lower is better.
deviation is the standard deviation of the results. This is a measure of how much the results deviate from the average.
median is the middle value of the results.
99th % - 99% of all the measured values are less than this value.

While running Benchee in this fashion can be useful for ad-hoc benchmarks, a much better method is to include Benchee as part of our unit tests.

Automate Benchee for Elixir and Run Tests

By default, all Elixir and Phoenix applications have a test directory and use ExUnit to run tests. Our goal is to get Benchee running as part of our test suite and test a different implementation of the Fibonacci sequence.

Start by creating a new file called test/benchee_unit_test.exs, and copy the following code into it:

defmodule BencheeUnitTest do
  use ExUnit.Case
  alias Application.TestHelper

  @tag :benchmark
  test "benchmark fibonacci list generation" do
    # capture benchee output to run assertions
    output = Benchee.run(%{
      "case_10_numbers" => fn() ->
        FibonacciBenchmarking.list(10)
      end
    })

    results = Enum.at(output.scenarios, 0)
    assert results.run_time_data.statistics.average <= 50_000_000
  end
end

Go ahead and run mix test on the console. Validate that the output looks like the following:

Operating System: Linux
CPU Information: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
Number of Available Cores: 8
Available memory: 62.76 GB
Elixir 1.13.3
Erlang 24.2.2

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 0 ns
reduction time: 0 ns
parallel: 1
inputs: none specified
Estimated total run time: 7 s

Benchmarking case_10_numbers ...

Name                      ips        average  deviation         median         99th %
case_10_numbers        1.31 M      765.77 ns  ±3552.51%         599 ns        1165 ns
.

Finished in 10.7 seconds (0.00s async, 10.7s sync)
1 test, 0 failures

Randomized with seed 612867

So far, we have integrated Benchee into our test suite and added the first test to validate one of the test cases. Let's add the second test case to compare. Update the test function to:

defmodule BencheeUnitTest do
  use ExUnit.Case
  alias Application.TestHelper

  @tag :benchmark
  test "benchmark fibonacci list generation" do
    # capture benchee output to run assertions
    output = Benchee.run(%{
      "case_10_numbers" => fn() ->
        FibonacciBenchmarking.list(10)
      end,
      "case_1000_numbers" => fn() ->
        FibonacciBenchmarking.list(1000)
      end
    })

    results = Enum.at(output.scenarios, 0)
    assert results.run_time_data.statistics.average <= 50_000_000
  end
end

Just like we did before, we can run the test suite with mix test, and validate that the output looks like the following:

Operating System: Linux
CPU Information: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
Number of Available Cores: 8
Available memory: 62.76 GB
Elixir 1.13.3
Erlang 24.2.2

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 0 ns
reduction time: 0 ns
parallel: 1
inputs: none specified
Estimated total run time: 14 s

Benchmarking case_1000_numbers ...
Benchmarking case_10_numbers ...

Name                        ips        average  deviation         median         99th %
case_10_numbers          1.33 M     0.00075 ms  ±3419.36%     0.00059 ms     0.00109 ms
case_1000_numbers     0.00010 M       10.21 ms    ±11.54%        9.83 ms       15.29 ms

Comparison:
case_10_numbers          1.33 M
case_1000_numbers     0.00010 M - 13598.03x slower +10.21 ms
.

Our second test scenario tries to compare the performance of the Fibonacci sequence with a list of 1,000 numbers; however, this is not a very practical way to test with multiple inputs. We can take advantage of the Benchee.run hooks and provide a list of inputs for each scenario.

Go ahead and open the test/benchee_unit_test.exs file and replace the contents with this code:

defmodule BencheeUnitTest do
  use ExUnit.Case
  alias Application.TestHelper

  @tag :benchmark
  test "benchmark fibonacci list generation" do
    # capture benchee output to run assertions
    output = Benchee.run(%{
      "generate_list" => fn(input) ->
        FibonacciBenchmarking.list(input)
      end
    },
    inputs: %{
      "case_10" => 10,
      "case_100" => 100,
      "case_1000" => 1000,
      "case_10000" => 10000,
      "case_100000" => 100000
    })

    results = Enum.at(output.scenarios, 0)
    assert results.run_time_data.statistics.average <= 50_000_000
  end
end

In this new version of the code, we have generalized our generate list case to accept a list of inputs, and we can now run the test suite with mix test.

However, because of the size of our last input, you will get a message like this:

❯ mix test

Operating System: Linux
CPU Information: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
Number of Available Cores: 8
Available memory: 62.76 GB
Elixir 1.13.3
Erlang 24.2.2

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 0 ns
reduction time: 0 ns
parallel: 1
inputs: case_10, case_100, case_1000, case_10000, case_100000
Estimated total run time: 35 s

Benchmarking generate_list with input case_10 ...
Benchmarking generate_list with input case_100 ...
Benchmarking generate_list with input case_1000 ...
Benchmarking generate_list with input case_10000 ...
Benchmarking generate_list with input case_100000 ...


  1) test benchmark fibonacci list generation (BencheeUnitTest)
     test/benchee_unit_test.exs:6
     ** (ExUnit.TimeoutError) test timed out after 60000ms. You can change the timeout:

       1. per test by setting "@tag timeout: x" (accepts :infinity)
       2. per test module by setting "@moduletag timeout: x" (accepts :infinity)
       3. globally via "ExUnit.start(timeout: x)" configuration
       4. by running "mix test --timeout x" which sets timeout
       5. or by running "mix test --trace" which sets timeout to infinity
          (useful when using IEx.pry/0)

     where "x" is the timeout given as integer in milliseconds (defaults to 60_000).

     code: output = Benchee.run(%{
     stacktrace:
       (elixir 1.13.3) lib/task.ex:794: Task.await/2
       (elixir 1.13.3) lib/enum.ex:1593: Enum."-map/2-lists^map/1-0-"/2
       (benchee 1.1.0) lib/benchee/benchmark/runner.ex:77: Benchee.Benchmark.Runner.parallel_benchmark/2
       (elixir 1.13.3) lib/enum.ex:1593: Enum."-map/2-lists^map/1-0-"/2
       (elixir 1.13.3) lib/enum.ex:1593: Enum."-map/2-lists^map/1-0-"/2
       (benchee 1.1.0) lib/benchee/benchmark.ex:103: Benchee.Benchmark.collect/3
       (benchee 1.1.0) lib/benchee.ex:48: Benchee.run/2
       test/benchee_unit_test.exs:8: (test)
       (ex_unit 1.13.3) lib/ex_unit/runner.ex:500: ExUnit.Runner.exec_test/1
       (stdlib 3.17) timer.erl:166: :timer.tc/1
       (ex_unit 1.13.3) lib/ex_unit/runner.ex:451: anonymous fn/4 in ExUnit.Runner.spawn_test_monitor/4



Finished in 60.0 seconds (0.00s async, 60.0s sync)
1 test, 1 failure

Randomized with seed 567196

As it happens, we hit a timeout error after 60 seconds. Fortunately, as part of the stack trace, we get a couple of suggestions on how to solve this problem. For now, update the test suite with this code:

defmodule BencheeUnitTest do
  use ExUnit.Case
  alias Application.TestHelper

  @tag :benchmark
  @tag timeout: :infinity
  test "benchmark fibonacci list generation" do
    # capture benchee output to run assertions
    output = Benchee.run(%{
      "generate_list" => fn(input) ->
        FibonacciBenchmarking.list(input)
      end
    },
    inputs: %{
      "case_10" => 10,
      "case_100" => 100,
      "case_1000" => 1000,
      "case_10000" => 10000,
      "case_100000" => 100000
    })

    results = Enum.at(output.scenarios, 0)
    assert results.run_time_data.statistics.average <= 50_000_000
  end
end

Note: Depending on your system, running that last scenario will take a while; feel free to remove it to continue with the tutorial.

Now that we have a baseline of our Fibonacci sequence generator's performance, a common and useful exercise is to compare the performance of different implementations of the same algorithm. In this case, we have an alternative implementation of the Fibonacci sequence generator based on a recursive function.

Start by updating the lib/fibonacci_benchmarking.ex file with the following code:

defmodule FibonacciBenchmarking do
  def list(number), do: Enum.map(0..number, &fibonacci/1)
  def list_alternate(number), do: Stream.unfold({0,1}, fn {a,b} -> {a,{b,a+b}} end) |> Enum.take(number)

  def fibonacci(0), do: 0
  def fibonacci(1), do: 1
  def fibonacci(n), do: fibonacci(0, 1, n-2)

  def fibonacci(_, prv, -1), do: prv
  def fibonacci(prvprv, prv, n) do
      next = prv + prvprv
      fibonacci(prv, next, n-1)
  end
end

Following that, we will update the test/benchee_unit_test.exs file to account for both implementations:

defmodule BencheeUnitTest do
  use ExUnit.Case

  @tag :benchmark
  @tag timeout: :infinity
  test "benchmark fibonacci list generation" do
    # capture benchee output to run assertions
    output = Benchee.run(%{
      "generate_list_enum" => fn(input) ->
        FibonacciBenchmarking.list(input)
      end,
      "generate_list_stream" => fn(input) ->
        FibonacciBenchmarking.list_alternate(input)
      end
    },
    inputs: %{
      "case_10" => 10,
      "case_100" => 100,
      "case_1000" => 1000,
      "case_10000" => 10000,
    })

    results = Enum.at(output.scenarios, 0)
    assert results.run_time_data.statistics.average <= 50_000_000
  end
end

The update test case will run two scenarios side by side for each of the prescribed inputs, letting us compare their overall performance. Go ahead and run mix test to see the results.

Compiling 1 file (.ex)
Operating System: Linux
CPU Information: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
Number of Available Cores: 8
Available memory: 62.76 GB
Elixir 1.13.3
Erlang 24.2.2

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 0 ns
reduction time: 0 ns
parallel: 1
inputs: case_10, case_100, case_1000, case_10000
Estimated total run time: 56 s

Benchmarking generate_list_enum with input case_10 ...
Benchmarking generate_list_enum with input case_100 ...
Benchmarking generate_list_enum with input case_1000 ...
Benchmarking generate_list_enum with input case_10000 ...
Benchmarking generate_list_stream with input case_10 ...
Benchmarking generate_list_stream with input case_100 ...
Benchmarking generate_list_stream with input case_1000 ...
Benchmarking generate_list_stream with input case_10000 ...

##### With input case_10 #####
Name                           ips        average  deviation         median         99th %
generate_list_stream        1.66 M      601.29 ns  ±3858.34%         441 ns         955 ns
generate_list_enum          1.35 M      742.83 ns  ±3201.26%         610 ns        1164 ns

Comparison:
generate_list_stream        1.66 M
generate_list_enum          1.35 M - 1.24x slower +141.55 ns

##### With input case_100 #####
Name                           ips        average  deviation         median         99th %
generate_list_stream      258.49 K        3.87 μs   ±512.17%        3.29 μs        8.04 μs
generate_list_enum         31.71 K       31.54 μs    ±26.84%       30.57 μs       41.17 μs

Comparison:
generate_list_stream      258.49 K
generate_list_enum         31.71 K - 8.15x slower +27.67 μs

##### With input case_1000 #####
Name                           ips        average  deviation         median         99th %
generate_list_stream       17.67 K      0.0566 ms    ±14.14%      0.0550 ms      0.0988 ms
generate_list_enum         0.102 K        9.84 ms     ±9.92%        9.60 ms       14.14 ms

Comparison:
generate_list_stream       17.67 K
generate_list_enum         0.102 K - 173.90x slower +9.78 ms

##### With input case_10000 #####
Name                           ips        average  deviation         median         99th %
generate_list_stream        501.16      0.00200 s    ±33.05%      0.00190 s      0.00370 s
generate_list_enum            0.27         3.75 s    ±12.05%         3.75 s         4.06 s

Comparison:
generate_list_stream        501.16
generate_list_enum            0.27 - 1876.93x slower +3.74 s
.

Finished in 65.3 seconds (0.00s async, 65.3s sync)
1 test, 0 failures

Randomized with seed 839542

As you can see, our list function's Enum implementation is much slower than the Stream implementation, especially when the input size is larger. Comparing the performance of the two implementations is valuable in understanding the trade-offs and will help you develop more performant applications.

When adding benchmarking tests to parts of your automated testing, consider the potential drawbacks, such as the increased time it takes to run the tests. In this case, the benchmarking tests are tagged with :benchmark and can be excluded from the default test suite. This allows us to run the benchmarking tests separately from the unit tests and only when we need to.

A much better approach is to take advantage of CI/CD pipeline integration like GitHub Actions and run the benchmarking tests as part of the pull request validation process. This way, we can run the benchmarking tests as part of the CI/CD pipeline and get the results without having to run the tests locally.

Improving Benchee Reporting

Now, while seeing results on the console can be useful for a quick glance, the console is not the most convenient way to share results with your team. Benchee provides a number of different ways to export your results to a file.

For this example, we will use benchee_html to generate an HTML report with our benchmarking test results. To do this, we will add the benchee_html dependency to our mix.exs file:

def deps do
  [
    ...
    {:benchee_html, "~> 1.0", only: [:dev, :test]}
  ]
end

Next, we will update the test/benchee_unit_test.exs file to generate the HTML report:

defmodule BencheeUnitTest do
  use ExUnit.Case

  @tag :benchmark
  @tag timeout: :infinity
  test "benchmark fibonacci list generation" do
    # capture benchee output to run assertions
    output = Benchee.run(%{
      "generate_list_enum" => fn(input) ->
        FibonacciBenchmarking.list(input)
      end,
      "generate_list_stream" => fn(input) ->
        FibonacciBenchmarking.list_alternate(input)
      end
    },
    inputs: %{
      "case_10" => 10,
      "case_100" => 100,
      "case_1000" => 1000,
      "case_10000" => 10000,
    },
    formatters: [
      Benchee.Formatters.HTML,
      Benchee.Formatters.Console
    ])

    results = Enum.at(output.scenarios, 0)
    assert results.run_time_data.statistics.average <= 50_000_000
  end
end

Let's go ahead and run the tests again:

mix test

On completion, you should see the following report open in your browser:

The HTML report provides a much more detailed view of the benchmarking results and allows us to share results with our team easily. For example:

In addition to the HTML report, Benchee also supports exporting results to JSON, CSV, and XML formats. Exporting results to a file is a great way to integrate them with automation, such as CI/CD pipelines.

Monitoring Your Elixir App in Production

Benchee can help you discover potential performance bottlenecks, but what about how fast things really are in your production app?

To be able to discover new and existing bottlenecks, and solve bugs and other issues your users may face, you need to use an APM. AppSignal has been supporting Elixir developers for years and seamlessly integrates with your app. Bonus: We're the only APM that ships stroopwafels to new users 😎

Wrapping Up and Next Steps

In this tutorial, we discovered how to benchmark Elixir applications with the Benchee library.

We also learned how to compare the performance of different implementations of the same algorithm.

Yet we have only scratched the surface of Benchee's capabilities. As a next step, I highly encourage you to explore the available Benchee configuration options and visualization plugins.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Using Abbreviations instead of Aliases

Allan MacGregor 🇨🇦 — Sun, 08 Nov 2020 00:00:00 +0000

I recently came across one of Fish shell best and likely most underrated features, abbreviations ; a great alternative to aliases and dare I say a full replacement.

The Problem with Aliases

An alias is a (usually short) name that the shell translates into another (usually longer) name or command. Aliases allow you to define new commands by substituting a string for the first token of a simple command.

The main issue is that aliases are expanded behind the scenes, take the following alias:

alias rm='rm -Rfi'

The way aliases work on most shells the following drawbacks become apparent:

They hide what is really happening as they are resolved behind the scenes
Makes copy-pasting a command-line for instructions to others, difficult of not impossible
Command history will be recorded as the alias, so history loses value
Aliases become less valuable if you have to edit the options

The better approach

Abbreviations work on the same principle as aliases but with the main advantage that an abbreviation will get expanded ‘live’ as is being typed. Let’s look at the following example:

abbr --add miex "iex --erl "-kernel shell_history enabled" -S mix"

I converted this from an alias that I used on a daily basis for elixir development, here are the advantages so far:

The abbr expands “live”, the git completions work as normal and commandline doesn’t have to lie or do any other hacks like that.
Clean history. Using abbr means other developers can understand your terminal history.
Easy to use a shortcut that’s close to what you want and edit it.

Summary

abbr make for a better and more understandable alias
I’m replacing all my aliases with abbreviations

Exploring Headless Commerce

Allan MacGregor 🇨🇦 — Wed, 04 Nov 2020 11:16:40 +0000

One of the interesting side effects of the COVID-19 pandemic is the unprecedented acceleration and adoption of ecommerce by many companies.

However, is my experience that headless commerce is a very abused term and as someone brilliantly put it:

Headless Commerce is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, sp everyone claims they are doing it.

The purpose of this article is to explore a bit of the general concepts of Headless commerce as well review some of the available technologies and platforms available for implementing a headless commerce solution.

What is Headless Commerce?

At it's core every ecommerce website is composed of two functional halves, the frontend which handles the presentation layer and the backend that handless the business logic.

Traditionally, both systems are tightly coupled meaning both the frontend and the backend are developed and served by the same framework/platform.

At its core, headless commerce is an architecture pattern that decouples the presentation logic from the e-commerce backend system that handles things like inventory, order fulfillment, payments, etc.

It helps to think of headless commerce as a strategy for building your ecommerce solution rather than a particular technology or approach, as headless commerce can be implemented in several different ways.

Why use Headless Commerce?

Headless can open up a world of possibility from a client acquisition standpoint, as well as a way to offer more digital ecommerce options.

Headless commerce promises a variety of benefits:

Customizability
Speed
Scale
Control

More importantly, a headless approach gives us the ability to provide customers with more individualized and personal experiences.

Having the frontend decoupled can drastically increase the speed of development to launch new features without having to touch the heavy and often expensive backend logic.

Having an independent frontend, allows brands to implement completely different strategies to present and market their products, for example, a content-first strategy where the transactional aspects are secondary and a high amount of emphasis is put on the content about the product.

Technology

Now to the fun part, implementation. For the purpose of this initial article, I won't into full detail on the implementation details rather the following is a list of available solutions that are currently available on the market.

Frontend Technologies

The following is a non-exhaustive list of solutions for building ecommerce presentation layers and storefronts.

Vue Storefront website

Vue Storefront is the open-source frontend for any eCommerce, built with a PWA and headless approach, using a modern JS stack.

Vue Storefront is probably one of the first, if not the first Headless commerce framework as well as the first PWA one.

One of the main advantages of Vue Storefront is that can work with any ecommerce backend and provides adapters for some of the most popular ones like:

Magento
Shopify
BigCommerce
commercetools

Front-Commerce website

Front-Commerce is a layer that gathers data from your eCommerce backend and related services to directly serve pages to consumers

Front-commerce has been around since 2018, and like Vue Storefront they are PWA storefront, but with a strong focus on Magento 1 and Magento 2.

Next.js Commerce website

The all-in-one starter kit for high-performance e-commerce sites.

Next.js commerce by vercel is fairly new but looks extremely promising. Right now it can only support BigCommerce as its backend but other backends are on the roadmap.

Additionally, this project is opensource so you can follow the development on their Github

Backend Technologies

The following is a non-exhaustive list of solutions for building ecommerce backends, which can include catalogs, payments, content management, inventory, order management, and so on.

BigCommerce website

BigCommerce is a SaaS ecommerce platform that provides some headless capabilities.

While BigCommerce is and started as an all-in-one solution they offer one of the most extensive supports for building headless ecommerce implementations and constantly adding more to the mix.

Their official documentation Developers Guide to Headless Commerce offers a variety of approaches for implementing a solution:

Headless without coding
Custom solution but without building from scratch
Extend an existing solution or build from scratch

Commercetools website

commercetools is a next-generation ecommerce platform build with a headless first approach and on a microservices architecture.

While commercetools is aimed at medium to large-sized companies, it is definitely an option to consider due to its API first and headless focused architecture as well its ability to scale and and extensive set of functionality.

One thing to keep in mind about Commercetools is that both the licensing and implementation cost will be prohibitive for some of the smaller merchants.

Nacelle commerce website

Nacelle is a headless eCommerce platform made for developers who want to create superior customer buying experiences.

Nacelle is one of the most interesting options I stumbled on while doing research for this article. Nacelle market's itself as a headless commerce platform, but in reality, it would be more appropriate to call them a middleware.

Nacelle handles the data flow between your back end to your PWAs and custom applications; which in my opinion fits perfectly the definition of middleware, and unlike the other two previous options you will still need your products and ecommerce backend set up in one of the following two platforms:

Magento
Shopify

Still, Nacelle in my opinion makes it a perfect solution for merchants looking to transition to a headless strategy without fully having to replatform.

References

HeadlessCommerce

Originally published in allanmacgregor.com

Circuit Breaker Pattern in Elixir

Allan MacGregor 🇨🇦 — Mon, 02 Nov 2020 00:00:00 +0000

Circuit breaker is used to detect failures and encapsulates the logic of preventing a failure from constantly recurring, during maintenance, temporary external system failure or unexpected system difficulties.

In the age of microservices, we are more than likely to have services that are calling and are dependent on external services outside of control.

Remote services can hang, fail or become unresponsive. How can we prevent those failures from cascading through the system? from taking critical resources?

Enter the Circuit breaker pattern; the pattern was popularized in the book Release It by Michael Nygard and by thought leaders like Martin Fowler

The idea behind this pattern is very simple; Failures are inevitable, and trying to prevent them altogether is not realistic.

A way to handle these failures is wrapping these operations into some kind of proxy. This proxy is responsible for monitoring recent failures, and use this information to decide whether to allow the operation to proceed or return an early failure instead.

This proxy is typically implemented as a state machine that mimics the functinonality of a physical circuit breaker which my have 3 states:

Closed: In this state the circuit breaker let’s all the requests go through while keeping track of the number of recent failures, and if the number of failures exceeds a specific threshold within a specific timeframe, it will switch to the Open state.
Open: In this state, any requests are not sent to external service instead we either fail immediately returning an extension or fall back to a secondary system like a cache.
Half-Open: In this state, a limited number of requests from the application are allowed to pass-through and call our external service. Depending on the result of these requests the circuit breaker will either flip to a Closed state or go back to the Open state reseting the counter before trying to open again.

The Circuit Breaker pattern offers a few key advantages worth noting:

The Half-Open state gives the external system from recovering without getting flooded.
The Open state implementation gives options for how we want to handle failure wether failing right away or falling back to a caching layer or secondary system.
This pattern can also be leveraged to help to maintain response times by quickly rejecting calls that are likely to fail or timeout.

Example

For our example, let’s imagine that we have the following scenario:

We are running a job board aggregator that will consume job postings from Github and other sources. However, since we are consuming a few API we run the risk of that API to hit limits or be down.

Let’s start by creating an example API connector to Github Jobs that retrieves the latest 50 jobs posted:

defmodule CircuitBreaker.Api.GithubJobs do 
    ...
    @spec get_positions :: none
    def get_positions do
        case HTTPoison.get(url()) do
        {:ok, response} -> {:ok, parse_fields(response.body)}
        {:error, %HTTPoison.Error{id: _, reason: reason}} -> {:error, reason}
        end
    end
    ...
end

file : lib/circuit_breaker/api/github_jobs.ex

All this connector is doing is making a request to jobs.github.com retrieving the json, parsing, and returning the list of jobs. If we want to test this we can manually call get_positions on our console:

iex(1)> CircuitBreaker.Api.GithubJobs.get_positions
{:ok,
 ["Software Engineer", "Backend Engineer (w/m/d)",
  "Senior Frontend Engineer (f/m/d)", ...]}

Circuit Breaker Switch

Now that we have ability to make calls to get the job postings we need to build our circuit breaker to wrap around the the Api adapter. Let’s take a look at a skeleton for our switch.

defmodule CircuitBreaker.Api.Switch do
  use GenStateMachine, callback_mode: :state_functions

  @name :circuit_breaker_switch
  @error_count_limit 8
  @time_to_half_open_delay 8000

  def start_link do
    GenStateMachine.start_link( __MODULE__ , {:closed, %{error_count: 0}}, name: @name)
  end

  def get_positions do
    GenStateMachine.call(@name, :get_positions)
  end
  ...
end

file : lib/circuit_breaker/api/switch.ex

For implementing our circuit breaker we could use the gen_statem behavior directly or in this case leverage the GenStateMachine package which gives us tracking, error reporting and will work with the supervision tree.

The first two functions we added are:

start_link: will start the circuit breaker with an initial state and a specific name.
get_positions: this is our public client api that wraps around the Github Jobs adapter we just built.

An important thing to note here is the first line:

  use GenStateMachine, callback_mode: :state_functions

In this callback mode, every time you do a call/3 or a cast/2, the message will be handled by the state_name/3 function which is named the same as the current state. In this case our state_name functions will be open, closed, half_open.

Let’s go ahead and start by adding our closed state code:

  def closed({:call, from}, :get_positions, data) do
    case CircuitBreaker.Api.GithubJobs.get_positions() do
      {:ok, positions} ->
        {:keep_state, %{error_count: 0}, {:reply, from, {:ok, positions}}}
      {:error, reason} ->
        handle_error(reason, from, %{ data | error_count: data.error_count + 1 })
    end
  end

file : lib/circuit_breaker/api/switch.ex

All we are doing is calling the Api adapter get_positions and depending on the results we are either returning the positions list or handling the error.

Let’s go ahead and jump into the terminal and try to get the list of positions through our circuit breaker:

iex(1)> CircuitBreaker.Api.Switch.start_link
{:ok, #PID<0.231.0>}
iex(2)> CircuitBreaker.Api.Switch.get_positions
{:ok,
 ["Software Engineer", "Backend Engineer (w/m/d)",
  "Senior Frontend Engineer (f/m/d)", ...]}

Let’s add the function for the other two states and review how the circuit state change works:

  def half_open({:call, from}, :get_positions, data) do
    case CircuitBreaker.Api.GithubJobs.get_positions() do
      {:ok, positions} ->
        {:next_state, :closed, %{count_error: 0}, {:reply, from, {:ok, positions}}}
      {:error, reason} ->
        open_circuit(from, data, reason, @time_to_half_open_delay)
    end
  end

  def open({:call, from}, :get_positions, data) do
    {:keep_state, data, {:reply, from, {:error, :circuit_open}}}
  end

  def open(:info, :to_half_open, data) do
    {:next_state, :half_open, data}
  end

And let’s add a couple of private utility functions:

  defp handle_error(reason, from, data = %{error_count: error_count}) when error_count > @error_count_limit do
      open_circuit(from, data, reason, @time_to_half_open_delay)
  end

  defp handle_error(reason, from, data) do
    {:keep_state, data, {:reply, from, {:error, reason}}}
  end

  defp open_circuit(from, data, reason, delay) do
    Process.send_after(@name, :to_half_open, delay)
    {:next_state, :open, data, {:reply, from, {:error, reason}}}
  end

Most of the magic is happening on the open_circuit function which is doing two things:

First, we schedule a message to set our circuit breaker state to half_open after our specified delay
Second, we return a new state setting the circuit breaker fully open

After 8000 milliseconds , the circuit breaker now on open state will receive our scheduled message and change the state to half_open.

Finally, during half_open state, we will try to make the calls to the api endpoint and in case of failure, we will switch automatically back to fully open and try again.

Conclusions

Circuit Breakers are a valuable pattern to have in our arsenal, as they can help increase system stability and have a more reliable way to handle errors with remote services.

This example just scratched the surface of what you can do with circuit breakers, there is plenty of opportunities to expand this pattern further, e.g.

Improve the logic for tripping the breaker by also looking at the type of errors, and frequency.
Add Monitoring and logging once the circuit breaker changes state
Fallback to a secondary service or cache layer before returning failure.

Finally, as with any pattern is important to keep in mind the use case and decided if this kind of behavior is desired.

The full code for this example can be found in circuit_breaker_example

5 Reasons to Love Elixir

Allan MacGregor 🇨🇦 — Sun, 01 Nov 2020 20:25:02 +0000

Whenever I have a new side project idea, the need to build a quick prototype, or even an opportunity to build something from scratch at at work, I keep coming back to Elixir time and time again.

Elixir has a strong pull on me, and not without reason. The language creators have done an exceptional job at building not only a fantastic language but also a thriving ecosystem.

The following 5 are some of the reasons why I keep coming back to Elixir.

Mix

Mix is a build tool that ships with Elixir and provides tasks for creating, compiling, testing, debugging, managing dependencies, and much more.

Mix is comparable to tools like rake or artisan if you are coming from the Ruby and PHP worlds respectively

Creating a new project

mix new myproject

Running that command will generate our new project with the boilerplate structure and necessary files. The output will look something like

- creating README.md
- creating .formatter.exs
- creating .gitignore
- creating mix.exs
- creating lib
- creating lib/myproject.ex
- creating test

The most important file that is generated is our mix.exs file which contains the configuration for our application, dependencies, environments and the current version. The file might look something like this:

defmodule Myexample.Mixfile do
  use Mix.Project

  def project do
    [
      app: :myexample,
      version: "0.1.0",
      elixir: "~> 1.11",
      start_permanent: Mix.env() == :prod,
      deps: deps()
    ]
  end

  def application do
    [
      extra_applications: [:logger]
    ]
  end

  defp deps do
    []
  end
end

Overall, mix is really powerful and easy to extend; and is one of the reasons why using elixir is so friendly to new developers.

Additional Resources

Credo and Dialyxir

Credo is a static code analyzer for elixir, that has a very particular focus on teaching and building code consistency.

Dialyxir is a mix wrapper for Erlang Dialyzer; which is another tool for static code analysis.

$ mix dialyzer
...
examples.ex:3: Invalid type specification for function 'Elixir.Examples':sum_times/1.
The success typing is (_) -> number()
...

While both tools are static analysers they fill different needs and offer different kinds of insights into our code.

Credo is better suited to see of our code follows the common 'good' code practices accepted by the community; while Dialyxir on the other hand let us catch things like type errors and unreachable code.

Additional Resources

Processes

One of Elixir's biggest selling points is the concurrency support, and the scalability and fault tolerance that comes with it; at the core of this concurrency model we have elixir processes.

Processes in the context of elixir are not the same as operating system processes, elixir processes are incredibly lightweight in comparision and can be describe as having the following characteristics:

isolated from each other
run concurrent to one another
communicate via message passing
run inside of the Erlang VM

Process can be created and spawn directly like:

pid = spawn fn -> 1 + 3 end

Which all it does it spawns a process to execute the anonymous function; to more complex uses cases with message passing and supervision trees. One of the main abstractiosn that builds upon these processes is Tasks.

Tasks which provide better error reporting and introspection:

iex(1)> Task.start fn -> raise "oops" end
{:ok, #PID<0.55.0>}

15:22:33.046 [error] Task #PID<0.55.0> started from #PID<0.53.0> terminating
** (RuntimeError) oops
    (stdlib) erl_eval.erl:668: :erl_eval.do_apply/6
    (elixir) lib/task/supervised.ex:85: Task.Supervised.do_apply/2
    (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Function: #Function<20.99386804/0 in :erl_eval.expr/5>
    Args: []

Tasks are incredibly handy and enable us to run work both synchronously with Task.await/1 and asynchronously with Task.async/1

Additional Resources

The Syntax

Elixir syntax is one of its main appeals; and at times you can clearly see the Ruby influence in the language and the clear focus on developer happiness and usability.

defmodule Article do 
    defp read(:content, [paragraph | article]), do: read({:content, article})
    defp read(:content, []), do: {:done, []}

    def read_article() do
        {:done, _} = read({:content, get_article()})
        for _ <- 1..10, do: something()
    end
end

But looking at the snippet above makes it clear that while syntax is highly readable this is not Ruby, there are a couple things happening in the code above:

mix of inline and fully expanded functions
multiple function heads with pattern matching
simple recursion
function parameter decomposition

Elixir, in my opinion can pack a lot of expressiveness into a very small amount of code that is also highly readable.

Additional Resources

Phoenix Framework

Phoenix is a web development framework written in the functional programming language Elixir. Phoenix uses a server-side model-view-controller pattern. Based on the Plug library, and ultimately the Cowboy Erlang framework, it was developed to provide highly performant and scalable web applications. - wikipedia

Saving the best for last we have the Phoenix Framework, which often gets compared to the Laravel or Ruby on rails; but in my honest opinion is much much better.

Frameworks like RubyOnRails and Laravel suffer of a capital sin, there is too much freaking automagic. By this I mean those frameworks try to abstract too much and hide too much complexity to the point where they become more of a hindrance in certain scenarios.

Phoenix, has the right amount of scaffolding and abstractions without taking anything away from your control. Getting a Phoenix project up and running is easy as:

$ mix archive.install hex phx_new
$ mix phx.new demo --live

Phoenix Liveview

Phoenix LiveView is an exciting new library which enables rich, real-time user experiences with server-rendered HTML.

LiveView enables developers to build realtime interactive apps without touching any Javascript; it's all done with server side logic.

Additional Resources

Originally published in allanmacgregor.com

The Thing Most Developers Get Wrong On Interviews

Allan MacGregor 🇨🇦 — Fri, 17 Apr 2020 15:24:51 +0000

As an interviewee your main priority is to help the interviewer visualize what kind of employee you'll be at their company, and how your previous experience and achievements makes you a great for the position.

One of the things that I still find surprising after years of hiring and interviewing engineers, is how poorly software engineers prepare in average for the behavioral and soft part of the interview.

As a hiring manager, I'm looking for software engineers that are flexible, that can perform under pressure, and above all adapt to their environment. Therefore, assessing soft skills, is a crucial part of the evaluation process.

Now chances are that if you are an engineer actively interview, you had been asked some of the following behavioral questions:

Could tell about a challenging problem that you had to solve in the last year or so?
Could you share with me an instance where you had to deal with conflict within your team and how you handle it?
Can you walk me through a difficult bug you had to solve?

Behavioral-based interview questions generally start with any one of the following phrases:

Tell me about a time when you…
Describe a situation when you…
Give me an example of a time you
Think about an instance in which you…
Tell me how you approached a situation where

In most cases people do a decent job of answering this kind of questions. However, in my experience there is still a lot of fumbling when it comes to this part of the interview so I would recommend the following:

Write a script.
Have specific stories in mind.
Practice makes perfect.
Slow down.

Behavioral interviews are fairly predictable, so putting the work ahead of time to practice and have a good narrative will help you make a great impression and stand out.

Now, there is a particular section of the behavioral interview that more often that not candidates get wrong or fail miserably to prepare for.

Why Us

I cannot stress this enough, before you prepare for anything research the company you'll be asked your reason for applying for the role or why you are interested to work for company X.

Many candidates give generic answers that could apply to hundred of other companies; at worst some candidates will come so poorly prepared that won't even have the faintest idea of what the company actually does.

As a candidate you want to show your interviewer that you're excited about their company and the problem they are tackling.

Part of the your interview preparation should include reading about the company and trying to answer the following questions:

What does the company do and why am I interested in this company?
What differentiates them from their competition?
What excites me about working on the industry?
What are the company core values?

Be Engaging

In this day an age, the myth of the lone curmudgeon developer on the basement of the company is long death; you need to show that you have the skills for the job and that you are good team member.

One of your goals as part of the interview is to turn the interviewer into your internal advocate. But how do you that? Easy, build a real connection with your interviewer.

At some point during the behavioral interview you'll be given the opportunity to ask a few questions. Be sure to be prepared with a few questions about the company and the product, but I would also recommend that you ask a few questions that give the interview an opportunity to talk about themselves.

People love talking about themselves, so demonstrating genuine curiosity about your interviewer’s experience will make you feel more personable and help you standout. Here are some good examples that you can use:

Can you tell me a bit more about your background and the thing you love more about this company?
How can you describe the culture at the company?
What are the opportunities for growth and personal development?
In your opinion what is the most important skill or quality for a candidate to be successful at this role?

Make sure to actually listen and engage with their answer. When you focus on actively listening, the conversation naturally becomes a two way exchange between to people, and will boost the chances of the interview connecting with you.

Finally, soft-skills are not different so remember, practice, practice, practice.

Ecommerce Platform Prototype with Elixir and Phoenix

Allan MacGregor 🇨🇦 — Sun, 20 Oct 2019 13:19:47 +0000

CrimsonCommerce is an experimental ecommerce platform build using a functional programming approach and leveraging the power of Erlang/Elixir and the Phoenix framework.

Building the Catalog(Category) context:

Quick 30 minute stream, where I'm working on the creation of a product and category models, inside the catalog context.

You can follow progress of the project here:

amacgregor / crimson_commerce

An OpenSource Ecommerce platform on the Elixir/Phoenix stack, which aims to leverage the power of functional programming for ecommerce.

Crimson Commerce

An open-source Ecommerce platform in Elixir and Phoenix
Explore the docs »

View Demo · Report Bug · Request Feature

About The Project

CrimsonCommerce is an experimental ecommerce platform build using a functional programming approach and leveraging the power of Erlang/Elixir and the Phoenix framework.

Built With

Goals

Highly scalable ecommerce platform
Provide tools and frameworks to reduce development complexity – both incidental and accidental
Build an ecommerce framework that developers are happy to use
Traceability, the ability to know what happened at any point in time to made decisions or solve problems.

Getting Started

To get a local copy up and running follow these simple steps.

Prerequisites

This is an example of how to list things you need to use the software…

View on GitHub

DEV Community: Allan MacGregor 🇨🇦

Advanced Dependency Injection in Elixir with Rewire

Introduction to Rewire

Why Use Rewire and Mox for Elixir?

Getting Started with Rewire and Mox in Your Elixir Project

Step 1: Create a New Elixir Project

Step 2: Add Dependencies

Step 3: Configuring Mox

Core Concepts of Rewire

Dependency Injection with Rewire in the EmailScanner Module

Mocking with Mox for Testing

Things to Consider When Using Rewire and Mox

Wrapping Up

Using Dependency Injection in Elixir

What Is Dependency Injection?

Types of Dependency Injection

Advantages of Dependency Injection

How Dependency Injection Works

Dependency Injection Diagram

How Can Dependency Injection Be Applied in Elixir?

Practical Application of Dependency Injection in Elixir for Testing

Injecting Mocks

Configurable Dependencies

Testing with Dependency Injection

Implementation with ExUnit

Testing EmailScanner with ExUnit

Common Dependency Injection Pitfalls and Best Practices

Wrapping Up and What's Next

Powerful Caching in Elixir with Cachex

Understanding Caching and Its Importance in Elixir

Benefits of Caching

Drawbacks of Caching

Caching Options in Elixir

Introduction to Cachex for Elixir

Adding Caching to Your Elixir Application with Cachex

Working with Phoenix

Caching Database Queries

Caching Views

Caching API Responses

Checking for Stale Cache Data

TTL

Fallbacks

Wrapping Up

How To Use Zig for Elixir NIFs

What We'll Cover

What Is Zig?

Setting up Zig for Elixir Development

Installing Zig

Configuring Zig for Elixir NIF Development

Verifying the Setup

Understanding NIFs in Elixir

Why Use NIFs in Elixir?

Advantages of Using Zig for NIF Development

Writing our First NIF

Writing a Simple NIF in Zig

Compiling the Zig Code Into a NIF

Integrating the NIF with Elixir

Loading the Shared Library

Defining a Fallback Function

Calling the NIF Function from Elixir Code

Writing Elixir Code to Use the NIF

Verifying That the NIF Works

Considerations

Wrapping Up

How To Instrument Your Elixir Application with AppSignal

Why Instrument Your Elixir Application?

Pre-requisites

Custom Instrumentation for Your Elixir App

Key Instrumentation Concepts

Instrumentation Using Helper Functions for Your Elixir App

Instrumenting Your Elixir App with the Function Decorator

Helper Vs. Decorator Functions

Wrapping Up

Benchmark Your Elixir App's Performance with Benchee

Why Do I Need to Benchmark My Elixir Application?

What Is Benchee?

Prerequisites

Elixir Environment

Setting Up Our Elixir Application

Implement Benchmarking on an Elixir Application

Dependency Injection with Rewire in the `EmailScanner` Module

Implementation with `ExUnit`

Testing `EmailScanner` with `ExUnit`