DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

Tech Tim (@TechTim42)
Tech Tim (@TechTim42)

Posted on

Redis Hackathon - Sanic Currency Exchange API

Overview of My Submission

api endpoint for Sanic Currency Exchange API

This submission is to switch my application (Sanic Currency Exchange API) from using PostgresDB to be able to use Redis Stack(RedisJSON) as data store, as shown in the diagram, it contains 3 parts of work.

refactor and data store switch

  1. Refactoring, to implement Dependency Injection to split Database logic and Application logic.
  2. Auto Test/Regression Test, to guarantee that before and after db switch the API behaves the same, this part of work was done in Postman, and the test scripts were shared in repo.
  3. Switch Data Store to RedisJson, to implement a Redis repository which keep the same behaviour as Postgres repository, to make the data store switch happen easily and smoothly.

The Idea

I cloned this Currency API (Github link below) about 3 years ago and kept maintaining it. The initial thoughts were maintaining the dependencies to avoid some security problems. Gradually, I started to re-write the application codes to make it more maintainable, readable. (This is a long way to go, I am sure it has not been finished yet, πŸ˜€)

This API is to fetch currency rates from Europe Central Bank, and offer the rates in RESTful API.

Since I saw the Redis hackathon announcement, I got this idea to switch data store from Postgres to Redis JSON. I used Redis before as a caching layer for application, but this is my first time to use the Redis modules. This implementation probably have a lot to improve, please feel free to leave comments to make it better and help me to learn more about Redis JSON and Redis Stack modules.

say thanks

Submission Category: MEAN/MERN Mavericks

  • Use Redis to the RESTful API to add caching and advanced searching capabilities using JSON module.
  • Use Redis as a primary database instead of Postgres.
  • Redis Stack docker image was used here instead of Redis Cloud, to keep the Postgres and Redis Stack have the same hardware spec.

How the data is stored:

The data is store as a document, for each day of the currency rates.

{
   "2020-01-01": {
      "USD": 1,
      "EUR": 1.1,
      "AUD": 0.97
   }
}
Enter fullscreen mode Exit fullscreen mode

How the data is accessed:

In python codes, the data is accessed through a simple get

r.json().get('2020-01-01')
Enter fullscreen mode Exit fullscreen mode

Language Used

  • Python (Sanic Framework)
  • Javascript

Link to Code

GitHub logo tim-hub / sanic-currency-exchange-rates-api

This is a self hosted, free, open source Python Currency Exchange Rate API fork.

Migrate Sanic Currency Exchange Rates Api to Use Redis Stack @Redis Hackathon

Use Redis Stack (RedisJson) to replace Postgres as main storage, plus some refactoring

stack change

How it works

How the data is stored:

The data is store as a document, for each day of the currency rates.

{
   "2020-01-01": {
      "USD": 1
      "EUR": 1.1,
      "AUD": 0.97
   }
}
Enter fullscreen mode Exit fullscreen mode

How the data is accessed:

In python codes, the data is accessed through a simple get

r.json().get('2020-01-01')
Enter fullscreen mode Exit fullscreen mode

Performance Benchmarks

postgres performance redis json performance

Load test is used here to measure the performance.

  • same Machine (my personal macbook laptop)
  • same spec for DB
    • 10GB Volume
    • Same ARM64 Instance

The 2 key results here are iterations (higher is better) and iteration_duration(avg)(lower is better)

Fact Postgres Redis Stack
Iterations 25.26/s 5.52/s
Iteration Duration 560.15ms 5.53s

This performance benchmark is surprising to myself as…

Additional Resources / Info

Collaborators

This is a solo submission.

the end


Top comments (0)

🌚 Browsing with dark mode makes you a better developer by a factor of exactly 40.

It's a scientific fact.