DEV Community: Oleg

Building custom Headless CMS that powers Blogging platform

Oleg — Wed, 05 Aug 2020 05:05:25 +0000

Being a beginner means exploring new horizons of programming by stepping into unknown and getting your hands dirty.

It's fascinating to start working on a project with a new piece of technology, isn't it? Problems you encounter and try to solve aren't as easy though, they're part of your learning path.

So, let me share with you the experience of building my first distributed system: headless CMS, API and a blog. It may help you to create a similar system by yourself avoiding mistakes I did. By the way, it's the most comprehensive article I could find on the web🥰.

I won't explain introductory information but will leave links to resources where you can learn more. Additionally, you can watch the talk in case you're not familiar with the advantages of distributed systems over monolithic ones.

The source code of the project along with the walkthrough the UI and API documentation (suggest not reading until you finish the article):

Vidzhel / Bluro

A headless content management system and a blog that was built with the use of the CMS in sake of learning and interest

Bluro

Bluro is a headless content management system (CMS) that was built with a clear goal to achieve - create something interesting and learn useful skills and technologies. As a part of the project a blog "Tech Overload" was build.

Backend uses neither no express nor no sequelize or other ORMs only bare bones, only hardcore

Frontend on the other hand makes use of React and bunch of other dependencies to simplify development

Table of content

Bluro

Installation

Go to folder where you want a new folder with the source code to be downloaded, run:

git clone https://github.com/Vidzhel/Bluro.git

Running dev server with docker

Running the server with docker requires creation of config files under configs directory:

View on GitHub

Exploring unit and integration testing in Redux Saga

Oleg — Tue, 14 Jul 2020 10:45:59 +0000

Redux is an extremely useful library that facilitates managing of an app's state. Among many middlewares, Redux-Saga fits me the best as in a React-Native project I'm working at the moment, I've had to deal with lots of side effects that would bring me endless headaches in case I put them in components. With the tool, the creation of complex flows becomes straightforward. But what about the testing? Is this as smooth as the usage of the library? While I can't give you the exact answer, I'm to show you the real example of problems that I faced.

If you're not familiar with testing of sagas, I recommend reading a dedicated page in the docs. In the following examples, I use redux-saga-test-plan as it gives me the full strength of integration testing alongside with unit testing.

A bit about unit testing

Unit testing is nothing more than testing of a small piece of your system, usually a function, that has to be isolated from other functions and, which is more important, from APIs.

Going ahead I'd say that I haven't seen the point of unit testing in my project yet. I moved all the business logic and APIs abstractions into external modules to let the sagas handle only the flow of the application. So I haven't had those big sagas that I couldn't split into smaller, clearly see what they do (they're quite self-explanatory).

There you can see the usual way to check our effect creators. If there had been any API calls that, I would've mocked them using jest.fn.

As we've done with tedious work, let's proceed to the main course.

Integration testing

The significant drawbacks of unit testing are external calls. You need to mock them. In case your sagas consist only of these calls and no logic, testing step by step, while abstracting all dependencies, becomes a dull task. But what if we only want to check the flow without dealing with each of the effects. What if we need to test sagas in the context of the state, with the use of reducers. I have excellent news, that's exactly what I wanted to share with you.

Test multiple sagas

Let's consider the following example, which is an adapted version of the code from my project:

Here we have the root saga sessionWatcher that initialize application by calling initApp right away after loading and also waits for the action to load a project by id. The project is loaded from storage after which, we save the project to the state and call an external function that saves the session and loads a map. The example shows all sorts of problems we can stumble upon during the testing process: working with multiple sagas, accessing the state, calling APIs that we'd like to avoid.

The test above tests all the sagas and split into few parts. The first part introduces objects that we'll use to test our sagas, the second, is testing itself. We call expectSaga that'll run the root saga and tests it against the checks listed below it.

The first function we see is provide, which uses matchers to locate the effect creators that will be mocked. The first tuple (pair of values) uses the effect creator from the Redux Saga library and matches exactly to select effect that calls getProjectFromStorage selector. If we want more flexibility, we can use matchers that are provided by Redux Saga Test Plan library as we do in the second tuple, where we say to match by function, ignoring its arguments. This mechanism allows us to avoid accessing the store or calling some functions and much, much more that I don't list here.

After that, we have a chain of effect cheks. Note that we don't have to put them in the specific order or include all the effects but rather list the effects that we expect to see. However, calls to dispatch must be in order.

The chain ends with silentRun function that does three things: runs our test, suppress timeout error and return a promise.

Simulate an error

To simulate an error, we can use already familiar providers and a helper function from redux-saga-test-plan/providers to replace an effect with an error.

Reducers and state

But what about the state, how would we test all together with reducers. With the library, the task is a no-brainer. Firstly, we need to introduce our reducer:

Secondly, we change our test a bit by adding withReducer, which allows us using dynamic state (you can provide state without reducer by calling withState), and hasFinalState, which compares the state with the expected one, functions.

More on using the magic library here. Hope you enjoyed the reading, thank you.

My story of building a server-client desktop application

Oleg — Tue, 30 Jun 2020 14:44:29 +0000

About a year and a couple of months ago, during my freshman year, I was at the start of my learning path and like others newbies, hadn’t much experience. The only way to gain skills and get better at something is practising, and this article was created, to share my experience that may help you.

I strongly encourage you to start building your project and gain hands-on experience in the technology or domain you’re learning through an unpleasant path of ups and downs, as the saying goes “Practice makes permanent”.

The article is regarded, as the sharing of my story of building a desktop application in C#. The first part is the introduction that precedes the main part and explains what it was all about. The discussion aims to help beginners or more advanced programmers who have never created complex client-server apps. You can freely access the code by the link but don’t judge me, in the time of building I had no idea about “clean code”.

Preamble

The story starts, when I and my classmates were given a list of term paper subjects that we could choose from. There were three dozens of ideas, and most of them were connected with creating a calculator or data management system, like a simple version of that which are used to store students’ rating. Scrolling my eyes through the list, I found nothing special except the only topic that caught my attention. It was about creating a desktop messenger. I had literally no idea how I was supposed to write it, but at the same time, my mind was drawing images of the final version. Of course, at the end, I got a completely different result, but it didn’t stop me from dreaming about something like Telegram, which I consider being the best messenger on the market. The language that we were supposed to write the project in was C#. However, a friend of mine managed to get a permit on writing it in PHP.

In order to at least start the project, I decided to dive into the subject and learn as much as I could. It was the first big project for me, so I thought that would be great if I did something mind-blowing. As you know, there is the WPF Framework in the sharp’s world which you can use to build gorgeous Windows applications, means no cross-platform though. It’s based on the .NET Framework which exists only for Windows. If you want something more flexible than UWP is your choice.

Background

I had started writing some code two months before I had to hand the project out. If you have ever worked with WPF you may be aware of XAML, which is a markup language that is used for building UI, it’s like HTML, which I had had a bit of experience working with by that time, but much much dreader. So without diving too much into details, I started designing my application. Now, when I have much more experience in building such apps, I understand that the logic was the only part I needed to concentrate on, especially in the early stages. The unwillingness of starting from the hardest part and desire to impress others got the better of me.

I remember the time when I created the first element of my design which was a button, a feeling of fascination was overflowing me because it took me much time to get it to the satisfying state. It may be compared, to the feeling of completion of a task that you struggled to finish for some time and is the cause of me, and probably you, being a programmer. However, writing in XAML took me a decent time and a bunch of articles to get into it. The flexibility of the language won’t let you just write a component. Regardless the all I said previously towards it’s a great language, even though you need to be patient to get at least some results.

By that time, I hadn’t had any plan on how to move forward. I wrongly prioritised tasks and put the main accent on designing the messenger instead of writing its logic when the time for building architecture came I was completely lost, in what I was needed to do next. Struggling to come up with something I was researching the net and stumbled across a couple of articles that were braking in pieces the process of software architecting. A simplified idea behind those articles was to split your system into blocks or layers and build them around the core. Equipped with new knowledge, I somehow managed to produce the first version. I’m proud to present you the architecture and hope it’ll help someone who is experiencing problems similar to mine:

Designing the architecture

Messenger consists of two parts: a client and a server. Each has its specifics. The Client-side gets user’s input, processes it, saves necessary data locally and form a request to the server, the request will be sent as soon as there is such a possibility. In turn, Server-side waits until a client connects and sends data, processes it and sends a response. Simple, isn’t it?

It was time to make diagrams and splits the overall project into layers and then into libraries and classes. Here is some quality documentation.

Both the Client-side and Server-side have three layers:

Data Access Layer — represents modules that provide access to a local (I chose SQLite as a database, due to its simplicity) or remote storage.
Mediator Layer — creates a junction between all other modules, to make the management other layers simpler, which allows us to scale up in the future (see Mediator pattern).
UI Presenter — If previous layers were components of the application core this one represents the UI and was separated because it has code that applies only to the desktop version and can’t be ported on other platforms.

I used the Mediator pattern to reduce the number of connections between the modules of the apps. Model View ViewModel to separate code that represents UI (XAML). Usually, that kind of architecture requires splitting a system working process into two stages: an initialization, when all your modules and components are constructed, and the runtime, when the components are used. The modules know nothing about each other because they belong to the lower level of abstraction. In this way, you prevent yourself from headaches when you decide to add some functionality.

Client-side

Referencing that I said previously, the Client-side should process a user input then save some data to the local storage and send to the server, but how we can accomplish it? First thing first, to make the program more efficient consider using threads. Split up your program, to introduce threads without producing race conditions is something that you want. My solution was to create separated threads for big chunks of the system. The first for the server connection, as you always need to wait for the server response and reconnect on connection loss. The second for the user interface, to avoid situations when the program stops responding after you click on a button and another one for the part that would process data, save and send it.

Ok, next thing is slightly trickier. Assume you had logged in and started writing a message. You were clicking the send button when the connection with the server interrupted, then when the connect established you understood that the message wasn’t sent. So, how you would deal with it? My answer was simple. You need to store data (e.g. messages, contacts, personal info) locally. It solves the problem with the absence of the connection. To resend data to the server I decided to implement two command queues that simply store all incoming commands that haven’t been processed yet and outcoming commands that are pending of sending. As soon as the client gets a new server command, responsible for the connection handling thread puts it into the queue then in case if the processing thread is not busy, the command will be handled and UI updated. The other way round works as well: a user clicks “send a message” then if the processing thread is free, some data will be saved locally afterwards a command is put into the queue where it will be grabbed and sent to the server. However, there is still a small problem which I haven’t solved: if a user closes the client all data in the queues is lost, so I suggest saving them into the database or somewhere else.

Server-side

The server-side is fairly simple. It creates a new thread for every new client and saves or grabs some data from the database.

Designing the data storage

Without saving data, your application appears to be stateless. And if you don’t want to create temporary chat rooms, you need to store user data somewhere. For the purpose, I used the SQLite relational database. In a moment, we will discuss the design of the data storage that I used on server and client sides, but before it, I’d say the following: When designing your data storage you should introduce as many tables as you need to prevent duplication of columns in tables. It’s called table normalization.

From the ER diagram, we can see that there are three main entities and their connections. Users can create many groups of two types: in channels, only admins can write messages, while in groups, all the members are allowed to share thoughts. Yet another thing that I need to mention is that password mustn't be stored as a plain text because if someone gets access to your database he will be able to see all the passwords. Instead, you have to store hashes. When a new user registers, you need to create a hash from the password and send it to the server. Every time the user logs in, you create the hash from the password and compare it with that is stored in the database.

However, there is a huge mistake, someone of you’ve noticed, in the way groups and users are connected. The many-to-many relationships that you can find there is being a bad design of relational databases. And actually, it’s pretty reasonable, as I stored lists of members, admins and user’s groups as values separated by a unique character. Whenever you want to check whether a user is a member of a group, you need to grab all the members, separate them and only then check. It’s extremely inefficient and can be solved by introducing a proxy table that will have two columns and store a user and a group (the same approach can be applied to all other many-to-many relationships). It isn’t an ideal solution. You can imagine how much records the table will store. Additionally, some databases allow you to store data in JSON. Also, you can consider using No SQL databases such as Mongo DB that stores data in already mentioned JSON.

Messages, which are sent by users, have groups as a destination point. Files and photos can be attached, but they need to be stored differently. Storing files in a database is a very bad idea. A better approach will be to store the files on the server, referencing them by their hashed name (to prevent overriding files with the same name).

Here it is, I hope that you enjoyed my first article. Leave your feedback down in the comments, who knows maybe you’ll help me to get better in writing and programming eventually.