DEV Community: Tipicalls

A Logging, Monitoring and Alerting platform for less than $20

Tipicalls — Fri, 26 Aug 2022 21:43:00 +0000

Can you track your application, server logs, monitor these, generate alerts from these with same platform and for few dollars only?

The answer is yes, and I have been doing that with Elasticsearch Cloud. Elasticsearch was originally built to track and store logs, and the cloud version allows you also to do the rest: Monitoring and Alerting.

1 — Tracking logs

Either you want to keep and track your servers or applications logs, Elasticsearch is a good place to do that. I have been using this platform on both On-Premise and on the Cloud for almost 6 years now. At the end, I would suggest you the cloud one, so you do not have to install anything and worry about the platform. Also, you can get an instance for less than $20 per month.

Regarding logs, there are some tools like Logstash that can help you to parse these on your server, so you send only what you need and within the format you want to. Still, I would suggest you to write your own script just posting the data you need. It will be easier to maintain, and you can add, modify the structure of data you send easily with a simple POST query.

You can get an idea of how to write this down in the following article: “10 lines of code that will save your life!”

2 — Monitoring and Alerting

Once you have your instance created, you can start building visualization and dashboards. From these, you can then on a daily basis look at how much CPU, bandwidth, number of users, … whatever you tracked, you have.

Great thing is that you can set up threshold in these visualizations and create alerts based on these. I have been using email alerts so far, and it helped a lot.

In Tipicalls.com, the meeting and collaboration platform we built, we are using Elasticsearch to monitor and alert on both web client usage, audio and video bridge usage and issues.

I hope this article helped.

Feel free to visit our 100% free online meeting room platform named tipicalls.com

10 lines of code that will save your life

Tipicalls — Thu, 25 Aug 2022 22:03:00 +0000

Do you really control how much your Web application is going to cost in terms of bandwidth being used? Have you anticipated potential huge bills?

Most of the cloud providers are now basing their pricing on bandwidth usage, which is really smart as this is not something as a developer we usually handle and monitor...

Unless you do the following below, you will never have any clue about how much data your application is retrieving from the database or from any server every day and so how much the bill will be for your boss at the end of the month for such database or server.

If bad things happen, it will be also very difficult for you to understand which query from the hundreds you do is generating too much traffic…

Here is an example of what you could do when querying a Firebase database. It could be another database or a server instead, the logic would be exactly the same:

What we do here is that for each query we do to the DB, we track the path of such query (could be the table name for a SQL DB) and how much data in Bytes we are retrieving from such query.

Then every 3 minutes, we build an JSON object with all these different path values we queried and for each, the total number of Bytes of data we retrieved from it.

Such JSON is then sent to Elasticsearch Cloud.

We can also add the information regarding the total session usage of the user and the user ID.

This way you can easily build a visualization in Elasticsearch giving you per user (or for all) how much data is being retrieved from the DB.

You can build a monitoring dashboard this way, and also set email alerts based on a total bandwidth threshold you may not want to go over.

When something bad happens, you can easily retrieve the usage peak, which query generated it and which user too.

You can get an Elasticsearch cloud instance for 20US$ and this will really save your life when things get bad…

If you are interested in understanding more about the Elasticsearch configuration part, let me know!

I hope this article helped.

Feel free to visit our meeting and collaboration platform tipicalls.com

Bye

Scaling servers on the fly with few lines of code

Tipicalls — Wed, 24 Aug 2022 22:47:00 +0000

You sometimes do not need complex architecture to scale servers automatically…

Why would you pay for some Cloud scaling services and learn how to use these if you could do it for free with just a few lines of code?

Moreover, you may want to scale your servers up and down based on something very specific that your application tracks, like number of logged users, network bandwidth, local weather… or scale only at a very specific day and time… In these cases, having the hands on your scaling mechanism let you create, remove servers based on whatever you need to.

In our free meeting and team collaboration platform Tipicalls, we had a scaling need for all the servers handling the audio and video pieces of the meeting rooms, and we wanted to scale based on the CPU usage on these servers.

So first we needed, on the conferencing servers, to build a service running and tracking the real-time CPU usage and pushing such information into our database.

This can be achieved with a simple NodeJS file called by a service every minute or 2:

So now you have, in your DB, the information of the current CPU being used by all your servers in real time.

Now you need to create a kind of Orchestrator, removing or creating servers based on this information.

To do so, create a server.js file in NodeJS getting info from your DB:

Here we are tracking CPU, but keep in mind you can track anything you want and write scaling rules based on anything. You can build much more complicated rules. You could for instance create scaling rules based on network bandwidth usage, this way you make sure that you never get billed for extra network usage being above your bandwidth threshold!

So with just few lines you have now your own orchestrator creating and removing servers on the fly automatically.

To create servers automatically, most of the Cloud providers now are offering the possibility of building servers using a predefined image you may have built first yourself.

The remaining steps to have your server created are to be able to give it an IP address, create a DNS redirection for it, and start the instance based on such image. The instance will need also to create its own SSL certificate when first started too.

I will show you all these steps in this article here, so this one does not get too long.

James,

Visit Tipicalls.com, our free meeting and team collaboration platform

Are Unit and QA testing necessary!?

Tipicalls — Tue, 23 Aug 2022 21:06:11 +0000

Is having dedicated people or code to test an app, not just a big loss of time, money and productivity?

I have been managing a QA testing team and multiple development teams for a long time. I believe today that investing in testing, being either unit tests or QA tests, are a loss of time, money and so a bad choice.

1 — QA Tests

First, would you employ someone extra to test a house being built by someone else? Would you ask such person to push the walls, jump on the roof, try to break the house being built in any possible way…? Maybe you should… but you do not because you suppose that person building the house already knows his job, that he is doing already such testing if necessary and does not need someone extra to do it.

I believe we should do the same supposition for a developer building an application. A developer, in my mind, should be responsible for code he is pushing to production and test it by his own. He should not rely on an external team or person to validate that the code is working properly and not breaking something else. Splitting responsibilities is a risk to have developers not caring about the code they do and relying on testers to find the bugs. A developer having a bad code quality should just spend more time on testing his own code and not rely on someone else to do so. I have seen in my career a clear relation between code quality and QA testing. The more we were planning for QA testing the mode the code we were received was bad…

QA testing phases also delay a lot the project timelines and make difficult the small feature and iteration development strategy.

2- Unit Tests

Also, your application should be easy to update, to improve, and implementing new features should not take months or years. Unit tests are making fast changes more difficult. When implementing unit tests, you just double the size of code you have, making your app more complicated to manage and update. Every small change will become a nightmare for the developer. Also, do you really need unit tests to check that your app works fine?

A developer may spend month or years to fully cover an application with unit tests and may just need only 5 minutes to do a full regression testing. This has been the case for most of the apps we built, implementing unit tests would have been a tremendous loss of time.

3 — Conclusion

If you do the choice of not implementing unit tests and not having any QA testing neither, you still need to make sure that you, as a developer, still do the necessary testing. For every change, feature implemented, the developer needs to spend time on testing it, making sure everything works as it is supposed to do and that nothing else has been broken. If as a developer you do not spend such time then some end users will find the bugs for sure. This time spent on testing should be planned in advance. When estimating development tasks, developers should always keep in mind about time they will need to do the necessary testing too.

We have built our meeting and collaboration platform named Tipicalls.com this way. If you find some bugs, please let me know, this will mean someone within the dev team did not spend enough time on testing :-) and that potentially we should review our development strategy and maybe recruit some QA tester or implement some Unit Tests!! :-)

I hope this article helped.
tipicalls.com

UiPath: Pros and Cons

Tipicalls — Mon, 22 Aug 2022 16:55:55 +0000

You may be in the phase of looking for an RPA (Robotic Process Automation) provider and may be looking at UiPath platform. Here is some feedback:

We have been using UiPath for almost 2 years now, and I believe an article like this one may have helped me during the phase we struggled finding the correct platform and tooling for the RPA work we had to do.

1 -UiPath Pros

. REF

UiPath offers to Bot developers something called REF: "Robotic Enterprise Framework". This is a kind of library, giving you the skeleton that your bot needs. It includes the logic you need to have to plug the bot to an orchestrator and all the good practices you need to follow too. I strongly suggest you use it if your bot need to perform retries and loop over multiple data.

. Support

As a developer, you will probably get blocked plenty of times in your RPA automation journey. Not finding how to do something in particular, how to set up your bot… Feedback we had, was always fast and efficient. This is an excellent point here, as sometimes when the bot fails in the middle of an execution, you do not have much time to fix things with nobody around you that can help.

. Certification

As mentioned in this article: ' New to RPA? Few mistakes to avoid…', I strongly recommend getting certified first (Advanced Developer) before starting any UiPath work. Certification is cheap and will force you in understanding how a bot should work, how to plug it to an orchestrator and will give you a good idea about all the features you have in your hands with UiPath. Time you will be spending on that will save you a lot of time later on I can tell…(see article)

. Cloud Orchestrator

We first installed the 'On-premise' version of the UiPath orchestrator in our own infrastructure. Installation and set up are not complicated but still, it is time-consuming, so you may better choose the Cloud orchestrator instead as you do not have anything to install. This is what we did and by moving from on-premise to cloud we also got the reliability, redundancy that we did not have in house.
Also, when you have multiple bots running being critical for the company and generating a lot of revenues or savings, you may feel better if the orchestrator piece is not in your hands but in the hands of people knowing such platform much better than you do.

Even if we saw some downtimes on the Cloud platform, I recommend it instead of on-premise.

** -UiPath Cons**

. Pricing

UiPath is expensive compared to Blue Prism and some other tools. You will need to work in a medium, large size company to be able to pay for this kind of platform. If you do not, you need to expect of a lot of savings from your future bots, so you can get the needed budget.

. Reports

The orchestrator managing and running the bots will allow you to report on executions very easily with nice graphs. But something annoying is that you cannot get very detailed reports of your execution. Even with the Elasticsearch module on On-Premise, the parameters of each execution are encapsulated into a block of JSON, which makes that you cannot build reports based on these parameters. For instance, let's say you need to send emails to customers and the customer ID information is one parameter of each run, then the issue is that you will not be able to build a report to understand how many emails have been sent to a specific customer ID.

. Contracting help

You may find, depending on where you are located, more available external contractors familiar with Blue Prism than with UiPath. Good thing is that if you don't find any, UiPath provides their own contractors.

I hope this article helped you

Visit our free meeting and management platform, Tipicalls.com