DEV Community: Ivan Dlugos

Rust looks awesome

Ivan Dlugos — Sat, 07 Sep 2019 16:42:44 +0000

I'm (finally) starting to look at Rust and it seems to have hit the nail on the head. Actually I haven't written any code yet (shame on me) but from reading "The Book", it looks wonderful and I had to share :). Solving almost every major pain point I can think of in the programming languages I've used for any reasonable length of time (namely C/C++, Go if we're talking system langugages).

Like resource management for example… No automatic deep copying nor wondering who owns complex members data after a struct copy. Borrowing instead of reference/pointer management. Compile-time data race prevention? Seriously?! Seems to good to be true. I guess it does have some learning curve but if that means being able to worry less about correctness? Worth. every. minute.

Also, those value-containing ENUMs? Together with compiler-enforced complete matches. I can’t count how many times I’ve seen code not matching all possible cases in a switch/if-else…

Needless to say I’m getting really excited to start putting Rust through its paces. Please share your thoughts on this :)

Go — build a minimal docker image in just three steps

Ivan Dlugos — Sat, 07 Sep 2019 16:03:53 +0000

Docker & Go image by github.com/ashleymcnamara/gophers

Go — build a minimal docker image in just three steps

When you build your Go application for docker, you usually start from some image like golang:1.13. However, it’s a waste of resources to actually use that image for runtime. Let’s take a look at how you can build a Go application as an absolute minimal docker image.

1. Choose a Go version

While it might be tempting to use golang:latest or just golang there are many reasons why this is not such a good idea but the chief one among them is build repeatability. Whether it’s about using the same version for production deployment that you’ve developed and tested on, or if you find yourself in a need to patch an old version of your application, it’s a good idea to keep the Go version pinned to a specific release and only update it when you know it’s going to work with a newer one.

Therefore, always use full specification, including the patch version number and ideally even the base OS that image comes from, e.g. 1.13.0-alpine3.10

2. Keep it minimal

There are two aspects to this — keeping the build time low and keeping the resulting image small.

Fast builds

Docker caches intermediate layers for you so if you structure your Dockerfile right, you can reduce the time it takes for each subsequent rebuild (after a change). The rule of a thumb is to order the commands based on how frequently their source (e.g. source of a COPY) is going to change.

Also, consider using a .dockerignore file which helps keep the build context small — basically, when you run docker build, docker needs to feed everything in the current directory to the build daemon (that Sending build context to Docker daemon message you see at the beginning of a docker build). In short, if your repo contains a lot of data not necessary for building your app (such as tests, markdown for docs generator, etc), .dockerignore will help to speed the build up. At the very least, you can start with the following contents. Dockerfile is there so that if you COPY . . (which you shouldn’t, BTW) doesn’t have to execute and invalidate everything bellow when you change just that Dockerfile.

.git
Dockerfile
testdata

Small images

Very simple — use scratch. Nothing else comes close (because it can’t). Scratch is a special “base” image in that it’s not really an actual image but a completely empty system. Note: in an older version of docker, an explicit scratch image was actually used as a layer, this is no longer the case as of docker 1.5.

How this works is that you use a two-step build inside a single Dockerfile, where you actually build your app on one image, called builder (as an example, it can be actually any name you fancy), then copy the resulting binaries (and all other required files) to a final image based on scratch.

3. Putting it all together

Let’s see how a complete Dockerfile looks like, shall we?

Please leave a comment if you find this useful and/or you would like to share a few tips or tricks of your own.

What do you think of the ObjectBox database for Python

Ivan Dlugos — Wed, 08 May 2019 13:16:02 +0000

After Android, iOS, Java, Swift, C & Go, first support arrives for Python as well.

https://objectbox.io/objectbox-python/

id = box.put(Person(first_name="Joe", last_name="Green"))  # Create
person = box.get(id)  # Read
person.last_name = "Black"
box.put(person)       # Update
box.remove(person)    # Delete

Storing data in Go

Ivan Dlugos — Mon, 10 Dec 2018 11:51:38 +0000

Most programs usually work with some kind of data that needs to be persisted - ranging from different kinds of user input through internal app state, runtime cache, to program configuration.

In this article, I'm going to provide an overview of the options available in Go, based on a few criteria we use to differentiate types of databases for various use-cases. Regardless of whether the project you're working on is a desktop app, a server program/micro-service, an embedded system/IoT solution, you should be able to find a storage solution to fit your needs.

It is not unusual to use multiple database systems in a single project because each might fit a different purpose, e. g. one for logging/tracing and another one for user-data. Keep in mind, however, that this might become a burden to maintain, because, as with any other type of dependency, there is always a learning curve, changes you need to keep track of during updates and last but not least, security and licensing implications. Additionally, if you find yourself in need of the data from two databases in the same place for some action, you might reconsider your architecture choices.

Where the data is stored

From the locality perspective, there are two main options based on what type of application you are building and what kind of data the application needs to store.

Remote (server) storage - either a database server or other form of API server to which you connect from your program. Useful in case of multiple clients accessing a common data or sometimes in multiple services working with a single database server (not that the latter is considered an anti-pattern in the service-oriented architecture because it breaks service isolation).
Local storage (embedded) - when the data is only specific to one application installation, or the data needs to be available off-line, it's desirable to have the data locally. In the embedded mode, the database libraries are part of the program and working with the data doesn't require any server/service (neither local nor remote).

Semi-Local storage - similar to local storage, the "semi" in this category comes from using a database server running on the same machine as the program instead of an embedded library (as that's not an option for some databases).

How the data is stored and accessed

SQL database - stores structured data in tables (one entry per row) and provides a query language to access and create new sets.

Key-Value store - uses an associative array (map/dictionary) to store arbitrary data as values (usually serialized) accessible by numeric/string keys.
Document store - stores "documents" (JSON, XML, or arbitrary data) and provides ways to group and search those documents based on various criteria.
Object database - combines approaches from the above-mentioned types to support representing data as objects, usually comes with a tight integration with objects (Go structs).
Graph database- represents data as a collection of nodes and edges - useful in applications where multi-tier relations are the most important part of the data.

The way the data is used (most of the time)

This is the category where it's actually most common for your project to end up with multiple databases in place because it actually distinguishes the purpose and the capabilities you need for your application.

data ingress database - logging, tracing, monitoring, time series,
caching store - operational cache to increase performance and decrease latencies,
analytical database - most useful to perform data analytics, usually come with tools for these purposes.

Use-cases

I've selected a few interesting databases for the most common usage scenarios.

Monitoring, tracing, analytics

Prometheus - remote server, data-ingress, analytical
InfluxDB - remote server, data-ingress, analytical
Jaeger - remote server, tracing, monitoring

General SQL storage

TiDB - remote server, distributed, scalable, ACID compliant, MySQL protocol compatible
CockroachDB - remote server, distributed, scalable, ACID compliant
rqlite - remote server, distributed, scalable, SQLite based

And a plethora of clients and ORMs for "standard" SQL servers so if you already have an existing SQL server infrastructure, you should have no problem connecting to it.

Key-Value stores

BadgerDB - local/embedded, ACID compliant, transactions
BoltDB - local/embedded, ACID compliant, transactions

Object stores

ObjectBox - local/embedded, ACID compliant, transactions, queries

This category seems to be quite underrepresented in Go, however, there are quite a few ORMs for key-value stores and SQL databases:

GORM - remote server, supports MySQL, PostgreSQL, SQLite, MS SQL
BoltHold - local/embedded, provides serialization over BoltDB

I hope this gives you a place to start with your next project. Let me know in the comments if you feel your favourite library should be listed or if you're missing something.

Storing confidential data in the cloud securely with Cryptomator

Ivan Dlugos — Fri, 03 Aug 2018 01:24:35 +0000

TL;DR: If you have some data you need to store safely in your cloud folder (OneDrive, Google Drive, Dropbox, etc.) jump directly to the Cryptomator setup.

I’ve been slacking off on backups of my personal things for quite a while, despite being a software engineer who knows the importance of backups and a working restore process and have been doing it on work-related stuff (sources, databases, etc.). I’ve had some backups on CDs & DVDs like ten years ago, then moved the “important” stuff to the cloud (Dropbox at first, then OneDrive, Google Drive), but it’s been inconsistent and disorganized. Still, lot’s of stuff have only been transferred from an old hard drive to a new one when changing PCs, without any backup at all.

Lately, I’ve finally decided to clean things up, removing all the unnecessary mess and archiving the rest in a cloud storage. However, I’ve ended up with a few documents & contracts that necessitated keeping, but I’ve not been content having them just sit there in a cloud folder, however trustworthy and secure the provider seems to be. I’m not so afraid of them getting attacked but of someone getting hold of my account through more usual means, such as social engineering, for example. What I’m looking for is a way to password-protect my data whenever I don’t need to access it.

End-to-end encryption (E2EE)

In plain English, this means that all your data is encrypted with some kind of password/key before it departs from your device (be it desktop, laptop, smartphone), is transferred and stored encrypted and then is only decrypted after you download it back to your device with the same password/key. Therefore, only a person who has the password/key can read that data, even if they get access to the encrypted version in the cloud. Of course this doesn’t protect you from someone who has access to your device but that is a different topic altogether.

See wiki on E2EE or google it for a more complete and correct explanation. I have considered a few options how to store my confidential files in the cloud:

VeraCrypt

Based on the TrueCrypt, VeraCrypt allows you to create an encrypted container/partition which you can then connect as a local drive and store data in it as if it was just another disk. While this is great and quite fast locally, the problem is that the container is a single file (e. g. 1 GB) and any change in it causes a full synchronization event — obviously very inefficient to upload the whole container each time you update a single file inside it.

E2EE cloud storage provider

There are providers that seem to address my concern by providing E2EE as a part of their storage offering, e. g. Tresorit or MEGA. However, I don’t really need so much storage to justify paying for it, if there are other, cheaper (free) options which I can use with my existing cloud providers.

Boxcryptor

Boxcryptor seemed interesting, doing the encryption on your device and storing the individual encrypted files inside the local synced folder of a cloud storage you are already using (Dropbox, OneDrive, etc.). However, I’ve been put off by a few things:

During installation, you need to accept a Data privacy license/contract in German. I’m not saying I fully read all those licenses we accept during software installation, but I’d at least like to be able to do so, without speaking German or depending on machine translation. I know not everyone knows English either but then they are not reading this article :)
Despite having a free tier, you need to register and login. I’m not sure why that would be necessary and honestly, I’m not willing to try and find out.

Cryptomator

Similar to Boxcryptor, Cryptomator stores your encrypted files in your local cloud folder (or any folder for that matter, you can use floppy disks if you still fancy those) in a so-called Vault. It ticks the boxes for what I’ve been looking for:

the files are stored separately, so there’s only need to synchronize the changed/added/removed files, not the whole vault,
client-side: no accounts, no data shared with any online service,
AES encryption with 256-bit key length,
file names get encrypted & folder structure gets obfuscated,
unlimited number of vaults, each with an individual password,
the vault’s size is flexible (unlike with VeraCrypt which can grow but doesn’t shrink back), limited only by the size of the underlying storage, with only a negligible overhead.

To be completely frank, there is one problem I’ve encountered and depending on the way you work it might affect you considerably. Working with the drive it attaches is not really transparent for some programs — they seem to have a problem opening the file directly from the drive. As an example, I can open a PDF file in Acrobat Reader but not in Edge. Or a video won’t play in the PotPlayer. I don’t see a clear pattern but I think it’s related to how that program implements file operations. I’d just have to file an issue on the project’s GitHub but for the time being and for my use-cases, I’m OK with this limitation.

Setting up Cryptomator

The setup is quite simple but the UX of the app is not that good at the moment, especially for the first time user, so here you go:

Download from the cryptomator.org and install
Launch the app and you are “greeted” with the following window:

Click on the plus button and select “Create new vault”
Browse to the folder where you want to create your vault (e. g. OneDrive folder) and write a name of the vault (e. g. _Confidential) — _it will be the name of a folder Cryptomator creates for you in the current directory.

After hitting save, you are prompted to create a password for the vault. This should be quite strong (long and complex) because if someone gets hold of your encrypted data, they have a lot of time to try to brute-force decrypt it (using dictionary attacks and other techniques). There are guides on how to create such a password all over the internet. By the way, maybe you might want to use a password manager if you like.
After your vault is created, you can choose how you want to work with the vault, most importantly whether to save the password locally and auto-unlock (only do this if you trust the security of your device) or you want to unlock it manually, entering the password each time. The latter might be safer, depending on your situation, whether you trust everyone who has access to your computer.
Now you just open the Cryptomator drive, copy files over and to be sure, you can take a look at your encrypted data in the vault.

Unlocked vault automatically mapped by Cryptomator as a network drive

Encrypted Vault as visible in the target folder (OneDrive in this example), already synced to the cloud

The file path, name and contents are all encrypted when stored

Don’t forget you can’t just copy files directly to the vault (e. g. on OneDrive) — they won’t magically get encrypted. You always need to work with an unlocked vault, i. e. inside the network folder Cryptomator attaches for you.

That’s it, from now on, your confidential files are much more secure with a little upfront and a minimum ongoing effort. Cryptomator supports all the major platforms — Windows, Linux, macOS, Android, iOS so you can even access the files on other devices, synced automatically through your cloud provider. I personally will also take a look on how to have the data automatically backed up to another cloud storage in an unlikely situation that it is lost by one provider for some reason.

All of the mentioned approaches have some issues, be it performance, ease of use, price, etc. Ultimately it’s your choice which one you choose, if any, but do yourself a favour and take at least some precautions; better safe than sorry. And remember:

“Just because you’re paranoid doesn’t mean they aren’t after you.”

― Joseph Heller, Catch-22

Fast-track to a new language for an experienced programmer

Ivan Dlugos — Sun, 15 Jul 2018 15:28:16 +0000

OK, so you are a software developer/engineer/coder, whatever you call yourself, and you are having some thoughtful pause between the tasks coming in a never-ending stream…

"Yeah, I can write a pretty decent Java/Python/… (fill in your favourite) but so what? This stuff is getting boring! I need some challenge! I gotta learn some new tech!"

And there you are, searching the internet for the "Best programming language to learn in 20XX", checking out the TIOBE Index, comparing salaries those languages could earn you, etc. Been there, done that. And I'm not going to discourage you, quite the opposite, let me give you some hints to guide you along the way.

Motivation

Make sure you know why you are doing it in the first place. Is it to sate your curiosity? Or do you feel your skill set is getting outdated? You really, really wanted to finally do some functional programming? Maybe you are even one of the pragmatics who just wants to have the best tool for the job :)

Whatever your reasons, remember them or write them down on a sticky note or something. You are gonna need this to steer you in the deep waters of choices ahead of you. You might come across, for example, this latest cool-kid-on-the-block which does everything c++ does, but much better, bringing a lot of new features, solving all kinds of problems. It might just so happen that they forget to mention all kinds of problems it's gonna bring (you know.. you don't know what you don't know).

There are all kinds of interesting languages and technologies that might seem very tempting at first, but be ready and remember what your original motivation was. Either stick to that or just update (aren't we agile after all?). Just have and keep some general direction or your noble journey to master a new language might end up like that failed project. If you've worked for some time in a company of any size, you know which one I mean.

Picking the right one

This is strongly tied to your reasons and expectations. Take a quick look at the few you are interested in. Check out their homepages. Scroll through the introductory tutorials/quick start guides. If you want to see the hard data, take a look at their popularity on Google Trends and TIOBE. Filter them on StackOverflow and GitHub to see the state and drive of the community.

After your quick research, you would ideally end up with a single language to tackle next. We all know it's not gonna be the last one so if you can't decide, just throw a dice.

OK, got it, whats next?

Learn by observation

Actually follow the quick start guide or tutorial. Watch a video on YouTube. For some languages, like Go, for example, you don't have to install anything to quickly try them out. You can do it online in the browser, on a site provided either by the language creators or by third parties (e. g. coding-challenges, tutorial sites, …).

At this point, you should get an idea what the language feels like, how it compares to what you have used before. In case you have any doubts, don't be afraid to revisit previous steps to make sure your reasons and expectations are met and you actually like your choice.

Learn by practice

Obviously, you can't learn to program in a language without actually doing it. Here are some tips to get you started.

use a coding-challenge site, e. g. Codility, HackerRank or similar. This is great for two reasons. Firstly, you have an online environment to practice in (no need to invest your time and set up a proper local environment yet). Secondly, you have actual problems to solve, with a proper specification, already defined tests, etc. This is a huge benefit because not only you are getting the syntax into your "muscle memory" in bite-size pieces, but you are also training and refreshing your algorithmic thinking (and that never hurts). Yeah, some of those live editors are not too good (though they are getting better each time I check) and if this is a stopper for you, just set up your dev-env, use ProjectEuler and do the scaffolding for each problem yourself.
create something, useful or useless, doesn't matter too much. There are many things the coding-challenges do not cover - databases, APIs, sockets, and rest of the real world stuff you are dealing with daily. So after your algorithmic thinking is up to speed, you should try to use the language on some actual project. Something that would keep you engaged consistently and provide you with much-needed practice and experience.
give back to the community, by working on open-source projects in the language you have just learned. There is always someone in need of a helping hand so if you can't think of anything to create yourself, why not code some PRs and make the world a better place? Take a look at this list of OSS projects (by language) looking for outside help.

Use it professionally

This is the real deal. Until you have used the language for a reasonable time on a (preferably larger) project, you are not going to cover its intricacies and really get to know it. Therefore, as soon as you feel comfortable, or sooner if you are a perfectionist, try to incorporate your hard earned knowledge wherever it makes sense. But please make sure you are not one of those guys with a big new hammer for whom everything looks like a nail :)

Déjà vu

Time has passed, a few months, a year or more… You have finished a project or a few in the new language. You know how it ticks, you know what you like and don't like…

And if you are the ever curious guy like me, you are going to be asking the same question that got you here.

What's next?