DEV Community: Guilherme Dugaich de Oliveira

AI-powered search with Python, MongoDB, and OpenAI

Guilherme Dugaich de Oliveira — Sun, 02 Mar 2025 15:05:38 +0000

With the advancements in AI technology in the past couple of years, it became clear that it is here to stay and is already impacting our lives and how we interact with technology. As a software developer, I was drawn to discover how to use this new tool to enhance my problem-solving capabilities and provide better solutions.

When studying the field, I came across similarity search as a technique that I could easily implement and would provide a new way for searching information, combining my own databases with the power of language models. In this article, I will explain similarity search and some of its underlying concepts, as well as a walkthrough of a practical implementation of a movie recommendation system using Python, MongoDB, and the OpenAI API. I hope this can help others to better understand one of the many practical applications of language models.

This project was inspired by freeCodeCamp’s video on the same topic. Shout out to freeCodeCamp for providing amazing free resources for learning!

Similarity search

It is a way to search for text data using its meaning. This is initially a confusing concept when compared with regular database searches. Normally, you would look for text matches to find similar words, something like regex, but semantic meaning is something different. Think about the words ‘sea’ and ‘ocean’ - they are completely different words but have similar meanings and a shared context, which make them close to each other in terms of meaning.

But how can we tell our software programs that two different words or sentences have similar meaning? Since this is a human concept, some very smart people came up with a way to represent meaning in a way a computer can understand: vector embeddings.

Vector embeddings

In essence, vectors are arrays of numbers. However, their true power lies in translating meaning, as perceived by human language, into proximity in the vector space. It is beyond the purpose of this article to explain the concept of vector spaces. For now, it is enough to understand that vectors that are close to each other represent similar meaning.

To transform text into a vector we need to use a text embedding model, which is a machine learning model that has been trained to do this task of converting a piece of text into an array of numbers representing its meaning. Models such as Word2Vec, GLoVE, and BERT transform words, sentences, or paragraphs into vector embeddings. Images, audio and other formats can also be transformed into vectors, but we will stick to text for now. To learn more about vector embeddings, check out this great article from Pinecone, a vector database provider.

MongoDB’s vector search

MongoDB is a NoSQL database capable of storing unstructured data at scale. It uses a JSON-like format to store documents with a dynamic schema, making it flexible to store almost any type of data, which is less easy to do with traditional SQL databases where the data must be structured in a tabular form.

One of the things that can be stored in Mongo are vector embeddings, which we already know are just arrays of numbers. Besides that, Mongo has implemented vector search in its databases, which allows us to perform similarity search.

To perform a similarity search, we need two things: a query vector, representing what we are looking for, and a database of vectors, representing our universe of possibly similar vectors. So in this case we need to embed our natural language query, normally coming from the user of our application, into a vector. We also need to have the database with embedded data.

We can use search algorithms like K-Nearest-Neighbours to find the database vectors that are closer to our query vector, and determine which are the most meaningful matches within our search space.

Hopefully this will get clearer with the practical implementation. Let’s get to it.

Generating Embeddings

In this example, we will use Mongo’s sample movie database, specifically the database sample_mflix and the collection embedded_movies, which contains data about movies including title, cast, plot, etc. The sample data is offered for loading whenever you create a new Cluster. If you are not sure how to create a new Mongo cluster, follow the docs to create a free cluster and connect to it.

The sample data for this collection already has a field called plot_embdding that contains embeddings of the plot field. In this case, the field is embedded by OpenAI’s text-embedding-ada-002 model, which generates arrays with 1536 numbers. Different models will output different array sizes.

In a production-like situation where you want to use your own project’s data for this, you would have to generate the embeddings and save them to the database, which can be done in several ways. In this example we are going to use the embeddings already present in the sample data for ease, but we could also generate the embeddings using an open source model with the library sentence-transformers, for example. MongoDB’s documentation has a comprehensive section on how to create embeddings for your data.

Keep in mind that generating embeddings can be computationally expensive, and a good strategy must be outlined if you are doing this for an entire database, especially a large one.

OpenAI’s embedding API models are not free, and you need to set up an account and add balance to it in order to use it, which is what I did for this project. This might be good for production since it is a reliable supplier of AI models, but using open-source models would also be a good strategy. It all depends on the needs of the project.

Since we already have our database field embedded, now we need to create the function that will generate the vector for our search query. We should use the same model used for the rest of the database - mainly because we need to have same-size arrays in our vectors and also because using different models can impact the quality of our search results. We need to get an OpenAI API key and set it up on our environment variables. Reminder to never upload API keys to public repositories!

# generate_embeddings_openai.py
from openai import OpenAI
from dotenv import load_dotenv
import os

load_dotenv(override=True)

openai_key = os.getenv("OPENAI_API_KEY")
client = OpenAI(api_key=openai_key)

def generate_embedding_openai(text: str) -> list[float]:
    response = client.embeddings.create(
      model="text-embedding-ada-002",
      input=text,
      encoding_format="float"
    )

    return response.data[0].embedding

Before running the vector search we need to create a search index in the MongoDB database, which can be done in the Mongo interface or programmatically via the pymongopackage in Python. The index is an essential part to perform the search efficiently.

from pymongo.mongo_client import MongoClient
from pymongo.operations import SearchIndexModel
import time
from dotenv import load_dotenv
import os

load_dotenv(override=True)

# Connect to your Atlas deployment
uri = os.getenv("MONGODB_URI")
client = MongoClient(uri)

# Access your database and collection
database = client["sample_mflix"]
collection = database["embedded_movies"]

# Create your index model, then create the search index
search_index_model = SearchIndexModel(
  definition={
    "fields": [
      {
        "type": "vector",
        "path": "plot_embedding",
        "numDimensions": 1536,
        "similarity": "dotProduct",
        "quantization": "scalar"
      },
      {
        "type": "filter",
        "path": "genres"
      },
      {
        "type": "filter",
        "path": "year"
      }
    ]
  },
  name="plot_embedding_vector_index",
  type="vectorSearch",
)

result = collection.create_search_index(model=search_index_model)
print("New search index named " + result + " is building.")

# Wait for initial sync to complete
print("Polling to check if the index is ready. This may take up to a minute.")
predicate=None
if predicate is None:
  predicate = lambda index: index.get("queryable") is True

while True:
  indices = list(collection.list_search_indexes(result))
  if len(indices) and predicate(indices[0]):
    break
  time.sleep(5)
print(result + " is ready for querying.")

client.close()

Now we have everything we need to run our search: our embedded database, our embedded user query and the search index created in the appropriate field. With all that, the search can be performed as follows:

# query_movies.py
import pymongo
from dotenv import load_dotenv
import os
from generate_embedding_openai import generate_embedding_openai

load_dotenv(override=True)

mongodb_uri = os.getenv("MONGODB_URI")
client = pymongo.MongoClient(mongodb_uri)
db = client["sample_mflix"]
collection = db["embedded_movies"]

query = 'movies about war in outer space'
query_embedding = generate_embedding_openai(query)

results = collection.aggregate([
    {
        "$vectorSearch": {
            "queryVector": query_embedding,
            "path": "plot_embedding",
            "numCandidates": 1000,
            "limit": 4,
            "index": "plot_embedding_vector_index"
        }
    }
])

for r in results:
    print(f'Movie: {r["title"]}\nPlot: {r["plot"]}\n\n')

The vector search query get several parameters, which include:

queryVector: the user query embedded by the model, in this example we embedded the query 'movies about war in outer space' with the same model we used to embed the rest of the database.
path: the collection field in which to perform the vector search.
numCandidates: the number of possible results that are candidates to being the closest one to the user query.
limit: the numbers of top results we will return from the candidates. The closest ones from the query in the vector space.
index: the search index we just created.

Results and conclusion

When running our script on query_movies.py we get the following results:

Movie: Buck Rogers in the 25th Century
Plot: A 20th century astronaut emerges out of 500 years of suspended animation into a future time where Earth is threatened by alien invaders.


Movie: Farscape: The Peacekeeper Wars
Plot: When a full-scale war is engaged by the evil Scarran Empire, the Peacekeeper Alliance has but one hope: reassemble human astronaut John Crichton, once sucked into the Peacekeeper galaxy ...


Movie: Space Raiders
Plot: A futuristic, sensitive tale of adventure and confrontation when a 10 year old boy is accidentally kidnapped by a spaceship filled with a motley crew of space pirates.


Movie: V: The Final Battle
Plot: A small group of human resistance fighters fight a desperate guerilla war against the genocidal extra-terrestrials who dominate Earth.

We can notice that all four results have plots related to war in outer space, including some that don’t even share the same keywords, proving our point that this is not a simple text search. This meaning-driven search is a powerful tool to help you create more sophisticated applications.

The true value of this approach lies in the fact that you can use your proprietary data to make this type of search, delivering much better results to users and adding value to your application.

A beginner's introduction to databases, indexes and transactions

Guilherme Dugaich de Oliveira — Fri, 21 Oct 2022 13:57:21 +0000

Introduction

The term “database” is widely used in different contexts in the digital world. Since I started my career, I've seen this term referring to different things: Excel spreadsheets, tables on websites, text files, and other formats. Indeed, all these things can be considered databases, at least according to Wikipedia's definition:

…a database is an organized collection of data stored and accessed electronically.

This definition from Wikipedia sounds simple. It can be said that a text file containing my name and phone number is a database. However, this is very fragile—what if the file is deleted or corrupted? What if I add some information there that doesn’t belong? When developing a real-world application, we need a robust system to manage our data.

Databases are the most important part of any software system. After all, this is where all the information of an application is stored, such as user names, email, address and even passwords (hopefully cryptographically hashed). But how is a database system better than a text file? This article gives an overview and introduction to the concept of databases in the context of software development.

Database Management Systems

DBMS are special pieces of software designed to manage the storage of data in a computer system. They can read, write, remove or edit data, in addition to controlling access to the information and other features. Everyone who works with software development will eventually deal with a DBMS, so it is important to understand how they work, at least superficially. Some of the most widely used DBMS software today are MySQL, PostgreSQL and MongoDB. I became interested in databases when I learned to manage my data using a DBMS simply by writing SQL queries, and I was curious to know what was happening inside the DBMS.

The DBMS is a mediator with which the user can manage data reliably. It can be accessed through a graphic interface client, through the command line in a terminal, or through an application’s code, where typically queries are created and sent for a DBMS to execute in a database.

This is DBeaver, an open-source database graphic interface, that allows you to access lots of different relational databases (in this case a Postgres instance) where it is easy to see data and manipulate it. Here we have a users table with 3 entries – we can see our database much like a spreadsheet, with several features and capabilities.

Here we have the same data being accessed through postgres CLI tool psql. This is also an easy and fast way to access data, but it can be impractical when managing large datasets.

The most common case you will find as a developer is accessing a DBMS through an application’s code. This example shows JavaScript being used to access the same postgres database using sequelize, an Object Relational-Mapping library. Here we are accessing the database, inserting the data and then reading it.

Those systems behave much like a server responding to client requests. They get a query from the client and respond with data, or an error message in case there were a problem with the query or with the DBMS itself. The query comes from the client as a string, which is parsed, and then executed by the DBMS. Database servers can run on the same machine as the application calling them, or on a different machine on the same network. They can even run on a distant computer in the cloud.

Types of databases

The description given before applies to pretty much all database systems, and there are lots of them today. Each flavor of database has its own pros and cons, being used for different types of features and applications depending on the user’s needs. They are typically separated in two categories: relational and non-relational.

The relational database model emerged in the 1960s, and is the most used to date. In this model, data is stored in tables that contain rows and columns. Each row is an entry, or an observation, and each column is a field, or an attribute of that data entry. This helps to avoid duplicate information and makes it easier to create relationships between different tables.

Queries for relational databases are normally built with a Structured Query Language—SQL, that allows you to specify exactly the data you want, in the format that you want. If you work with software development, it is more than likely that you had to work with SQL, or at least some type of abstraction to it like an ORM (Object–relational mapping) that allows you to make queries from your code without having to write SQL statements.

Non-relational databases are normally referred to as NoSQL—not only SQL, and they are hard to define. They encompass various types of databases that don’t fit in with the relational model. It's possible to write a whole article about NoSQL databases and all their different types. Some of the most famous members of this class are MongoDB and Redis. It is not in the scope of this article to explore all types of databases, but the principles explained here apply to most of them.

NoSQL databases are more flexible in the way they store data. In the relational model, you have to respect all the constraints and rules of your schema, which is basically the structure of your tables and columns. Most non-relational databases allow you to store data in any way you want, for example MongoDB can store your data in collections of documents, that are basically JSON objects. Redis on the other hand is a key-value store, where you save just keys and their values, looking to optimize space and performance.

Transactions and their properties

A database transaction is a unit of work that is composed of a collection of queries that have a logical reason to be together. For example, consider a money transfer between two people. We need to have the amount of the transfer subtracted from the sender’s balance—if there is enough money—and added to the receiver's balance. This pair of operations constitutes one transaction.

To guarantee the validity of the data being recorded, database transactions should be compliant with the ACID properties.

Atomicity defines that transactions need to happen completely, or not at all. This is especially critical in systems that deal with financial data, like in our bank account example—if this money transfer transaction breaks after subtracting the sender’s balance and before adding to the receiver’s, we lose track of money. After a transaction starts, if everything runs correctly it will commit the changes, meaning that they will all be written to disk permanently. If anything goes wrong, it will rollback and none of the changes made by the transaction are persisted.
Consistency ensures that the data stored in the database is actually correct, meaning that it guarantees referential integrity and all other rules applied to a database system, so the information registered there makes sense. For instance, in the bank account example, you cannot have a money transfer to a user that is not on your users table.
Isolation guarantees that all database transactions happen independently of each other. This is especially important in systems where there are a lot of simultaneous queries hitting the database. The DBMS cannot allow that the changes of an uncommitted transaction interfere with another one.
Durability is the property that guarantees that the data, after a transaction is committed, is actually persisted to disk and will be available even after a system crash. This is a more complex topic and involves knowledge in hardware and other topics, but the general idea is straightforward.

There are a number of problems that can happen if transactions don’t follow those properties, like incorrect reads, interference between separate transactions and even the loss of data. Most database systems nowadays have these properties natively implemented and have features that guarantee those properties. This is a vast topic to cover in just a few paragraphs, so I encourage you to do more research if it interests you.

Indexes and how they can improve performance

Much like the index of a book that tells you where the content you want to find is located, a database index stores the memory addresses of your data in an efficient manner, so it can handle searches much faster. When you index a database column, its data is stored in a separate space in the disk, according to a different data structure.

We need indexes when our databases grow too big, and searching them becomes slow. This is an algorithmic problem. Imagine if a database had to scan all its entries to determine the largest value of a column, for example. If there are billions of records, that can take a lot of time. In essence, indexes narrow down the space where you search for your data, so your DBMS can get what you want from a smaller set of query results.

Indexes normally convert rows in a table into nodes in a binary search tree. If you are familiar with computer science, you know that BSTs are a data structure where searching has a time complexity of O(log n), which is much faster than the O(n) complexity that comes with a simple, sequential search.

If you want to learn more about binary search trees, algorithmic time complexity, database transactions and much more, check out Computer Science Distilled. It will give you the fundamental concepts to understand computer science and make you a much more efficient coder. Besides, it is a fun and interesting read.

Like everything in software development, there is always a trade-off. We shouldn’t just index all our columns and hope our database will be super fast. Every new entry on a table requires its index to be updated as well, and that can be computationally expensive, so inserting an entry becomes a slower operation in the end. Besides that, indexes occupy additional disk space, which can pose an extra cost to your application.

There are also a few situations where adding an index might be useless. One example is when the indexed column has a lot of repeated values, for instance a nationality column in a users table of a service that is mostly used by customers from a single country, let's say Brazil. If almost all of the users are Brazilian, the index won’t do much in terms of narrowing down your search space, since the database will have to do a non-indexed search among all those rows where nationality equals Brazilian, which will be practically the entire column. However, if you query for users from other countries, then this index is useful, because it will narrow down your search to the few non-Brazilian users there.

Likewise, an index is useless when you make some sort of change in the entire indexed column right before querying it. For example, searching for a value in a name column right after you apply an uppercase function to all the names. If the index was created on a lowercase name column, then this query will basically skip the index and do a regular search for uppercase values. This happens because the index stores the data it was told to, and a lowercase string is different data in memory than an uppercase string, even if they are the same word. It can also happen with the CAST statement in SQL, that converts the type of an entire column. Those kinds of statements force what is called a table scan, where the whole column is processed and transformed into something else that the index won’t recognize.

Only create indexes for the columns where you need them. Each index can make the read queries faster, while making insert, update and delete queries slower. As a software developer, it is your job to figure out which tables and columns are worth indexing. This is a critical part of database design, and a bad indexing strategy can significantly impact your application’s performance.

Conclusion

The topic of databases is incredibly vast and complex. This article was a brief introduction to someone who might know a little bit about SQL and databases but wants to explore some more.

Databases are powerful tools and a fundamental part of every application. Knowing how they work is key to having a healthy, performatic software that can serve users quickly. A bad decision on indexing strategies or transaction management can make your application slower or even worse, corrupt your data and negatively impact your project.

It is the programmer’s job to decide how and when to use each type of database system that best fits the needs of their application. It is important to know what problems you are trying to solve, so you can pick the best tool for the job.

I switched careers to software development and this is what I learned

Guilherme Dugaich de Oliveira — Mon, 05 Sep 2022 12:24:17 +0000

I wrote this article to tell the story of how I switched from a career in the finance industry to software developer and share some insights I had along the way that might help you, if you see yourself doing something similar. This can relate to people who had a few years of experience in another field and decided to switch to a career in technology, specifically software development.

Coding was not something new to me when I decided to do my career transition - the first lines of code I wrote were probably 10 years before I started doing it for real. I was introduced to programming in a couple classes in college, and to be honest I didn’t fall in love with it at first (it was interesting at best), but at the time I didn’t realize how useful it would be to me. This helps to bust the myth that programmers are people who are completely obsessed with coding since they were children, who can stare at a computer screen for hours and be amazed. That was not my case.

I went to college for a mechanical engineering major, and later on switched to economics, where I got interested in finance and got my first job. Right there I started realizing that programming could be a game changer for me - that is because finance work has a lot of data manipulation, and what better tool for that than IT (information technology)? I worked 5 years in finance and in that time I did several programming courses and tried to apply my knowledge wherever I could, but I still lacked a professional approach to it, my knowledge was still very basic.

During the 2020 pandemic, I realized it was the right time to really shake things up and make the move. So I planned everything I needed and quit my job in December that year to start my software development journey in 2021. I joined a 12-month long full-stack web development bootcamp, and before finishing the course I got my first job. I am now completing 1 year of experience in the field and I feel comfortable enough to share some things I've learned in the process, as I believe that it can be useful for other career-switchers out there.

First, patience.

Switching careers is a long term game, this means nothing is going to happen overnight. I waited a whole year before quitting my job to study full-time, and then 8 more months of preparation to get my first tech job. Since learning to code at a professional level takes a lot of time, you should plan things in the long run, being aware that the benefits will come as a reward of the time and effort you put into your goal.

Don't fall for unreal expectations of very high salaries in a short time. There are a lot of people trying to profit from the booming job market, and they will promise you things that might not happen. There is no magic formula, you have to put in the time.

Also, it is important to mention the financial impact this will have. If you already have a steady salary working on something else and decide to change, it is likely you are going to face a decrease in your monthly income. After all, you are joining a new industry as a beginner and will be paid accordingly. Try to save some money before making any decision, so you can be financially secure during the process of change.

Use your experience in your favor.

Whatever that is. If you are coming from another field, you certainly have some skill that will help you in your software development career. Whether you are a great communicator, or someone who can keep things organized, there is room for all kinds of profiles and backgrounds.

As Steve Jobs says in his famous speech from 2005:

“You can’t connect the dots looking forwards, you can only connect them looking backwards.”

Only looking backwards you can see that the choices you made in the past created the path to where you are today. So whatever you did with your life up until this point, it can be used in some way or another in your new career pursuit.

Jobs was talking about how a typography class he took in college helped him years later when he was designing the fonts for the Mac personal computers. In my case, my background in economics and finance helped me in my work as a developer in a Fintech company, where I was already familiar with the business jargon and terminology. Also, I worked with customer support, which is a valuable skill in technology, especially if you are building customer-centric products.

Show that you have something different, and since you will lack practical experience in the beginning, try to differentiate yourself with the skills you already have. Find out what those are and capitalize on them.

Consistency beats intensity.

There is no way around, you will have to study hard. IT is a highly technical field and you will have to understand complicated concepts. Also, you need to be comfortable with not knowing everything, because you never will. No one does.

I found out that studying and practicing a couple hours a day, every day, is more effective than having a very intense routine for a short time and then burning out of stress. It takes time and repetition for your brain to absorb new concepts, so you should not flood it with new information every waking hour.

Find a study routine that you can keep for 6 months or more and stick to it, the consistency will pay off. If you are serious about getting into IT, you will be learning new things for the rest of your career, so you should find a way to do that where you keep yourself happy and motivated.

Focus on something instead of trying to learn everything.

This was key for me. It is very easy to be seduced by all the new shiny frameworks and technologies that appear every day and try to learn everything - the more you know, the better, right? Not quite.

For me, it was better to focus on a topic, (in my case back-end development and Node.js) and try to understand it deeper, than to have a very superficial knowledge on lots of topics. I don't wanna be a jack of all trades, and a master of none.

I'm not an expert in anything, I just learned enough about the back-end so I could get an entry level job. If I studied back-end, front-end, DevOps, cyber security and design, all at the same time, I probably wouldn't get a job in any of them.

Build things.

I've talked a lot about studying, but software development is a very practical endeavor. The best kind of learning is the result of trying to build things - making mistakes and trying again, over and over. As soon as you learn something new, you should try to apply it somewhere.

Learned about HTML and CSS? Go and build a simple web page. Learned about databases? Create your own and try to make queries to update and read data. It is only through practice that the things you learn will resinate. At least that is my case, and literally every developer I know has said that at some point.

It is likely that at some point you will be stuck in tutorial hell, where you keep watching lots of videos and consuming content on a topic, while not putting it to practice. You should be aware of this and try to apply everything you know in a project, even if it is just a personal side quest of yours. Hopefully, the bootcamp I joined had lots of practical assignments and I had to deliver a new project every week.

Be part of a community.

The career changing journey can be lonely. If you decide to do this completely by yourself, be aware that you will face more difficulties than if you are part of a community, where people help each other and try to learn together. That can be anything: an online forum, Twitter, Linkedin, your town’s square, or any place where you will find people looking to achieve the same goal as you.

When you are looking for a job, it's hard to get an opportunity just by shooting your resume to every job post you see on LinkedIn. I tried that and it can be especially frustrating if you are a beginner. Connections with people that can refer you to a job or being part of a group can go a long way in getting you your first interview.

Use the Internet.

Almost all the information available on software development is free and online. Every major programming language, framework and tool has its documentation available a click away. You can find classes and tutorials for literally anything on Youtube, and answers to most of your questions will be on StackOverflow or other resources you can find on Google.

Using the internet wisely is one of the best skills you can have as a developer. Even the most seasoned of professionals will look for answers on the internet on a daily basis. Learning to code is about solving problems, not remembering little details such as language syntax or method names. You should focus more on understanding the core concepts instead of trying to remember everything.

Find out what works for you.

Those tips are a reflection of my own personal experience, which is unique to me. Other people will have different stories and situations, and that is fine. You should find the strategy that best fits your profile and execute it. I believe the tips I gave here are applicable in different situations, and you should try to extract what applies to your reality and use it. Only you know what is best for you.

For example, if you can't quit your job to study full-time like I did, try to fit in as many coding hours as you can on your schedule. You will be amazed by what an hour a day can do in the long run.

I hope this was useful in some way. If I can help at least one person in switching to a career in software development with this article, I will consider it a success.

What is Docker? Creating a container for a Node.js application

Guilherme Dugaich de Oliveira — Sun, 17 Jul 2022 23:43:12 +0000

Context and Motivation

A software is, basically, a set of files that is read, interpreted and executed in some way by the computer. This basic definition leaves room for a question: what about running the same program on different machines? They must share a similar environment, with the same resources needed to run such software.

This is an age-old problem in the computing world. The famous “on my machine runs” meme shows that if some code is executed locally on a developer's machine, it does not guarantee that the same program will run correctly on another machine, or on a server in a production environment.

Before talking about Docker, it's important to talk about the problem it solves and what was the tool used before it. The challenge is to be able to run the same program in different environments, different machines. Any software has dependencies, which are libraries of code that the software needs to function. Also, it needs executable binaries to run. In order for your program to run successfully on a given machine, you need to make sure that its dependencies and binaries are installed.

If a developer writes Python code on their machine and pushes that code to Github, making it public on the internet, and someone else clones that project on their machine and tries to run it, will it work? Only if the dependencies are installed and Python is working on its correct version. What if the project was developed on a Windows computer, and the other person tries to run it on a Linux machine? Some adaptation will also be required.

In the example of just two developers, this doesn't seem to be a big problem, but on larger projects, with hundreds of people working and multiple development, staging, and production environments, this can become a nightmare. This article intends to give an overview of one way to solve this problem, which is with Docker. To be able to follow the example tutorial that will be done below, you need a basic knowledge of Node.js, Linux systems and REST APIs.

Virtual Machines

As a rule, computers have a single operating system, at least that's how they come from the factory. To try to use more than one operating system without having to buy another computer, there are some alternatives. You can install another system on the same machine, sharing the same hardware, and make a dual boot setup, where the user chooses between two systems when starting the machine.

This is a good solution, but it does not allow both systems to run simultaneously. For this, another type of solution emerged, virtualization. A single machine can have its resources (memory, storage, CPU, etc.) divided between virtual machines, which are simulations of other computers. This division of resources is done by a special type of software called a hypervisor. And even with virtualization we still have a default machine operating system, which is called the host system (host OS). And the hypervisor is installed on it.

A hypervisor is able to do the following division: allocate 2GB of memory, 100GB of disk storage and 2 CPU cores for a Linux (Ubuntu) system, and 4GB of memory, 200GB of disk storage and 4 CPU cores for a Windows system, all on the same hardware. Obviously, the hardware in question has to have enough resources to run the virtual machines. Virtualized systems, running on top of the hypervisor, are called guest operating systems.

The user can, while using the host OS, open a hypervisor window and use another system, as if it were running natively. This opens up the possibility of running multiple machines simultaneously, as many as the hardware can handle, which is a very powerful utility. However, it is still an expensive option in terms of hardware and processing, as each virtual machine builds its own operating system from scratch.

This is a very basic explanation of virtual machines, but it allows you to understand how this solution that came up long before Docker, and is still widely used. Virtual machines virtualize the hardware, booting an entirely new operating system from scratch. On the other hand, Docker virtualizes the operating system.

Docker

According to the official documentation, Docker is an open platform for developing, shipping and running applications. It allows you to separate the application from the infrastructure for faster software delivery. With Docker it is possible to manage the infrastructure in the same way that you manage the code.

For a more practical definition, Docker is an application that you install on your machine, like any other, and it has both a command line interface (CLI) and a graphical interface on the desktop. It allows you to package your applications in isolated environments called containers. The properly configured container has everything needed to run an application, including the previously mentioned binaries and libraries.

Unlike virtual machines, Docker is not virtualizing hardware resources, but simulating an isolated environment to run an application. This concept will become clearer with examples.

The container can be thought of as a microcomputer running on top of the Docker execution engine, and that microcomputer is isolated from the rest of the machine. An application running in the container does not know about the machine's resources, or how it is being used by other applications. Containers are fast and lightweight, allowing for a great software development and deployment experience.

A detail that differentiates containers from virtual machines is the fact that they can be easily shared through their images, which are files that contain all the information about a given container, and Docker uses them as a starting point to create a new one. Anyone can send and receive container images and have them running on the docker engine in their local machines or cloud environments.

Docker sets out to do three things: build, push, and run images. That is, it can create a container from the image, send this image to other developers, in addition to cloud environments and other remote container repositories. And of course, it also has the ability to run these images, as long as Docker is properly installed.

The idea is really a little abstract, but it is important to understand that the container behaves as if it were an isolated machine, like a normal computer, where there is a file system, folders, executable programs and everything else. This concept will be important when explaining Docker commands.

Creating a container for an application

Now, let's build a container for a Node.js application with Express and see in practice how it all works. To keep the focus on Docker, the application will be very simple, a single endpoint that returns a message. Make sure you have Node and the npm package manager installed on the machine. To create the application, start a new directory with a name of your choice and inside it execute the following commands.

$ npm init -y
$ npm install express

The first command creates a Node.js project in the current directory, starting a package.json file. The second installs Express, the framework we use to create the REST endpoint. Then create an index.js file in the project root with the following code:

const express = require('express');

const app = express();

const PORT = process.env.PORT || 3000;

app.get('/', (req, res) => {
    res.send('I S2 Containers');
});

app.listen(PORT, () => {
    console.log(`Node app running on port ${PORT}`)
});

Here is our Node.js application! A single GET endpoint that returns the message “I S2 Containers” to the client. To start the server and make the endpoint available, run the command node index.js from the project root. It is now possible to call http://localhost:3000/ directly from the browser or any HTTP client to see the magic happening.

Okay, we already have an application, but what if we want another developer to run this application on their machine, before deploying it? We would have to upload the application on Github, or on any other open platform, the person would have to download the project, install Node, install the dependencies and only then run it. Docker makes this process simpler. To turn the application into a container, we need to have Docker installed locally. If you don't already have it, follow the instructions in the official documentation and install.

First, we need to create a file called Dockerfile at the root of the project. This is where the instructions for building and running that application will be. It works as a sequence of steps, or commands, that Docker will follow to build and run the image of the application. After creating this file, your project should look something like this:

Now, lets write the Dockerfile and check what each command means

FROM node:17

WORKDIR /app

ENV PORT 3000

COPY package.json /app/package.json

RUN npm install

COPY . /app

CMD ["node", "index.js"]

FROM node:17 - This command tells Docker which base image we are using for our application. Here it is important to mention Docker Hub, which is Docker's remote repository on the internet, where users can download pre-made images. In our example, we are using the image called node, which is the image of a container that already has all the Node.js dependencies we need installed, and we also pass the tag 17, which is the version of Node used. With this command, Docker understands that it will start creating the container from an image that already exists. From here, every command in the file will be run from that base image. Every Dockerfile must start with a FROM command.

WORKDIR /app - Defines which is the main directory of the application, inside the container. This is where the subsequent commands will be applied. The container has its own file system, and the /app directory will be at the root of that file system.

ENV PORT 3000 - Sets the PORT environment variable to the value 3000.

COPY package.json /app/package.json - Copies the package.json file to our previously defined working directory.

RUN npm install - Runs the Node dependency installation command. It is worth remembering that this command is being executed inside the /app directory, which contains the package.json file.

COPY /app - Copies the entire contents of the local root directory into our application's directory.

CMD [“node”, “index.js”] - Defines the default command to be executed when the container starts. When we tell Docker to run our image as a container, it will look at this command and understand that when starting the container, it will run the command node index.js, which is the command that spins up the HTTP server we built.

Ok, now that we have our Dockerfile ready, we can create our image.

$ docker build --tag i-love-containers .

With this command, Docker understands that it has to build the image. The tag option passed defines a name for the image, i-love-containers, and the period at the end of the command is defining the path where the Dockerfile is located, which is in the project root.

After executing the command, the logs of the things that Docker has done will be shown in the terminal. It is clear that it executes the commands specified in the Dockerfile. And now that we have our image built, just use the docker images command in your terminal to see the images available on the machine. With the image ready, let's run it as a container.

$ docker run -p 5000:3000 -d i-love-containers

The parameter -p 5000:3000 is used to indicate that port 3000 of the container must be mapped to port 5000 of the machine where Docker is running. That is, to access our endpoint on the local machine we use http://localhost:5000/. This is evidence of the container's independence from the rest of the computer, it needs to explicitly know the port we are going to request. The -d parameter is to run in detach mode, which means the process will start in the background.

Now we can run docker ps to see which containers are running. Notice that docker gave your container a name, something random, in the NAMES column. This command only shows currently running containers, and to show all available containers, including inactive ones, use docker ps -a.

Calling the endpoint on port 5000, we see that it returns the expected message, our application is running inside the container. It is important to note that the Node installed locally on our machine is not running, only the one that is in the container.

You can stop the container from running with the docker stop <container name> command and similarly get it running again with the docker start command.

Deploy

We have a few options to make our application available to the world. First, we can upload our image to the aforementioned Docker hub, which is a central repository of images on the internet, where anyone can download images that they have access to. Docker Hub is a very complete tool and has several features. If you're interested in how it works, and how you can easily make your image available on the Docker hub, study the tool's documentation.

With a Docker image, it is possible to deploy the same container on several cloud platforms such as Heroku, AWS, Google Cloud, and others. The subject of deploying containers is quite extensive and deserves a post dedicated just to that. For now, it is interesting to know that all major cloud platforms have container deployment mechanisms, which makes your application very adaptable from one platform to another.

Why Docker?

First, containers are much lighter in terms of memory and processing when compared to a virtual machine that needs to spin up an entire operating system, since containers share the same host OS, used by the Docker engine. To be even more specific, they share the same kernel, unlike virtual machines that each have their own.

For those unfamiliar with the term, the kernel is the brain of an operating system, it is the part of the software that communicates with the hardware. When we talk about a Linux system, we are actually talking about a system that uses the Linux kernel, and there are several operating systems that use it. A system that uses the Linux kernel is commonly called a Linux distribution, like Ubuntu, CentOS, Kali and others. When building a virtual machine, it is necessary to create a kernel from scratch, which is much more cumbersome than simply starting a Docker container, which already uses the hardware's kernel resources.

Here it's worth mentioning a small disadvantage of Docker. Since containers share the same kernel, it is only possible to run containers that are based on images from the same host OS. So we can only run Linux-based containers on Linux machines, and the same for Windows and MacOS. A container of a Windows image would not work on a Docker installed on Linux, and vice versa.
As we saw in the example, this is not such a big problem, since it is possible to run Docker inside WSL 2 running on Windows. There are several mechanisms to work around this problem. One of the biggest use cases for Docker is to deploy applications to cloud environments, where Linux is most often used.

Currently, many companies use containers for microservices architectures, where parts of the system are separated into smaller applications with well-defined responsibilities. This makes maintenance, testing and understanding of complex systems easier. We can have a container running Node.js, another running PostgreSQL or another database, another running a front-end application with React, all within the same business logic, but divided into independent containers, each with it's own deployment strategies and details.

I hope this article has been useful for those of you who didn't know Docker, or knew and had some doubts about how it works. Knowing Docker today is a fundamental skill for developers, to increase the power of their applications, making them scalable and easy to deploy.

To give credit where credit is due, this article was inspired by NetworkChuck's YouTube video.

What is REST?

Guilherme Dugaich de Oliveira — Sun, 03 Jul 2022 17:24:19 +0000

Since I started getting more into back-end engineering, the term REST has appeared many times, often in the context of API design. Everyone wants a REST API, or a RESTful API, but what is that anyways? This humble post tries to explain it in simple terms.

As I do with many things in the tech world, I like to start explaining it by saying what the acronym stands for. In this case, according to Wikipedia, Representational State Transfer is a software architectural style designed specifically for the World Wide Web, our beloved internet. It aims to define how a web application should behave: as a state machine, where a user requests that state in the form of web links, or URLs, and gets the representation of that state in the form of data.

History

Besides unraveling acronyms, another useful way to explain tech concepts is to find out why they were created in the first place. Most technologies were created to solve a specific problem and the REST style is no exception.

In the early 90’s, when the internet was this very new thing everyone was talking about, there was already pressure for the creation of standardized interface protocols that members of the network could use to interact with each other. Organizations such as the W3C and IETF working group, came together to try and organize it. A guy named Roy Fielding worked on such groups and, over the following years, he worked on the REST architectural style and defined them in his PhD dissertation titled “Architectural Styles and the Design of Network-based Software Architectures”. Read more about the history on Wikipedia.

Properties

First, it is important to say what REST is not. It is not a new programming language or framework. It is not software that runs on a computer and it is not a communication protocol, like HTTP. It is a style, meaning that it is a way to organize a software system that makes sense in the context of web applications. In order for an API or system to be considered RESTful, meaning that it applies the REST style, it has to follow some properties by definition.

Client-server architecture. Establishes a separation of concerns between user interface and data storage. This allows for flexibility on both sides, where UIs can be designed independently from the server in many different platforms (Web, mobile, IoT) and the server side can scale horizontally and vertically.

Statelessness. This means that no session information is retained by the server, and every piece of information that is sent across the network exists in isolation from each other or from the context of the session. Every resource is unique and independent. This is important for data-intensive applications where retaining session data could be expensive.

Cacheability. The ability to cache responses allows clients to make better requests to servers and avoid outdated data, for example. This can improve performance.

Layered System. A REST call happens between client and server, regardless of what is in the middle. Load balancers and proxy servers can improve scalability, performance and security of a system without jeopardizing the actual data transaction.

Code on demand. This is an optional property and allows servers to modify or extend functionalities on the client side by sending over compiled code.

Uniform interface. This is probably the most fundamental constraint to define a RESTful service. The communication interface between client and server must be consistent and well defined, so it can simplify and decouple the architecture.

Usage

More than enabling agents in a given network to communicate with each other, the REST standard intends to do so on a large scale, so the true power of the internet can be unleashed. The idea is that the resources on a server where clients will be making requests should be as lightweight as possible.

A resource is a piece of information in the server that will be handed to the client, abstracting all the internal work this server might be doing. For instance, if a client requests a list of names from his Facebook friends, they don’t care if the information is coming from Facebook’s database, from their filesystem or from another service on the web, they just want the information. The REST service works as a layer of abstraction for the client to get the information they need, and that is what makes it simple.

Coming back to the uniform interface property, that interface consists of an endpoint (URL) , a return type and an HTTP method or verb. The endpoint is the location of the resource, where the resource can be anything you are looking for, like JSON data, an image or a file. The return type specifies the type of data being sent over, normally it is JSON, HTML or XML. And finally the verb or method, which is an important part of the request.

The verb or method that comes with a REST call via HTTP defines the type of action we are going to perform with the resource. The verbs are pretty self-explanatory and they are very helpful when building requests:

GET: used to read resources;

POST: used to add new resources;

PUT: used to edit existing resources;

DELETE: used to delete resources;

Finally, the anatomy of a REST call would be something like GET https://mysite.com/api/resource and that would get the data from the specific resource. POST and PUT calls normally have a body of data sent along with the request, that contains the information to be added or edited to the state of the server.

After you call a RESTful service, you should receive a response containing the resource itself and also a status code, that is a standard code that defines the status of the request. The categories for status codes are defined as follows:

1xx: Information;

2xx: Success;

3xx: Redirecting;

4xx: Client Error;

5xx: Server Error;

You probably came across some of those codes when surfing the web, probably the famous 404 Not found code that many websites sometimes return with a funny and creative page. Check out this website for a creative explanation of the status codes.

I hope this article was helpful for clarifying the REST idea and its usage on the web. It is incredibly important for web applications to follow these standards in order for us to have more shareability of resources and information across the internet.

Discovering the front-end and React

Guilherme Dugaich de Oliveira — Sun, 03 Jul 2022 17:10:35 +0000

As a curious person, I have always been interested in programming. When seeing a computer or a smartphone work, one should wonder what is going on inside it.

At a certain point, I started pursuing that curiosity. At the age of 18 I took my first programming course, an introduction to computer science discipline at the university, that used C++ as the language of choice. It was pretty fun. I could study about variables and their types, repetition loops, conditional statements, and it all made a lot of sense, but nothing more than that. The course consisted mainly of writing command line programs that performed some sort of calculation. It was still pretty unclear to me how those little programs could become something like a website or mobile app.

I was still curious, but for whatever reason in life, I didn’t try to make a career in the field at the time. Over the years I did more programming courses in Java, C and Python. Those were mostly introductory courses and I reviewed those same basic concepts by building similar command line programs. It still made a lot of sense, but at the same time I didn’t know how those would be useful or how one could make a career out of it. It seemed far from the real world. I was working in finance at the time and tried to apply some of those concepts there, but the impact was still limited.

One of the projects I did was a tic-tac-toe game built in Java that could be played in the command line. I was pretty excited about it, after all it was a challenge for a beginner like me. I remember showing it to my family and friends and I could clearly tell that they didn’t think it was a big deal. Of course, who would open a terminal and run some weird commands in order to play a poorly looking game with terrible usability? At this point, I didn’t fully understand the importance of a friendly user interface. Technology is all about creating solutions, but without a good UI, your solution becomes much less useful and more difficult to use.

When I decided to study web development and go all-in into the tech career path, approximately 10 years after that first C++ introductory course, things started to make more sense. I started to understand how the internet works and began to discover the true value behind those lines of code. Besides that, I discovered how to build an interactive web page and put it up on the internet. I could build something that someone could actually use for something, anywhere in the world. That was a game changer.

Learning the basics to build a Web user interface

I started learning the basics of the front-end with the three technologies that compose the core of any modern web page: HTML, CSS and JavaScript. This article is not supposed to be a tutorial, as I will only describe what those technologies are used for. For those who want to learn it, there are tons of good resources over the internet. I personally recommend freeCodeCamp.

HTML, or Hypertext Markup Language, is the content part. It defines the building blocks of the page, like a header, a sidebar, a dropdown menu and so on. It does so with a tag pattern, where each tag represents an element in the screen.

CSS, or Cascading Style Sheets, is the style part. It is what gives those HTML elements some visual appeal. With it you can add color, position the elements, shape them and make the page beautiful.

With HTML and CSS alone it is possible to do lots of things, but only static pages. Meaning that there is no user interaction and the page is the same for anyone who accesses it. That is where JavaScript comes into play. It is a programming language that can manipulate the web page through something called the DOM (Document Object Model). It can add and remove HTML elements, add behaviour to things like buttons and dropdown selectors and basically anything that involves specific user interactions and data.

This very brief description summarizes how every web page works. Everything that happens in the browser in terms of user interface can be narrowed down to HTML, CSS and JS. That means you can basically build anything knowing just that. Correct, but if you are a habitual web user you know that there are some pretty complex pages out there, and building those with just these three technologies would be possible, but incredibly complex and time-consuming. This is why something like React was born.

What is React and why it exists

Quoting React’s own documentation, it is an open-source JavaScript library, developed by Facebook, used for building user interfaces. A library is basically a set of functionalities developed and published by a third party that are not native to the language. React exists to facilitate the work of developers who need to create interactive user interfaces in web applications. The three main concepts of React are components, props and state.

Components are the building blocks of React. They are independent, reusable pieces of code that can be used separately. For instance, a single page can be broken down into smaller components such as a navigation bar, a button with a specific functionality and a form.

You might think that React components are similar to HTML tags, and they are in a certain way. However, a component can have several tags and, more importantly, they can manipulate the way those elements interact with the user. An <img /> tag in HTML represents a simple image. In React, you can create a component that still displays the image, but you can change its source dynamically according to user-specific data, or even put conditions in the display or style of that image. React allows you to manipulate the inner mechanics of the elements in the screen, something you can’t do in plain HTML.

Props, which is shorthand for properties, are pieces of data that can be passed to a component, just like parameters to a function. The render of that component, that will be displayed on the screen, can be a result of manipulations of that data passed as props. For example, a component can receive the user name as a prop and render it in some way.

State is data that is created within the component and used only inside it. It is supposed to represent the actual state of the component in a given moment. For example, whether a button is clickable or not. Managing state is one of the biggest challenges in front-end projects. Complex applications have several pieces of state and sometimes managing it requires help from other libraries, like Redux, or even React’s own Context API.

Fully understanding these three concepts and being able to play around with them has an incredible power. It allows you to create basically anything that comes to mind. Of course the level of complexity will change from one project to another, but the basics will stick.

When it all comes together

Besides learning about the front-end, I also discovered what an API is and how it distributes data between the front-end and the back-end. The React application I know how to build will consume the data from an API and display it to the user. I can also understand what is the role of a database and how it can be used to store all the data of the application in a safe, reliable way.

Being able to build friendly, intuitive user interfaces gives a whole new meaning to programming. All those fundamental concepts I learned early on my journey are now proving themselves useful in a real context. Now I know that all that code made in C, C++, Java and Python can be translated into useful data that can be integrated into an API or a database, which then can be consumed by the front-end to develop real software products that real people can use to improve their lives.

Seeing something I have built with code being used by someone is, for me, the greatest motivator to keep studying and discovering more. After learning the basics of the front-end it is time to dive into the back-end, where all that data and logic can be manipulated in order to better serve the users. Understanding this logic of how web applications work was certainly the biggest transformation on my journey, it allows me to have a big picture vision of how things work and where each piece of the puzzle goes.

What have I learned about Git and Github

Guilherme Dugaich de Oliveira — Sat, 02 Jul 2022 13:46:33 +0000

Introduction

In the beginning of 2021, I dove into the world of software engineering and technology in general. I learned amazing tools and languages like Linux and its distributions; the Visual Studio Code; the triad of web development with HTML, CSS and JavaScript; React for creating user interfaces and much more. However, the one thing that caught my attention the most was Git. Not only for its prevalence in the market, but for its elegance and efficiency. In this article I intend to present the basics of how Git works, as well as show why it is so powerful. I do not intend to make a tutorial, but rather a theoretical explanation of the main concepts. The example images in this article are in portuguese, the original language this text was written in.

Git came to solve the file version control problem. You can relate to this problem if you ever had a folder on your computer with file names like work.pdf, work1.pdf, work2.pdf, work-final.pdf, work-really-final.pdf? When we are working individually with a single document this may seem harmless, but in a project with dozens of files and many people working simultaneously, it can become a nightmare.

Before explaining how Git and GitHub work, and why they make life so much easier, it’s important to check some definitions. Git is the version control software used in most development projects today. It keeps track of all changes made to a file since its creation, allowing the user to “go back in time” to each of those changes if they want to. Git is a free, open-source program, initially developed by Linus Torvalds, who also started the development of the Linux kernel. It is mostly used as a CLI(Command Line Interface) program, or as a desktop application with a GUI(Graphic User Interface).

GitHub is a website that runs Git under the hood, and offers a user-friendly interface for versioning our files. GitHub allows your projects to be public, so anyone on the internet can view it, suggest changes, and make comments. Later in this article, I’ll explain how the interaction between Git, used on a local machine, and GitHub on the web happens.

How Git Works (in a nutshell)

The moment we create a git repository, we have a folder in our computer that will keep track of everything that happens there, both adding and removing content. Inside the repository a hidden .git folder is created, which is where the internal working mechanics takes place. For those interested, the official Git documentation is quite complete and has an entire book explaining in detail how it works.

Git usage can be represented by four main concepts (or commands): commit, branch, pull request and merge. And three others related to the interaction between Git and GitHub: clone, pull and push.

Commit is the packaging of a change to the file. Each change made and confirmed generates a commit, which is a snapshot of the difference between the file’s previous state and its current state.

In this case, we have a test.txt file, which initially had 3 lines. A new commit has been made and 2 new lines were included. This image was taken from GitHub, where I created a repository that contains only the test.txtfile.

Git keeps track of all commits, in a chronologically ordered sequence. It is possible to check all the commits of a file since its creation, and if necessary restore the file to its state in a given point in the past. Each node in the image below represents a commit in the file's history. In this case, 3 were performed in total.

You might be wondering about the meaning of the word main in the image, and that brings us to Git's second fundamental concept: branches. Each sequence of commits is present in a branch, and by default the repository starts with a primary branch called main. It allows us to create new branches for the change history of a file, in case it is necessary to make edits without impacting the main branch. This is very useful when testing something new that we don't know if it will stick around.

An interesting fact is that not long ago the main branch was called master. Git changed the name because the word master has ties to slavery times, when people called slave owners their masters. Way to go, Git and Github!

In this example, a new branch called new-branch was created and 3 new commits were made to it. Commits made in main remain there, untouched, while commits from the new one are included in the new one. In it we have all the commits made in main plus those made exclusively in new-branch.

Commits in main and new-branch. Note that the first 3 commits from the new branch are the same as those from the main. The 6-digit identifier on the right is called a hash. Which is a unique identifier for each commit.

The concept of branching can be a little hard to grasp at first. I recommend that you look up for a tutorial or read the official documentation, if it’s not clear.

Branching opens many doors. We can test whatever we want without harming our work on the main branch. However, the point of having branches is to eventually merge them. How do we do it when one of these branches turns out great, and we want it to become our main work?

The pull request is a request from the new branch for the main one to pull its work. In other words, the main branch can incorporate the work done in the newer one. In this case, we'll open a pull request to ask main to merge the work done in new-branch.

Pull Request: at the top we can read that my user wants to merge into main 3 of the additional commits coming from the new-branch.

There is a discussion among users that the pull request name is not the most intuitive. After all, we are asking for a merge to be performed, correct? Some say that the most intuitive name would be merge request. And other platforms similar to GitHub actually use this name.

Finally, we accept the pull request and merge the new branch with the main one. Our main now incorporates all the content we created in new-branch, in addition to the content it already had.

Now, new-branch can be deleted, as its content is already embedded in the main branch.

How Git and GitHub work together

This entire process of creating files, branches and merges can be done either on your local machine, using the Git software, or directly on the GitHub website. But the great power of these tools lies in their use together. The same git repository can have a local copy, on your computer, and a remote copy, which is stored on a GitHub server in the cloud, associated with your user account on the website.

When you clone a remote GitHub repository on your local machine, you have a copy of these files, and on it you can do whatever you want: commits, new branches and merges. After working locally, it is common to upload this work to GitHub so that it is available on the internet. That's where the push comes in.

In the diagram, you can also see the pull, which is exactly the reverse path: bringing changes from the remote repository to your local machine. This operation is very useful when there are teams with many people working on the same project, each on their own machine, but using the same shared remote repository. In this scenario, which is the standard in teams using GitHub, each person can make a new branch, copy it to their local machine, do the necessary work, push it to the server, and then open a pull request to have their work merged.

Collaboration and organization

Version control functionality is interesting and becomes an extremely powerful tool when used on a large scale. Most of today's most important open-source projects, such as the Linux kernel, Visual Studio Code, React and many others, are made in this distributed model, where developers from anywhere in the world have access to the source code, they can work locally on the project, propose changes and improvements, and all without compromising the integrity of the original project or the work of others.

The open-source world is fascinating and deserves an entire article just about it. The Ingenuity robot, which recently took flight in the atmosphere of Mars, internally runs software that relies on open-source libraries. Those have contributions from approximately 12,000 developers on GitHub worldwide.

An important point to make is that Git and GitHub are not just tools for programmers, they can be used with any kind of file. Certainly, the prevalence in the technology market is much greater, since it's not intuitive for non-programmers and has a considerable learning curve. But they are definitely worth it to keep the work structure more organized, collaborative and dynamic.

To clarify this last point, I recommend watching the YouTube channel The Coding Train, by Professor Daniel Shiffman from New York University, called Git and GitHub for Poets. In the videos, Professor Shiffman, who has an incredibly engaging way of explaining things, shows how to use Git and GitHub to work on a text file of a poem, without writing a single line of code. I strongly recommend checking it out.

Why I started my career in Web development?

Guilherme Dugaich de Oliveira — Sat, 02 Jul 2022 13:17:52 +0000

“Any sufficiently advanced technology is indistinguishable from magic.” — Arthur C. Clark

This famous quote well describes the impact that new technologies have on us, sometimes so impressive that it makes us question how it can be real. How does this Instagram filter make me look older? How does Facebook know I’m looking to buy something and keep showing me ads? How can an Uber car, driven by a person I don’t know, who doesn’t know where I’m going, manage to arrive so fast and take me to my destination? The most magical thing of all is that there is no magic at all. They are just ones and zeros, binary signals being transmitted by cables and over the air, which properly organized can be translated into text, image, sound and much more. This is the main reason that made me want to move to the technology field: the possibility of solving complex problems in a simpler, more scalable and elegant way.

By saying that ones and zeros must be organized, we are talking about something quite complex. There are countless operating systems, programming languages and applications to allow us humans to communicate effectively with an inanimate object such as a notebook or a smartphone. From this need to define the interface between human and machine comes the fantastic world of software development, where the role of a developer or programmer is to solve problems through technology and the creation of computer codes. Undoubtedly, there are many ways to solve the same problem, but software development, combined with specific knowledge of the problem at hand, seems to me to be one of the most efficient tools that a 21st century professional can have.

The coronavirus pandemic in 2020, certainly the most tragic and challenging event in our recent history as a society, was a catalyst for the technological revolution, which had already been underway for a couple decades and was brutally accelerated. Virtually overnight, many were forced to stay at home and find new ways to do the same things as usual. In the mood for food from your favorite restaurant? Get Uber Eats. Want to buy something from that store in the mall? Order it on Amazon and have it delivered the next day. Need to have a meeting with your co-workers? Open a room in Zoom. These companies spend millions of dollars and many hours of developers sitting in front of a computer, trying to solve that problem efficiently.

Not surprisingly, the search for this type of professional is on the rise in the pandemic. In Brazil, the supply of qualified developers is not enough to meet the growing demand. In this context emerged Trybe, a school that aims to meet this demand by qualifying developers for the market, and where I placed my trust to help me in this career transition to technology. This demand translates into better compensation, which is another advantage of the field. Another interesting benefit is flexibility. Remote work was already a reality in technology companies even before the pandemic, and it is now widely accepted and consolidated in many as the main form of work. It is possible to work for companies from anywhere in the world, sometimes being paid in the contractor’s local currency. Can you imagine getting paid in US dollars living in Brazil? This aspect was certainly an advantage for me, who greatly value the freedom to be where I want to be.

That’s why I decided to start my career as a developer. More specifically in Web development, which is the area that combines the power of software with the scalability of the Internet, which allows your solution to be used anywhere in the world where there is a computer with a browser and a network connection. I’m just at the beginning of my journey, but just knowing the possibilities that exist is enough motivation to continue studying and evolving every day.