DEV Community: Vandy in the Vandyverse

Como se tornar um Dev influencer com o mínimo de esforço

Vandy in the Vandyverse — Thu, 11 Feb 2021 17:44:23 +0000

Poste polêmicas no twitter periodicamente. Engajamento é o que você precisa.
Atenha-se a assuntos simples e rasos. Assim você vira uma figura de autoridade para iniciantes e fica longe do radar dos profissionais mais experientes, minimizando o risco de ser corrigido, e por fim mostrar que você não sabe tudo. Por exemplo, fale sobre novidades de tecnologias, afinal é basicamente ler o changelog
Regurgite discursos prontos de autoridades da nossa área como verdades absolutas, ao invés de guidelines que devem levar o contexto em consideração. Assim você supre a necessidade da maioria pela resposta fácil, afinal, pensar dá muito trabalho e tá todo mundo ocupado.
Siga a moda e fale que o jeito famoso é o jeito certo, independente da situação. Monolito? ERRADO! Tem que botar microsserviços em tudo! Tua aplicação de 10 usuários precisa de NoSQL pra escalar!
Vincule sua marca pessoal a uma tecnologia, mas esteja pronto para pular pra próxima silenciosamente. Ao fazer a mudança, pareça que você já sabia tudo há tempos. Ninguém se interessa por quem está aprendendo, e sim por quem já sabe. Não se esqueça disso.
Pratique gatekeeping, utilizando qualquer critério arbitrário no qual você se inclua na parte privilegiada. Ex: “Quem não usa VIM nem programador é” - Assim cria-se um grupo no qual você faz parte, botando todo o resto pra baixo. Se der sorte, ainda arruma uma galera para te proteger dos seus detratores.
Use toda sua pompa e escreva sobre outros assuntos, afinal, todos sabemos que o desenvolvedor é capaz de falar com autoridade sobre qualquer tema, pois somos mais inteligentes que o resto do mundo. De política à saúde pública, nada escapa de seu intelecto implacável.
Abuse do viés do sobrevivente: Se funcionou pra você, funciona para todos.
Nunca fique tempo suficiente num emprego para lidar com as partes ruins de suas decisões, pois você só quer caso de sucesso para contar.
Fale mal do trabalho alheio, é o caminho mais fácil para você parecer fodão, além de botar medo em quem quiser te contrariar
Coloque tudo o que você faz ou fez na sua descrição em redes sociais, mesmo que você seja um bosta na maioria. Vale tudo, hobbies, trabalho, etc. Ex: "Esgrimista, CEO, Programador, Cozinheiro, Campeão regional de dominó"

Interviewing the interviewer

Vandy in the Vandyverse — Fri, 16 Oct 2020 14:14:45 +0000

The interview is not only getting asked to invert a binary tree for working on a Rails shop or designing a microservice architecture with Kafka to solve the business imaginary scaling challenges. It is also an opportunity for you to understand little bit about the company and people you'll work with.

This is a compilation of questions and what you might learn from them when interviewing for a job. Consider this a list of suggestions to use as a foundation to write your own.

Team and company structure questions

Here you check how different parts and teams connect to each other and how your team fits in the big picture.

What's the team structure? Who I'll report to?

This is to learn how the company works. Here you can learn if the hierarchy is explicit or implicit, the later also known as "Oh, we don't have a hierarchy here".

How's the onboarding process looks like and what are the expectations?

Once you start on the job, if you're going to receive help fitting into your role and in how much time the company expects you to be productive. Here you can check how much support will be provided for new employees.

What is the turnover? How many people left the team?

If a considerable amount of people left the team, it can be one or more of the following, though not restricted to:

Toxic environment
Team is not a priority
Job is not interesting
Mismanagement

Did you have a conflict within the company? How it was handled? Someone was fired?

If yes, ask them to tell the story. You can learn a bunch of stuff here:

How management deals with a**holes
How conflict resolution works there

What is the journey the code takes from my machine to production?

Code reviews, trunk based development, CI/CD...? This is where we check some indicatives about the quality and real development processes

Career

How's the career ladder?

Can you advance your career inside the company without having to go into management roles? Usually larger companies have this well defined.

Do you have training/books/conference budget?

Do they invest in their workforce?

Can developers contribute to open source tools you use?

Perhaps not only the company gives back to community, but it is also invests on technology it depends on.

Business

Most of people are very fond of getting paid for their work. That said, it is important to know where the money comes from and goes to.

What is the churn rate?

How many users/clients you lose every year/month? You might ask this both about the company in general or your specific team, in case it is a multiple product company.

How many new users? Is it a net positive?

Customers always come and go, but if there is no net positive, business might be struggling.

How were you affected by corona virus?

In some countries (like Germany) you have a trial period where is surprisingly easy to fire people during it. When the first wave hit, guess who was fired to cut costs?

Did the company achieve break even? If not, what is the projection? If yes, what's the profit in % ?

"Break even" is the term for: "revenue >= costs"
This is one of my personal favourites, and I've seen founders get uncomfortable with this one.

Given you keep your current revenue + funding, how long can the company continue its operations?

This is a important one for startups that didn't achieve break even.

Closing thoughts

There are a lot more questions you can ask. Remember, use your own judgment to pick the time and the person you'll ask each one. I've seen some of these questions leaving a few interviewers uncomfortable(though they shouldn't).
I believe some of these questions balance the power dynamics of the interview process between both parties and some interviewers don't like that.

Creating a queue system using only Python 3

Vandy in the Vandyverse — Sun, 29 Sep 2019 09:37:44 +0000

I was looking for some practice, but wasn't feeling like solving coding challenges. I wanted instead to build something and learn/remember stuff on my way, on a experimental fashion.
How can one do such thing, you may ask. What about picking something from your daily work and put an extreme constraint on it?
That said I came up with the task: “We need a queue, but nothing beyond python and files is allowed. Also, no third party libraries".

Disclaimer: This is meant as an exercise. The code here is by no means production-ready or close to it.

Move your big toe

The problem and constraints are set. But now, what is the first step? After being stuck for a while, I came up with it: "Persist the messages in a binary file, taking care to compress them"

First version and the problem of atomic writes

How it works: We reserve the first 50 bytes on the beginning for the indexes. They're responsible for keeping track of what messages were already consumed and which are next on line.

At the same time, we're avoiding filling up the whole memory by reading the file in chunks

Every time we call self.q.write, we’re writing data to the python’s internal buffer, without really writing the file, avoiding system calls that could make operations slow.

So, in order to really write the file in the disk, we call flush(), right? Well, more or less. The flush method copies the file internal buffer (which is 8192 bytes by the way, but it can use the file preferred block size when defined) to the operating system file buffer, what even so, does not guarantee that the data will be written to the disk.

That said, to really write the file in the disk, we call flush(), right? Well, more or less. The flush method copies the file internal buffer (which is 8192 bytes by default, but it can be changed) to the operating system file buffer, what even so, does not guarantee the data will be written to the disk.

The function call os.fsync() at line 82, writes all the buffers from a file on the disk. If you want to know a little bit more about it, see the module docs and the system call docs.

Yet, this approach has a few problems. In case of system failure (like a power outage) we might have an incomplete file write and/or index update. This happens because there is no atomicity in the operations (more on this later). Also, we're not deleting the consumed messages, bloating the file during usage. Finally, the file is not being locked for exclusive access by an instance of Queue

This answer on StackOverflow says atomicity can be achieved by making a copy of the file, writing on it and then replacing the original. Being this also the common approach, because according to the POSIX standard, renaming a file is an atomic operation. Click here if you want to know more about it.

Seems great, but also quite costly approach for big files. Thus, we can keep a separate index file, with the position of the last successfully written bytes. Since this will be a tiny file, perhaps might not even need to use the rename operation?

File systems, blocks and sectors

File systems usually store data in blocks of 4096 bytes. This means that even if you create a file having 200 bytes of size, it will fit inside of a 4096 bytes block.
Blocks are an abstraction of the disk sectors, where each one of them has 512 bytes usually. So, a block can span many sectors, and a sector may be part of more than one block. According to an interview with Dr. Stephen Twiddle, even in a power outage scenario, disks have energy to finish writing a sector:

“If you start a write operation to a disk, then even if the power fails in the middle of that sector write, the disk has enough power available, and it can actually steal power from the rotational energy of the spindle; it has enough power to complete the write of the sector that's being written right now. In all cases, the disks make that guarantee.”

So, even fitting our index in a single disk sector, we’re only writing in blocks. That said, we cannot guarantee that the whole block will be written in a case of a system failure.

Solving Atomic writes

Well, looks like using rename() is the way to go. The index file is small, so copying -> renaming won’t be as costly. Let’s see the code.

We can notice some changes here. Now we keep an index file, where the os.rename() operation (line 107). Also it is worth to remember o.rename uses the system call rename, which we said before it is atomic. That said, it is nice to have this as a transaction commit point.

Another nice thing, this change makes the queue more robust, since it not only points to the next message to be consumed, but also to the last successfully written byte. This means if a write wasn't finished, the index will continue to point to valid positions in the file, so we’ll always have a contiguous block of messages.

More safety has its price. The chart below show operations per second:

Enqueue / dequeue

Since dequeuing updates only the index file, it allows more operations per second. So, how can we improve that? Perhaps avoiding a file write per operation might help. Let’s see bellow.

Buffers

A buffer is a region in memory where we store data for reads and writes. Remember the Youtube gray bar when you’re watching a video? That’s one of the many applications of a buffer. Unlike a cache, the data will not be available once consumed.

Here we need two buffers. One will perform reads (consuming messages) and other for writes (adding messages). Also, the messages inside the write buffer should be accessible, in case they’re requested before being persisted in the file. Let’s see the code:

This is not a functional implementation. That's because it’ll only update the files once the buffers get filled, meaning we’ll lose some messages if a system error happens in-between. But, it serves as demonstration of how much performance we can have if we avoid writing a file constantly. Let’s see:

Enqueue/Dequeue

This is a huge performance gain compared with the previous versions. However we end up with some reliability risks. There are strategies to mitigate this problem, making this queue both reliable and fast, but implementing them has enough topics to write a book!

Problems, Improvements that I'll never make and final thoughts

This was written as a Python library, but it should run as a service, accepting connections via socket.
Implementing async IO might improve performance, but the code would be much more complex, since the messages order should be guaranteed. The most popular approach on that is via Thread pools, and if you want to know more, see this project.
Messages consumed are not erased, causing a file bloat. Perhaps something in the lines of PostgreSQL's vacuum might be a solution?
Buffers are flushed to the disk only when they reach a limit, instead of also having a delay to do so.
We don’t have a message size limit, which can cause us some troubles.
The queue files are not being locked for usage, meaning we may have two processes using the same queue at a time. Causing all the types of trouble (corrupted reads and writes, for instance)

Tackling this problem was a lot of fun, even without ending up with a production ready solution. However, it can be a very fun and educational experience to try to reinvent things we use in our everyday work. Hope you had fun reading this and get some ideas of your own to make your own experiments.