DEV Community: Renan Moura

Beware of excess of "best practices"

Renan Moura — Thu, 20 Jan 2022 12:58:14 +0000

Unlike other disciplines that are more rigid and regulated like Civil Engineering, Software Engineering doesn't have a set of rules to follow by law enforcement.

You won't go to jail if you don't do TDD (Test-Driven Design), or even write tests for your code.

Your system can work just fine if you don't follow any agile method.

You could deploy your next app in a bare-metal Linux machine instead of setting up Kubernetes on a cloud provider to auto-scale your pods, and no one will complain about performance and such.

We have all read Clean Code, Clean Architecture, and some other "bibles" of the software development world which tell us what are the "best practices".

We have all spent hours learning about Scrum or some similar methodology to manage a software project as opposed to old-fashioned cascade-like project management because that's what works "best".

We do tons of meetings, in Scrum alone we have: sprint planning, daily scrum, sprint review, and sprint retrospective.

There are days when I wonder if I attend meetings for a living instead of developing software.

So, what is all this rant about?

It is about not following blindly the so-called "best practices".

I do X because Google does, I use Y because Facebook uses it.

This is a silly reason to do or use anything.

If you are not working with lots of people, in a huge project, with thousands of simultaneous users, following all the "best practices" can be a drawback instead of an advantage.

If you are working alone, in a small app with a handful of users, it is ok to commit directly into the main branch in your GitHub repo.

If your project is not database intensive, or only has to deal with very little data, you can use an embedded H2 database stored directly in your file system and it will work more than fine.

There are projects and teams where it makes sense to follow every single piece of advanced architecture and people management processes there are, but that's totally not the case for your startup of three people.

It is easy to be misled by reading about the latest tech and thinking "everybody is using this", when in fact they are not.

FOMO (Fear Of Missing Out) as a Developer is real and you should be aware of this.

Choose the right tech for the right context in a rational way.

Choose the right project methodology for the right context in a rational way.

You don't have to ditch Scrum in its entirety, you can adapt it and do only 2 of the meetings described instead of all of them.

You can spawn a cheap Linux VM and deploy your app by hand instead of spending hours setting up CI/CD for something you can do manually in 2 minutes.

If your app screen is a simple feature with one text field and a button (like Google's home page), React/Angular/Vue is an overkill.

Learn all of those things, read the books, but use your best judgment to adapt the "best practices" to take advantage of what makes sense to your project and team, and not just because someone else said you should use it.

You also can subscribe to my newsletter.

Learning Programming is Non-Linear

Renan Moura — Sat, 20 Nov 2021 12:28:14 +0000

Learning is by no means a linear process, even in hard sciences like Math.

It is very common to see people asking in groups, Reddit, and other forum-like places "What path should I take to become a Software Developer".

Unfortunately, learning programming is not linear.

You will find many lists and articles giving you a path like: learn HTML, CSS, JavaScript...

They are not wrong in the sense that you should learn JavaScript before React, but they don't warn you that you should not worry about mastering the previous topic before jumping to the next.

Those topics are all interconnected, which means they will "click" in your head only after you have some knowledge of each topic.

Learn some JavaScript and then some React.

If you decide to "master" Python before trying Django, good luck, you will never get to Django.

Python is a massive language, you can learn the basics very quickly, but mastering it is another matter.

That is what I had in mind when I wrote The Python Guide for Beginners.

You can learn the core of the language as fast as possible, and then move on to dive into Web Development, Data Science, or general use of Python as a programming language to automate your everyday tasks.

If you choose, say, Machine Learning, you will need to understand some details of the language specifically for some tasks, but you will have enough knowledge of the basics to google yourself out of any situation.

The "click" also happens as you expand your arsenal of tools.

Learn More Than One Programming Language and some concepts that you learned in language X will make much more sense after you learn language Y.

I'm challenging myself at the moment to dive more into web3, which is a new realm of software development and solutions revolving around blockchain and the crypto space.

Adaptation is The Top Skill for a Software Developer and you have to keep improving yourself as new opportunities arise.

This means I won't waste hundreds of hours to learn Solidity or understand every single concept about Smart Contracts.

The Only Way To Learn Programming is to take action with what you know.

So I will learn some things, apply them to a project and then I will be able to adapt myself to any situation that comes to me.

If you wait until you are 100% prepared, you will be already too late to take the best opportunities.

You also can subscribe to my newsletter.

State of Data Science 2021: Popularity of Python

Renan Moura — Sat, 04 Sep 2021 18:00:09 +0000

Python continues to be an excellent choice if you are entering the data science field.

Python still dominates and is the most popular language, particularly among younger generations.

88% of students surveyed are learning Python in preparation for a data science career.

63% of the respondents said they use it frequently or always.

71% of educators are teaching Python.

It is also interesting to notice SQL raking 2nd place right after Python.

Most structured data is still in relational databases, so a good knowledge of both Python and SQL are a must to deal with data.

The good news is that they are both very accessible and good to begin working with code.

Comments about the other languages

R is an alternative to Python, but I don't see any advantage in learning it if you are already in the Python path since R won't bring anything to the table that Python doesn't.

Then we have JavaScript and HTML/CSS, which makes sense since your results won't live in a Word document on your computer, a good way to display them is on the web with nice interactivity.

Bash/Shell are super useful, the command line is one of the most powerful tools in a coder's tool belt, not only that, but many tools that deal with data engineering like Hadoop rely heavily on the command line interfaces that can be easily automated with a nice shell script.

If you are wondering why Java ranks so high in this list, Hadoop, Hive, HDFS, etc. are made in Java, for instance, and many data pipelines depend on JVM powered tools like Kafka.

So while you may never touch Java as a Data Scientist, you will most probably have to deal with it as Data Engineer at some point.

C/C++ ranks high due to the number of libraries coded in these languages for high performance.

Python's most used Machine Learning frameworks and libraries like Pandas are implemented in C/C++ while Python just provides a nicer API to work with.

The other languages (C#, TypeScript, PHP, Rust, Julia and Go), although they have their place, of course, would not be the subject of further studies from my point of view at the moment.

They are used for more specific use cases or simply fall into "that's what I and my team knows best".

The best contender here would be Julia to replace Python, but it still has ways to go before deserving the time and energy to learn it.

Go would be the high level performant alternative to Java, but it doesn't have the ecosystem with as many tools behind it yet.

So, out of this list, the ones I think will pay you the most dividends for your investment in time and effort are Python, SQL, JavaScript, HTML/CSS, Bash/Shell, and Java.

These languages are more than enough to put you in any stage of a Data Science project or pipeline.

You can read the full report on State of Data Science 2021

Read this article directly on my blog.

Edge Cases are the hardest parts of a Developer's job

Renan Moura — Sun, 29 Aug 2021 18:46:04 +0000

If you are a developer with not much experience you might feel like a lot of the parts of your job are really hard.

The framework doesn't behave as it should.

The database is locking.

The language you are using has a weird way to deal with some situations that should be simpler or easier.

Maybe you are fighting to center that div on your screen.

All of those problems are legitimate and can drive you crazy.

But once you get past these issues and become a master of your tools and how they behave in each and every situation then, and only then, you will feel the hardest part of a developer's job: Edge Cases.

What I mean by edge cases?

An edge case is a situation that you (and maybe not even your end-user) never thought would happen.

You create a page where everything fits perfectly, you test it on mobile, on your computer, and on your super-sized TV, great!

And then you deploy your app in production to find out how a user is having rendering issues when using Safari on an iPad and, curiously, on the same iPad, it works perfectly on Firefox, go figure.

Edge Cases are the hardest parts of a Developer's job.

The previous example is quite technical, but you got the idea.

The same rationale applies to any layer of your app.

If you implement business logic in your backend, you have to think about all of the possible combinations that might happen and test each and every one of them for your system to work bug-free.

Most problems in software arise from edge cases.

How to identify and avoid problems with Edge Cases?

Communication.

When writing business logic, the first thing to do to catch edge cases is to talk with your users, customers, clients, or any person who understands the problem in depth.

Ask as much as possible, understand the situation and think about the possible edge cases.

Remember there is a distance between regular manual work and automating this work with a system.

There are edge cases your users might not even notice because they are so obvious to them, but not to you or anyone else.

As said by Bill Gates:

The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency.

Most operations are inefficient, so be careful before automating anything.

The second thing to avoid falling into an Edge Case trap is simple attention to detail.

Attention to detail avoids problems related to the technical part while Communication avoids problems related to business logic.

Attention to detail is for when you don't test your application with all kinds of screen sizes, or all browsers, or when all the planets align in a leap year and your app crashes (who knows).

So, if you ask me, how do you evolve from a junior developer to a mid-level developer?

My answer is: learn how to communicate clearly and learn to develop attention to detail.

After learning general technical skills, these skills are the ones required to jump to another level in your career.

String Manipulation in Python

Renan Moura — Sat, 27 Mar 2021 20:47:53 +0000

String manipulation is one of those activities in programming that we, as programmers, do all the time.

In many programming languages, you have to do a lot of the heavy lifting by yourself.

In Python, on the other hand, you have several built-in functions in the standard library to help you manipulate strings in the most different ways you can think of.

In the following articles, I showcase these many features of the language regarding strings specifically along with some nice tricks.

You can also download a PDF version with all of these tips and a few more tricks on Python String Manipulation Handbook.

Machine Learning Project – How to Analyze and Clean Data, Create an ML Model, and Set Up an API

Renan Moura — Fri, 26 Feb 2021 15:59:34 +0000

This is a series on Data Science and Machine Learning applied to a House Prices dataset from the Kaggle competition House Prices: Advanced Regression Techniques.

You can download a PDF version of this Data Science and Machine Learning Project with the full source code repository linked in the book.

In this series we begin with the EDA (Exploratory Data Analysis) of the data, we create a script to clean the data, then we use the cleaned data to create a Machine Learning Model, and finally we use the Machine Learning model to implement a prediction API:

You can download the complete code in the Github Repository with clear instructions to execute this end-to-end project.

>>>You can also watch how to run this project on Youtube<<<

How to Learn Machine Learning and Deep Learning: a guide for Software Engineers

Renan Moura — Tue, 02 Feb 2021 20:39:40 +0000

This article was first posted on renanmf.com

This is a video where I share the resources I used to learn Machine Learning and Deep Learning online.

If you are looking for a path to follow, this is your guide!

I give you guidelines to learn:

Python
Calculus
Linear Algebra
Machine Learning
Deep Learning

I hope you find it useful!

Python Project: Guessing Game

Renan Moura — Mon, 04 Jan 2021 14:09:54 +0000

This numbers guessing game is a simple game that allows you to practice many of the fundamentals of the Python language.

It is a great Python project for beginners and in this article, you will find not only the source code but also an explanation of how it works.

The game works like this:

You choose a level: 1 (25 chances), 2 (15 chances), 3 (5 chances)
The game will randomly pick a number from 1 to 1000
In every turn, you pick a number
The game will always tell you if your guess was lower or higher than the right number
You win if you guess the right number before you run out of chances
The game saves the highest score in a file named "score.txt".

Prerequisites

Although simple, this game involves many basic programming skills, and if you have any trouble following this code, don't worry, there is an article for every single thing you should know on this list:

The Game

My suggestion is to open a text editor like VSCode, copy and paste the code, or open this article in another browser window, and put it side by side with this window, this way you can read the explanation while following along with the code.

The code starts at the bottom, on the main block.

Choose a level: `choose_level()` function

It calls the function choose_level() on the main block and goes to the beginning of the code where the actual function choose_level() is and where you pick a level and get the number of chances accordingly.

The input() function is inside a while loop that only breaks if you pick a valid integer, otherwise, it will catch the error with the try/except block and print "Invalid choice", restarting the while loop so the user can choose a level again.

If the input is valid, but not 1 or 2 or 3, the if block will print "Not a valid level." and restart the loop using continue so the user can choose a level again.

If the input is valid and either 1 or 2 or 3, the program moves on.

The core of the Game: `play_game(chances)` function

Then the play_game(chances) function is called on the main block with the number of chances returned from choose_level() as an argument and the program goes to the implementation of play_game(chances).

Initializing the game

play_game(chances) starts by picking a random number from 1 to 1000 and by initializing the number of points to 9999.

The for loop uses the range(1, chances + 1) function giving a turn starting at 1 and going up until the number of chances you chose based on the difficulty level.

Note that range() does not include the number of the second argument, meaning that if you have 25 chances, it will only go until 24, that's why we have to make "chances + 1".

The code then uses f-strings to print your current turn and how many chances you have.

Receiving input from the player

Then you guess a number with the input() function inside a while loop.

Note the try/except that catches an exception in case the user tries a value that's different from an int.

If the value is invalid, the except block catches the error, prints "Invalid guess." and the while loop restarts.

If you chose a valid value but it isn't between 1 and 1000, the if block will be triggered, print a message, and use continue to restart the loop.

If you choose a valid number that is between 1 and 1000, the break command will be triggered, breaking out the while loop, and the program then proceeds to check if your guess matches the right number on if guess == number:.

Checking your guess

The if\else block to check the guess against the right number goes two ways.

If the user nails it, the code will print the message with the score and call save_highest_score(points) to save the score on a file (I will explain save_highest_score(points) later).

If the user doesn't guess the right number, it will go to the else block.

If the number is higher, the game prints "Your guess was too high, try a lower number.", if it is lower the game prints "Your guess was too low, try a higher number.".

The program then calls the function calculate_score(points, number, guess) (I will also explain this one later) to calculate your current score based on how many times you guessed wrong.

You always start the game with 9999 and then lose points every turn you guess wrong.

When you run out of chances and wasn't still able to guess the number, the program goes to if turn == chances: where it checks if you exhausted the number of chances, and if you did, it prints "You ran out of chances!" and gives you the answer.

Finally, the game prints "Game Over!".

Calculating the Score: `calculate_score(points, number, guess)` function

Let's understand calculate_score(points, number, guess).

The function calculate_score(points, number, guess) takes as arguments the current points the player has, the number they are trying to guess, and their current guess.

The code subtracts the right number from guess, and then we have a lost_points that is then subtracted from points.

Notice the use of the abs() built-in function, it returns the absolute value of a number.

The absolute value of a number is the value without considering its sign. So both -790 and 790 become 790.

This is important because if the right number is 200 and we guess 1, lost_points will be 199 which is alright, but if we guess 990, lost_points will be -790, which will increase the total number of points instead of decreasing it due to the code doing subtraction with a negative number.

To see this in action, try removing abs() from lost_points = abs(number - guess) so it will look like this lost_points = number - guess.

Saving the Final Score: `save_highest_score(score)` function

Finally, let's analyze what save_highest_score(score) does.

It opens the file "score.txt" to read the highest score recorded in the game.

If there is no such file, the except block prints the message "There was no champion" since there is no previous score to compare.

If there is a file and it is not empty, the variable record stores the highest score.

Then we check if the current player's score is higher than the previous one.

If it is, the program prints "You are the new champion!" and saves the new record on "score.txt".

If the current score is not higher than the one stored in the file the program prints "Try again to beat the champion!".

The Code

import random

def choose_level():
    print("How hard do you want the game to be?")
    print("1 - Easy - 25 chances to try")
    print("2 - Medium - 15 chances to try")
    print("3 - Hard - 5 chances to try")

    while True:
        try:
            level = int(input("Choose the difficulty level (1 or 2 or 3): "))
            if(level < 1 or level > 3):
                    print("Not a valid level.")
                    continue
            break
        except ValueError:
            print("Invalid choice.")

    if level == 1:
        chances = 25
    elif level == 2:
        chances = 15
    else:
        chances = 5

    return chances

def calculate_score(points, number, guess):
    lost_points = abs(number - guess)
    points = points - lost_points

    return points

def save_highest_score(score):
    record = 0

    try:
        with open("score.txt", "r") as score_file:
            record_line = score_file.read()
            if record_line:
                record = int(record_line)
                print(f"The record for this game is: {record} points.")
    except FileNotFoundError:
        print("There was no champion")

    if score > record:
        print("You are the new champion!")
        with open("score.txt", "w") as score_file:
            score_file.write(str(score))
    else:
        print("Try again to beat the champion!")

def play_game(chances):
    number = random.randint(1,1000)
    points = 9999

    for turn in range(1, chances + 1):
        print(f"-> Chance {turn} out of {chances}")

        while True:
            try:
                guess = int(input("Type a number between 1 and 1000: "))
                if(guess < 1 or guess > 1000):
                    print("Your guess must be between 1 and 1000!")
                    continue
                break
            except ValueError:
                print("Invalid guess.")

        if guess == number:
            print(f">> You nailed it! Final score: {points} <<")
            save_highest_score(points)
            break
        else:
            if guess > number:
                print("Your guess was too high, try a lower number.")
            elif guess < number:
                print("Your guess was too low, try a higher number.")

            points = calculate_score(points, number, guess)

        if turn == chances:
            print(">> You ran out of chances! <<")
            print(f"The right number is: {number}")

    print("Game Over!")

if __name__ == "__main__":
    print("###############################")
    print("Welcome to the Guessing game!")
    print("Guess the number from 1 to 1000")
    print("###############################")

    chances = choose_level()

    play_game(chances)

Testing the Game

The trick to winning in this kind of game is to divide the numbers in half when guessing, this way you eliminate 50% of the numbers in every turn, making your space of possibilities smaller quickly.

Let's say the game starts with the random number 616.

Since this game goes from 1 to 1000, the logical first guess is 500.

Then the game will tell you guessed too low, so you know the right number is between 500 and 1000.

Then you try 750, the number in the middle of 500 and 1000.

The game will say you guessed too high, now you know the right answer is between 500 and 750.

You keep doing this until you either nail it or run out of chances.

To run the game, copy and paste the code in a file with any name, I chose the name "guessing_game.py", and then run:

python guessing_game.py

Or, depending on your Python installation:

python3 guessing_game.py

Here is a sample output of when you beat the game:

###############################
Welcome to the Guessing game!
Guess the number from 1 to 1000
###############################
How hard do you want the game to be?
1 - Easy - 25 chances to try
2 - Medium - 15 chances to try
3 - Hard - 5 chances to try
Choose the difficulty level (1 or 2 or 3): 1
-> Chance 1 out of 25
Type a number between 1 and 1000: 500
Your guess was too high, try a lower number.
-> Chance 2 out of 25
Type a number between 1 and 1000: 250
Your guess was too low, try a higher number.
-> Chance 3 out of 25
Type a number between 1 and 1000: 375
Your guess was too high, try a lower number.
-> Chance 4 out of 25
Type a number between 1 and 1000: 312
Your guess was too high, try a lower number.
-> Chance 5 out of 25
Type a number between 1 and 1000: 281
Your guess was too low, try a higher number.
-> Chance 6 out of 25
Type a number between 1 and 1000: 296
Your guess was too low, try a higher number.
-> Chance 7 out of 25
Type a number between 1 and 1000: 304
Your guess was too high, try a lower number.
-> Chance 8 out of 25
Type a number between 1 and 1000: 300
Your guess was too high, try a lower number.
-> Chance 9 out of 25
Type a number between 1 and 1000: 298
>> You nailed it! Final score: 9631 <<
There was no champion
You are the new champion!
Game Over!

Here is a sample output of when you lose:

###############################
Welcome to the Guessing game!
Guess the number from 1 to 1000
###############################
How hard do you want the game to be?
1 - Easy - 25 chances to try
2 - Medium - 15 chances to try
3 - Hard - 5 chances to try
Choose the difficulty level (1 or 2 or 3): 3
-> Chance 1 out of 5
Type a number between 1 and 1000: 4
Yout guess was too low, try a higher number.
-> Chance 2 out of 5
Type a number between 1 and 1000: 500
Your guess was too high, try a lower number.
-> Chance 3 out of 5
Type a number between 1 and 1000: 250
Yout guess was too low, try a higher number.
-> Chance 4 out of 5
Type a number between 1 and 1000: 23
Yout guess was too low, try a higher number.
-> Chance 5 out of 5
Type a number between 1 and 1000: 2
Your guess was too low, try a higher number.
>> You ran out of chances! <<
The right number is: 289
Game Over!

If you want to truly understand what is going on in this code, the best thing you can is to modify it and see what happens.

Try to change the messages in the print() functions, change the number of chances for each level, come up with a different approach to calculate the score on calculate_score(points, number, guess), etc.

There are many changes you can do to make this game even better!

The Ultimate Python Guide For Beginners

Renan Moura — Wed, 09 Dec 2020 22:18:20 +0000

Python has become one of the fastest-growing programming languages over the past few years.

Not only it is widely used, it is also an awesome language to tackle if you want to get into the world of programming.

This Python Guide for Beginners allows you to learn the core of the language in a matter of hours instead of weeks.

You can download a PDF version of this guide: The Python Guide for Beginners.

Duck Typing in Python

Renan Moura — Sun, 06 Dec 2020 22:01:27 +0000

If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck.

Dynamic Typed languages like Python and JavaScript have an interesting characteristic of letting an object respond to a call according to the definitions of methods and attributes it has independently of its type.

This means that the type of the object, i.e, its own class, or the class it inherits from doesn't matter, the interpreter only cares about the interfaces of the object.

Code Example

class Duck:
    def quack_and_swim(self):
        print('quack and swim')

class Bird:
    def quack_and_swim(self):
        print('quack and swim')

class Frog:
    def ribbit_and_swim(self):
        print('ribbit and swim')

def check_duck(animal):
    animal.quack_and_swim()

if __name__ == '__main__':
    duck = Duck()
    bird = Bird()
    frog = Frog
    check_duck(duck)
    check_duck(bird)
    check_duck(frog)

The output of the code above is:

quack and swim
quack and swim
Traceback (most recent call last):
  File "<stdin>", line 7, in <module>
  File "<stdin>", line 2, in is_duck
AttributeError: type object 'Frog' has no attribute 'quack_and_swim'

The first and second objects duck and bird could successfully respond to the of quack_and_swim() while the frog object couldn't and throws an exception.

In terms of code, the classes Duck and Bird are not related at all, they don't share a parent class through inheritance, but they have the same interface for the function quack_and_swim().

The type of the object is not important, the interface of the object when you use it is what matters.

In the end, if the object behaves like a duck, for practical purposes, it is a duck, no matter what bird it is, as long as it has the right interface.

But what do we do about the thrown exception?

Since we don't check the type of the object or if it has certain attributes, the Pythonic way to handle any exception is to use a try/except block:

try:
    check_duck(frog)
except AttributeError as error:
    print(error)

The treated exception above will output:


Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
NameError: name 'check_duck' is not defined

So we always assume the object has the method or attribute we need, if it doesn't, we catch the exception with the try block and treat it accordingly.

If you had any trouble following this article, I recommend this series of articles about Object-Oriented Programming:

Also, consider reading Handling Exceptions in Python to learn more about how to deal with exceptions in Python.

Think like a programmer

Renan Moura — Mon, 14 Sep 2020 19:56:39 +0000

A programmer is not someone who types super fast (yes, you have been fooled by movies and TV series).

Programming is about problem-solving.

Most of the time, a programmer is thinking about the problem and how to solve it.

Problem-solving is a skill that requires creativity.

And just like any other skill, it can be learned.

The best way to learn is to practice with different problems and projects.

Solving different problems will teach you different skills.

Don't worry too much about syntax, you can always google it if you don't remember, but the reasoning behind solving a problem is something you have to develop by yourself.

The general problem-solving flow is like this:

Analyze and understand the problem
What are the inputs
What is the required output
Plan the solution
Break the problem into smaller parts
Break the parts into tasks
If you get stuck, step back a little and rethink your approach
Test the solution
Fix errors that might appear, face them like challenges, not annoyances
Iterate the steps until success

Programming teaches you to think better.

Problem-Solving flow

A program doesn't exist without a goal.

You code because someone in your company or an external client requested a solution for a problem they have.

Developing a system is a project and as such, must be treated as one.

You have a new project to develop, a new program, there some important things you should check before coding to maximize the chances of success of your endeavor.

The first thing is:

Stakeholders and their problems

Ask yourself: who is using this program? what for? what problem do they have that I am solving?

Stakeholders can be your boss, a client, or the actual person who is going to use the program, the so-called end-user.

Talk to these people.

Don't assume you know what they need, you will waste time and resources and feel frustrated when you find out that what you built does not fill the gap as needed.

Take notes in an organized way of everything they tell you.

These notes will turn into the second thing:

Requirements

Every software has requirements like:

"I must be able to manage my customer data"
"I need the input X to become output Y"
"I have thousands of spreadsheets with sales information and want to know how much did we profit monthly on average in the last 5 years combining their data"

Analyze and understand the problem and turn it into requirements you have to fulfill, describe in words what has to be done for your program to be successful.

The third step is to:

Break down the requirements into tasks

The requirement "I have thousands of spreadsheets with sales information and want to know how much did we profit monthly on average in the last 5 years combining their data" translates to:

Create a way to input/import all the spreadsheets
Process the data to get only what is related to sales profit, since the spreadsheets have a bunch of information I don't need such as names and addresses
Group the sales profit by month and average them
I have to output another spreadsheet with the result

With these written down, you have a clear path on what to do and you are able to track your progress and report it back to the stakeholders and show them you are advancing in each task.

If your requirement includes a screen or a form in a website, you can also draw a rough sketch of how it is going to look like, which field goes where, to serve as a model.

You then discuss these tasks and sketches with the stakeholders to check if you understood the problem well and if you are on the right track.

This is a good point to find some mistakes and correct the direction before wasting resources coding a solution.

The fourth step is to:

Design and code your solution

Now you can work on your code.

Define which libraries you will need, which functions should you create, what approach should you use.

This varies a lot from programmer to programmer, my solution will differ from yours and they will be both right.

There is no single solution to a problem in programming.

The fifth step is to:

Test and iterate

First, test it on your own and check your test cases.

In the example of the spreadsheets, you can try to use just a few files, say 3, to check if your code outputs the same results that you achieved but doing the process manually.

Doing the process manually is simply taking each spreadsheet and making the process of taking the average monthly profit by hand, just like the end-user would, and yes, it usually takes a long time to do it.

It is usually better to ask the end-user to provide this manual test case instead of doing it yourself because they will most likely do it faster and more accurately.

If your code doesn't output the same, you have to iterate, find the error, fix it, and test again until it matches the expected output.

If your test ran alright, give the solution to the stakeholders, so they can try it with more test cases and give you feedback if there is something to fix, enhance, or if your solution is just right.

Embrace this process and you will be way more successful than just sitting and coding in dark.

Finding help

What to do if you get stuck?

It does not matter if you are a beginner or a pro, you will eventually get stuck.

I've been coding for a long time and I still get stuck.

Finding a way out of a problem by yourself is a major ability for a programmer and you should get used to it.

If you see an error, the first thing is to stop and read it carefully.

Sometimes the error is very explicit in telling you what went wrong and what you should do.

See error messages as a friend pointing you in the right direction.

Some errors are not so friendly though.

In this case, Google is your best friend.

Chances are, someone has been through this error before and there is a solution for it on the internet.

Just copy and paste the error message on Google and look for the solutions available.

You might need to open more than a few blogs and forums, but trust me, this is a part of the process, and getting good at it is really important.

I encourage you to spend some time reading the official documentation of anything you are learning in tech.

Some documentations are just bad, but most major technologies have good docs and Python is one of them as you can see here Python Documentation.

Learning from the source is a great advantage in this ever-evolving world of programming.

Puzzle Solver

Renan Moura — Mon, 22 Jun 2020 21:05:28 +0000

A few weeks ago I sent an e-mail to my subscribers and ended it asking the following question:

What's your motivation for coding? What do you want to learn?

I had many incredible replies and took my time going through each of them and replying them all.

There was one reply that stuck in me head (I won't identify the person since I didn't ask his/her permission):

Two main reasons why I’m learning to code are: 1. problem solving
and 2. looking to change job sector.

Problem solving because I want to be able to find automated solutions
to everyday problems that I find to be a nuisance.

Change of jobs because I feel no passion about what I am currently
doing. I’ve felt more alive in the few months I’ve been learning
to code and solving problems in Python and other programs than what
I’ve felt in my entire career as a lawyer.

To which I replied:

Problem-solving is truly one of the main skills in programming.

The job market for programmers is only growing and far from its peak, so I would say you are at the right time to make a change.

This feeling you are describing is what I call "Puzzle Solver Feeling".

Programming is about solving puzzles daily.

A programmer is someone paid to solve puzzles.

The problem you are given is often a total mess and you have to make sense out of it.

It doesn't matter the language or the tools you are using, learn problem-solving and the rest will follow.

How to learn problem-solving?

By practicing with problems and projects the general problem-solving flow:

Analyze and understand the problem
What are the inputs
What is the required output
Plan the solution
Break the problem into smaller parts
Break the parts into tasks
If you get stuck, step back a little and rethink your approach
Test the solution
Fix errors that might appear, face them like challenges, not annoyances
Iterate the steps until success