DEV Community: AIDRI

15 Python tips and tricks every beginner should know !

AIDRI — Wed, 11 Aug 2021 09:44:38 +0000

Hy guys!
Today I'm going to share with you the best tips and tricks to master in Python!

These tips are based on my experience on Codingame, during ClashOfCode (I was in the top 100 at one time :) )

1/ Create a number sequence

Sometimes you may want to create a sequence of numbers: a fairly intuitive way would be to create a loop and perform n calls of the append() method.

my_list = []
for i in range(0, 10, 1):
    my_list.append(i)

In reality this operation is quite time consuming, and it is better to write :

my_list = list(range(0,10,1))

>>> my_list = [0,1,2,3,4,5,6,7,8,9]

This is not only faster to write but also faster to compute!

2/ Overlaying two dictionaries

Concatenating two dictionaries can be a useful operation to group information and avoid getting lost in lots of variables...
So we can use the update() method:

a = {“alpha”: 3, “beta”: 5}
b = {“gamma”: 1, “delta”: 12}

a.update(b)

>>> a = {“alpha”: 3, “beta”: 5, “gamma”: 1, “delta”: 12}

Warning: if there are two identical keys, then the value will be that of dictionary b.

3/ Create a dictionary of a sequence

By creating a dictionary of a sequence, I mean easily creating (in one line) a dictionary where the key depends on x and where the value also depends on x. However, this method can be modified to create the dictionary according to a list, inputs, etc...

my_dic = {(x+2): (x**2 + 1) for x in range(4)}

>>> my_dic = {2: 1, 3: 2, 4: 5, 5: 10}

4/ Reverse a list

Inverting a list is one of the most useful things you can do in Python. You must know this operation !

my_list = [1, 2, 3, 4]
my_list = my_list[::-1]

>>> my_list = [4, 3, 2, 1]

It is much faster to write, much more readable, and especially much faster to execute than the built-in function reversed().

5/ Unpacking a tuple

Unpacking a tuple is an interesting operation especially to perform operations on the values, and avoid having to retrieve the value by its index each time:

my_tup = (0, 1, 2, 3)
a, b, c, d = my_tup

>>> a = 0
>>> b = 1
>>> c = 2
>>> d = 3

6/ Filter a list

Filtering a list is a useful process in algorithms or in more common programs. You can keep the values you want by passing a list into a function that acts as a filter. This function returns True (the value is kept) or False (the value is deleted).

my_list = [1,2,3,4]

def my_filter(x):
    if x==3 or x%2==0:
        return True
    else:
        return False

my_list = list(filter(my_filter, my_list))

>>> my_list = [2,3,4]

7/ Return multiple values from a function

We know that to return a value from a list, we must use return. However, the function stops after returning a value. However, we can use yield to continue to execute the function. Useful for returning variables, for debugging, etc.

def my_func(x)
    for i in range(x):
         yield x**2

for k in my_func(4):
    print(k)

>>> 0
>>> 1
>>> 4
>>> 9

8/ import this

The this library is more a joke than a trick, but it's nice to know the little easter eggs of its language:

import this


The Zen of Python
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
. . .

9/ Basic operations on sets

If we have two sets a and b such that :

a = {1,2,3}
b = {3,4,5}

then we can perform the following operations that you must know !

# Union 
print(a & b)

>>> {1, 2, 3, 4, 5}

# Intersection 
print(a | b)

>>> {3}

# Symetric difference
print(a ^ b)

>>> {1, 2, 4, 5}

10/ Code if : else : in one line

It's not the most useful trick, but it's always useful to know how to write an if : else : in one line to make the code cleaner, or in shortest code competitions on codingame !

i = 1 if True else 2

>>> i = 1

11/ Limit the number of recursion of an algorithm

Limiting the depth of recursion of an algorithm is important. It is even the first thing to do when you know the maximum depth of recursion! You can do it with the Python library sys:

import sys
sys.setrecursionlimit(1000)

12/ Print text faster

When we have to print text in Python, we use by default the print() method ! However, when you have to print thousands of lines, this method can be slow. In this case we use :

import sys
sys.stdout.write(…) # only string

You can use a similar method for the input, but it's a bit more complex :)

This method is up to 8 times faster than the normal print

13/ Have the middle items of a list

Having the items in the middle of a list is a little trick to know when unpacking this list... Indeed, depending on the number of variables (or underscore if you don't want to keep the variable) at the beginning and at the end, you can have a variable containing a list of values containing only the "middle" one, in fact, the one that have not been put in other variables!

l = [1,2,3,4,5,6,7]
a, *b, c = l

>>> a = 1
>>> b = [2,3,4,5,6]
>>> c = 7

14/ Separate big numbers by _

For more readability, and because Python does not allow spaces between the digits of a number, we can use _. For example, 1 000 000 000 can be written in Python :

1_000_000_000

15/ Exchange keys / values of a dictionary

Exchanging the keys and values of a dictionary is a technique that can be useful, especially in the field of AI :). Here is how to do it:

my_dic = {“a”: 1, “b”: 2}
my_dic = {v:k for k, v in my_dic.items()}

my_dic = {1: “a”, 2: “b”}

Conclusion

That's all for today, I hope you liked the article and that you were able to improve your coding skills! Don't hesitate to share the article with your friends and to give me your feedback in comments!

Adrien

Getting started in AI

AIDRI — Fri, 06 Aug 2021 06:52:04 +0000

Hi guys,🖐

Today I wanted to give you some tips and answer some of your questions so you can get started in the world of AI !

By the end of this article, you will no longer have any fears about the difficulty of the field, one person will have become your god, you will have dozens of hours of videos to watch, and you will have plenty of project ideas for all levels !

I hope you like the article, let's get started !

I/ Is it necessary to have significant knowledge in maths ?

It is true that at first sight, one can say to oneself that one needs advanced knowledge in maths to create a brain... In fact, not really ! It all depends on the job you want to do. Obviously, if you want to become a researcher, you will need to have a good background in mathematics. But if not, you will soon realize that high school or basic undergraduate knowledge is enough !

Among this knowledge, I think you need to be comfortable with linear algebra, including: vectors, matrices, eigenvalues and eigenvectors, functions, derivatives, and vector and matrix calculus. Moreover, some basic notions of probability can be interesting: conditional probabilities, dependant and non-dependent elements, variance, etc...

If you don't have all this knowledge, don't worry, there are plenty of mini-videos or courses to explain everything ! I would recommend three of them :

Khan Academy
3Blue1Brown on linear algebra
MIT courses on linear algebra (there is also one I'm doing on probability).

II/ Where to learn AI ?

That's probably why you're here: where to learn AI. In this section, we'll talk about theoretical AI, but right after, I'll give you the best tools to code them !

To learn AI, we notice three different methods :

First of all, MOOCs (Massive Open Online Course). These are online courses, which last several weeks, but where everyone can start them when he wants, but especially go at his own pace. There are videos, with teachers and speakers, additional documents, links to useful information, but also exercises and tests. Some of these MOOCs offer certificates that can be useful in a CV / on Linkedin.
Here are some MOOCs that may be interesting to get started in AI :

As you will see, one person often comes up in MOOCs: Andrew Ng, he is in some way the god of MOOCs, we all already listened to him when we started AI ! There is no more reliable source.
As you have noticed some MOOCs are expensive, you can claim financial aid, which is often granted.

Then there are the books, there are plenty of them that sometimes contain exercises with answers. Again, this allows you to go at your own pace, but you can also study a particular chapter or find information easily. Here are some books to read: (a little expensive, but the content is worth it)

Deep Learning, Aaron Courville, Ian Goodfellow and Yoshua Bengio
Hands on Machine Learning with Scikit-Learn and Tensorflow, Aurélien Géron
Pattern Recognition and Machine Learning, Christopher Bishop
Reinforcement Learning: An Introduction, Andrew Barto, Richard Sutton

Finally, the research papers. There are plenty of them, but this is not what I would recommend for a beginner. Indeed, it is easier to follow a course where all the notions are covered than to go and find the different points by yourself (at the risk of forgetting them) and study them in depth.
However, when you feel capable, and you see that you have a good level, you should start to detach yourself from the MOOCs to go towards papers specific to your project: it remains relatively well explained and will give you tracks to carry out your idea !
You have different places to read them: arxiv (the most famous), researchgate, paperswithcode (code is provided with it), but also the blogs of the big ones in the field: deepmind, openai, facebook research, etc...
Here are some interesting ones :

Mastering the game of Go with deep neural networks and tree search, by David Silver & al
Attention is all you need, by Ashish Vaswani & al
Faster R-CNN: towards real-time object detection with region proposal networks, by Shaoqing Ren & al
Adam: A Method for Stochastic Optimization, by Diederik Kingma & al
Language Models are few-show learners, by Tom Brown & al
you will find dozens of others very interesting, but I must continue my article :)

III/ How to code AIs ?

I won't teach you how to code AIs here, but I will give you some tips to get started:

First of all, I advise you to code in Python, it is the language where the most libraries are developed. A new language is emerging : Julia, it's really nice if you want to implement IAs from scratch, but otherwise some libraries are missing !
Then you have to choose a framework : to start it is interesting to have one, but when your project will become more complex, you will have to choose several. Your choice must be made between Tensorflow and PyTorch. Eventually you can use Scikit Learn.

Read the documentation of your framework carefully, especially the important functions.
Look at implementations on github, it is even recommended by big names (Andrew Ng). Indeed, this will not only allow you to discover your language, but also to understand in another way the paper / course you are working on.
Use datasets and RL environments already made. As far as datasets are concerned, you will find plenty of them on the internet in general, which correspond to your project (you may have to rearrange the data, but not much more). Think to look on Kaggle, there is a "contest" section but also a "dataset" section gathering thousands of datasets ! On the RL side, you will need environments, use OpenAI-gym, they are very good and easy to use. For autonomous cars, I advise to use Carla. It is quite complete, and many tools are available (Lidar, camera, etc...).

IV/ Interesting projects to work on

At some point, you will have to get out of the "tutorial spiral", and start creating your own projects: you will become more and more confident in your skills, you will build a portfolio, etc...
Here are some ideas :

Beginner :

prediction of the price of a house
image classification (cat / dog)
chatbot
classification of tweets according to emotion
spam checker

Intermediate :

translation
personal assistant
driving a car (by reinforcement learning)
Moon Lander (by reinforcement learning)
detection of emotions on a face

Advanced :

summarize a text
chess player
Robotics (on OpenAI gym)
text generation
math solver (addition, subtraction, modulo, etc...)

Finally, you can try the Kaggle contests, many contests are available with lots of dataset. The goal is of course to make the best AI possible, using an AI model (no need to invent one) and to "tune" it !

V/ Conclusion

The field of AI is therefore quite accessible to everyone. You just need to have a basic knowledge of mathematics, and to start studying courses and books. Coding is also the best way to know your skills, and to reassure yourself about the apparent complexity of a project !

I hope I've motivated you to get into AI !

Adrien

Is Github Copilot a friend or a foe ?!

AIDRI — Fri, 30 Jul 2021 18:37:05 +0000

Hi guys,🖐

Today, I wanted to share with you his experience on the Github Copilot assistant that helps developers.

I hope you'll like it. Let's start:

I/ Introduction

On June 29, 2021, Github announced on its blog and on Twitter the release of a revolutionary assistant, able to help developers to code faster, discover APIs, and solve problems. Its name: Github Copilot. Immediately, this assistant made a lot of noise in the programming field. That's why we decided to create this article to summarize what happened last month, and the opinions about this AI.

II/ What is GPT3 and OpenAI Codex

Github Copilot has been developed in collaboration with OpenAI, the company founded by Sam Altman and Elon Musk, and managed by Ilya Sutskever. GPT-3 is an AI developed by this company, but it is especially the big brother of GPT-2, another similar AI developed a few years before. This AI is based on the technology of the Transformers, which uses a mechanism of attention (Attention Layer). It is composed of 175 billions of parameters, which makes it one of the biggest AIs ever created. To take advantage of all the capabilities of GPT-3, OpenAI has created alternative versions: DALL-E, GPT-F, and OpenAI Codex. Here, the AI has been trained on billions of lines of code from projects available on the internet. Github, led by Microsoft which has the exclusivity of GPT-3, decided to make OpenAI Codex available via VS Code, thus creating one of the most attractive private betas!

III/ The positive points of Github Copilot

Github Copilot has many advantages:

- You can write code faster : 💨

Indeed, OpenAI's AI runs on very powerful hardware technologies, which allows to predict results quickly. Moreover, many results are generated just as quickly, which allows me to introduce the following point...

- We have access to many results:

If the first result generated by Copilot is not suitable, it is easy to choose another one that may be more relevant. The number of results generated can even be changed in the AI settings.

- The AI takes into account the context: 👂

It can be given feedback to help it find relevant results. Moreover, it takes into account the previously written code, which allows it to adapt to our code. It constantly tries to understand our project. A funny example: the AI even adds its own comments! It's magical!

- It allows to discover APIs more easily: EZ

Copilot allows you to generate functions to call APIs, it's great if you want to save time, but personally I don't recommend to do that if you are still a beginner. Don't forget that it's Github Copilot, not Github Pilot ^^.

- Copilot is improving day by day: 💪

As announced by Github, when you accept the proposed code, it may be sent to Microsoft services for the AI to continue training. The results will be more and more relevant.

IV/ The negative points

However, all is not rosy with this AI ^^ :

- There is a legal blur : ⚖️

The AI is trained on code available on the Internet, but the owners may not necessarily agree to this use. However, the answers of the AI are based on these codes: you understand that on some points, it's a bit vague.

- The AI can repeat itself: 🔁

Although the AI has a system to avoid repetition, it happens that the AI creates two similar functions which can quickly become ugly.

- The AI does not use the latest conventions:

Indeed, Copilot tends to use old practices that nobody uses anymore and that make the code ugly. It can even become annoying when using some APIs, databases, etc...

- Unfortunately, Copilot can give the impression to beginners to know how to use it: 🧞

Copilot helps developers to code, but beginners shouldn't use it too much : indeed, they will learn bad manners, and will think they know how to code, when in reality, it's not the case. So Copilot is great for senior developers, but probably not for beginners.

V/ My opinion about Github Copilot

After using Github Copilot, I quickly realized that the AI was great to help the developer, but that it was not going to take our work. Indeed, it needs to be oriented, redirected to the solution. However, I was able to do some repetitive tasks, like coding a function to read data and show it, quite easily, which I liked. So you don't have to be afraid of Github Copilot.
If I had to summarize in one line what Github Copilot is now, I would say that it is a form of intelligent StackOverflow, a little bit like a little brain that would know all the languages, algorithms, and that would try to help us by all means.

VI/ Some screens

I let Copilot code a basic NN, here is the NN by Copilot... The good point is he even managed to create some functions that he coded separately for me right after !

I just asked him to code me a class to create Polynomes : in addition to defining them, I can do elementary operations with them, it's rather successful...

But hey, not everything is perfect, and sometimes he makes a few little mistakes! He didn't respect the order here ^^ !

DEV Community: AIDRI

15 Python tips and tricks every beginner should know !

1/ Create a number sequence

2/ Overlaying two dictionaries

3/ Create a dictionary of a sequence

4/ Reverse a list

5/ Unpacking a tuple

6/ Filter a list

7/ Return multiple values from a function

8/ import this

9/ Basic operations on sets

10/ Code if : else : in one line

11/ Limit the number of recursion of an algorithm

12/ Print text faster

13/ Have the middle items of a list

14/ Separate big numbers by _

15/ Exchange keys / values of a dictionary

Conclusion

Getting started in AI

I/ Is it necessary to have significant knowledge in maths ?

II/ Where to learn AI ?

III/ How to code AIs ?

IV/ Interesting projects to work on

V/ Conclusion

Is Github Copilot a friend or a foe ?!

I/ Introduction

II/ What is GPT3 and OpenAI Codex

III/ The positive points of Github Copilot

- You can write code faster : 💨

- We have access to many results:

- The AI takes into account the context: 👂

- It allows to discover APIs more easily: EZ

- Copilot is improving day by day: 💪

IV/ The negative points

- There is a legal blur : ⚖️

- The AI can repeat itself: 🔁

- The AI does not use the latest conventions:

- Unfortunately, Copilot can give the impression to beginners to know how to use it: 🧞

V/ My opinion about Github Copilot

VI/ Some screens

If you liked my article, do not hesitate to share it, but also to give me your feedback

Adrien