DEV Community: Taeho Jeon

How to implement Fibonacci Sequence with Python

Taeho Jeon — Sun, 09 Jun 2019 05:59:35 +0000

Fibonacci sequence is one of the most popular interview questions. There are several ways to implement it with Python. Let's see how to do that.

Fibonacci Sequence

The Fibonacci Sequence is the series of numbers:

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ...

The next number is found by adding up the two numbers before it.

The 2 is found by adding the two numbers before it (1+1)
The 3 is found by adding the two numbers before it (1+2),
And the 5 is (2+3),
and so on!

The rule of Fibonacci Sequence is

x(n) = x(n-1) + x(n-2)

where:

x(n) is term number "n"

x(n-1) is the previous term (n-1)

x(n-2) is the term before that (n-2)

source : https://www.mathsisfun.com/numbers/fibonacci-sequence.html

Method 1 : With for loop

# using for loop 

def fibonacci_loop(num):
    if num == 0:
        return 0
    elif num == 1 or num == 2:
        return 1
    elif num > 2:
        a = 1 # variable for (n - 1)
        b = 1 # variable for (n - 2)
        for _ in range(3, num + 1):
            c = a + b
            a, b = b, c

        return c

%time fibonacci_loop(40)
    CPU times: user 6 µs, sys: 0 ns, total: 6 µs
    Wall time: 8.11 µs
    102334155

With recursion

We can solve the problem with for-loop, but it is not so intuitive.

From the rule of fibonacci sequence x(n) = x(n-1) + x(n-2) ,

we can make a function that call itself,
it is called recursive function.

# Recursive Method 1 : traditional recursive function

def fibonacci_recursion(num):
    '''Return a fibonacci sequence value of num'''
    if num == 0:
        return 0
    if num == 1 or num == 2:
        return 1

    return fibonacci_recursion(num - 2) + fibonacci_recursion(num - 1)

%time fibonacci_recursion(40)
    CPU times: user 26.5 s, sys: 64 ms, total: 26.6 s
    Wall time: 26.6 s
    102334155

Quiet slow??

It takes more and more time while the number is increased.

Because the function itself is needed to calculate same value over and over.

We can reduce total number of calling the function with saving the result to cache.

# Recursive Method 2 : With using explicit cache
cache = {}

def fibonacci_cache(num):
    '''Return a fibonacci sequence value of num'''

    if num in cache:
        return cache[num]

    if num == 0:
        result = 0
    elif num == 1 or num == 2:
        result = 1
    else:
        result = fibonacci_cache(num - 2) + fibonacci_cache(num - 1)

    cache[num] = result
    return result

%time fibonacci_cache(40)
    CPU times: user 2 µs, sys: 0 ns, total: 2 µs
    Wall time: 5.01 µs
    102334155

same result but super faster than normal recursive function

Using LRU cache

Python provides a functools library helps the recursive function calls,

let's use Least Recently Used cache (lru_cache) from functools

# import library
from functools import lru_cache

# Recursive Method 3 : with implicit cache provided by functools

# set cache with 1000 (need to set a big values, small cache is NOT useful)
@lru_cache(maxsize = 1000)

def fibonacci(num):
    '''Return a fibonacci sequence value of num'''

    if num == 0:
        return 0
    if num == 1 or num == 2:
        return 1

    return fibonacci(num - 2) + fibonacci(num - 1)

%time fibonacci(40)
    CPU times: user 3 µs, sys: 0 ns, total: 3 µs
    Wall time: 5.25 µs
    102334155

Little differ time values, but it could be ignored. (just some micro seconds)

Conclusion

We need to set lru_cache when using recursive function for performance

Source : https://github.com/Teosoft7/fibonacci_python.git

Computer Vision: A new way to see the world

Taeho Jeon — Fri, 07 Jun 2019 18:23:42 +0000

How does Tesla's autopilot work? How does the car recognize the signals, other vehicles, pedestrians, lanes and so on? Basically, Tesla is using several cameras on the vehicle to understand what obstacles are around of the car. It enables the car to detect traffic, pedestrians, road signs, lane markings, and anything else that might be in front of the vehicle. This information is used to help the car drive itself. Tesla is using its self developed Full Self Driving(FSD) computer that handles image processing and recognition.

This technology came from kinds of artificial intelligence(AI) technology called computer vision. Computer vision is the science and technology of machines that see. As a scientific discipline, computer vision is concerned with the theory and technology for building artificial systems that obtain information from images and/or videos.

Then, how computer vision recognize the objects in the image/video? It involves numerous tasks including recognizing what objects are present, localizing the objects in 2D and 3D, determining the objects' and scene's attributes, characterizing relationships between objects and providing a semantic description of the scene.

In terms of data science, the task of computer vision is a kind of classification problems that validates an object is on the target dataset or not. (It is the same problem with spam detection!) The model could be quite complex because of the basic characteristics of the image/video data. The classification model consists of an algorithm and a dataset for training algorithm. We can use state-of-the-art algorithms something like classifying with neural networks.

Let's think about how the classification model is working especially focus on the dataset.

Generally, the computer vision algorithm works with video and/or image. The data structure of images in the computer can be presented with n by m matrix based on its resolution. For example, resolution of full high definition image is 1920 x 1080, it could have 2 million pixels, we can put it into the matrix. If we set 2 Bytes per value, the size of the matrix could be around 7.6MB. And the video is a combination of images, usually, we are using 30 frames(images) per second. It means, over 200MB size of data could be processed for analyzing the 1 seconds video. (This is a simplified example of image/video processing, it would quite different in the real world.) Because of these kinds of difficulties, the dataset must be light and clear. The number of training data also does matters. In the universe, there are tons of objects. If it is too huge, it means it needs a huge computing power to perform a computer vision analysis. It is impossible to put a supercomputer into a car.

Microsoft COCO(Common Object in Context, http://cocodataset.org/#home) dataset is a new large-scale dataset for detecting and segmenting objects found in everyday life in their natural environments. COCO dataset has 91 objects types, a total of 2.5 million labeled instances in 328k images.

The images could be sorted by a)iconic object image, b)iconic scene image, c)non-iconic image. Most of the dataset for object recognition have focused on image classification, object bounding box localization or semantic pixel-level segmentations with iconic images. But COCO dataset focus on segmenting individual object instances even if it is placed on the non-iconic image.

Here is an example of object detection with COCO dataset.

Conclusion

The human eyes are one of the most sophisticated organs in the body. There are a lot of studies about computer vision are progressing in the world and the skill level of computer vision can understand is improving so fast. With computer vision, someday I wish that we could have another eye for us.

Sources

Microsoft COCO: Common Objects in Context ( https://arxiv.org/abs/1405.0312 )
Object Detection with Mask RCNN and Coco Dataset ( https://youtu.be/dviGTND6a6s )

Advanced Python: for more attractive code

Taeho Jeon — Mon, 20 May 2019 16:15:59 +0000

Tips and Tricks for Advanced Python

Python is a high-level, interpreted, interactive and object-oriented scripting language. Python is designed to be highly readable. It uses English keywords frequently whereas other languages use punctuation, and it has fewer syntactical constructions than other languages.

The syntax of Python is clear, intuitive and easy to learn compared to other programming languages such as C/C++, JAVA, and Swift. Python has a lot of built-in standard features like comprehensions, dictionaries, sets, and generators to provide the most common functions required by modern programming languages. And Python also has tons of libraries provided by its own community such as 'pandas', 'numpy', 'matplotlib', and so on. Because of these, Python is one of the most popular programming languages, especially for data science.

Besides the simplicity and user-friendly of Python, learning how to write code is not an easy task. It is very similar to learn a new unfamiliar language. It needs to be practiced again and again until could feel comfortable within it. And to write more clear code, it needs to understand how Python interpreter works, but it could be out of scope for the data science. But here is seven tips and tricks for more attractive Python code as a data scientist.

1. Underscore(_) separator for Large Number

'_' can be used as a separator for expressing a large number

ten_billion = 10_000_000_000

print(f'{ten_billion:,}')
10,000,000,000

2. Assign a value with if statement

When 'if' statement has only an assignment code inside of it

# general
isHappy = True

if isHappy == True:
    result_string = 'Happy'
else:
    result_string = 'Not Happy'

print(result_string)

Happy

# advanced
isHappy = True

result_string = 'Happy' if isHappy else 'Not Happy'

print(result_string)

Happy

3. Swap values between two variable

Python can swap the values between two variable without temp variable

# general
low = 10
high = 9

if low > high:
    temp = low
    low = high
    high = temp

print(low, high)

9 10

# advanced
low = 10
high = 9

if low > high:
    low, high = high, low

print(low, high)

9 10

4. Enumerate function

If you need index value inside of for loop, use enumerate function

# general
grades = ['A', 'B', 'C', 'D', 'E', 'F']

i = 1
for grade in grades:
    print(f'{i} : {grade}')
    i += 1

1 : A
2 : B
3 : C
4 : D
5 : E
6 : F

# advanced
grades = ['A', 'B', 'C', 'D', 'E', 'F']

for i, grade in enumerate(grades, 1):
    print(f'{i} : {grade}')

1 : A
2 : B
3 : C
4 : D
5 : E
6 : F

5. List Comprehensions

a concise way to create lists

# beginner
numbers = [1, 2, 3, 4, 5, 6, 7]

squared = []
for number in numbers:
    squared.append(number * number)

print(squared)

# advanced
numbers = [1, 2, 3, 4, 5, 6, 7]

squared = [number * number for number in numbers]

print(squared)

6. Unpacking

unpacking values inside of tuples


# from 13 cards
cards = ('A', '2', '3', '4', '5', '6', '7', '8', '9', '10', 'J', 'Q', 'K' )

# extract ace, 2, 3 only
ace, two, three, *_ = cards
print(ace, two, three)
A 2 3


# extract ace, [numbers], J, Q, K
ace, *numbers, J, Q, K = cards
print(ace, numbers, J, Q, K)
A ['2', '3', '4', '5', '6', '7', '8', '9', '10'] J Q K

7. For~Else

else statement also be used after for loop, it is executed after for loop

# For ~ Else
grades = ['A', 'B', 'C', 'D', 'E', 'F']

my_grade = 'A+'

for grade in grades:
    if grade == my_grade:
        print('grade found')
        break
else:
    print('grade not found')

grade not found

Data Visualization: the best way to understand the data

Taeho Jeon — Mon, 06 May 2019 04:00:18 +0000

To analyze the data, we need to understand the characteristics of data first. But reading the data is not an easy task. Spreadsheet application can help us to read data efficiently with table or sheet but if the data is huge, we could get some difficulties from it. For example below is the Wage (Github link) data which contains income survey information for males from the central Atlantic region of the US, part of the book 'An Introduction to Statistical Learning'. It has 3,000 rows and 12 columns containing the summary of wage data by year, age, region, and so on.

We can check with pandas like below,

df = pd.read_csv('Wage.csv') df.head()

And we can see the basic statistical values from data with below python statement,

df.describe()

we can get some information from data with tables, but it could not give a clear understanding of the data. Because the table and text do not tell us the story of data.

But with some image like below, we can see, the change of wage by ages, year, and education level at a glance.

As we saw above, it is becoming much easier to read the data by visualization with a chart. We can have much information through the data simultaneously. In these days, there are lots of convenience tools for drawing a chart from data. Microsoft Excel and Tableau is a famous and powerful application for organizing the data and draw and it is the best fit for most business cases. And there are also online tools that support the various style of charts such as Google Chart, Online chart tool, and so forth. By using these kinds of chart tools we can have various chart without much difficulty.

All of the tools are very convenient but still have some limitations. We need to follow their rules and methods for them.

But by writing code with python, we can draw a chart much freely. One of most strength of Python is, it has lots of chart libraries that we could get the various visualized form of data. We can make a general chart with matplot, plotly, seaborn, and we can also draw some plot on the map with folium, basemap, and there are a bunch of more chart libraries that make our life easier.

Most of the data can be presented by the chart but how about the text data. There are a few methods that visualizing the text with the chart. But the most interesting method is called 'Word Cloud' that expressing the kind of chart with text. A word cloud is an image made of words that together resemble a cloudy shape. The size of a word shows how important it is e.g. how often it appears in a text — its frequency. Here is an example of Word Cloud.

Word clouds typically used to easily produce a summary of large documents (reports, speeches), to create art on a topic (gifts, displays) or to visualize data (tables, surveys). But it was a little hard to produce word cloud, you should count the words in some text and draw it with art tools such as Photoshop or Illustrator. But with wordcloud Python library, it is super convenient. You can make a word cloud within several lines of code.

texts = 'Dog Cat Animal Carmel Snake' wordcloud = WordCloud() wordcloud = wordcloud.generate_from_text(texts) plt.figure(figsize = (8, 8), facecolor = None) plt.imshow(wordcloud, interpolation="bilinear") plt.axis("off") plt.tight_layout(pad = 4) plt.show()

Try to make a creative word cloud with Python codes!!!

Data Scientist: Who connect people and data

Taeho Jeon — Thu, 25 Apr 2019 05:42:45 +0000

In March 2016, AlphaGo, artificial intelligence(AI) software of Google DeepMind defeats a South Korean Go grandmaster Lee Sedol. Due to Go's complexity and the importance of reaction and intuition, most of Go expertise expect that AI never going to beat the human. But it was wrong, AlphaGo won the series 4-1 and astonished the world with the power of AI. It was a major milestone for artificial intelligence, made the AI more popular to the public.

Before this event, most people did not recognize the words "Machine Learning(ML)". I also think that ML is very far from my life even though having some knowledge about that. But right now we can find the result of AI very easy, many times we could not realize it is from AI. For example, voice assistant service such as Siri(Apple), Alexa(Amazon), the autonomous driving skill of Tesla, the recommendations at Amazon first page, movie recommendations from Netflix is the output of AI.

Who made these things are happening?
The answer is the Data Scientist. From Wikipedia, data science is a multi-disciplinary field that uses scientific methods, processes, algorithms, and systems to extract knowledge and insights from structured and unstructured data. Does it sound like a completely newer job? No, I believe it is just a new word. Data science is a newer field based on the use of computers, technologies, and statistical methodologies. But it is an inheritance from 'data mining', and also called 'big data analytics'. Both are the process of sorting through large data sets to identify patterns and establish relationships to solve problems through the analysis. They are tools allow enterprises to predict future trends.

What is the role of a data scientist?
Basically, data scientist gathers and analyzes the data with the goal of reaching a conclusion. They do this through several different techniques. They could present data with a visual context for a user to look for meaningful patterns that wouldn't be noticeable from the spreadsheets with a large number of numbers and characters. And they also create some algorithms that are used to predict some patterns and events will going to happen. Their main purpose is the practice of finding a meaningful conclusion among the huge amount of data.

Why data science became so hot?
First of all, we can collect data from everywhere easier than before, because of technology innovation. The price of sensors, microprocessors, memories, and storage is going cheaper and cheaper, the cost of using a network is also down every day. For example, if I want to collect the temperature data at some place in Olympic National Park, I need a solar panel, battery, smartphone and temperature sensor for that. Definitely, it costs under $400, and I also need an internet connection to get the data remotely. It also cost under $50 per month. Many telecommunication companies such as Verizon and AT&T offers 4G unlimited data transfer under $100 per month. In terms of cheaper storage, companies can also collect and store much more than before. This kind of change brings the big data era. The power of utilizing its data is becoming the most important point for evaluating the value of the company.

How could be a data scientist?
Arguably data scientist is the hottest career today and being a data scientist is absolutely challenging for everyone. But it is not so hard, any way you don't need to be a Ph. D. If you have some understanding of certain industry based on your experience, you can be a data scientist with learning about some computer science skills especially for handling the data and some knowledge about statistics. It is not super complicated.

Why Should I be a data scientist?
My major is computer science and did work for over a decade as a software engineer. Most of my career is handling and analyzing the data from the advertising industry such as TV rating, advertising spending of some brands, number of impressions of a digital campaign and so on. I worked for several companies around the adverting industry as a data engineer, database administrator, frontend and backend engineer and more. Even though I worked as a media planner with the understanding of data. My last position was creative technology in the second-largest advertising company of Korea. My role was a kind of bridge between tech. people and business people. I could say I'm kind of prepared candidate for data science. In the next couple of weeks, I'm trying to refresh my knowledge about computer science and study more about the statistics and math features of data science for a data scientist.

Would you like to connect the data and people?