DEV Community: Kei

I hate making resumes so much that I made a resume builder, and it was a blast.

Kei — Tue, 03 Jan 2023 06:22:59 +0000

Hello everyone, this is Kei.

How do you make your resume when you are looking for a new job?

I think it's important to have a good resume design, so every time I change jobs, I have to redesign my resume, but it's a lot of work.

There are services that can create resumes for you, but it's hard to find a service that is suitable for me because, design-wise, they are either not suitable for the culture of the country or too classic.

Even if you want to modify an existing template a little, you have to learn how the template works from scratch, which is not easy.

So, I thought it would be nice if I could use a template that is super simple to manage resumes and easily customizable, and since I had some free time during New Year's, I made one myself from scratch.

This is what I made.

About Features

The functions available are all basic: creating the basic information needed for a resume. Saving. Loading.

One of the unique features is that as long as you load a theme that is compatible with resume builder by URL, you can use that theme immediately.

Each theme can use its own attribute values, and if the values corresponding to the theme are set in the custom attributes, the values will be automatically applied when the theme is changed.

The information entered in the biography can be saved, so there is no need to enter custom attributes for each theme again as long as the data is loaded from the next time onward.

Incidentally, the background data is saved in JSON format, so it will be more convenient if you save it in github or other file system.

About Themes

Themes can be applied by entering a URL in the input form or selecting one from the dropdown.

Currently, only one theme is available since this application itself was created during the New Year's holiday, but the theme repository is open to the public, so we would be happy if anyone who thinks he/she is the one to create a theme would open it to the public.

I am also planning to create a theme when I can take the time.

Have a great job search, everyone!

My own mistakes that I should have done at that time

Kei — Mon, 12 Dec 2022 09:03:03 +0000

First of all, this article is a completely reflective post that focuses on my past mistakes, just talking about what I might have done differently. Some readers may know better ways to do this, etc. However, this is only based on my experience, so I would appreciate it if you could keep it warm.

If you get thrown a project one day, be prepared.

That may not be the best way to put it. But when a project is thrown at you one day, it is usually because the project is not going well. If it is a simple staffing increase, it may not be. But if you look around and see that your predecessor is missing, the odds of going to hell are high. In such a situation, I would say to run anyway. If you can't escape, you might as well go buy a sleeping bag for the season. A sleeping bag is the safest way to sleep on an office chair.

Be the first to check for test codes

When you take over a project, the major difference between you and your predecessor or your team is not coding skills, but domain knowledge. Clients tend to lump us all together as engineers, so they don't take into account the difference in domain knowledge that your predecessors have accumulated since the early days of development and their understanding of the current code. Therefore, they make demands based on the same assumptions as before, such as development time.

The test code is to fill in these differences in experience as much as possible. Therefore, when you take over a project, the test code should be the first thing you look for. If you cannot find the test code, you should refrain from changing the code. Then, after passionately talking about the dangers of the current situation, tell them that they should prioritize writing tests first or refactoring now. Because we can't guarantee that it will work, and there is a possibility that another function will break without our knowledge. Besides, by writing tests and refactoring, it is often possible to get a sense of how the code was intended to be implemented. If you skip this step and proceed with development, and a bug occurs, it is not your predecessor or your customer who takes responsibility. In the end, it is you. Wouldn't you like to go home every day?

Don't hate the code. If you hate it, rethink how your project is progressing

The most common cause of project flame-outs is often an overloaded project progressing faster than they think it should. Especially in agile development, for example, there are too many feature requests from the customer, and the requests are carried to the engineers without being discarded. That is natural. If the development time is fixed, the customer wants as many functions implemented as possible. I would probably do the same.

However, just like a conveyor belt in motion, if the speed is not properly adjusted, the amount of work per iteration will increase many times over, and eventually, we will choke ourselves. Let's act now to slow down the conveyor belt.

Customers don't know what's in the code

Customers only know what they experience firsthand, such as whether the software is working properly. In other words, they have no other criteria to evaluate software. However, it is not always the case that the part of the software that you are in charge of is the easiest to evaluate. This is especially true when you are working on the back end or on a large function. In other cases, what looks surprisingly simple from the client's point of view may actually be based on an unbelievably complex process. No matter how hard you try to explain the difficulty of the process to the customer, they will never be able to empathize with it. This is not surprising because they are not engineers. If you are ever told by a client or your boss that you are unappreciative, calm down and smile back, because there are people in the world who feel the same way as you do. At least I am on your side.

To achieve functionality, you need to broaden your vision.

With legacy code, you don't know what's going on in the first place, so trying to add a new feature often doesn't work the way you want it to. In fact, you may think it would be faster to start from scratch. But reality is merciless and will not allow us to do so. Perhaps, though, if we wait 20 years, it might be possible.

In such a case, let's just try to find out if there really is another way to realize the request. It is okay to take the plunge once and move the data out of the current application and let another tool handle the process. There must be other ways to achieve the objective.

And always keep the following thoughts in the back of your mind." The engineer's job is to provide the methods and things to solve a problem. And we should not determine for ourselves the extent to which we look for solutions. Because we are so desperate that we only see what is in front of us."

Are you sure you got that estimate right?

When you are asked to estimate a feature, are you sure you are getting the correct estimate? Remember that software development estimates are based on the uncertainty of the system. If you think you can do it in 10 days in your estimate, you should actually estimate 14 days or around. The reason for this is that in many cases, the actual work is different from what they imagined after they actually started working on it. When asked, "Will it really take that long?" They will be convinced if you tell them that there are many uncertainties and that you want to give them plenty of time to work on it. Besides, they will not blame you if the work is completed earlier than you imagined, so there is no disadvantage in estimating more than enough time.

Don't let them throw away the business logic

It is a common practice in contracted development to have the business logic for a service developed by the client thrown to the engineers. This is not so common in cases where the company is conscious of digitalization, such as when the methods it has cultivated in-house are incorporated into software. On the other hand, it is very common in cases where the client wants to create a new service. In such cases, even the clients themselves do not have a clear idea of what they actually need to do and how to put it together to create what they want to create. And the engineers are forced to think about the process. And in the worst case scenario, the product is not at all what they had in mind.

Furthermore, not knowing the logic of their business is like driving a car without a driver's license. It would be unwilling for a developer to proceed with development knowing that the worst possible future awaits them. So, let's help our clients to think about the logic together with us. There is no need to use complicated terminology. You can use a ubiquitous language that is a common language between the customer and the engineer. Anyway, help the customer to understand how the software system works and to think about the logic on their own.

How to improve your Database performance

Kei — Mon, 12 Dec 2022 06:09:42 +0000

Hello everyone, I am Kei and I'm here to talk about database optimization in PostgreSQL

One question before We start.

How do you store data when developing applications these days?

Recently, many small application developers are using so-called NoSQL, such as Firestore.

I myself use Firestore as my main data store for both work and pleasure, so I have recently moved away from the strong SQL and PostgreSQL RDBs that I am familiar with from the past.

But I'm starting to think lately that this is not a very good thing.

Naturally, NoSQL is convenient at first glance because of its fast implementation and transaction speed, but after using it for a long time, you start to realize that the running cost of NoSQL is surprisingly high.

Especially when handling core business data, it is difficult to ensure the relevance of data, and you have to be even more careful. Therefore, I believe that NoSQL cannot be a perfect substitute for RDB, and that it is important to use it where it is essential. There is no silver bullet.

So, recently I've been switching where to store data from Firestore to PostgreSQL for some of my projects, but it's been a few years since I've used it and I've forgotten a lot of things, so I wanted to relearn some things.

So, I'm going to share some SQL How To's with a focus on improving database performance over the next few posts. (By the way, I will be using PostgreSQL.

Flow of Query

Now, since this is the first article in this series, let's start with what we are most likely to work on when performing database performance: query optimization.

To do this, the most important thing is to understand how DMBS receives and evaluates queries.

Raghu Ramakrishnan, Johannes Gehrke, Database Management System 3rd ed, McGraw-Hill, 2002, p.405

The figure above simulates the query processing flow.

Parser

The role of the parser is to parse SQL statements. The parser parses the SQL statements that we normally write and execute, and checks them for syntax errors. After parsing is complete, the parser also converts the SQL to a format that can be processed by a DBMS.

Optimizer

The Optimizer, as the name implies, creates the best way to access the data, and it selects the least expensive execution plan based on various conditions such as the presence or absence of Indexes, the degree of data distribution, and information on the internal parameters of the DBMS itself.

Catalog Manager

The information needed to develop an execution plan in Optimizer is done through the Catalog Manager. The Catalog Manager stores statistical information about the tables, for example, the number of records.

Plan Evaluation

After the Optimizer has generated the execution plans from the SQL statements, it is responsible for selecting the best plan from these execution plans. Unfortunately, the plan selected by Plan Evaluation is not always the best.

Now that you have come this far, I am sure that you somehow understand the importance of the Optimizer function in improving the performance of the database.

But whatever the case may be, let's write and execute some SQL at hand to see how the whole process works!

create table users(
    id serial primary key,
    user_name varchar(10) not null
)

do $$
begin
for r in 1..1000 loop
insert into users(user_name) values(r || 'user');
end loop;
end;
$$;

explain
select * from users where users.user_name = '5000user';

First, without thinking, create a user table and then create records for 1000 users.

Now let's add the word explore to the SQL statement and run it.

Interestingly, the information was different from what we normally see on our screens. Yes, this is the execution plan itself, which shows how the query is being executed, and this is how we, as engineers, can make sure that the execution plan is properly formulated.

Here is how we see this information

Seq Scan

This represents the type of operation, which can vary depending on the situation. Also note that the notation varies depending on the type of DBMS. In this case, since we are using PostgreSQL, this Seq Scan indicates that a full scan is being performed.
users

This is the name of the operation target. In this case, we are operating on the users table, so it is denoted as users.
In parentheses

Information about the operation target is displayed here. The cost and number of rows to retrieve, and width is the average size of the rows to be retrieved.

Those with good senses may have already noticed here that this time, user_name is set sequentially numbered, so rows=1 should not be the case. But this is no mystery; the plan itself is created in an optimal way. However, as I explained earlier, the execution plan is created based on information from the catalog manager, so the current situation is that the information from the catalog manager is incorrect, i.e., the execution plan is created based on the wrong information.

Although we said that the catalog manager creates statistics based on the information in the tables, the fact is that statistics are not updated immediately after data is stored in the database. (It is usually done automatically on a regular basis.

(Usually, it is done automatically on a regular basis.) Also, if the amount of data is small, it does not have a significant impact on performance even if the statistics update is not done immediately. However, if you have a large amount of data at one time, as in this case, or if you want to measure performance properly after the statistics are updated, the current state is not appropriate.

So I have to wait until the next update? When? Don't worry! Don't worry! postgreSQL has SQL to update the statistics.

analyze users;

This is all it takes to update the statistics in the users table. Try executing the same SQL statement as before and check the information.

As we expected, you see that rows is displayed as 1, which is the correct number of rows. However, it is important to note that the statistics are only a guide and do not necessarily show the correct number of rows. Especially for a simple query like this one, I am sure it is correct, but the more complex the query is, the more suspicious it becomes, so always keep this in mind.

Let's play a little more interestingly.

create table items(
    item_id integer not null,
    item_name varchar(10)
) 

do $$
begin
for r in 1..100000 loop
insert into items values(r, (r ||'user' )); 
end loop;
end;
$$;

analyze items;
drop table items;

create index item_name_idx on items(item_name);

explain
select item_name from items i
where i.item_name = '50000user';

Remember this value! Next, let's put something called index against user_name in the users table.

create index user_name_idx on users(user_name);

The results have changed from Seq Scan to Index Scan.

Now, here is a question for you: Have you ever heard of B-Tree?

(I'll spare you the details of B-Tree, as you can find plenty of information about it on the Internet.)

PostgresSQL also uses B-Tree, so it would be faster to put an index on user_name of items like this.

It should be faster to retrieve the data. However, when we look at the actual values, we see that the cost is lower before pasting the index.

Actually, this is because postgreSQL's EXPLAIN does not actually issue SQL, but returns an estimate of the execution plan. Therefore, if the values are off by a large amount, as in this case, you can get the correct information by specifying that you can actually execute the query.

drop index item_name_idx;

explain analyze
select item_name from items i
where i.item_name = '50000user';

Now let's add an index.

create index item_name_idx on items(item_name);

explain analyze
select item_name from items i
where i.item_name = '50000user';

With explain analyze, we can now see the correct execution time and it is clear that the index is effective.

Now that we know how to read the execution plan, let's try a more complex query.

What we will do here is split the categories into 1 and 2 according to the birth year of the monsters in the Pokémon table (the release date of each Pokémon).

create table monsters(
    id serial primary key,
    monster_name varchar(10),
    year integer
)

insert into monsters(monster_name, year) values('Picachu', 1996);
insert into monsters(monster_name, year) values('Chikorita', 1999);
insert into monsters(monster_name, year) values('Torchic', 2002);
insert into monsters(monster_name, year) values('Piplup', 2006);
insert into monsters(monster_name, year) values('Oshawott', 2010);
insert into monsters(monster_name, year) values('Fennekin', 2013);
insert into monsters(monster_name, year) values('Rowlet', 2016);
insert into monsters(monster_name, year) values('Scorbunny', 2019);

Also, let's write an SQL statement that gets exactly the same result in the two different method pictures.

case1

explain
select monster_name, '1' as category
from monsters m
where m.year <= 2006
union
select monster_name, '2' as category
from monsters m
where m.year > 2006

case2

explain
select monster_name,
case when year >= 2006 then
'1' else
'2' end as category
from monsters;

Let's run the queries for case1 and case2. To our surprise, we found that the results of the execution plan are very different between case1 and case2.

Why does this happen?

The reason is obvious: when we are doing union, we are using the select clause twice to combine the results of each. On the other hand, with the case method, the select clause is issued only once, resulting in fewer scans and lower execution costs.

So, what do you think? We have focused on the optimizer, but this is just a brief example, and we are only at the beginning of some very interesting database performance tuning.

Thank you!