Intro to HTTP

Igor Perepilitsyn — Thu, 08 Sep 2022 08:36:51 +0000

Hypertext Transfer Protocol is a set of rules for sending data across the internet. It was invented to standardize communication, e.g., how browsers should request data from servers and how servers should respond. HTTP is very widespread, so as a web developer you will interact with the protocol a lot.

The basic request

It’s a text protocol, so a human can easily read it. Here is a simplified version of a message that a browser generates to ask for the main page of dev.to:

GET / HTTP/1.1
Host: dev.to
User-Agent: Chrome/102.26.590.728
Accept-Language: en-US

The first line, called the request line, contains the request. We tell the server that we want to read (GET) a resource at the root page / (as in dev.to/). The line ends with version (1.1) of the HTTP protocol.

Headers

The second and the following lines contain additional information called headers. They are technical bits of data needed for communication to work out correctly. A header consists of a name and a value separated by a colon. One line holds precisely one header, and there is no limit to their number, apart from the max request size a server can handle.

Headers are optional, except for Host, which specifies the domain of the requested resource. The last two headers in the example above specify our browser and preferred language.

The complete list of all available headers is long, and you don’t really need to know all of them. Among other reasons, a browser will handle them for you in most cases.

Response

And here’s how a simplified HTTP response may look:

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 31890
Content-Encoding: gzip
<response body>

The first line consists of an HTTP version and a status code, which briefly describes the request result. Same as in requests, the following lines contain headers. For instance, the server tells our browser that the response has an HTML page (Content-Type: text/html).

An HTTP response can have a response body: a file (a bunch of bytes), an HTML page, or it can be empty. That’s what we see when our browser renders this information on our screen.

That’s it for the basic layout of an HTTP message.

Cookies

Cookies — are small pieces of information that persist across requests. They are used to “remember” a user on a website. For example, your login information is saved in cookies. That’s why you don’t authenticate each time you visit some website. That is also the reason you see targeted ads. Cookies are stored in your browser and sent to the server with every request.

In terms of HTTP, cookies are headers. They are initially sent with the response:

Set-Cookie: <cookie-name>=<cookie-value>

If browser has saved cookies, it will pass them in subsequent requests as headers:

Cookie: name=value

Methods

Methods or HTTP verbs describe which action we want to perform on a resource. An action is executed on a server-side; hence methods are only used in requests. There are 9 methods as defined in the standard. Some of them may seem identical, but there are 3 properties to help us determine the difference between them:

Safety — action only reads the resource and does not change it.
Idempotence — doing the same action repeatedly will produce the same result.
Cacheability — the response can be saved somewhere and retrieved for later use.

Now, let’s have a look at each method. I’ll leave out CONNECT and TRACE because they are pretty rare.

GET

Safety	Idempotence	Cacheability
+	+	+

GET is the most frequently used method on the internet. It is used to read a resource. When navigating a website in a browser, we make a GET request.

GET / HTTP/1.1
Host: httpbin.org

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 9593
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>httpbin.org</title>
...

POST

Safety	Idempotence	Cacheability
-	-	sometimes

POST method sends data to the server. That data is placed in the request body, which works the same way as in an HTTP response. POST method alters resource state, often creating a new instance of something. It is the second most frequently used type of request on the internet. If you submit a form on a website, your browser will send a POST request.

POST /post HTTP/1.1
Host: httpbin.org
Content-Type: multipart/form-data; boundary=--------------------------878806948835616453637143
Content-Length: 285

----------------------------878806948835616453637143
Content-Disposition: form-data; name="login"
kazauwa
----------------------------878806948835616453637143
Content-Disposition: form-data; name="password"
supersecret
----------------------------878806948835616453637143–

PUT

Safety	Idempotence	Cacheability
-	+	-

PUT is like POST, but idempotent. That is, subsequent PUT requests will produce the same result. POST, in contrast, can lead to unexpected effects. Imagine submitting a registration form multiple times and creating multiple identical users.

PATCH

Safety	Idempotence	Cacheability
-	-	-

Also known as partial PUT. Unlike POST, it cannot be used in forms. PATCH is meant to send only partial changes to a resource (say, we only want to change a username), but in reality, not many servers implement it.

DELETE

Safety	Idempotence	Cacheability
-	+	-

DELETE is used to delete a resource. It is allowed to send data in the request body but is not required. As with PUT, it should produce the same result for subsequent requests. After all, you can’t delete something twice.

HEAD

Safety	Idempotence	Cacheability
+	+	+

HEAD works almost like GET, except that the response should not have a body. If it is present, it must be ignored. HEAD is used to fetch headers that can be used to plan the subsequent request. For instance, if we want to download a file, it may be a good idea to check its size (Content-Length header) and see if we can save it.

OPTIONS

Safety	Idempotence	Cacheability
+	+	-

OPTIONS asks a server which methods can be performed on a resource. They can be found inside Allow header in the response. You probably won’t use it directly, but it is used by a browser sometimes. Not many servers implement this method.

OPTIONS / HTTP/1.1
Host: httpbin.org

HTTP/1.1 200 OK
Allow: OPTIONS, HEAD, GET

Status codes

Every HTTP response has a status code, a 3-digit number that indicates whether it was successful or not. They are divided into 5 groups:

1хх — informational
2хх — successful
3хх — redirection
4хх — client errors
5хх — server errors

Let’s take a look at some of the common codes. The complete list is available here.

1хх

Informational codes notify a client of an intermediate status of their request. You probably won’t ever see them.

2xx

Status codes from this group indicate that the request was accepted and successfully processed by a server:

200 OK — success. Most of the time, you will see this status code.
201 Created — request resulted in creating a new resource. It is usually sent in response to POST and PUT requests.
202 Accepted — request is accepted but not yet processed. For example, the operation takes a long time to finish and is handled by some other application that runs once a day.
204 No Content — the body is empty, but headers may be useful. You may see it in response to a HEAD request. It can also be used with idempotent and unsafe requests to indicate that the state wasn’t altered.

3xx

These codes indicate that a client needs to take action to finish the request.

301 Moved Permanently — The URL of a resource was changed. The new one is in the response.
302 Found — similar to the previous, except the change is temporary. For example, search engines won’t change the old URL to a new one in the results.
304 Not Modified — resource contents are cached and haven’t changed since the last request. It is safe to stop the current request there and use the cached data.

4xx

Status codes from this group indicate that there was a mistake in the request:

400 Bad Request — the request is malformed, and the server cannot process it. There may be many reasons for that, but the error is often somewhere in the request body.
401 Unauthorized — authentication is required to proceed
403 Forbidden — user is authenticated but unauthorized to proceed to the resource.
404 Not Found — the famous “page not found” code. Indicates that resource doesn’t exist. Some websites may return 404 instead of 403 to hide the existence of some pages from unauthorized users.
405 Method Not Allowed — resource does not support the request method. Remember the OPTIONS method? You’ll see this status code if the requested method is not among those listed in the Allow header.
429 Too Many Requests — there are too many requests from the client, and they are being rate-limited. Used against DDoS and brute-force attacks.

5xx

These status codes indicate that the server has encountered errors during request processing:

500 Internal Server Error — unhandled error on the server side. Usually happen due to bugs in the application.
503 Service Unavailable — server is not ready to process the request. The response may have a hint on when it will become available.

If you have any questions, feel free to ask them in the
comments.

9 shell tools for productivity

Igor Perepilitsyn — Sun, 04 Sep 2022 18:38:22 +0000

So recently, I spent a few days trying to recreate my usual environment on a new laptop. Many of the tools I use became so natural that I forgot that they weren't there in the first place. After a couple of days of intense memory exercises, I finally managed to get everything together. At first, this post was a memo to save my future self from more brain push-ups. But then I thought that it could be helpful to someone else. Behold, community, this is the list of the tools I use daily:

1. Oh My Zsh

I use zsh shell because it's ~~the default option on macOS~~ really powerful. But there is a way to take it to the next level — oh-my-zsh. It comes with a bunch of handy commands and a powerful way to configure and extend your shell with plugins. There are also themes if you want a different shell flavor.

2. powerlevel10k theme

I really like how it looks, but I also like how fast it is. P10K can magically skip a lag between you hitting an enter key and a prompt appearing on the screen. It also works during shell startup, e.g., if it suffers from plugin bloat. Speaking of which, powerlevel10k natively integrates with many of them. For example, it can detect your current Python virtual environment or AWS user and display them in the shell prompt.

3. bgnotify

This small tool fires notifications when a command finishes executing in a terminal. No more switching between windows every 5 seconds to check on rust compilation!

4. direnv

A must-have if you use environment variables a lot. You export them in a .envrc file and place it in the root directory of your project. Then those variables are automatically loaded upon entering that directory. Direnv also provides handy functions for .envrc file, and it can load envs from .env!

5. fzf

Command-line fuzzy finder. I use it mainly with reverse search by command history. Outputs a list of results and has an extended syntax for searching.

6. pyenv

Pyenv lets you install different versions of python on your system simultaneously, without breaking a thing. You can switch between any of them in one command. You can also bind a specific version to a directory, which will activate every time you enter it. There is an extension that adds support for virtual environments.

7. lazygit

Lazygit is a terminal UI for git. It's less wordy than the CLI and not as bloated as a GUI tool or an IDE plugin. I like it because it has most of the information I need on a single screen and provides many shortcuts for frequently used commands. Here is a video that shows more interesting things Lazygit can do.

8. z script

A simple script that gathers statistics on your filesystem tree "movements" and uses it to teleport you to the most "frecent" directories. For example, if you frequently navigate to your project directory cd ~/work/company/myapp, you can use z myapp to jump right in.

9. wuzz

A useful tool for exploring HTTP requests. Unfortunately, it's rarely updated, but it does the basic stuff.

And that wraps things up. If you know other amazing tools that help you be more productive, share them in the comments!

DEV Community: Igor Perepilitsyn