Implement HTTPS using Sockets and OpenSSL

The Internet Protocol Stack

Most developers are familiar with visualizing the Internet stack as a layered cake. Each layer aims to abstract away the complexity of the lower layer. There are two main conceptual models: OSI and TCP/IP. Despite using different concepts for layering, the models closely resemble each other.

Both models built around the encapsulation method. The main idea is logically separate functions in the network are abstracted from their underlying structures by inclusion or information hiding within higher level objects.

Each layer builds a protocol data unit (PDU) by adding a header containing control information to the data, referred as service data unit (SDU), from the layer above.

The result of encapsulation is that each lower layer provides a service to the layer or layers above it. At the same time, each layer communicates with its corresponding layer on the receiving end.

One of the most widely used protocols residing on the top level is HTTP. It is a protocol which allows the fetching of various kinds of resources. It is the foundation of any data exchange on the Internet.

Hypertext Transfer Protocol (HTTP)

HTTP is designed to be simple, and human readable (HTTP/2 is an exception). It uses client-server architecture. When a client wants to communicate with a server it first creates a TCP connection, and then send a message in HTTP format.

It can be something as simple as:

GET / HTTP/1.1
Connection: close

The usual response can look like that:

HTTP/1.1 200 OK
Content-Length: 12
Content-Type: text/html

Hello world!

Sockets

For the actual implementation we need some programming primitives to abstract transport layer, TCP protocol. In modern operating systems such an abstractions is provided by the sockets.

A socket is one endpoint of a two-way communication link between two programs running on the network. A socket is bound to a port number so that the TCP layer can identify the application that data is destined to be sent to.

The following examples will use Ruby, but all modern programming languages have some library providing access to the underlying operating system sockets.

Look at the the simple HTTP client making a request to http://google.com:

require "socket"

socket = TCPSocket.new "www.google.com", 80

socket.print "GET / HTTP/1.0\r\n"
socket.print "\r\n\r\n"

while line = socket.gets
  puts line
end

socket.close

Transport Layer Security (TLS)

The obvious issue with HTTP, it's not only human readable, but the data is transferred as a plaintext. It makes it impossible to use the protocol to exchange any sensitive data. To solve this issue, the modern Internet stack introduces TLS.

The primary goal of the TLS protocol is to provide privacy and data integrity between two communicating applications. The protocol is composed of two layers: the TLS Record Protocol and the TLS Handshake Protocol.

One advantage of TLS is that it is application protocol independent. Higher-level protocols can layer on top of the TLS protocol transparently. In the Internet stack is resides between TCP and HTTP.

OpenSSL

The actual implementation of the TLS is really complex. It is not a human readable protocol, and the data is transferred as bytes. The extensive use of cryptography makes it even harder to grok.

OpenSSL is a robust, commercial-grade, and full-featured toolkit for the TLS protocols. It is also a general use cryptographic library. OpenSSL is widely used by web servers, including Nginx and HAProxy, and operating systems, including many Linux distributions. OpenSSL derivatives, like BoringSSL and LibreSSL power even greater number of systems and applications, like OsX, Android, Chromium, and many more.

OpenSSL module for Ruby provides a wrapper for C API, and can be used to implement a simple HTTPS client making a request to https://google.com:

require "socket"
require "openssl"

socket = TCPSocket.new "www.google.com", 443
context = OpenSSL::SSL::SSLContext.new
secure_socket = OpenSSL::SSL::SSLSocket.new socket, context

secure_socket.connect
secure_socket.print "GET / HTTP/1.0\r\n"
secure_socket.print "\r\n\r\n"

while line = secure_socket.gets
  puts line
end

secure_socket.close