One evening I was thinking a topic for a small Elixir + Phoenix project and came up with an idea of implementing URL shorten-er with Elixir and Phoenix. The implementation is quite easy and quick, and overall the application is going to be very small.
The technology stack consists following technologies, Elixir, Phoenix framework and PostgreSQL. This stack is run in docker containers.
If you want to follow through this article I have public GitLab repository which contains the application.
Features
I started defining key user paths:
- User is able to create new link
- User is able to navigate to created link and redirected to original URL
- User is able to review statistics of the created link
These 3 paths are pretty trivial to implement, so lets dive in to it!
Implementation
I'm not going to go through the initial project set-up in this article. You can do so here!
Resource
The only resource we are going to need is Links. We can create one running the following command inside our docker container. If you didn't follow the earlier link for setting up the local docker + docker-compose environment ignore the docker-compose related commands.
$ docker-compose run web mix phx.gen.html Links Link links
This command will create Links and Link modules, Link-controller, migrations and test files.
A lot of files are generated when creating new resource, but take your time to go through the files. You can read more information of the phx.gen.html here.
Migration and schema
In previous step we generated resource for our application. The mix phx.gen.html accepts fields as parameter for the database table etc. You can put them there or define the fields manually in migration and schema file.
Migration
Our migration file looks pretty neat. The only notable change is that we are saying "Don't create primary key" for the links-table, since we are defining the id-field to be the primary key of the links table and setting its length to 8-digits.
defmodule Shorturl.Repo.Migrations.CreateLinks do
use Ecto.Migration
def change do
create table(:links, primary_key: false) do
add :id, :string, size: 8, primary_key: true
add :url, :text, null: false
add :visits, :integer, default: 0
timestamps()
end
end
end
Schema and changeset
Here we define the schema for our model/struct. Pretty basic stuff, but we need to define that we are not using the default primary key.
defmodule Shorturl.Links.Link do
use Ecto.Schema
import Ecto.Changeset
@primary_key {:id, :string, []}
schema "links" do
field :url, :string
field :visits, :integer
timestamps()
end
...
end
Below the schema block we have our changeset function with a custom validation for the user defined original URL.
defmodule Shorturl.Links.Link do
use Ecto.Schema
import Ecto.Changeset
...
@doc false
def changeset(link, attrs) do
link
|> cast(attrs, [:id, :url, :visits])
|> validate_required([:id, :url])
|> validate_url(:url)
end
def validate_url(changeset, field, options \\ %{}) do
validate_change(changeset, field, fn :url, url ->
uri = URI.parse(url)
if uri.scheme == nil do
[{field, options[:message] || "Please enter valid url!"}]
else
[]
end
end)
end
end
The validate_url/3 checks if the original URL is an URL with scheme (http, https). If not, error message is attached to the :url field of the changeset.
Routes
If you are not familiar with the Phoenix routing, I suggest you to read this.
When creating a new Phoenix project, there is already some routes defined in routes.ex. The LiveView routes for metrics dashboard are enabled only in dev and test environments, we can leave it there. The initial route for PageController can be removed, since we are only using one controller in this application, LinkController.
At this point we need following routes
- Path to show the form of the new shortened URL
- Path where to post the form
- Path which redirects the user to original URL
- Path where the stats of the shortened URL are shown
defmodule ShorturlWeb.Router do
use ShorturlWeb, :router
pipeline :browser do
plug :accepts, ["html"]
plug :fetch_session
plug :fetch_flash
plug :protect_from_forgery
plug :put_secure_browser_headers
end
scope "/", ShorturlWeb do
pipe_through :browser
get "/", LinkController, :new
post "/links", LinkController, :create
get "/:id", LinkController, :redirect_to
get "/:id/stats", LinkController, :show
end
...
end
Now we have defined our routes for our application, wonderful! Each atom after LinkController is the action where the request is passed to.
LinkController
New
This is the action which is responsible for rendering the front page of the application. The form helper in the view needs Link changeset to be available. If you look the corresponding template file (new.html.eex), you can see that we also pass @action variable for the form-template. This is used to tell the form helper where to send all the form data.
defmodule ShorturlWeb.LinkController do
use ShorturlWeb, :controller
alias Shorturl.Links
alias Shorturl.Links.Link
def new(conn, _params) do
changeset = Links.change_link(%Link{})
render(conn, "new.html", changeset: changeset)
end
...
end
Create
All the magic happens here what comes to the creation of the link. First we need to create random 8-digit identifier for the link, this is the earlier mentioned primary key for the database entry. We put the identifier to our parameter map and try to create the link. If the link is successfully created and persisted in database, we redirect the user to :show-action, otherwise we just render the front page (:new-action) with the Link changeset which holds the possible errors and display them in the UI.
defmodule ShorturlWeb.LinkController do
use ShorturlWeb, :controller
alias Shorturl.Links
alias Shorturl.Links.Link
...
def create(conn, %{"link" => link_params}) do
case create_link(link_params) do
{:ok, link} ->
conn
|> put_flash(:info, "Link created successfully.")
|> redirect(to: Routes.link_path(conn, :show, link))
{:error, changeset} ->
render(conn, "new.html", changeset: changeset)
end
end
...
defp create_link(link_params) do
key = random_string(8)
params = Map.put(link_params, "id", key)
try do
case Links.create_link(params) do
{:ok, link} ->
{:ok, link}
{:error, %Ecto.Changeset{} = changeset} ->
{:error, changeset}
end
rescue
Ecto.ConstraintError ->
create_link(params)
end
end
...
defp random_string(string_length) do
:crypto.strong_rand_bytes(string_length)
|> Base.url_encode64()
|> binary_part(0, string_length)
end
end
Edit:
I totally forgot to handle situation where the randomly generated ID is already taken by some other URL. I updated the above code to handle also this case with a nice recursive function! I abstracted the actual business logic of this case to it's own function, so the create controller function stays clean and readable. Thank you Roberto Amorim!
Show
This is purely just displaying the information of the shortened link. Original link, shortened URL and visits are shown in the view. In the controller we fetch the Link with the id from our path parameters and pass the link for the view. If the query results in Ecto.NoResultsError the user is redirected to front page with error flash message. Flash message documentation. We also pass domain to the view, this is used for prefixing the shortened URL.
defmodule ShorturlWeb.LinkController do
use ShorturlWeb, :controller
alias Shorturl.Links
alias Shorturl.Links.Link
...
def show(conn, %{"id" => id}) do
try do
link = Links.get_link!(id)
domain = System.get_env("APP_BASE_URL") || nil
render(conn, "show.html", link: link, domain: domain)
rescue
Ecto.NoResultsError ->
conn
|> put_flash(:error, "Invalid link")
|> redirect(to: Routes.link_path(conn, :new))
end
end
...
end
Redirect_to
This is the action which is responsible for redirecting the users to original URL when they click a shortened URL/link somewhere. The structure is pretty much the same as in :show, but we need to update the visits count for the associated link. The Task.start/1 is used for side-effects in Elixir applications and we don't care what the task returns. Also the function call in the task is run on its own process, so it doesn't block the execution of the following code.
defmodule ShorturlWeb.LinkController do
use ShorturlWeb, :controller
alias Shorturl.Links
alias Shorturl.Links.Link
...
def redirect_to(conn, %{"id" => id}) do
try do
link = Links.get_link!(id)
# Start task for side-effect
Task.start(fn -> update_visits_for_link(link) end)
redirect(conn, external: link.url)
rescue
Ecto.NoResultsError ->
conn
|> put_flash(:error, "Invalid link")
|> redirect(to: Routes.link_path(conn, :new))
end
end
...
defp update_visits_for_link(link) do
Links.update_link(link, %{visits: link.visits + 1})
end
...
end
Templates and styling
I'm not going through styling and templates in this article, but you can look for them in the GitLab repository which were linked earlier in this article.
Testing
There is small portion of tests done in the repository, go look it up! For the large part they are generated by the mix-task and modified to meet the functionalities of our application.
Are we done? No.
Lets say that the application gets huge amount of links daily and you persist all of them in your database. At some point your DB will blow up. We need to clean old URLs away which are not used any more, sounds fair?
To begin with the implementation, I took Quantum library and configured it according the documentation. This will be responsible for scheduling jobs and running them.
We want to delete all the links which are not visited during the last 7 days and we want to run this cleaning job daily.
config :shorturl, Shorturl.Scheduler,
jobs: [
{"@daily", {Shorturl.Clean, :delete_old_links, []}}
]
defmodule Shorturl.Clean do
alias Shorturl.Links
require Logger
def delete_old_links do
Logger.info("Running job for old links")
{count, _} = Links.delete_all_old()
Logger.info("Removed #{count} old links")
end
end
In links.ex
def delete_all_old() do
from(l in Link, where: l.updated_at < ago(7, "day")) |> Repo.delete_all()
end
Just like that, with a few lines of code the database is cleaned daily. The Ecto query API is just wonderful and we don't have to delete the links one by one when using delete_all/1.
Conclusion
This was a nice tiny project which was fun and quick to implement, no biggie. I tried to make this article beginner friendly as possible!
The next step is to deploy this application to AWS with some GitLab CI/CD and Terraform magic!
Top comments (4)
Well done! Just curious: how would you handle possible duplicate random string ids? In the code above, it would fail silently (that is, just display the form again after submit without any error message), would it not?
Thanks! And you are totally right! It would just fail and user will be greeted with an error message. If that happens, it is like winning in a lottery 10 times in a row. If you calculate how many permutations 8-char/digit string can have with upper- and lowercase letters, symbols, numbers etc. The number of total permutations would be enormous.
I think the current solution is viable option how to do things for this exact case. In the create function we could check if the custom id exists already in DB, but it will slow down the creation of the new shortened url.
Good point anyway! :)
I think checking is not necessary; you can just proceed with the insert, and, if any unique violation occurs, generate a new key and try inserting again, repeat until it works. That way you make sure it works while not incurring in any penalty on the "happy" path.
Yes! That would be even better! I’ll update the article at some point to handle it like this. 👍