Under the hood of dev.to (Part 1)
This article series will uncover the secrets of dev.to's source code, helping the world understand and improve this application.
The source code is available on github, and you get a cool badge for contributing!
Disclaimer: I don't know ruby, nor ruby on rails, so there might be parts of this post which are incorrect or lacking. Feel free to point these out and I'll do my best to correct them!
Introduction
Submitting an article is easy, right?
All you need to do is press the SAVE POST button, and there we go!
There is much more complexity to it, and in this post I'll uncover the magic happening behind the scenes!
Application overview
Dev.to uses Ruby On Rails for its back-end, and Preact on the front-end.
The back-end hosts a REST api, and the front-end uses those to access and publish data.
The front-end is a Single Page Application, but is also Server Side Rendered.
This means that if you access dev.to/new directly, the server will generate all of the HTML for you, ready for you browser to display it.
Then, whenever the bundled preact scripts are loaded, we gain the SPA functionality: When trying to access a new page, it will be fetched by JavaScript, and preact will update the page content with the received html.
Showing the new article view
Alright, so you want to write an article.
First, you head up to dev.to/new.
Ruby on rails check its route in /config/routes to find /new using the GET protocol.
This route tells it to load the articles controller, and the new method.
get "/new" => "articles#new"
get "/new/:template" => "articles#new"
get "/pod" => "podcast_episodes#index"
get "/readinglist" => "reading_list_items#index"
This controller can be found under /app/controllers/articles_controller.rb.
Before loading the new method, few permissions check will be executed.
Those are declared on top of the controller, and includes method such as ensured you are logged in and preventing banned users from creating articles.
class ArticlesController < ApplicationController
include ApplicationHelper
before_action :authenticate_user!, except: %i[feed new]
before_action :set_article, only: %i[edit update destroy]
before_action :raise_banned, only: %i[new create update]
before_action :set_cache_control_headers, only: %i[feed]
after_action :verify_authorized
// ...
Once those are done, the new method is called:
def new
@user = current_user
@tag = Tag.find_by_name(params[:template])
@article = if @tag&.submission_template.present? && @user
authorize Article
Article.new(body_markdown: @tag.submission_template_customized(@user.name),
processed_html: "")
else
skip_authorization
if params[:state] == "v2" || Rails.env.development?
Article.new
else
Article.new(
body_markdown: "---\ntitle: \npublished: false\ndescription: \ntags: \n---\n\n",
processed_html: "",
)
end
end
end
it is quite straightforward: It checks if you are using a template (Aka. using the path /new/:template), and loads either this template, or creates a generic Front Matter body.
The Article.new represents the New Article View, available under /app/views/articles/new.html.erb
<% title "New Article - DEV" %>
<% if user_signed_in? %>
<% if params[:state] == "v2" || Rails.env.development? %>
<%= javascript_pack_tag 'articleForm', defer: true %>
<%= render 'articles/v2_form' %>
<% else %>
<%= render 'articles/markdown_form' %>
<% end %>
<% else %>
<%= render "devise/registrations/registration_form" %>
<% end %>
This loads the correct view based on our conditions, typically articles/markdown_form
<%= form_for(@article, html: {id:"article_markdown_form"}) do |f| %>
<% if @article.errors.any? %>
<div id="error_explanation">
<h2><%= pluralize(@article.errors.count, "error") %> prohibited this article from being saved:</h2>
<ul>
<% @article.errors.full_messages.each do |message| %>
<li><%= message %></li>
<% end %>
</ul>
</div>
<% end %>
<!-- ... -->
This form renders the HTML you usually see when accessing dev.to/new, we're finally there!
The generated HTML is used as body in the /app/views/layouts/application.html.erb at some point in Ruby On Rails's magic.
Saving an article
Alright, you've written your awesome article about how good Ben Halpern's website is, and you now wish to publish it for everyone to see!
You've set the published value to true, and you press this big blue SAVE POST button. What happens then?
Your HTML was loaded, Preact loaded, and it listens to the click event for the SAVE button.
Front-end
We're now in the front-end code, under /app/javascript/article-form/articleForm.jsx.
The button itself is under elements/publishToggle.jsx, and our articleForm.jsx added an event listener for the click.
publishToggle.jsx:
<button onClick={onPublish}>
{published ? 'SAVE CHANGES' : 'PUBLISH' }
</button>
articleForm.jsx:
<PublishToggle
published={published}
onPublish={this.onPublish}
onSaveDraft={this.onSaveDraft}
onChange={linkState(this, 'published')}
// ...
/>
articleForm.jsx:
onPublish = e => {
e.preventDefault();
this.setState({submitting: true, published: true})
let state = this.state;
state['published'] = true;
submitArticle(state, this.handleArticleError);
};
The submitArticle function is imported from ./actions.
actions.js - submitArticle
export function submitArticle(payload, errorCb, failureCb) {
const method = payload.id ? 'PUT' : 'POST'
const url = payload.id ? '/api/articles/'+ payload.id : '/api/articles'
fetch(url, {
// ...
body: JSON.stringify({
article: payload,
})
})
.then(response => response.json())
.then(response => {
if (response.current_state_path) {
window.location.replace(response.current_state_path);
} else {
errorCb(response)
}
})
.catch(failureCb);
}
Therefore, once you click the SAVE ARTICLE button, the following happens:
- An article is created based on the current
statevariable - The article is sent to
/api/articles - Once the save is complete, we're redirect to its new URL.
We can now start digging into the back-end!
Back-end
We're now receiving an article from the front-end in the form of a JSON file, at the /api/articles route via a POST.
Routing
Once again, in the /config/routes.rb file, we need to search for our endpoint.
There is an api namespace which contains our articles resource.
A Ruby on Rails Resource maps few default CRUD verbs to their respective methods, so in our case the POST method will call the articles#create method.
routes.rb
namespace :api, defaults: { format: "json" } do
scope module: :v0,
constraints: ApiConstraints.new(version: 0, default: true) do
resources :articles, only: %i[index show create update] do
collection do
get "/onboarding", to: "articles#onboarding"
end
end
resources :comments
// ...
Controller
We now are in the /app/controllers/articles_controller, under the create method:
def create
authorize Article
@user = current_user
@article = ArticleCreationService.
new(@user, article_params, job_opportunity_params).
create!
redirect_after_creation
end
Service
This method calls the ArticleCreationService, which will create our article!
def create!
raise if RateLimitChecker.new(user).limit_by_situation("published_article_creation")
article = Article.new(article_params)
article.user_id = user.id
article.show_comments = true
if user.organization_id.present? && article_params[:publish_under_org].to_i == 1
article.organization_id = user.organization_id
end
create_job_opportunity(article)
if article.save
if article.published
Notification.send_all(article, "Published")
end
end
article.decorate
end
This services creates a new instance of the Article model, and saves it.
Model
With Ruby on Rails, our models are Active Records, and have a bit of magic attached to it.
While I won't dive into the database mapping part of the object, what I find interesting are the before methods, called when creating or saving an object.
before_validation :evaluate_markdown
before_validation :create_slug
before_create :create_password
before_save :set_all_dates
before_save :calculate_base_scores
before_save :set_caches
after_save :async_score_calc, if: :published
The before_validation methods will be called before ensuring the object is valid.
- evaluate_markdown will convert our markdown to HTML
- create_slug will create a most-likely unique slug for the URL
- create_password will make a unique preview password value
The remaining methods should be quite explicit by their names.
The model will also perform many validations on its properties.
validates :slug, presence: { if: :published? }, format: /\A[0-9a-z-]*\z/,
uniqueness: { scope: :user_id }
validates :title, presence: true,
length: { maximum: 128 }
validates :user_id, presence: true
validates :feed_source_url, uniqueness: { allow_blank: true }
validates :canonical_url,
url: { allow_blank: true, no_local: true, schemes: ["https", "http"] },
uniqueness: { allow_blank: true }
Conclusion
Phew, this article is now saved! That was a lot of work for a simple action.
As a quick recap, to view an article, we load the correct Controller, which loads a View and renders it to the page.
When trying to perform CRUD operations, we find the correct route based on our API Resource, which loads a Controller. This controller can interact with the data using Services, themselves using Models to interact with the database.
Now that the technical side is covered, I would like to get some feedback on this post.
I have few objectives with this serie:
- Help people navigate through big codebases and understand their architecture
- Lower the contribution entry-barrier for open-source projects such as this website.
This is why feedback is important.
Did it help you understand the source?
Perhaps there is something specific you would like to see?
Please tell me in a comment below, and I'll do my best to improve this serie!
Top comments (10)
This is a great post!
One element possibly glossed over in the current implementation is that we have a new version of the editor not yet the default in production, because it's a work in progress, but is the default in development.
The one you're describing is the new one. You can see this in prod by visiting dev.to/new?state=v2. It still has some features to be ironed out, but should be a slightly cleaner experience than the current one, though we plan to keep offering the current one in settings.
Just a detail, but you can see the code that decides this in the snippets.
It is so helpful to have the code audited and described in this way. I was just thinking about writing a similar post on some of the Rails internals.
Hi Ben, Just looked at the new editor you guys are working on. It's very clean and nice. Looking forward to it's general release.
i got a question , i'm still a beginner learning web dev using spring mvc , is it worth to learn ruby to understand how the code work , or do i have to stick with spring , sorry my english kinda bad
If you want to understand MVC, either frameworks both framework are fine.
If you want to understand dev.to's source, you can start reading it right now, and learn the parts of ruby which don't make sense.
I don't know ruby, but its syntax is explicit enough for anyone to read its source code in my opinion.
thank you that's really helpful answer
Really enjoyed this post! Would love to read one about authentication. When I went to play with it, it looked like I was going to need to set up a GH app for OAuth, which felt like a lot of work, so I instead edited the seeded posts to see if my feature worked. But that will only take me so far, so at some point I'll probably need to figure out how to get authenticated in dev mode. If you're looking to do more like this (I assume so, since this is titled "Part 1"), then that would be a great topic!
Yep, you need to get the GitHub keys to authenticate locally (or the keys for Twitter authentication)
See docs.dev.to/get-api-keys-dev-env/#... for how to
Great post. Can I borrow you to make documentation for my projects? With such nice pictures? :) Iām kidding. Well done.
Waaoo nice job Brother!
Looking into dev.to source code is in my todo list this post is great.