Creating a CMS with Firestore, GCS, and Sinatra

A Content Management System (CMS) is a software application that allows users to create, manage, and modify content on a website without needing to know HTML, CSS, or other programming languages.

It provides a user-friendly interface where you can create web pages, add content, upload images, and perform lots of other functions without directly manipulating code. Popular examples of CMS's include WordPress, Drupal, and Joomla. These platforms make it possible for individuals and businesses to run and maintain their websites with minimal technical knowledge.

In my quest to build a super-app, I decided to implement a CMS that would make adding new services a breeze.

Let's talk about how data would flow in a CMS.

You could store HTML in Firestore directly, but Firestore has a maximum limit of 1 MiB (1,048,576 bytes) for a single document, including the document key, metadata, and any indexed fields. This is generally more than enough for most HTML content, but if your HTML content is very large, you may run into this limit.

Storing HTML in a database can have some performance implications. The amount of data that needs to be transmitted and processed can slow down your app, especially if the HTML content is very large.

Also, keep in mind that databases are not really designed to serve static content like HTML. As an alternative, we will store HTML content in a cloud storage service like Google Cloud Storage. It's designed to serve static content and can handle large amounts of data more efficiently.

To create a minimally working CMS, we will render a form that lets us upload a local file. The file will then be given a url and stored in GCS:

<form action="/upload" method="post" enctype="multipart/form-data">
     <input type="file" name="file">
     <input type="submit" value="Upload">
</form>

The action of this form post's to a method called "/upload" found in our app.rb:

post("/upload") do
end

First, we will fetch the "file" from params (we named the file input "file"). We will then give it a unique filename. Suppose we are storing HTML for a header in GCS:

post("/upload") do
  # Get the file from the request
  file = params.fetch("file")

  # Create a unique filename
  filename = "header.erb"
end

Now, let's create GCS and bucket storage objects:

post("/upload") do
  ...
  storage = Google::Cloud::Storage.new
  bucket = storage.bucket "cisco-vlahakis.appspot.com"
end

In production, caching is a great way to save you from being charged queries. Both the browser and GCS can cache data. While this is a great feature in production, it is annoying in development. You will probably be editing files you store in GCS quite often as you develop, which means we want to disable caching for now. If we didn't, we may or may not see changes to our code updated until the specified time has passed. Luckily, we can specify Cache-Control metadata to prevent caching:

post("/upload") do
  ...
  # Specify the Cache-Control metadata
  cache_control = "no-cache, no-store, must-revalidate"

Now we are ready to send the HTML to GCS. We send a ref to the file, the filename, our cache rules, and finally, we will set permissions to "publicRead". For now, we would like to set objects created in GCS to Public so that we have permission to use them in development. You can edit these permissions later when ready:

post("/upload") do
  ...
  # Upload the file to GCS
  gcs_file = bucket.create_file(file[:tempfile], filename, cache_control: cache_control, acl: "publicRead")
end

Now, we will store the url pointing to the GCS HTML file in Firestore. Let's store the header HTML url in a collection called components:

post("/upload") do
  # Get the public URL of the file
  url = gcs_file.public_url

  # Add a new document to Firestore with the URL
  doc_ref = $firestore.doc("components/#{filename}")
  doc_ref.set({ url: url })
end

Finally, we will redirect to "/" when done:

post("/upload") do
   ...
   redirect "/"
end

Suppose we also have our pages stored in GCS and Firestore. We can now match all routes and have one function to parse and display the components associated with the page:

get("/*") do
end

First, we will get the route from our browser and fetch the corresponding Page in Firestore with that route:

...
# Get the route from the URL
  route = request.path_info[1..] # Remove the leading slash

# Fetch the corresponding Page record from Firestore
  pages_col = $firestore.col("pages")
  matching_pages = pages_col.where("route", "=", route).get
  page = matching_pages.first

If the page is not found, we will redirect the user to a 404 page. Otherwise, we will get the components to display for the Page. This is an array of urls to GCS components:

...
# If the page is not found, redirect to a 404 page
  if page.nil?
    redirect "/404"
  else
    # Get the GCS URLs from the Page document
    gcs_urls = page.data.fetch(:components)
  end

We will now create a helper function called fetch_html_from_gcs that returns the complete, joined HTML of the fetched components. Afterwards, we will finally render the HTML:

...
else
    # Get the GCS URLs from the Page document
    gcs_urls = page.data.fetch(:components)

    # Fetch the HTML content from GCS
    html_content = fetch_html_from_gcs(gcs_urls)

    # Render the HTML content
    # The :page template should be set up to display the raw HTML content
    erb :page, :locals => { :html_content => html_content }
  end

Let us build the fetch_html_from_gcs function now. It will take an array of GCS component urls:

def fetch_html_from_gcs(components)
    Array(components).map do |component|
end

Let's fetch the url and continue to next iteration if there is no url. Let's then check that the url starts with http:// or https://:

...
# Fetch the HTML template from GCS
url = component.fetch(:url)

# Continue to next iteration if url is not present
next unless url

# Check if the url starts with http:// or https://
unless url.start_with?('http://', 'https://')
  next
end

We will use the http module to get the HTML template response and parse it to a string:

...
response = HTTP.get(url)
html_template = response.to_s

Then, we will fetch the properties Hash from our component object taken from the Page object in Firestore. The properties Hash defines the custom properties that the Page passes to this component (such as "Home" for :title):

# Get the properties for this component
properties = component.fetch(:properties, {})

If you need to send over details about the signed in user and session, this would be the place to do so:

properties["current_user"] = current_user()

# Add session data to the properties
properties["session"] = session

Suppose a component has nested components, like a Sidebar using a Link component. We can store a Hash as a property that stores a url to the nested component HTML in GCS. We can then recursively call fetch_html_from_gcs on the nested components:

# Look for placeholders in the HTML template and replace them
      # with the HTML content of the nested components
      properties.each do |key, value|
        if value.is_a?(Hash) && value.has_key?(:url)
          # This is a nested component
          nested_html = fetch_html_from_gcs([value])
          html_template.gsub!("<%= #{key} %>", nested_html)
        end
      end

Next, we will render the HTML template with the properties:

html_content = 
 ERB.new(html_template).result_with_hash(properties)

Finally, we join all the HTML strings with line breaks together together:

html_content
    end.compact.join("\n")

Whew! We have successfully stored components (and Pages) in GCS, their urls in the associated Firestore records, and brought them to render in our web app!

We used a file upload form to send files to GCS, but an extra step would be to create a code editor in the web app itself that submits the code via submit button (and could also let you see the output).