Ok, so this story begins a couple of years back! I’ve always been interesting in building FinTech solutions and have been doing it for about 16 years (since 2008) (wow time flies). The reason for being interested in FinTech is really weird. I don’t have a financial background (engineer and university dropout). I think I just love the amount of data that is needed to build something.
So now to the platform. First an introduction to the idea:
A Financial News site that analyses the pressreleases that are published by publicly traded companies, write an article and publish it on the site. The site should also have commenting functionality and also AI portfolios that buy and sell (real prices but not real money) to inspire users.
The goal is to make savings and investments a part of every persons daily life!
If you want to have a look at the platform:
Investoract is for the UAE market (in English) and Börs360 is for the Swedish market (in Swedish).
The Tech Stack:
- Back-End
- Python
- Django
- Django Rest Framework
- Requests
- Scrapy
- Front-End
- Next.js (App Router)
- React
- Infrastructure
- DigitalOcean App Platform
- Redis Stack
- Postgres
The Django Models are quite simple. A couple of apps with a couple of models. Some models are connected to RedisSearch using signals and serialize the data and store it on Redis on each change. Redis indexes that data and make it searchable. Some Django Views that act as DJRF views but instead of fetching the data using the ORM, they instead fetch the data from RedisSearch. The speed this gives me is simple unprecedented (for me).
Ok so thats the tech part, what about the SEO?
So to make the platform SEO friendly, the first thing i needed to decide was if I should use App Router och Page Router in Next. As it was a hobby project, I though “why not just try App Router”…
The Developer Experience was awesome. It was really easy to build pages that fetch the data from the back-end and provide it as Server Side rendered pages to the client. So after a couple of weeks I had the first pages built using SSR.
So as a checklist I thought that I would need to fix the following before releases the site and asking Google to index it:
- Static Site Generation
- Make sure to add some of pages to Static Params (do not add all pages as it impacted the build time)
- Meta Data
- Make sure that we have title, description and stuff populated
- OG Tags
- Make sure we have OG tags and fallbacks for all pages
- Comply with all (most of) the recommendations the LighHouse provides
- Add Rich Result JSON
- Create a Sitemap
- Test the site using Google Page Speed
- Add Site to Google Search Console
The above checklist is probably not a complete checklist, but simply the tasks that I needed to complete before releasing the site
The big day!
So I released the site a couple of days after new year and Google picked up the site quite quickly. Analyses the data on Google Search Results for a couple of weeks I could quite quickly see that something was wrong. Some pages were being indexed but some were just simply crawled and not indexed.
Hickup
Going through the code (especially the raw html that was being sent to the client) I saw that Next.js was sending the data to the client as a JSON object and not prerendered HTML. Thats not good, as Google crawl the pages with no data and simply not index them as they do not contain any content that the user would be interested in.
So here is where it gets a bit messy. So Next.js allows us to add Client Side components quite easily. The only problem with this is that as soon as a client side component is rendered, the data for this component is (quite obviously) not rendered in the HTML tags. This was quite a bummer, as I had built an Infinite Load list on the start page that automatically loaded more articles as soon as the user reached the end of the list. I thought this was quite cool, but Google didn’t. This meant that almost all pages that contained a list of articles weren’t been indexed.
To solve this issue I went back to the drawing board and changed my Infinite Load List to a paginated list instead that used URL Query parameters instead. By doing that I could remove ‘use client’ from list components and make Next.js render these as pre-rendered HTML.
So just to clarify, the page was still considered SSR when I had use client but the server didn’t actually render that HTML but instead provides a JSON object (saved as a JS variable) that was available for the client to render the data.
Things I still need to fix asap
The Design
I’m really not happy with the UI yet. The first version was created using next-ui but I quite quickly moved over to just tailwind instead.
Login Experience
Yeah, if you try it out, you will probably see that the login experience needs a revamp.
Interested in some code examples from the project? Leave a comment and I'll create a new post with some code snippets :)
Top comments (2)
Did you see improvemets in your web analytics making this change?
Absolutely! Saw an increase in both Google Search Console and Analytics. Search Console started to index more pages, which meant that the site appeared on more search terms, which led to more visitors.