If you have a single-page application (SPA) and wanna do some SEO for it, Dynamic Rendering is a good option.
Unlike server-side rendering, you can put the web pages on search index without changing server-side code.
Google provides rendertron to achieve Dynamic Rendering.
And there's a deployment configuration of Google App Engine (GAE) in the repository, so you can deploy it easily to GAE.
Actually, I used rendertron with a production application for over six months. It worked well for SEO.
But the problem was the cost.
At that time, the server cost was $1,500 / month, for only rendertron.
Moving from GAE to Cloud Run
I've used Cloud Run for some times and consider moving rendertron to it.
In terms of instance-per-hour cost, Cloud Run is much cheaper than App Engine:
App Engine (F4 instance, 2.4GHz, memory 1GB): $0.26 /hour
Cloud Run (cpu v1, memory 1GB, tier 1): $0.095 / hour
Cloud Run needs a docker image, but Rendertron doesn't have dockerfile now.
I cloned rendertron repository and added Dockerfile
:
FROM node:12-slim
RUN apt-get update \
&& apt-get install -y wget gnupg \
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 libxtst6 \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /rendertron
COPY . /rendertron
RUN npm install
RUN npm run build
ENV NODE_ENV production
CMD ["npm", "run", "start"]
And it needs puppeteerArgs option in config.json
:
{
...
"puppeteerArgs": ["--no-sandbox", "--disable-dev-shm-usage"]
}
Also I set cacheMaxEntries to keep cache (config.json
):
"cacheConfig": {
"cacheMaxEntries": -1
},
I built docker images with Cloud Build and deployed it on Cloud Run successfully.
nginx configuration
In our case, I need to proxy requests from search engine crawlers to rendertron.
My nginx config is like:
server {
...
location / {
try_files $uri/index.html $uri @prerender;
}
location /static/ {
expires max;
add_header Cache-Control public;
}
location @prerender {
set $prerender 0;
if ($http_user_agent ~* "googlebot|bingbot|yandex|baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator|Linespider") {
set $prerender 1;
}
if ($args ~ "_escaped_fragment_") {
set $prerender 1;
}
if ($http_user_agent ~ "Prerender") {
set $prerender 0;
}
if ($uri ~* "\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot)") {
set $prerender 0;
}
# WORKAROUND: there's something wrong with accept-encoding: deflate. rewrite accept-enconding header here to avoid it.
proxy_hide_header Accept-Encoding;
proxy_set_header Accept-Encoding "gzip";
if ($prerender = 1) {
rewrite .* /render/https://$host$request_uri? break;
proxy_pass https://rendertron-xxxxxxx.a.run.app;
}
if ($prerender = 0) {
rewrite .* /index.html break;
}
}
}
Result
I did some tuning such as changing the number of cpu cores and tweaking concurrency.
The result was 10x cheaper than before: $150/month (before $1,500/month)
It's been running without any maintenance for over 3 months.
Top comments (2)
Now how do you do this for search bots only?
There's nginx as a reverse proxy in front. nginx sees if the user agent of a request is a search bot.