DEV Community

Cover image for Letting bots crawl your dynamically rendered page
Deepjyoti Barman
Deepjyoti Barman

Posted on

Letting bots crawl your dynamically rendered page

Meta tags are one of the most important aspects of a blog. In the recent years, crawlers such as from Google and Bing etc have advanced a lot, they are able to crawl pages that render content dynamically.

One important role meta plays is to give the page an identity to all the crawlers from social share sites like Twitter etc. These crawlers are not yet that advanced as to recognize dynamically rendered content.

How do they work?

Whenever you share a post on Twitter, Facebook etc, you'll see a nice little card showing you the title of the post, some description and the cover image (possibly). Ever wondered how they are able to show that? Yep, they use the meta tags from the posts URL.

Technically, how do they work?

All right, let's get a bit technical here, how does it actually get all that data?

Well, when you share an URL, the crawlers basically send a GET request to that URL and the server sends back the HTML content of the page. Once the page is received, the crawlers extract all the content from it and accordingly you see those data on your Twitter feed.

What is dynamic rendering of content

Okay, so what is this dynamic rendering I'm talking about? In order to understand how dynamic rendering works, you need to know how the static pages work.

You can read the full article on here

Top comments (0)