robots.txt As a beginner in react , we all must have come across this robots.txt file when we create our react application. Most of us never bothered to know what it was and delete it , thinking it as a unnecessary file in our react app . Yaa I've also done it 😂 😂.
Now lets know what it is .
*Robots.txt file is a file located on your server that tells web crawlers which pages they can and cannot access. *
The pages we blocked , when the web crawler crawls in it , we get soft 404 as error.
The two action we can do is
- Allow: /
- Disallow: /
we can allow and disallow the pages to access.
Adding robots.txt file in your application , positively increases your app's SEO Performance . So Know the stuff and increase your apps SEO Performance right away with robots.txt . ☮️
Top comments (1)
Informative 👍