DEV Community


Discussion on: Growth of the Web in 2020

imkevdev profile image
Kevin Farrugia

Thank you, good read. A tiny comment that CrUX websites also need to be publicly crawlable in addition to publicly available, i.e a robots.txt with no-index will stop the origin being available in CrUX. (Just learnt of this recently)