DEV Community

64BitDev
64BitDev

Posted on • Originally published at linkedin.com

2

Robots Exclusion Protocol finally penned

#website #robot #webcrawler #html

After 25years out in field and available in 500M+ websites, Robots Exclusion Protocol finally drafted to added to #IETF- https://www.ietf.org/id/draft-rep-wg-topic-00.txt

Top comments (0)

profile
Coherence
 Promoted

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

Read next

kaja_uvais_a8691e947dd399 profile image

Top Free Hosting Services for Web Developers in 2025

Kaja Uvais -

tobidelly profile image

Interactive Quiz App

TD! -

tobidelly profile image

Rock, Paper, Scissors Game

TD! -

udanielnogueira profile image

Como Inserir Som em HTML

Daniel Nogueira -

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay