[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: robots.txt and rss



> opinion:  RSS aggregators probably *should* respect robots.txt.

I agree. If someone were to post a BDG, like the one [1] Simon Fell 
posted for Etags, I would put together support for it in Radio 
(assuming it was easy, which I assume it is).

Dave

[1] 
http://www.pocketsoap.com/weblog/stories/2002/05/19/bdgToEtags.html