[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [syndication] Compromise time....
On Sun, Nov 16, 2003 at 01:37:39PM -0500, Ian Graham wrote:
>
> So, back from holidays. I spend some time digging around trying to
> find a good summary of historical proposals for robots.txt, but
> didn't find anything.
[snip]
> Basic conclusion: robots.txt not worth changing, all other issues are
> really about providing metadata about site contents, and this doesn't
> belong in robots.txt (but no one figured out where it should go...)
Thanks for the digging. It's really quite a mess, isn't it? :-(
Jeremy
--
Jeremy D. Zawodny | Perl, Web, MySQL, Linux Magazine, Yahoo!
<Jeremy@Zawodny.com> | http://jeremy.zawodny.com/