Re: [syndication] Re: Standard RSS location?

[Mark Nottingham <mnot@mnot.net>]

On Wed, Jul 26, 2000 at 04:37:44PM -0700, burtonator wrote:
> > Now I'm not complaining.  It's just that the reason I am not
> > publishing a list of all my RSS feeds (and I think I publish more of
> > them than any other site on the net - not that that means very much -
> > like I said it only took me an hour or two) is that I cannot currently
> > trust the bulk RSS aggregators to behave in a sensible manner with
> > such a list.
> 
> hm... maybe multicast RSS/OCS... humm. :)

Just some random thoughts; 

there are mechanisms in the HTTP that basically say, "Come back later", with
the Retry-After response header and a 503 status. Granted, this isn't widely
supported, but there's no reason why can't be, and it would be one mechanism
to shed this unnaturally bursty load. This depends somewhat on the nature of
the load; if it's because your server is swamped with connections, but
transaction processing is cheap, it doesn't buy you much.

Bursty traffic when objects expire is one reason HTTP/1.1 shys away from
Expires times, which happen at a particular time, and encourage delta times
in the form of Cache-Control: max-age. If you need to have expiry happen at
a particular time, it may be useful to introduce some jitter in the expiry,
so that all the requests aren't bunched together.

Of course, there are other solutions, such as hosting services
and content delivery networks, which could easily soak up that sort of load.
Disclaimer: I work for one of the latter.

Cheers,


-- 
Mark Nottingham
http://www.mnot.net/