Re: [syndication] Accounting for aggregated views

["Bill Kearney" <wkearney@syndic8.com>]

A bit long and decidely rant-ish but I'm open to debate...

> That definitely would be a train wreck. However, that is not what I
> was asking about.

Ah, sorry them, it's tangental to the concept.

> > CDF attempted to address this issue by providing a way for a consuming
program
> > to 'upload' a log report.  That degree of functionality was never
implemented in
> > RSS (any of the versions).
>
> Thanks for the pointer -- I wold've never thought to look at CDF.

Yes, there's well-blazed territory there.  Contrary to what some folks might
have you think, RSS did not evolve in a vacuum, let alone from any one source.

> > Combine this with a general sense from the audience that they do not want to
> > tell you.  The readers don't want to be tallied up.
>
> A very valid point. I would just add that most of the time it comes
> down to some sort of trade-off between privacy and desire to access
> content.

Yep, the good part about RSS is that this time the user is in control of the
process, not the publishers.  I bear no animus toward the time-honored
profession of commerical publishing.  But some of the drivel I've heard from
their ilk lately makes it clear they've lost focus (not in this thread but they
do read the lists).  The push for user-tracking is not something that should be
tolerated without resistance.

> > To most folks that decry the lack of page view counting, I encourage them to
> > make use of feed-specific URLs and teaser text in the feeds.  Put text into
the
> > feed that invites the user to come back to the website to learn more.  And
use a
> > feed-specific URL so you'll know the landing page was arrived at from
something
> > that consumed the feed XML (instead of regular HTML browsers).  You could,
> > conceivably, generate an entirely unique RSS feed for each 'entity' that
> > retrieved it.  This would probably be more of a drain on the server's CPU
that
> > is practical.  Not to mention the usual hassles of determining just whether
or
> > not the remote requester is 'unique' or not.
>
> Yes, this is a possible solution. But don't you think it solves a
> generic problem with a specific implementation? In other words, there
> is no standard way of implementing this behavior. Is there?

There doesn't need to be a standard behavior and possibly having one would be a
bad thing.  Think about it.  If there was a standard way to force such things on
users then there would surely come a way to break it.  By using plain old URLs
in site independent ways you're eliminating at least one part of the risk.
Sure, the "word could get out" about a feed stooping so low as to use such
things (I'm kidding, of course) but it wouldn't make it "easy" for aggregators
to take wholesale steps at skirting around the attempts (aka newsmonster).

> > There are, from time to time, various people that make noises about RSS not
> > being capable of surviving or growing unless it succumbs to the use of
cookies
> > or other view-tracking techniques.  It seems these folks are WRONG.  We've
seen
> > better than 20,000 new feeds come online just the first half of this year
alone.
> > Sure, a lot of it's fluff or personal weblog stuff but there are a lot of
> > heavy-hitters like BBC and Reuters playing along.
>
> Umm.. I am not saying that RSS is not going to survive -- I don't have
> enough credentials to say something like that, but both you and I know
> that fast rate of the format adoption does not necessarily equate to
> this format being perfect.

Plenty of folks have tried 'perfecting' formats.  Suffice to say there are more
bones from the dead attempts at perfection than the those continuing to rumble
on "imperfectly".  Just look at how much the 0.91 format continues to thrive.

> > To the folks that say they hate using teaser feeds then my response is then
> > you'll have to live without having more detailed web statistics.  This is
also
> > how I respond to the folks that refuse to play along with RSS unless they
can
> > get the stats.  Provide lead-in copy in the free feeds and require that the
> > users login to use full content feeds.  All decent aggregator programs
support
> > user authentication.  This won't give you per-item view stats but it will
let
> > you know if a user is downloading the feed.
>
> But why? Why do they have to live with that? If there is a real need
> for something, why not attempt to address it?

To serve whose ends?  The content providers?  Given the tremendous amount of
content that's emerging, it's utterly ridiculous to think the consumers will
tolerate the intrusions on their privacy when they can just get the content
elsewhere.  Serve US, the consumers, or you're toast.  Well, maybe it's not
quite that extreme but it's certainly not going to slide toward better serving
the needs of the publishers.

> > And if the site has enough resources to setup and run user logins then it
> > probably also has enough to support altering the feeds themselves to have
> > per-reader unique URLs in the items.   I'm sure some folks would find this a
> > horrendous intrustion on their so-called privacy.  Yeah, just like the
police
> > from Casablanca "shocked, shocked!".  Then it's a simple matter to handle
> > cross-referencing the inbound use of those URLs back against the user
profiles.
> > This still doesn't help if you want page-views on the feeds.  To get that
fine
> > grained a sampling you need to play games with embedded web bugs (invisible
> > gifs) and such.  These are likewise considered quite the pox on humanity but
> > alas some folks insist on trying to use them.  A great many newsreader
programs
> > don't display the content using HTML so these bugs won't work (and are just
a
> > waste of embedded HTML nonsense).
>
> I am glad you brought these up. All of this hacky nonsense is exactly
> what motivated me to make the initial post on this topic.

Right, but recall that if you build a "proper way" to do it then you're also
giving a clear sign on what to avoid.  Don't think for an instant the tool
makers won't just skirt right around it.  Some tools already offer a filter that
immediately throws away in-feed advertising.  This without the user ever seeing
it.  Don't think they won't do the same with usage reporting.

> I am going to go out on a limb here, but from my observations, the
> following is true most of the time:
>
> The absense of an accessible, standardized way to address a specific
> need results in having this need addressed using workarounds, the
> consequences of which are usually less than desirable and produce
> exponentially more grief than developing the standard in the first
> place.

I don't hear many complaints from the people that standardized RSS.  Nor from
the developers decrying a way to force their users to push usage data back to
publisers.  And certainly not from the consumers demanding to have their reading
habits dissected.

> The Internet has given us junk email, pop-ups, and numerous lesser
> evils of the Web exactly because of that.

Ha!  And that's part of what's great about RSS and why so many users have gotten
wise to using it.  You can't use those things with RSS because the people that
created RSS and it's tools knew better than to let those evils take root in it.

> So, what I am trying to do is start a discussion on the "right" way to
> account for aggregated views.

And I'm saying that without a whole lot more explanation as to why the users
should put up with the burden it's just not going to happen.  There's an awful
lot of side-band communication that would have to take place.  Right now it's
one request.  Pull the feed and read it to your hearts content.  To start
pushing back usage data would require a whole other level of effort on the
reader side of the process.  The reader (this being a portal or a user client)
would then have to start keeping track of usage.  Then that data would have to
get pushed back up to the consuming site.  The consuming site would also,
presumably, have to start keeping track of it.  And to what end?  Making the
advertisers happy?  Pardon me if I'm less than motivated.   And don't think
you'll be able to get away with just pushing up a simple integer.  I'd bet a lot
more would be demanded.

Pushing up a usage integer could, conceivably,  be done with an HTTP header.
Most client software (portals or readers) could add this without a lot of
hassle. Servers expecting to receive it would have to know to adverise their
interest in it and to use it if presented.  The HTTP spec allows for this sort
of thing but some folks want to take the lazy way out and cobble it into the
user-agent string.  Talk about hacks.  That and this sort of thing wouldn't lend
itself to sending more detailed usage info.  That'd be a hack layered onto a
hack.

Honestly, CDF attempted to do this pretty well but recious little ever rose to
take advantage of it.  For any number of reasons I suppose.  Firstly due to the
immaturity of the market of course.  But also because of no user interest.  The
developers didn't see any user interests being served and wisely decided not to
bother implementing something that didn't enhance the user experience.  Heck, if
someone wanted to I'd fully support some sort of optional element that a feed
could contain the indicated a way to send info back to it.  It could be
something as simple as sending back up a text file formatted in the W3C common
log format.  The trick would lie in how to do this effectively.

<someNS:logging rdf:resource="mailto:usagepatters@example.com"/>
<someNS:logging rdf:resource="http://www.example.com/usage/";>

The push up the common log.  Either as an appropriately MIME-typed attachment or
as POSTed data.

Consider what a pain in the ass it would be for an RSS reader on a cell phone to
push this data back to a site.  The SMS bill would be astronomical.  Try
explaining THAT to an outraged cell phone customer.  Likewise, consider the user
on a detached laptop or one that's behind draconian firewall configurations.
What should those be using?  Why should their reader program have to jump
through extra hurdles to collect and deliver data back to the source?  What's in
it for them?

Now, someone that wants to start publishing feeds and offering cash back for
usage is more than welcome to try.  I'd daresay there's potential in it.  But I
can also imagine just how thorougly hackable it would be.  Heh, the script
kiddies would have a field day gaming the numbers.

Also imagine what a pain in the ass it would be for the receiving sites to deal
with potentially bogus data.  Whats to stop the script kiddies from cobbling up
bots that mercilessly hammer the receiver APIs with bogus data?  Hell, referral
spam is all the rage I can only imagine usage spam would have some game-able
angles as well.

Right now RSS is succeeding because of many things.  It's easy to implement and
easy to retrieve.  Adding usage reporting is neither.  I recognize that it's
something a lot of people think they want.  But their wish for it doesn't stand
up to the hassles it would put on the consumers.

There's too much good content available elsewhere such that there's no traction
in threatening the users by denying them data.  They're already too smart to buy
that line.

Thus I keep falling back to the idea that if you want to trap the consumers into
reader-viewing-usage hell it can be done with the browser.  If the user wants to
play that game let them use the tools already sutied for it.  Don't come trying
to wreck a fine thing like RSS without having something more in it for the
audience.

-Bill Kearney
Syndic8.com