Kos Traffic Numbers Inflated by 60%
by Patrick Ruffini :: October 3rd, 2007 1:20 am 
Yesterday, I had the good “fortune” of being frontpaged on Daily Kos. The post sat atop the site for two hours. According to Google Analytics, the link produced 1,164 visitors yesterday.
For the traffic behemoth Kos is portrayed as, that seems low, especially since he linked to two posts of mine. I went and looked at other notable traffic spikes this year, and this one isn’t really even close to some other blockbuster links. For instance,
- A Marc Ambinder link (linked to by Andrew Sullivan) to my 2008 Wire videowall produced 2,205 visits on June 5th.
- A link from the Real Clear Politics homepage on August 17th produced 1,889 visits.
Now, Kos was semi-complimentary in his link. He wasn’t direct in telling his readers to rape and pillage my servers. And I know well that the wording of a link and how it’s presented can dramatically impact the clickthrough rate. If there’s a site I need to link to but I don’t want to reward with lots of traffic, I hyperlink certain words and not others to minimize clickthrough (in addition to sticking in a rel=”nofollow” directive).
But this got me thinking, especially since a sub-theme in the linked post suggested Kos was not all he was cracked up to be in terms of his audience size.
Earlier today, it was pointed out to me that Kos’s average visit length was all of 2 seconds, suggesting either a coordinated attempt to bomb the site with fake traffic or an extremely low level of engagement on the part of his readers. This seemed as remarkable, given the breathless huffing and puffing about his community platform being a game changer in terms of traffic and audience reach.
So I started to do some digging around his SiteMeter stats and those of other big bloggers.
My source was right. The SiteMeter numbers are indeed fishy. But the reason is far from nefarious: a design flaw in how SiteMeter counts visits that systemically overcounts unique visitors on extremely high traffic blogs like Daily Kos… by a lot.
First of all, I looked at the Detail view showing the last 100 visitors. Overwhelmingly it showed visitors hitting the site only once, with a visit time of zero (you need to hit a second page for it to register any time spent). Contrasted with my traffic, with an average visit length of three minutes, this seemed highly improbable.
Then it hit me: SiteMeter only accounts for the last 100 visitors individually. On a site like Daily Kos, the 100th most recent visitor could have been 15 seconds ago. If you are the 101st most recent visitor and you click on a new page, you are counted as a new unique visitor in SiteMeter’s all important count. On a normal site, this wouldn’t matter, since it’s highly unlikely you’ll stick around long enough to have 100 others show up after you. On a site with hundreds of thousands of page views a day, it’s extremely likely you will.
Other corroborating evidence of this includes the Daily Durations chart and the Page View / Visitor chart by hour. During slow traffic periods (early mornings and weekends) the ratio of page views to uniques returns to more normal levels (up to about 5 to 3). Also there is an odd spike in daily durations up to 3 seconds from 2 that only happens on weekends and is very consistent — a spike that you don’t see on medium traffic blogs. What you see there is a telltale sign of the longer time horizon required for double counting (and triple and so on).
Currently, Kos’s average daily “visit” count stands at about 454,000 and his daily page views at 538,000, a low 1.18 ratio. This number has fed the huge mythology surrounding Kos that he has “half a million” readers a day (I used the number 600,000 as recently as 48 hours ago), while top conservatives are stuck in the muck at about 100,000 to 150,000. These are the numbers used to populate N.Z. Bear’s frequently referenced traffic ranking.
We now know that the only thing we can trust about the SiteMeter numbers are the page views. And from that we can arrive at a more realistic number of daily unique visitors for Daily Kos and other leading blogs.
How so? The best guide we probably have are other netroots blogs like MyDD (stats) and OpenLeft (stats) built on open community platforms. They have low enough traffic that SiteMeter’s inflationary effect is minimal at best. Using Scoop (what Kos uses) and SoapBlox respectively, both have a ratio of about 1.9 page views for every visit (itself a less stringent measure than “unique visitor”). On Red State, where there is likely a little bit of this effect, it’s about 1.8 to 1. On a Wordpress-style blog without diaries, the ratio averages 1.5 page views per visit.
Extrapolating from Kos’ page view number, a more accurate “visitor” number for Kos would be in the neighborhood of 283,000. If Kos is a stickier site than MyDD or OpenLeft (a fair assumption), that number is probably lower. That works out to an artificial inflation in the accepted Daily Kos traffic number of about 60%.
By the way, this is not some theoretical exercise. This is the number SiteMeter would show if they didn’t have this quirk in counting traffic to high-velocity sites. Sites with as more traffic than Kos show a similar skew. That includes gadget and lifestyle blogs like Gizmodo with 2 million page views and a 1 second visit length and Lifehacker at 3 seconds.
To be fair, some conservative blogs probably fall in this boat, though the skew is probably no more than 20-30% in the most extreme case simply because we don’t fall in quite the same traffic league as Kos. Most author-led blogs average about 1.5 page views per visit, and that would peg Michelle Malkin’s actual visit number at about 130,000 (down from 140K). In effect, that means Kos is twice the size of Michelle. That’s not something you’d necessarily want to hang your hat on, but it is dramatically different than the 4 or 5 to 1 number that is in reporters’ minds (and was in mine until tonight). So we still have a hill to climb, but it doesn’t look quite as big as it did 24 hours ago.
Why does this matter? Because if someone uncovered a 60% ratings inflation in Rush Limbaugh’s or Bill O’Reilly’s numbers, we’d never hear the end of it.
Tagged: blogosphere 
conservatism kos netroots
Comments (63) 
Trackbacks (20) 
del.icio.us 
digg it 
subscribe
Both comments and pings are currently closed.
As inflated as his ego!
It’s the same phenomenon you see in MSM. During the election cycle, you’ll see poll after poll showing Democrats with leads, yet on election day they lose. MSM and their allies in the DNC always show surprise (and usually claim fraud). It’s because they’ve engaged in this false puffery and then begin to believe their own hype.
It probably explains why Kos-backed candidates are 0 for 20 in elections, too!
1 second average time visiting?
1 second?
doesn’t that sound a little odd?
1 second? what the heck can you do in 1 second?
open the site to inflate the traffic number?
seriously, 1 second?
I’m not a web-guy, but come on, no one visits a site for 1 second. And if that is the average, that means that some number of people are visiting for less than 1 second? please.
talk about something wrong with the numbers.
Are advertising rates set by the sitemeter count?
It seems there could be a problem, there, if rates are charged due to faulty numbers.
FYI: This weekend my report on the “Troops Out Now” Protest here in DC was linked to by many sites and I got nearly 30,000 hits in total over the last few days as a result. As an example, the visitors I got from Michelle Malkin’s site alone produced 7,852 visitors and continues to climb. I got far more from LGF, but then again I had multiple links from that site.
My current average duration is ~ 30 seconds, but over the weekend when the links were more active my average duration seemed to be around 2 minutes. My average page views are currently 1.2.
If a lot of visits to Kos’ website last only 1 second, that might indicate that lots of people follow links from other sites, then react “Oops, I don’t want to read that lefty nutjob Kos”, and quickly hit the “Back” button on their browser. Meaning that Kos hasn’t exactly won the “hearts and minds” (or votes) of these one-second visitors.
I’ve been wondering about this for a long time, but didn’t know how to check it out.
well, you can also check the numbers at these three sites that have a ‘panel’
www.quantcast.com
www.compete.com
www.alexa.com
quantcast estimates that kos is at 250K unique visitors per month.
while these numbers are flawed, they are much much different than sitemeters 454K visitors per day.
Patrick–
If you wanted to verify this, compare Malkin’s new format to her old page. I pick her because her site is probably one of the most frequented Righty blogs giving that “100 reset” we’d expect to see on Kos. Also because her format changed from an “all on one page” format to a “main story-click through” format for the additional content.
Now, if you look at her statistics, you’ll see that her page views dropped before June, but then jumped by about 600,000 in June when the new format was introduced. If you look at the data, April was a high-traffic month, but the preceding few months and May were not as high as April, but now her site is generating regular April numbers, which supports your hypothesis that the many of the visitors are “non-unique”. As much as I love Michelle, her stats are inflated if what you say is true.
Now go to Kos who gets 350,000 more hits a day than Malkin, and we’re talking a doubling or tripling of the “unique visitor” error.
It’s a good catch. Nice work. The only criticism I would have is to compare this theory to a couple of premium SiteMeter accounts (which store up to 4,000 visitors) and see if the same pattern emerges. Kos, that cheap bastard, only uses SiteMeter basic. And an old version of it as well.
As for my blog, it’s all about the Chris Benoit autopsy. *sighs*
It probably explains why Kos-backed candidates are 0 for 20 in elections, too!
This stat wasn’t true before 2006 (Kos put a lot of effort into the Chandler [KY-6] and Herseth [SD-AL] special elections), but it’s strikingly untrue now. The official Kos-backed candidates on this 2006 page include Reps. Hodes [NH-2], Walz [MN-1], Murphy [PA-8], Sestak [PA-7], and McNerney [CA-11], as well as Sens. Webb [VA] and Tester [MT]. That’s 7, not zero.
The inflated traffic report is interesting, though.
I have been suspicious of that for a long time. The number of comments is another tipoff. The number of comments on LGF often exceeds Kos and Charles Johnson has a very accurate count - typically in the 125 -150,000 range. Kos numbers have always seemed out of proportion given the rabid nature of the Kos following - not that LGFers are, ahem, uncommitted. I think true Kos numbers are in the vicinity of LGF.
I was linked on Kos’ front page one time and received about 1,200 hits over 48 hours.
“Extrapolating from Kos’ page view number, a more accurate “visitor” number for Kos would be in the neighborhood of 283,000.”
Looking at the Alexa rankings, Kos has about 2.5 times the traffic of Malkin. Using Malkins sitemeter numbers as a baseline, that would give Kos around 360,000 visitors a day. Of course, you’re saying that Malkins numbers are skewed upwards also.
The effect you’re describing does seem to exist, exaggerating traffic more as real traffic increases.
Apparently this kind of stuff makes you guys feel better? Ok, whatever it takes.
Patrick: excellent research on a fairly opaque issue…and a good rejoinder to one of the primary tenets of the Left blogosphere: that they’re an unstoppable orange wave of resurgent liberalism and that Kos can reshape the political debate in a fundamental way.
We obviously have a hell of a lot to do on our side to mobilize and activate GOP voters, but in terms of web traffic and message we shouldn’t let Kos and his brethren intimidate us. It’s hard work, not impossible.
The Daily Kos adserver, blogads.com, show an estimatd 3.9 million impressions for a one week ad. That works out to an average of 557k/day. I assume they are counting number of ads served and don’t rely on third party traffic stats, but if what you are saying is true, blogads has some explaining to do to their advertisers.
I do not understand.
when I go to a site like yours or instapundit or etc I scroll down the page and may click to another site but I don’t usually go to a different page of yours. Does that mean that sitemeter counts me as a zero time vistior, or as many 0 time visitors once for each time I am there after a hundred others?
when does my time start to count if I only look at one page. What counts as a page?
Another method to check stats would be to find the number of hits an off site linked photo had the day it was posted on Kos. The Iraq map currently on the front page is from Photo Bucket, but I can’t find a way to get the hit stats.
BTW - While it might point you in the right direction, using link click throughs as a basis for estimating total visits may not be very accurate. All it tells you is the number of people who had the interest in or took the time to hit it. That being said, I took a look at stats for linked YouTube pieces on Kos and it seems to back up your assertions.
And this page tells you how Site Meter calculates time on site. Again, it’s not what you think it is.
I would’ve thought that if you waned to find out how Site Meter worked, you would go to the Site Meter site and look it up. It’s what I did. A little clicking around and Googling isn’t hard. Sure, it’s harder than just making stuff up off the top of your head, but still not that much harder.
The facts, whatever they are, Oliver, are what I care about. I have a non-partisan attachment to reality and data that my political commitments (full disclosure- libertarian, so everybody feel free to commence hating my politics) will never overrule. Kos is influential and important and useful to an extent. No one would bother to monitor traffic to my blog, because it is essentially zero. The interest alone shows the influence, but a difference of +/- 60% is pretty significant.
But the data are interesting. Politics aside, the details of how such numbers are generated is important, and the variance in the numbers arrived at by various methods is interesting.
Politics not aside, traffic=influence=calculation on the part of political figures to decide which outlets to monitor and how to weight them. It makes sense to disseminate your message through the most efficient vehicles possible, and the details matter when picking the mix you need. From a completely reptilian, partisan point of view, progressives should be just as glad to get an accurate read of webstats as anyone who gets ideological jollies seeing a minor notch-down of Kos.
If anyone spends time pimping numbers like this, then decry them as somehow unimportant when they are found to be inaccurate, this is also a useful data point.
Oliver Willis has commented?
Speaking of grossly inflated…
I think that putting various sites into Alexa gives you comparative traffic sizes. Alexa’s flaws even out if you use it in a comparative fashion, rather than comparing Alexa’s numbers to another system’s numbers.
I think that Patrick is underestimating things because he thinks that DailyKos is the main page, like a blog on the Atlantic, rather it is a ton of individual subpages with the diaries, that’s where most of the action takes place.
I dropped Site Meter about eight months ago. They miscounted my visitors to the tune of 8,000 plus per day and this went on for at least four weeks. In addition it is possible to “back door” some sites through their logo.
Jose, before you go calling people stupid, perhaps first you should read Patrick’s article (and read it slowly, so you can understand it) then go read those pages you linked to.
Unfortunately for you, the sitemeter pages aren’t saying what YOU think they are saying. Sitemeter “tracks” the last 100, if, as Patrick said, traffic volume is high enough, by the time the person in the #1 slot clicks to another page or outclicks the site, they very well will be ouside of the 100 “count” window, and therefore will be “tracked” as another unique visitor.
The only one who looked stupid in this exchange Jose, is you.
It should also be noted that Sitemeter “tracks” items that are found via image searches and many robots as visitors. Someone searching for images, is generally grabbing the image and not “visiting” the site (reading the material there), you’ll see a number of “image grabs” on Kos’ sitemeter as well. Robots, well, they’re just indexing the site. Given the nature of the numerous “google-bombs” originating from Kos’ kids and linked to in their comments, alot of that “traffic” is indeed inflated.
The point of this well researched article is that Sitemeter is incapable of reporting time-spent properly AT ALL for a high-traffic site like Kos.
When you read a web page, the web server and/or an analytics service assign you a session number. The time spent metric is determined by looking at the time stamp from your final page request and subtracting the time stamp from your first page request. To save resources, Sitemeter only keeps track of the most recent 100 sessions. Visits that only include one page are typically not counted in the time-spent metric. So, if 100 people came to the site after you did that day (which is almost always the case), as far as Sitemeter is concerned, your next click is your first one.
Kos gets so much traffic that Sitemeter’s methodology cannot properly count time-spent, period.
The one-second duration has nothing to do with reality. You can’t make assumptions like “people click and then leave,” or that somebody’s sending Kos bad traffic.
Also, “ad impressions” are not the same thing as page views or any other stat. Pages can have multiple ad impressions.
This isn’t any kind of “notch-down” on Kos. He clearly quotes better numbers to his advertisers, and he almost certainly makes internal strategy decisions based on his Google Analytics numbers.
Take a look at Alexa - shows less than 4,000 per day visit DailyKos at
http://www.alexa.com/data/details/traffic_details?url=http%3A%2F%2Fdailykos.com
I think the real problem is Kos doesn’t use serifs…
I’d go further and say that no analytics can accurately say amount of time spent on site for anyone. The assumptions you make when this number is estimated far outweigh the actual data you know (time between clicks). Especially true on blogs where most of the recent content is posted in full on the front page.
I think it’s an interesting problem and I’m surprised that Sitemeter hasn’t noticed/fixed it already. And that it isn’t a well known problem already given the plethora (google analytics, feedburner pro, etc..) of other reporting methods to sanity check these numbers.
And I wouldn’t put much stock in the Alexa numbers… 
Just to be clear, Alexa’s numbers are worthless. Saying that they “even out if you use it in a comparative fashion” means that they use a completely valid, random sample, which is patently untrue. They’ve got a huge sample bias. For example:
SEOChat.com vs WebMD.com, Alexa-style: http://tinyurl.com/2×5ku8
Patrick, I noticed this on SiteMeter’s website:
http://support.sitemeter.com/index.php?_m=knowledgebase&_a=viewarticle&kbarticleid=8
Two Site Meter Accounts are available:
Site Meter - Basic(free) - The last 100 visitors are tracked.
Site Meter - Premium($6.95/month) - The last 4000 visitors are tracked.
If Kos has been paying the extra $6.95/month, then his stats should be pretty accurate.
So I think Site Meter is aware of this issue, and has a solution for premium sites. Are you sure Kos isn’t paying the extra cash? Perhaps he isn’t, and perhaps he isn’t doing it on purpose because he likes the high numbers. But perhaps he doesn’t know, or perhaps he is paying and his numbers really are that high. I certainly don’t know, but I think you should be sure before you make any accusations.
h/t to Jose. Though that commentor was clearly ignorant and partisan, at least he got me looking down the right path.
Since my average visit is 2:22 and Kos’s is :02 I guess I have actual visitors and he use this:
http://scriptmonster.org/CGI-and-Perl/Exchanges/alexa-ranking-improver-send_c.htm
or something like it.
I looked into it some more, and I do think Kos has the basic Site Meter package that only logs the last 100 visitors. When you look at his Site Meter logs, you only see the last 100 visitors, and the longest “visit” for any people on the page is always less than the view time between visitor #1 and visitor #100 (which implies they are no longer tracked after they drop off the list). So I think Patrick has the right idea, and I think Kos should reach into his pocket, pull out $6.95, and get his numbers right.
However, it is fair for Kos to claim ignorance up until now. We should give him the benefit of the doubt. But there is an easy, cheap way for him to fix the problem. And if he doesn’t fix it then I think we can fairly claim that he is knowingly inflating his numbers. Give him a few weeks and see if he corrects the problem. Instapundit, amongst others, has linked to this article, so there is no way this issue will slip under Kos’s radar.
One other thing - Kos’s main page is huge. It is quite possible that the majority of his readers find all they need on his front page and never click through any links even though they spend a long time reading. You might find that his numbers aren’t nearly as inflated as 60% once he fixes the problem. Personally, I put it at 33%, giving him a true readership of around 340,000-350,000. If Kos openly agreed to fix this problem and pay for Site Meter Premium, this might be a fun betting pool!
I’ve been linked by all of the following, with the corresponding observed daily traffic effect:
Fark: 30,000 - 40,000
Boing Boing: 15,000 - 20,000
Instapundit: 8,000 - 12,000
NRO: 6,000 - 7,000
WSJ Best of the Web: 5,000
LGF: 4,000 - 8,000
Front page at DKos: 2,000
Just sayin’.
This discussion is starting to move into some interesting territory — that “unique visitors” and “time spent” are deeply flawed metrics that can differ wildly based on the methodology used. While making the unique visitors/time spent window bigger will help, uniques and time spent (*especially time spent*) are still “educated guesses we all agree on.”
Viva la Page View!
Mr. Ruffini,
You are well known as a right-leaning hack…Why would you expect the core readership of Daily Kos to come to your site? Why would they want to? Unlike the lemming-like actions of the right, we don’t just follow a link if it happens to appear on a progressive blog…
Hard to wrap your mind around, I know….
I made the comment that follows on Meryl Yourish’s post. Yourish moderates comments before allowing them to appear on the site, so I’m cross-posting the comment here rather than risk its loss.
October 3rd, 2007 at 10:11 pm
The question is one as to whether there is a defect in Sitemeter’s tracking software. Meryl Yourish makes Sitemeter’s documentation authoritative as to how Sitemeter’s software really works; this seems such an obvious error that I disdain further comment on it. Yourish also relies on experience at Lucent as a basis for a claim as to how visitor tracking usually works. But whether Yourish’s experience at Lucent is relevant depends partly on whether Sitemeter’s software has the defect Patrick Ruffini thinks he has detected. Whether Sitemeter’s tracking works the way tracking supposedly usually works also depends on whether Sitemeter’s software has that defect. Moreover, the way software usually works is this: Software has defects. Meryl Yourish doesn’t appear to have settled anything.
Wow Hank, you completely ignored the technical issues and want straight to partisan name calling. You go girl!
Kralizec - I don’t think it is a defect at all. Site Meter would like to make some money. If your website has so much traffic that you have more than 100 visitors online at a time, they figure you can afford to shell out $6.95 a month and track up to 4000 visitors instead of only 100. Site Meter tracks statistics for free quite well if you have a small to mid traffic site. For the big boys, Site Meter requires some payment in order to give accurate statistics. Frankly, I think that is quite fair.
What I would like to see are some big name bloggers reading this (and I know you are!) to start paying the pocket change required for Site Meter’s premium service, and advertise your statistics as accurate. That way, you can point out that your left-wing competition is hiding behind inflated numbers. Fun for the whole Blogosphere!
Matthew,
The question is why would they knowingly pay for something that’s going to publicly show them having less traffic than they have. Particularly when the smart ones know exactly what their uniques are through Google Analytics?
Well naturally, it is not in Kos’s best interest to pay a monthly fee to LOWER his traffic numbers (and possibly his ad revenue!!!). But if leading conservative blogs switched to using honest numbers, it might be possible to launch a public shame campaign to force Kos to be honest too. And who isn’t up for a little manufactured morale-rightousness from time to time?
If nothing else, a particularly evil Republican (not me - I am the “diet coke” of evil) might try to launch an email campaign to Blogads to let them know they are paying Kos too much!
Quite honestly I don’t know enough about the subject but the bottom line seems to be that Kos pulls about 2.5 to 6 times the traffic of the most visited GOP leaning sites. I’d be interested to hear his response since this is a highly technical matter and with due respect to Mr Ruffini he does have an agenda. Even if it the figures around 3 times is that good news?
I’m far from an expert in these matters, but isn’t it possible to design a program that would effectively send random hits to a website? Possibly hits that are in the tens of thousands? Would that explain the visits that last approximately one second?
Your article started me doing some research into the site meters I am using. I switched to Site Meter Pro this morning. We’ll see if that makes a difference. My bottom line is Blog Harbor’s stats are completely different from Site Meter. Their count for the past year is less than my entire month’s count on Blog Harbor. Something is very fishy here and I don’t think I’m that stupid.
Including Blog Harbor and Site Meter, I am using at least two additional hit counters. I don’t get the same numbers on any of them. I may be deluding myself but I’m sticking with my Blog Harbor meter, especially when the meter on TLB for “average daily visits” for the past month is so far off that all I can do is laugh.
Thanks for the article.
SJ Reidhead
the Pink Flamingo
http://thepinkflamingo.blogharbor.com/blog
There is a completely reasonable explanation for low “average visit length” and for why that number will plummet during traffic surges.
Average visit length is the time between a visitor’s first page view and a visitor’s last page view.
A visit is a series of page views by a unique IP address with less than 30 minutes in between each page view (by SiteMeter — this is not a universally standardized metric).
So what happens when you visit a page on a site and then leave without clicking any internal links? Your page view gets registered, and a new “visit” is initialized. But after 30 minutes you haven’t viewed any more pages. Your visit expires. Your visit length is calculated and it comes out to zero seconds. That’s right — any visit with only one page view has a visit length of zero seconds. You can’t take that number at face value. Of course the visit was longer than that. But without a second data point, there is no way of knowing. Thus, the average visit length is a metric which is always lower than it should be. Even for people who visit multiple pages, the length of their stay at their last page won’t be counted, so their visit length will be abbreviated.
What happens during floods of traffic is that people get funneled to a specific page, deep linked by other sites. Since these people are readers of another site, and not DailyKos, most of them will read the linked page, and then leave. All those zeroes severely depress the average visit length metric, even to the point of it nearing zero.
When visit length nears zero, your page views per visit ratio nears 1. That is, one page view, one visitor. Normally, most sites will have a page views per visit ratio of more than one (and note that it can never dip below 1). But when you get a flood of traffic to a deep linked page, the foreign viewers (readers of someone else, coming in to read just that link) will generally only read that one page, and then close their browser tab/window (or click the back button).
Patrick says he has an average visit length of three minutes. This tells me that he has a fairly loyal readership, or that his site encourages the clicking of internal links. And that’s great for him, but it doesn’t mean that people with lower average visit lengths are presenting bogus numbers or that SiteMeter is malfunctioning. It just means that they get a lot of deep linkage that drives a lot of “casual” traffic.
You cannot compare the page views per visit numbers on one site to another. The “60%” number that Patrick conjures up is a complete fabrication based on how sticky he thinks the Daily Kos is.
Furthermore, I see a lot of you saying that SiteMeter says “The last 100 visitors are tracked” for basic accounts. Look again. It says “visitor details are tracked.” All that it takes to track visits is an IP address and a time stamp. So just because they’re not showing details for more than the last 100 visitors, it doesn’t mean that they’re not tracking IP/timestamp for visit-counting purposes. Indeed, SiteMeter explicitly says that visits time out after 30 minutes. It mentions no other method for the expiration of visits. It is wrong to assume that SiteMeter’s tracking ability is limited by what limited data it shows you on the backend.
And before people start: I’m a libertarian. I do not read the Daily “screw ‘em” Kos. I’m just a techie (in fact, one of the lead developers of WordPress, the software that powers this site).
Jason says:
It should also be noted that Sitemeter “tracks” items that are found via image searches and many robots as visitors. Someone searching for images, is generally grabbing the image and not “visiting” the site (reading the material there), you’ll see a number of “image grabs” on Kos’ sitemeter as well. Robots, well, they’re just indexing the site. Given the nature of the numerous “google-bombs” originating from Kos’ kids and linked to in their comments, alot of that “traffic” is indeed inflated.
This is not true. SiteMeter uses a JavaScript snippet to record viewers. Images do not call JavaScript. Robots do not execute JavaScript. The only thing that will trigger a SiteMeter page view (and initiate/continue a visit) is a JavaScript-capable browser loading an HTML page. Thus, SiteMeter’s numbers are actually deflated, since people with JavaScript turned off don’t count, and people on mobile browsers without JavaScript support don’t count.
As for Michelle Malkin’s jump in page views, yes, that happened because of the site relaunch (which I coordinated). There was an overnight increase in page views due to the new format of the site. Only one full article is shown on the front page, so more page views are registered as people click through to the article pages.
let me propose a simple test that I think answers this - just open three copies of the “Detail” page listing the last 100 visitors and refresh the three pages about twenty seconds apart. (This works when Kos is getting 30,000 visitors per hour, or 100 every 12 seconds.)
*IF* Sitemeter tracks IP addresses separately and for half an hour (regardless of how long the list becomes), there ought to be *zero* duplicates on the three lists.
But if (the Ruffini Hypothesis) Sitemeter tracks IP address for the last half hour subject to a hidden constraint that it only tracks a maximum of 100 addresses, and *if* some of the visitors wait more than about twelve seconds before clicking a second page, then then the lists will contain multiple entries for the same IP.
I copied three updates of the Detail page into Excel and sorted - Duplicate IP addresses appeared in adjacent rows paired with their correct time stamps.
A lot of the sample had to be dropped because all I was given was the ISP, e.g., “Comcast” or “Ameritech”.
However, among a list of 118 ostensibly unique visitors I had twelve duplicates - 9 double hits and three triple hits.
That is hardly exhaustive, but I think it is conclusive - I had a hard time getting the page refreshes to overlap, so I expect some visitors slipped through the cracks, but clearly the phenomenon seems to be in play.
I would exhort someone with some tech savvy to duplicate my impromptu effort with screen captures and the like.
Bonus thought - if you know your own IP address you can go to Kos, wait twenty seconds, open a new page, then look for yourself on the multiple Detail pages.
Tom,
I think that the lists would contain multiple entries for the same IP whether or not Patrick is right. The differences you need to look for are the numbers in the Visit column on your double hits. Were the durations of the visits long, encompasing both hits? or was each hit 0 seconds, indicating that Site Meter tracked each one as a new Visit? Patrick’s hypothesis (and mine) is that if the second page view took place after the IP dropped off the top 100, then it would be a new visit with a visit length of 0 seconds. Thus inflating Kos’s numbers.
Hey, I came up with another evil trick. Do you think Site Meter would let me sponsor Daily Kos and pay for his upgrade to Site Meter Premium? I’ll have to cut back on 7 I-tune downloads a month, but as long as Jethro Tull doesn’t put out another album I should be OK.
I think that the lists would contain multiple entries for the same IP whether or not Patrick is right.
I don’t follow that.
Patrick’s hypothesis (and mine) is that if the second page view took place after the IP dropped off the top 100, then it would be a new visit with a visit length of 0 seconds.
That I do follow. I am going to just create a new batch; I did the first set at the public library and couldn’t really save anything.
Mark Jaquith,
I understand that the Google image search page doesn’t generate the hit, but when people click the image to grab it, it opens the page in a frame within a google window. Users don’t necessarily have to “visit” the page (although technically they do, since it’s opened in the frame) and they aren’t necessarily viewing the content of the page, they’re just grabbing the image, which Google makes convenient by allowing you to grab the image either in it’s thumb’d size or in it’s original size without haveing to scroll down into the page.
My point is that alot of sitemeter “traffic” is inflated by image searches and not site “visitors”. You may think that having your page open in a frame while googlers grab images from the top constitutes a “visit”, I call it more of a theft.
I understand that people would like to count these Google Image results as “visits” because it makes their numbers bigger, but it’s pretty intellectually dishonest to do so.
Just run a google image search and look at the results. Keep in mind that your goal is not to read the content of the page, but merely to grab the image. You should be able to easily see what I’m referring to.
Counting “visits” like these is just inflation in my view.
I should note that you need to run the google image search, click on the picture you want, and THEN look at the results. The java snippet gets loaded at that point.
Patrick,
A lot of the technical jargon is way over my head but this appears to have some legs.
Now, does Kos have any potential problems with his advertisers?
There is always a difference between Analytics and Sitemeter. Analytics is 100% javascript dependent. Sitemeter will default to image tracking if javascript is not enabled. There is also the issue of referrers. Considering that Norton’s is still one of the most popular IS suites out there, that is never accurate. Norton’s loves blocking referral headers from being sent.
In all honesty, the best traffic rankings come from the server access logs. Using Webalizer or AW-Stats is also the easiest way to analyze that data. The problem is that I don’t know of any site that makes those stats public.
Quantcast, the open source internet ratings service reports monthly uniques of 262,000 for Kos. This is in line with your estimates. I compared the NY Times SEC disclosure for traffic on their about.com site and it was close to qauntcast’s number. At the end of the day the sitemeter is meaningless. In short order only acredited services like CommScore and Nielsen and Quantcast will be used to set ad rates.
http://www.quantcast.com/dailykos.com
This reminds me of the misleading 10X monthly internet traffic growth touted in the 90s to zoom the internet and telecom stocks. I guess Kos is kind of like Enron.
whammer says:
“I have been suspicious of that for a long time. The number of comments is another tipoff. The number of comments on LGF often exceeds Kos and Charles Johnson has a very accurate count - typically in the 125 -150,000 range. “
One reason may be that if you are even slightly critical over a small number of posts on Kos, your posting privileges are revoked. They don’t notify you or anything, you just can’t post.
All part of Kos’s effort to create an “echo chamber” exclusively for the loony left.
Bob C
This answers for me what has been a nagging question for some time…. When visiting KOS or other lefty sites, the posts are clearly left. When visiting LGF or Powerline, they are clearly right. When viewing various stories from “non-partisan” web sites, the reply posts have always seemed to be overwhelmingly from the right. Now I understand that folks on the right tend to get more of their news from the net than regular folks, so that may account for some of it, but I think this explains the rest…..
You think those traffic numbers are inflated? You should see how much I inflate the traffic cost numbers for my website.
I claim 30,000 per month for a text only site that only has unpaid moderators!
interesting to see this. I’m a liberal and am appalled by the diaries posted there that contain false information. I don’t believe it has much credibility, and is a liberal blog in name only.
Oliver Willis: You (an obvious libtard) speak of “feelings” the essay here speaks of FACTS, something with which a libtard like yourself has enormous trouble coming to terms.
Ruffini on Inflated Kos Numbers…
Who’s to blame? Patrick Ruffini blames SiteMeter: Currently, Kos’s average daily “visit” count stands at about 454,000 and his daily page views at 538,000, a low 1.18 ratio. This number has fed the huge mythology surrounding Kos that he has……
[…] Patrick Ruffini talks on KOS’s inflated traffic numbers! Gateway Pundit has the latest on Burma AJ Strata: US Has al-Qaeda Membership List From ME and EU GM Roper: Invasion of the Truthers! The Nose on Your Face: Back To School Islamic Rage Boy Lunch Box […]
[…] Is Duckie inflating his numbers? Filed under: Just a total douche, Liberals Are Corrupt — firstfriday @ 11:44 am Noted douchebag, solutions avoider and left-wing blogger Markos Moulitsas, aka Duckie (see the video), is being accused of lying about the number of hits his DailyKos website gets. The evidence is pretty strong that, like in the majority of his posts, he’s not telling the truth. […]
[…] Even if he’s right, dKos is still doing as much as us and MM put together. Is he right? I’ve heard other people complain, as Ruffini does, that a front-page link at dKos makes for a surprisingly small spike but that may be less suggestive of hits being overcounted than readers pouring into individual diaries instead of the front page. The crux of his argument is that SiteMeter only “remembers” 100 visitors at any one time, which means that if you click over to dKos and then wait for a few seconds while 100 new people stream in, the next click you make will count as a totally separate visit. So assuming that dKos receives 100 new visitors every 12 seconds, you could conceivably account for five visits yourself in the span of one minute if you space out your clicks appropriately. To wit: Then it hit me: SiteMeter only accounts for the last 100 visitors individually. On a site like Daily Kos, the 100th most recent visitor could have been 15 seconds ago. If you are the 101st most recent visitor and you click on a new page, you are counted as a new unique visitor in SiteMeter’s all important count. On a normal site, this wouldn’t matter, since it’s highly unlikely you’ll stick around long enough to have 100 others show up after you. On a site with hundreds of thousands of page views a day, it’s extremely likely you will. […]
The DKos visit count non-issue: Ruffini’s wrong…
There is no controversy regarding the Daily Kos visit count. Patrick Ruffini is misunderstanding the difference between what SiteMeter shows you on a summary page, and what is counted as a visit or a hit. SiteMeter is not inflating DKos’ site sta…
[…] Patrick Ruffini :: Kos Traffic Numbers Inflated by 60% […]
[…] Patrick Ruffini :: Kos Traffic Numbers Inflated by 60% It’s not just Kos: Ruffini excellently notes a flaw in how SiteMeter counts uniques on heavily visited sites. (tags: dailykos sitemeter stats) Post a Comment […]
[…] Patrick Ruffini :: Kos Traffic Numbers Inflated by 60% […]
[…] I just finished conducting a test that proves my hypothesis of Sitemeter’s overcounting of visits to high traffic blogs. […]
[…] Tom Maguire at Just one Minute: Patrick Ruffini identified an intriguing puzzle that leads Sitemeter to inflate traffic statistics for high-traffic sites. […]
[…] The poll is also another indication that the strength of the radical progressive strain in the Democratic ranks is significantly overstated, particularly the ultra-liberal web sites. […]
[…] The poll is also another indication that the strength of the radical progressive strain in the Democratic ranks is significantly overstated, particularly the ultra-liberal web sites. […]
[…] Daily Kos numbers inflated by 60% […]
[…] Wednesday, Patrick Ruffini blogged that he thought he had discovered a problem with the way blog traffic on the DailyKOS was being tabulated by SiteMeter: SiteMeter only accounts for the last 100 visitors individually. On a site like Daily Kos, the 100th most recent visitor could have been 15 seconds ago. If you are the 101st most recent visitor and you click on a new page, you are counted as a new unique visitor in SiteMeter’s all important count. On a normal site, this wouldn’t matter, since it’s highly unlikely you’ll stick around long enough to have 100 others show up after you. On a site with hundreds of thousands of page views a day, it’s extremely likely you will. […]
This meter runs funny…
Patrick Ruffini has figured out that a limitation of SiteMeter has “inflated” the daily counts at DailyKos, which obviously isn’t Kos’ fault and doesn’t affect his King of the Hill status — though, as Ruffini says, the hill isn’t quite……
[…] Note to Dems… you’ve been duped. […]
[…] Now our measly couple thousand doesn’t seem so, well, insurmountable. […]
[…] read more | digg story […]
[…] […]
wrestling action figure…
I Googled for something completely different, but found your page…and have to say thanks. nice read….