Login to download the latest version of Mint and your favorite Pepper, purchase additional licenses, or post in the Forum. Don't have an account? Create one!

In Partnership with Media Temple

Mint Forum

Inaccurate Groupings in Many Panes

There’s a problem that I’m experiencing on two of my five installations of Mint. These two installations are totally unrelated; different websites, servers, etc. Both are running 2.16 (v64). (The other three installations do not exhibit this problem.)

When grouping traffic by a time span (i.e., Referrers -> Repeat -> Past hour) the top item is always inaccurate. For instance, one of my blogs says that I’ve received 222 visits from kottke.org in the past four hours, my top referrer in that time frame. This is plausible. But when I change the time frame to eight hours, kottke.org is nowhere to be found, despite that the list includes referrers with substantially lower counts.

Ditto for Pages (i.e. Pages -> Most Popular -> 48 hours). That claims that there have been 2,903 visits to a year-old article on our site, or our number one page in that period. That’s pretty unlikely, though possible. But when I toggle to 24 hours, that article no longer makes the cut, despite that it ought to be #1 for that period.

My guess as to what’s going on here is a that Mint is using the traffic data incorrectly in a pretty basic way. Everything that doesn’t make the cut for the top 18 lists that are used throughout—that is, everything at #19 and below—is all being grouped together and counted collectively. Whatever happens to bubble up to the top of that list (presumably the referrer or page name or whatever is serving as the unique element) as a result of being arbitrarily first in MySQL’s storage order is made the bearer of that collective statistic.

So if a site has gotten a thousand referrals in the past 24 hours, evenly distributed among 100 referrers, Then that should be ten each. But on Mint’s Top 18 lists, the bottom 17 would each have ten, and the number one referrer would show 830 visits (1,000 - (17 * 10)).

I’ve checked out Mint’s tables in MySQL. They are without error and optimized.

Has anybody else experienced this problem? Is this perhaps a result of something that I’ve done wrong? Perhaps something a Pepper is causing?

Shaun Inman
Mint/Pepper Developer
Posted on Jan 07, '09 at 10:40 am

This problem sounds consistent with moving from a 32-bit server to a 64-bit server. Any chance you’ve done this recently? (I think Dreamhost is silently updating their servers.)

All the *_checksum columns (and the ip_long column) of any mint_-prefixed table in your database should be UNSIGNED on a 64-bit server. They would be SIGNED on a 32-bit server. You can use this script to change the problem columns to the correct sign and update their content’s checksums for the new server (just upload the unzipped script to your Mint directory and visit it in a browser).

Fascinating! I never would have guessed that to be the problem. Yes, one of these sites did just move to a 64-bit server. The other site is on DreamHost, and I suspect I’m one of those silent upgrades that you mention. I’ll convert those columns now. Thanks so much for your help, Shaun.

mitchoyoshitaka
Third-Party Pepper Developer
Posted on Jan 12, '09 at 03:55 am

Hi Shaun, I just noticed a similar phenomena after moving to mediatemple. I followed your instructions and things seemed fine, except in the referrers pane, where I’m still seeing anomalies (the #1 referrer in a certain time frame shows up with less hits in a wider time frame, for example. Any ideas?

Shaun Inman
Mint/Pepper Developer
Posted on Jan 12, '09 at 10:32 am

Hmm, was the script allowed to run to completion? Also, are you sure the referrer_checksum column was updated to the correct type? Not sure what else would cause this symptom.

mitchoyoshitaka
Third-Party Pepper Developer
Posted on Jan 12, '09 at 08:45 pm

Yup, referrer_checksum is unsigned, as I believe it should be, but to no avail…

mitchoyoshitaka
Third-Party Pepper Developer
Posted on Jan 12, '09 at 08:48 pm

Shaun, you use mediatemple… do you know if the servers I’m running on are completely 64 bit?

Shaun Inman
Mint/Pepper Developer
Posted on Jan 13, '09 at 11:17 am

You can check by hovering over Mint’s version number in the footer. Your server appears to be 32-bit so you appear to have moved from a 64-bit server to a 32-bit one.

mitchoyoshitaka
Third-Party Pepper Developer
Posted on Jan 14, '09 at 10:00 am

Thanks Shaun - flipping the switch back to signed and running the checksums script again fixed it all.

Shaun, I ran the checksum script a week or so ago and it worked like a charm however I’m experiencing irregularities again that are similar to before: Visits numbers do not equal Crushes numbers as well as odd pages receiving high numbers over 72 hours but not over 24.

Do I need to re-run the checksum script at regular intervals or is something else going on behind the scenes?

Shaun Inman
Mint/Pepper Developer
Posted on Jan 23, '09 at 08:48 am

Visits numbers will not equal Crushes numbers because they use two different means of tracking (Visits is aggregate based on time, Crushes on a completely separate session cookie).

Pages with higher numbers over the past 72 hours compared to the past 24 hours is normal—there’s been 48 more hours for people to visit them!

I’m sorry, I mixed up my words. The numbers in Pages are mixed up, but the other way around. For example, in the past 2 hours 4 people have visited Page A however that page doesn’t register in the past 4, 8, 24 hour pages. The Past 24 hours pages lists two pages with over 20 visits that don’t show up in at all in earlier time frames.

There are pages that show up with very large numbers over the 48 and 72 hour pages, but they don’t register at all on the high end of the 24 hour pages.

These are the irregularities that I’m seeing. I didn’t realize Visits and Crushes looked at different things, but the Pages numbers seem completely out of whack.

Shaun Inman
Mint/Pepper Developer
Posted on Jan 26, '09 at 09:19 am

Did you just run the checksum script? Did you also make the prerequisite changes to the *_checksum columns in your Mint database tables too?

No, I had assumed the script would do that. I’m going through and making the changes now. Should I re-run the script or will I be good after the changes?

Shaun Inman
Mint/Pepper Developer
Posted on Jan 27, '09 at 08:02 am

You will need to rerun the script.

It seems as though you can check whether you’re on a 64-bit system (via footer hover), and you could inspect the MySQL column status. Is there a chance you could automate this process within Mint? (ala the Upgrade button).

I had assumed that Mint just stopped tracking referrers properly and accepted it as such. It was by chance I came upon this thread to find a solution.

I have a feeling a great number of your customers are on shared hosts, and hardware has been moving to 64-bit systems for some years now — I think moving from a 32-bit system to a 64-bit system is extremely common in shared hosting environments.

I just fixed my tables and ran the script. My Crushes disappeared but the place where I had noticed a problem — Seeds — did not disappear and are still not accurate. That is, most popular and most recent do not match. Is this normal?

I noticed that there was an error log in my Mint folder. All 3518 lines are the same:

[19-Feb-2009 12:34:41] PHP Notice: Undefined index: session in /home/user/public_html/mint/checksums.php on line 101

Is this Mint’s version of “All work and no play makes Jack a dull boy?”

Shaun Inman
Mint/Pepper Developer
Posted on Feb 20, '09 at 10:11 am

The script does not touch optional Pepper (like Bird Feeder) tables. I’m not sure how Most Popular and Most Recent Seeds would match. One is sorted reverse chronological, the other most hits descending. They don’t share displayed data.

Don’t worry about the error log, it only contains notices.

Well, what I mean is that the site that I most recently installed Mint on has had little enough traffic that I can figure out what should be appearing on Most Popular. If I see a page from the site has been hit three times recently on Most Recent, that page should be bumping off one of the one hit pages on Most Popular. But it isn’t.

I am also seeing hits on the Feeds pane that are not showing up on the Seeds pane. Each Feed hit should show something on Seeds, right? I’m guessing that I must not have something configured right here. I can work on that, if that is the case.

Thanks for the info on the error log. I wanted to make sure that the script ran correctly.

Okay, I’m guessing that the discrepancies between Most Popular and Most Recent have to do with the pre-table fix data just being wrong. Ignore the bit about Seeds, I didn’t understand that they only registered when a post in the feed was clicked on.

Ignore me. Everything is fine here. Thanks for the response about the error log and making such a great stats program.

Hey Shaun … I’ve read all the replies and am not sure it this applies to our setup. We have two servers: web [64-bit] and DB [32-bit] and we are experiencing similar issues. [We have mint installed on our web server, with the tables on our DB server, to be clear]. Would any of the above remedies help with our stats? If so, which ones?

Mint is reporting a 64-bit arch, BTW — so maybe internally it is treating integers differently?

Martijn
Minted
Posted on Jul 24, '09 at 09:22 am

Hey thanks for this. As I changed permalinks my results messed up. I ran the checksum script after messing around with the database and it fixed all of it nicely.

Incredibly glad I found this thread, my stats have now returned to normal after a long time being completely off. Thanks very much Shaun!

You must be logged in to reply. Login above or create an account