As I was flipping through Twitter this morning, I caught a link to an infographic that a friend of mine shared. The giant thing can be viewed here.

I’ve been seeing a lot of these lately – reports where the “value” of a fan vs a non-fan get broken out. However, every single one of these reports falls directly into the problem of self-selection bias, and encourages completely false assumptions.

Take this recent report, that detailed how McDonald’s makes ~$160 more in sales from a Facebook fan than a non-fan. The comparison was actually $310.18 from fans vs $150.39 from non-fans. The section was prefaced with the following:

“In a recent study conducted by Syncapse, 20 brands were evaluated to see the economic impact of having fans on Facebook.”

That line implies causation with “the impact of having a fan”. It insinuates that if you were to get a McDonald’s customer to “Like” McDonalds on Facebook, they would start spending $160 more on crappy food.

That. Is. Moronic.

There are fanatics of every brand. If I can remember correctly, it’s claimed in Super Size Me that McDonalds refers to its super-regular customers as “heavy users”.

There’s something very simple happening here that makes all of these social media reports useless.

Heavy users are more likely to become a social media fan in the first place. Being a social media fan does not transform you into a heavy user.

Simply looking at what a fan spends vs a non-fan tells you nothing. Your sample group (fans) selected themselves, and already proclaimed to be aligned with your brand enough to “like” you on Facebook or follow on Twitter. Your control group (non-fans) like you enough to make purchases, but not enough to go out of their way to get company updates, deals, etc through a social network. No fucking kidding one of those groups is going to spend more than the other.

Now, it isn’t just social. I’m just picking on social because most of these reports/infographic are social-media based because everyone’s desperate to try and show some ROI. However, these reports being held up as “wins” is just as asinine as:

  • Conversion rates from branded term PPC ads.
  • Conversion rates from email marketing to your internal list.
  • Conversion rates from coupon affiliate sites.
  • View-through conversions from re-targeting ads.

In all of these situations, the consumers have already flagged themselves as buyers.Looking at their conversion rates etc vs other channels is a totally useless comparison. (Save, maybe, trying to figure out what your “max attainable” conversion rate could be.)

At some point, someone with a voice louder than mine is going to call out these circle-jerk reports that brag about the obvious.