This post takes a lighthearted and self-deprecating look at analytics and metrics.
I will use myself as the first example (not the woman in the picture), because I recently caught myself in a surprising analytics story. I will also get more serious toward the end.
As many of you know, I took on a recent project to move my blog to self-hosted WordPress. One thing I learned in this move was how I had slowly become addicted to analytics:
- How do Page Views (on wordpress.com) change when I posted various kinds of content, or at different times of the day?
- How am I tracking day-to-day, week-to-week, month-to-month?
- Are my numbers going up?
So here’s what happened
When I moved looseneurons.com, I lost the built-in analytics, so I switched to good old Google Analytics thinking about how much more control I would have, how much more information I could gather, what great new questions I could answer…
Almost immediately I noticed my logs tracking Visitors as the key metric. The Page Views are still there, but the emphasis shifted.
I learned some neat things, like on my young and growing blog, visitors tend to view an average of 4 pages during each visit.
That’s cool… but my philosophy about what was measured and why it mattered hadn’t shifted yet. I mentioned to a friend that looking at a Visitor count that was 1/4 the size of my earlier Page Views was triggering my psychology more than I would have expected.
I was totally unprepared. And that was a great event to notice!
By the way, his response to me: “If you don’t like what GA is telling you, turn it off.”
Hmm. Perfect. Thanks for the help, Dave.
Turning situation into action
So I wondered:
- Am I the only one this happens to, or does it happen to my customers?
- Do metrics that are weak, missing, flawed or incomplete nevertheless build some emotional connection in people (besides me)?
- When introducing new metrics, how long does it take to build new thinking at more than a superficial level, where metrics take on meaning, relevance and value beyond simple numeric signals of better or worse?
Have you ever thought about other people’s need for a philosophical shift when introducing new metrics? Do you have examples of this at work?
How do you act in this kind of situation, as a customer who became self-aware or as an analyst? It seems more to me than just a training issue.
What specific actions do you take when metrics change, to ease their adoption and quick semantic value?
Getting to fundamentals
As I consider this, I notice that I DO pay attention to shifts in metrics when they involve incentive compensation (e.g., designing a commission plan for sales engineers).
I was just triggered in this case that the metric I am looking at (Page Views or Visitors) is about operational performance… which at a larger scale drives enterprise value and the ultimate incentive for everyone employed!
Now, I work in web sites and metrics all day, every day, but the blog numbers I was looking had taken on meaning that I confess was not completely rational.
When given “better” numbers, I continued to press them into my old mold of reasoning and measuring, even though I could rationally understand the reasons things appeared to change.
(Incidentally, I am not serious about turning off GA. I have way too many numbers I can fixate on now!)
An example in software
Many of you also know TDD is a software development practice I truly love. One of the greatest teams I have worked with built an automated test harness that exercised 100% of the code for an automated call center back in 2001.
That “code coverage” metric became so well-known by the people in and around that team… that many of them set the metric (lowered to an 80% threshold) as a “target” against which future projects would be measured.
However, we had never set out intending to produce “100% code coverage” when we wrote the software – it had been an ancillary outcome to our intention of writing an awesome product.
Over the ensuing years, I have seen the code coverage threshold blind development teams and management to poorly written software and fragile test harness… though they do exercise a lot of the code.
It has been a personal struggle of mine to shift management, team and developer philosophy since.
(If you are curious about some other lighthearted posts I have written, check out Post-it Notes, The IT Super-Genius, Corvettes and Google+, Flattened by a Panda and Serendipity and Social Media. They help me channel my Loose Neurons creatively, and like this one they also carry a note of importance.)