Trendly & Google Analytics Intelligence: Sometimes 10% > 200%
Like many Google Analytics users, we were eagerly waiting to try out Google's new Analytics Intelligence feature. Now that we've had the chance, we wanted to share some thoughts on what it does and doesn't do, and how it compares to our own analytics product, Trendly. It turns out that they're quite different beasts, and we think most analytics users will want to use both.
The short version is this: Analytics Intelligence looks for outliers: individual days, weeks or months that buck the overall trends, whether or not they have much absolute impact on your traffic. Trendly looks for changes in the trends themselves, especially those that, over time, have a large cumulative impact on your overall numbers.
Here's the long version. Web analytics data is noisy: every day, for example, the number of visitors to your website will be slightly different. Usually, any given day's visitors won't deviate very much from the mean, that is, the average number of visitors per day. If you see 950 visitors on Monday, 1150 on Tuesday, and 900 on Wednesday, we could say that you have a mean of 1000 visitors per day, and that the deviations from this mean were 50, 150, and 100 on Monday, Tuesday, and Wednesday respectively. If the deviations are small, they probably don't mean anything - your website didn't suddenly get worse between Tuesday and Wednesday, it's just random chance that they were different. Over time, you can get a sense for what the usual range is, and not worry about fluctuations within it.
What about when the deviation is larger? Say that on Thursday you get 3000 visitors, a deviation of 2000. This is well outside the standard range. Google Analytics Intelligence is designed to notice and alert you to these outliers. That's a useful service; maybe you've been featured on some popular blog, or everyone's tweeting about some new promotion you announced. You'd want to know, and Analytics Intelligence will tell you.
Trendly might notice that outlier too, but it's really designed to look for more subtle changes. Say that the following week, you start getting more days with 1150 and 1050 and not as many with 900 and 950. You might say that the deviations are getting larger, but at some point, it will become clear that what's actually happened is that the mean has shifted: instead of getting 1000 visitors per day on average, now you're getting 1100. That's the kind of event Trendly will notice and alert you about. It may not seem like a big deal on any given day - which is why Analytics Intelligence wouldn't even bother to tell you about it - but over a few weeks, that 10% increase in average traffic will have a much bigger impact than a single day 200% spike to 3000. It's also much less likely that you would have spotted the change in average traffic yourself, making Trendly's alert (in our opinion) much more valuable.
 It's also worth asking what happens when you have two changes to different segments: for example, say traffic from a small country such as New Zealand goes from 2 to 6, while traffic from the US goes from 500 to 550. From what we've seen, Analytics Intelligence would be more likely to give you an alert about New Zealand (a 200% increase) than the US (a 10% increase, but reflecting many more new visitors). All of these alerts about smaller segments can be distracting, and it would be surprising if Intelligence didn't soon add an option to filter them out (this is not something the alert sensitivity slider currently seems to take into account).
Trendly, on the other hand, judges an event's importance by its absolute impact on your total traffic. This doesn't mean smaller segments are ignored; they're intelligently clustered together, and changes to them are only reported in aggregate, where they can meaningfully move the needle for your business.