Filtering out the noisy robots – Google Analytics

Google Analytics is already pretty good at being more or less accurate and not including search robots – however it isn’t perfect, discovering this pollution and filtering it out typically isn’t as hard to do as people think it is – but most people never bother to check. There is a string created by @philpearce which is worth implememnting.

phils GA filter list and regex guide.xlsx

Which is an excellent start, it won’t do it all unfortunately – there are thousands out there and your network admin or someone else might have added a few to the list – not all accounts have this issue, but out of the 500 or so I checked, typically, the larger sites suffered, and suffered enough to impact the key metrics.

Typically I do the following –

• open the profile you want to look at,
• Click into Technology >> Network
• Use an advanced filter that looks at the ‘near’ bouncers (1.1 pages / visit),
• typically the top few will be robots.

No idea who some of these robots are, but my previous place of work (Adfero) actually used PSInet as the ISP – so I never saw this issue until I left.

A lot of these services are dynamic liable to change host or company at anytime so it is worth repeating this once a month, maybe automate this (Dan Barker did this see below)

It is surprising how many robots do actually get tracked, which means that they must activate JavaScript and use cookies, it is also interesting some of the robot like visits that are tracked – including Google (what are they doing?), ComScore and many others that shouldn’t be…

Out of the box Google works, pretty well in fact, but still – audit the xxx out of it, and if you need help with that, drop me an email!

UPDATE 

@DanBarker – has created a way of looking at getting an alert when you get a rise of the “autobots” – http://p.barker.dj/autobots

Scanning the site to ensure that Analytics is on every page!

A second thing to check is that analytics is installed on every page – http://www.stateofsearch.com/checking-google-analytics-is-installed-on-every-page-with-screaming-frog/  – whilst putting it in your standard templates it is easy to miss the odd few pages that aren’t included in the standard template.