Article printed from SiteProNews: http://www.sitepronews.com
  HTML version available at: http://www.sitepronews.com/archives.html

Your Logfiles May Be Missing Important Data
By Ross Dunn (c) 2007

In this article I am going to explain logfiles and their
importance in website analytics from my perspective as a
ClickTracks (http://stats.stepforth.com/) user. Before I
begin, however, I want you to know that although I offer
essential analytic consulting, I am a certified ClickTracks
Analytics Professional (http://www.clicktracks.com/services/
certification.php) and have dabbled in books on analytics, I
don't consider myself to be an analytics expert. In fact, I
constantly find myself humbled by how much more there is to
know. That said, I do know more than the average site owner
and I hope that this article can shine a little light on this
often confusing subject and save you some future headaches.

As many of you may know I am a huge fan of the logfile version
of ClickTracks Professional (http://stats.stepforth.com/), a
website analytics package that I find indispensable for myself
and my clientele. ClickTracks can do a lot to determine what is
or is not working on a website; much more than expected in most
cases. The one thing, however, that ClickTracks or any other
logfile-based analytics tool cannot do is interpret information
in your logfiles if it is not recorded. Unfortunately this is a
common occurrence and many site owners have no idea that their
hosting company is not saving information that could help them
now or later when they find they need it.

The reality is that over ninety percent of the hosting companies
I have dealt with have not been saving the vital data that a
higher level analytics program needs; to work at peak
performance if at all. In this posting I will provide an
overview on this issue so you have enough information to
approach your hosting company about making the required updates
to their systems. If you are unsure, you can even refer them to
this article and/or the set of questions and details I provided
below.

What is a Logfile?

First, let me explain the very basic idea of what a logfile is
and how one is created.

Whenever you visit a website your browser requests information
from the server hosting the website. This request is passed onto
the server and in turn the server delivers the information
requested by your browser. Whenever this exchange takes place
your server saves the request along with a host of information
about the requesting browser such as:

  * the internet address (loosely connected to location) known
    as the IP
  * browser type (Internet Explorer or Firefox or..)
  * the screen resolution of the browser used
  * time and date of the request
  * the page requested for viewing
  * the website the visitor came from (known as a Referrer)
  * if applicable, the keyword(s) that were used to find your
    website on a search engine
  * etc.

Once this data is collected it is saved on the server in a
logfile for later use and over time it is often overwritten with
new data so the files do not get too large; they bulk up very
quickly especially on high-traffic websites.

How Can Logfiles Help Improve Your Website?

Now that you know what data is collected it is time to explain,
in general terms, how this data can be used to help your
website. There is a wide variety of information that can be
gleaned from a complete logfile such as:

  * How long visitors stay at your website or on a particular
    page.
  * What pages they visited.
  * Where visitors are viewing your website from geographically.
  * What keywords were used to visit your website and which
    search engines were driving the highest volume and/or
    quality traffic.
  * Which pages had the highest or least traffic.
  * The average time a visitor stays at your website: often a
    great indication of the 'stickiness' of your website.
  * You can determine the effectiveness of your pay-per-click
    campaign by tracking visitors specifically delivered from
    the campaign.
  * Identify potential pay-per-click fraud using tools like
    ClickTracks Professional that has a click fraud reporting
    tool.
  * and much more...

So What's the Issue?

Many hosting companies are smart enough to include a basic web
analytics program with every account. These programs are decent
for anyone who wants to simply find out the traffic to their
website and a myriad of other basic stats. However, there is
often a pitfall to these basic programs. You see in order to
save on computer performance the hosting company usually sets
their servers to collect only the minimal data these basic
systems require. As a result, more complex logfile-based
analytics programs may find themselves starved of the data they
need to operate fully. This is where my clients have found
themselves before; they have sub-par logfiles and are forced to
try and convince their hosting company to change their data
collection methods to meet more advanced standards.

If you have no interest in website analytics you may find this
whole scenario to be a non-issue. I completely understand,
however, put yourself a year or even a month down the road when
your website is taking off and you need to know more about the
visitors to your website. You just might find yourself in this
same frustrating scenario and it will seem absolutely insane how
hard you have to push to get this data properly collected.

Unfortunately, unless you are leasing your own private
(dedicated) server from the hosting company they tend to set up
their shared servers with only the basic needs of the majority
in mind. As a result, the only way to force change is if more
customers consider it a basic need - thus the reason for this
article. Help me affect change so that you save yourself a
headache in the future!

How to Be Sure Your Server is Collecting the Right Information

Most of you cannot check your logfiles for completeness with an
analytics program so you will have to trust your server
administrators to do their due diligence based on the following
question.

Note: If you like you can just copy and paste the following
question (noted in red) and send it to your hosting company
support staff:

Hello,

I would like to make sure my website's logfiles have the
necessary information to run a higher end web analytics program.
Is your server set up to collect the data on my website? I need
this data to properly analyze the traffic on my website.

  * Date and Time
  * Client IP Address
  * HTTP Method
  * Requested file and Query string
  * User Agent
  * Referrer
  * Status code
  * Cookie (preferable, but not required)

If you are unsure of the answer or you need to set this up then
please review the settings that need to be enabled on Apache
servers (http://way.clicktracks.com/help/en/an/
index.html?apache.htm) or Microsoft Internet Information Servers
(http://way.clicktracks.com/help/en/an/index.html?configuringiis.htm);
these pages include instructions if you need them.

Sincerely,
<>

My Hosting Company Disregarded This as Nonsense

I fully expect some will and that is because many website owners
still care little or nothing about web site statistics so they
have not even used the basic data to its fullest yet - and
hosting companies are aware of this. In fact, a good friend who
owns a hosting company himself guessed around 95% of his website
clients never even look at their stats. This is all true,
however, does that mean that important data should not be
collected for those who do want to delve deeper into analytics?
I don't believe so and the changes you are requesting will only
increase the size of the logfiles for your website a small
amount. Unless of course you don't even have logfiles which is
enough for me to recommend you take your services elsewhere.

Why Not Use Google Analytics Instead?

Google Analytics is an awesome solution for many small
businesses. It does not require logfiles and it takes a marginal
amount of work to begin acquiring proper data. In fact, I think
it is a great tool for the majority of businesses that want to
wade into a mid range analytics solution providing you are
comfortable with Google having access to your stats. That said,
there is one MAJOR flaw in using Google Analytics... it does not
have reliable click fraud reporting. You see many of my clients
use ClickTracks to monitor their pay per click campaign for
click fraud which is not something I would ever trust Google to
police itself on. That does not mean I do not use Google
Analytics. In fact, whenever possible I use both ClickTracks and
Google Analytics in tandem for redundancy especially when
certain capabilities such as cookie tracking are not available
from a hosting provider - Google includes cookies by default.

In Summary

Many website owners have no idea what they will or will not need
in the future to properly administrate their online marketing
campaigns. This article discusses a simple adjustment to the
accumulation of website logfiles that I strongly believe all
competent hosting companies should implement in order to provide
scalability for their clientele. The adjustment will provide the
additional information that a competent analytics solution will
need to provide accurate statistics.
================================================================
Ross Dunn is the CEO and founder of StepForth Web Marketing
Inc (http://www.stepforth.com), a web marketing company founded
in 1997 and based in Victoria, British Columbia, Canada.
StepForth provides cutting-edge web marketing services that
provide highly successful, targeted results for its clientele.
Ross Dunn is a Certified Internet Marketing and Business
Strategist (CIMBS) with a background in web design and online
marketing. His broad Internet experience in combination with a
talented staff has made StepForth a name synonymous with top results.
================================================================

Copyright © 2007 Jayde Online, Inc.  All Rights Reserved.

SiteProNews is a registered service mark of Jayde Online, Inc.