Back
to TopThe Troubleshooting Guide for SurfStats Log Analyzer on the SurfStats website contains the most up-to-date information.
How can I analyze different sites using the same Log?
Please refer to the "How To..." section in the SurfStats Help.
How does the second method of database population work?
This method checks for existing entries in the database and only add entries that are not already in the database. You cannot use this mode if you want to combine results from different web sites.
Notes on Database Population
A typical log file line (or hit) in a log file looks like this (Note that this is one long line and the bold part of the log line contains the referrer information ):
193.65.100.100 - - [25/Feb/2000:03:39:41 -0500] "GET /samples/clienterrors.htm
HTTP/1.0" 200 3547
"http://www.altavista.com/cgi- bin/query?sc=on&hl=on&q=HTTP+errors&kl=XX&pg=q"
"Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)"
Surfstats parses the log line and applies the set filters in FAST mode before entering the fields into its database. This means that if you change filters, the second option would not work properly. It is advisable to always use the first option (overwrite all) of database population. You should never change the filters if you use the second option of database population in FAST mode.
Surfstats applies filters and then enters ALL fields in the database in Interactive mode. This is why this mode is slower, the database is bigger and the queries are more powerful.
Why is the average number of hits more than the total hits on the summary page?
Surfstats assumes that the day is measure from the first time:date entry of the log to the last time:date entry of the log which can be less than 1. The average is the total divided by the integer value, e.g. the average on a total hit count of 200 hits in 6 hours is 400 (200/0.5). The average value will be incorrect if all the hits of the day was just generated in a few hours of the day and your log file does not reflect it.
Why do the results differ between log analyzers?
Report results differ between most log analyzers. There are a number of possible reasons:
Can I generate graphs in gif format?
No, you can generate graphs on reports in png format only. This format is supported in all the popular browsers. If you write a program that generates gif files, you have to pay royalties to Unisys who owns the gif compression patent.
Most free browsers (and Opera) can read the png format.
Do I need to be logged on to generate reports?
The Scheduler can generate reports without a user being logged on.
How to switch off shading on reports
Netscape does not display graphs and tables with the shading correctly. Create a theme with a light background image (with the theme editor) and use this theme on reports.
You can adjust the time zone of your log files to your time zone. Some log files are generated in Greenwich Mean Time (0 GMT) and you might want to see it in your time zone, e.g. GMT-7. To do this, select your site profile, click on Edit Site, then on general and enter your time offset. Also refer to the online help.
Does SurfStats save a copy of the reports if I generate to screen?
Yes. When the output option is set to screen and the report has been displayed, a copy of the report has been generated and can be found in the folder \Program Files\Surfstats\Sites\SiteName\Html where SiteName is the name of the site profile.
Which fields need to be logged to get all reports?
See the SurfStats online help to see how to set it up your server for extended logging. A sample line should look like this:
How do I use filters to include and exclude files, referrers and visitors?
Filters can be set in Edit Site - Filters or in the Control panel in Interactive mode.
You can set filters prior to generating reports. Select the site you want a report on, click on the Report button, click on Options, click on Filters, enter the filters and click on Start. The filters entered here will be saved.
In Interactive reporting mode, a control button is also available on the toolbar. Click the Control button, click on Filter, enter the filters and click on Apply. The reports requested after this will be filtered per specification.
Use the Include filter to include hits on reports to files, referrers and visitors only and the exclude filter to exclude hits. Multiple filters can be used in the same field. A filter must be flanked by * and multiple filters must be separated with a space like *filter1.htm* *filter2.htm* You can use a unique part of a field and use an * to complete the field.
To show only information from an IP Address in reports, enter the IP address in the Include IP Address field. To exclude information from an IP Address, enter the IP Address in the Exclude IP Address field.
You can also use wildcards, e.g. *206* *216* to filter on IP Addresses that includes the numbers 206 and 207. To enter multiple filters, use a space as separator. The current version cannot filter on a range of IP Addresses and the filter fields can take a maximum of 255 characters per field.
Tip. All hits are not equal. Robots from search engines can frequently increase the number of hits quite significantly. If you have a robots.txt file on your web site (we suggest you do), you can filter these hits by inserting *robots* in the Include or Exclude fields in the control panel of SurfStats. For more information on the robots.txt file and to generate it, see http://reallybig.com/robogen/index.shtml
How can I find out more about my visitors?
You have to do a DNS (Domain Name Server) lookup of a visitor's IP Address, also referred to as Resolving IP Addresses. You need to be connected to the internet to do this. The DNS (Domain Name Server) lookup queries database at DNS servers on the internet.
The country of origin will then be displayed in reports. Note that all IP addresses cannot be resolved due to various reasons like some ISP's allocate dynamic IP addresses to their clients, the host is not available, etc.
If you have received e-mail from a visitor, you can also look at the suffix of his e-mail address and then look-up the country from the ISO country code list. This list is available in the online help in SurfStats (Help-Contents-Appendix-List of country and ISO internet codes).
You can also find out more about your visitors by clicking on the hyper-linked IP Address in a report and thereby visiting your visitor.
V7.0 and later of SurfStats Log Analyzer features an optionally available pre-looked up database which eliminate the need for Reverse DNS lookups.
How do I make SurfStats do Revers DNS lookups?
There are two ways to make SurfStats do a Reverse DNS lookup:
Please note that you have to be connected to the Internet when resolving domain names. Surfstats has a global DNS database that is empty after installation. Every time when you do a DNS lookup, the database is populated with the domain names that are found. When a DNS lookup is done, the database is first queried before doing a DNS lookup on the internet.
For more information on DNS lookup, refer to the online help and:
How do I set up SurfStats to use FTP?
Tip. Use the sample AutoFTP profile provided with SurfStats as a reference when you are unsure about setting up a profile to use FTP.
How can I reduce errors on my website?
Use the Daily Error Report to see the trends in the amount of errors, the Requests Causing Errors Report and the new Error Referrers Report to see where errors originated from. Usually you will get a larger number of errors just after publishing your web site. You can then select the date in the control panel to see which errors occurred on a specific day or time period.
File not found errors - Check that the link to the file is correct. We recommend that you do not use capital letters in links and file names. Also make sure that you have a robots.txt file and favicon.ico file in every directory that was submitted to search engines.
Unauthorized Access Errors - Do not place a link to password protected directories on pages that search engines has access to. This includes the page you submitted to search engines and all the links from this submitted page. Also inform your clients to switch off or configure proxy servers correctly when accessing password protected directories.
Where can I find descriptions of terms and definitions?
If you call up the help by pressing the Help button from the main toolbar, you can go to General Information - Glossary of terms as well as Reports - Report Descriptions. Most reports also has a short description at the top.
BTW, you can print a selected topic or the heading with all sub-topics from the help.
What is the difference between the Editions?
Please refer to the SurfStats Log Analyzer product page for more information.
I want to analyze log files for third parties. May I customize the reports...?
Yes, you may customize reports. You can add your own logo (Edit Profile - Appearance - Logo) on reports. You are not restricted to the number of users that you want to create reports for per user license (except for Standard Edition) but you must have a user license for every computer that you want to install the program on.
How do I select only certain reports to be generated...?
Select the site profile, click on the Edit profile menu item, then click the Views menu item and select the reports you want.
How can I create a report showing more than 20 entries...?
Set the number of records (table and sub-table entries) to report on by editing the profile, clicking Views and specifying it on the right. Note that the number of graph entries is limited to 20.
What does the item "line number to report on when processing files"...?
The line number is for helping users to determine errors in their log files. If an error occurs when populating the database, the program will stop every X lines and prompt the user. If you use "1", you can use it to determine exactly where the error in your log file is and then edit the log file and remove it.
What does the word SHARED in the Client column on some reports mean?
A client session starts with the first file that is requested. Files requested by the same client (after the first file request) is still part of the client session. The reason why a client is shown as SHARED is because it is counted on the first page request, i.e. it has been counted on a previous page. If it is counted with every page (as some log analyzers does), the result is a much higher but incorrect client count. Some log analyzers show this as Not Applicable. If a file shows up as Shared in the Client Session column, it basically means that it was not an entry or first file request.
If you get a request for the file robots.txt, it means that a search engine robot accessed your site. This text file is used to tell robots which directories to exclude when cataloging your site.
When you get a request for the file favicon.ico, it means that a visitor using Internet Explorer 5 visited and bookmarked your site. If found by the visitor's browser, this 16x16 favicon.ico icon file replaces the default IE icon in the Address bar at the top of the browser. If you use *favicon* in the Include Filter, you can see how many Internet Explorer 5+ visitors bookmarked your page and is a good indication of how interesting your web site is.
Check out the excellent Web Developers Virtual Library and Favicon.com site for more information on these files and how to add them to your site. We recommend that you add these files to your site. You can then filter on these files to see how many visitors book marked your site and also how many search engine visited your site. See also the effect that search engine robots can have on a site.
Can SurfStats run on a Macintosh ... ?
No, SurfStats can only run on Windows 95, 98, NT4, 2000, 2003 and XP. The reports SurfStats generates can be read by any frames capable browser but Internet Explorer 4 or higher must also be loaded on your machine. You can analyze log files from Unix or Macintosh machines by linking a Windows 95/98/NT PC on a network to the Unix/Macintosh machine, FTP the files across, analyzing it and then FTP'ing the reports to specific destinations or e-mail it to clients.
Note that you do not have to FTP or download the log files if you can access it directly on local or remote drives. Rather use the File method for retrieval.
Which log file formats can SurfStats handle ... ?
SurfStats can handle most common log file formats. Refer to the features for the most updated list. If your log file format is not included in the list, please contact us and send us a small sample.
Can SurfStats handle compressed and uncompressed log file formats ... ?
Yes. SurfStats will extract compressed files before adding it to its database. It will also download compressed and uncompressed log files from the host's server. SurfStats supports most compressed file formats. Please refer to the features list. SurfStats also supports multiple log files in a compressed file.
Can SurfStats handle referrer information in log files ... ?
Yes, SurfStats will automatically detect if referrer information is available in log files. If referrer information was not supplied but later added to log files, SurfStats will report on referrer information from the time it was added.
How does SurfStats handle overlapping time periods in log files ... ?
When you view a log file for a specific date in SurfStats' log file viewer, you will notice that log file information of a previous or next day is also included. SurfStats will detect overlapping and will not add the same data to its database.
Note that the first method of data population does not check for duplicate entries in log files.
Can SurfStats join more than one log file in a report ... ?
Surfstats reads data (log files) from a local or network hard drive. To get the data from a remote host, you have to use ftp to retrieve it first. When you specify a site and click on report, Surfstats then parses the information from the specified log files and inserts it into a database that is created per setup site.
Basically you have to specify more than one file in SurfStats'
Site Manager with a semi-colon (;) character between files or you
can use wildcards, e.g. access*.* or you can use the new method
of log file specification to specify multiple files and
directories. Files can be a mixture of compressed and
uncompressed log files. When the Site properties panel is
displayed (Click on Edit Site), press F1 and select Log Files
Source Configuration. There are some examples how to do it.
Example 4 from the help file follows:
To download all files starting with access from a local directory
named \Logs on your Computer, use the following syntax:
C:\Logs\access*
Note the ‘\’ character which must be used for DOS. To include all
files from different directories, e.g. from \temp and \logs, use
c:\temp\*.*;c:\logs\*.* Place this: c:\temp\*.*;c:\logs\*.*
in the log files entry field.
I have set up my log directory as follows:
c:\Logs In this directory I place all my recent
log files that I want reports on. When a new month's logs start, I
move these files to a new directory that I create for the new
month.
c:\Logs\199908 In this directory I place all the logs files
for August 1999
c:\Logs\199907 In this directory I place all the logs files
for July 1999 etc. Now you can specify
1. A site for every directory (month). You can use wildcards for this.
2. A site for all and by specifying multiple files and directories, you can analyze log files over a large period. If the log files are large, use the summary mode for this.
My clients generate 65 MB log files per day, how can I expect them to download it ...?
Surfstats is one of very few log analyzers that uses a database to store log file information. The log file information is first parsed and filtered and then entered into a database. Although this method of analyzing log files are normally slower than directly parsing to reports, the database structure gives it its powerful filtering and reporting capabilities. Surfstats has two reporting modes:
Interactive - This mode is is used to query the database interactively, e.g. you would change a filter to display only hits to certain files and then select a report. This mode is for small to medium sized log files and should not be used to analyze large log files. You would get errors like "Tabledoc full" if your log files are too large for this mode to handle. Use the Fast mode for large log files.
Fast - This mode is used to generate "canned reports" like most other log analyzers and should be used for large log files. Less information is kept in the database with this mode and INTERACTIVE (dynamic) filtering is not possible in this mode. The reports can then be automatically e-mailed to clients or placed in certain directories for access by a browser.
The Professional Edition was developed to generate reports for clients. With it you can now fully automate log file filtering, generating reports and e-mail or ftp reports to specific folders.
My host does not supply log file information. Can I still use SurfStats...?
No. If you are serious about your website, insist that your your web host makes the log files with referrer information available to you or find a host that does.
Is it possible to have options/filters effect all output options...?
Filters are saved and used when set as follows:
When filters are set when output is set to screen, it is not saved. You can then change the filters dynamically as follows:
All the html reports are generated under the Html directory. If you use the default setup, use Win95/98 and your site name is Mysite, the files should be generated in:
C:\Program Files\SurfStats\Sites\Mysite\HTML\
The index file to the reports is called AllReports.htm and if you have a frame capable browser, open the default.htm in you browser.
The DNS lookup results are saved in a database. During a DNS lookup, IP addresses will first be resolved from the database and if not found, from the Internet. You can clear/flush the DNS database/cache if it gets too big by selecting the site, click on Options, click on DNS cache and then click on Flush the DNS cache. There are a number of Free DNS lookup tools that resolves IP Addresses and then creates a new log file with the resolved IP Addresses in the log file. See the links page for more information.
How can I save my settings and the report database...?
The Site Profile settings are saved in the SurfSite. file in the \Windows or \Winnt directory. Before you install a new version, save the site profiles from File - Save Site Profiles, uninstall the older or existing program (Add/Remove programs from the Windows Control Panel), install the new program and restore your site profiles from File - Restore Site profiles.
The report database cannot be saved and restored because:
We suggest that you compress your log files and use it as a backup.
Can I automatically generate reports every month to a different directory...?
You can automatically create directories relative to your computer's system date and then place reports in it.
Search in the help on "date macros" and see how to do it in the section "Specifying directories to be created relative to the system date".
You would have to create an index page that points to the different directories of monthly reports yourself.
Why can't I get DNS resolving to work through our firewall...?
Surfstats uses the default DNS Resolving port, port 200. Please make sure that this firewall port is open outwards to the internet.
How can I set up report generation for my users to access the reports by browser...?
You would need to create a site profile for every
user. Then specify the source of the log files (by file retrieval
or ftp) and specify the output (to folder or ftp).
You can also use the cloning feature in the Professional Edition to
duplicate sites and then change the relevant information for
faster site profile creation.
You can use the date macro feature to analyze files relative to
the current system date and also to create subdirectories with a
name relative to the system date, e.g. \Feb2001. The reports are
generated in html format and the default.htm file is the start-up
directory that lists all the reports. You would have to create a
menu if you want a user to access his/her reports in different
directories.
If you want users to access the reports over a web server you would
need to set up a virtual directory to the location of the generated report
files as well as set up security on the virtual directory. Alternatively you
can use the SurfStatsLive product if users need to access reports over a
web server.
The 'Most active countries' does not work on our reports. Is there a way to make this work...?
If running a version prior to V7.0 you need the IP Addresses resolved with the DNS lookup. The suffix of the domain name is used in the Most Active Countries report. Note that you can set the server to resolve IP Addresses. The resolved entries will then be in the log file and you do not have to do DNS lookup from the program.
How can I change the host names or domains of IP Addresses...?
You have two options:
1. Edit and change the IPResolved field of every IP Address of the
PC's.
2. If you have a list of the IP Addresses and their host names,
you can use the import feature to import the list. The format of
the file should be in the sequence IP Address, Resolved name,
Country Code and each should be followed by a carriage return as
follows:
IP Address
Resolved name
Country code (this field can be blank but should then contain
only a carriage return)
IP Address
Resolved name
Country code
etc.
as in the following example:
195.223.91.133
Peter
196.124.90.130
ns.ppc-electronic.uk
uk
Note the empty line below Peter.
Keep a backup of this list by exporting and saving it.