SharePoint Dragons

On a recent consultancy job, we were told that a certain site collection was used by a total group of 100 people. Big was our surprise when we turned to the SharePoint 2010 Web Analytics feature and found the site collection had a total of around 1000 unique visitors per day. Strange, no?

The definition of the total number of unique visitors per day can be found on the Microsoft Enterprise Content Management (ECM) Team Blog: http://blogs.msdn.com/b/ecm/archive/2010/05/03/web-analytics-in-sharepoint-2010-insights-into-reports-and-metrics.aspx : the total number of unique visitors per day consists of all SharePoint authenticated users plus anonymous users. In the last case, each unique IP address counts as a unique (anonymous) visitor.

Then, we saw something strange on one of our dev machines. The total number of unique visitors/day on a given site collection was 7, whereas there is only one user account on this machine. So, 7 different unique visitors was impossible on this machine. Were we on to something?

Intrigued, we took a look at the database. After doing some snooping around, we expected that the database WebAnalyticsServiceApplication_ReportingDB_[guid] is the database that holds the web analytics data.

After running a trace, we knew for sure. Executing the following table valued function returned the contents of our Site Collection Web Analytics Reports – Summary page:

— Get info for Site Collection Web Analytics Reports – Summary
exec sp_executesql N’SELECT TOP (2000) [propertyname], [currentvalue], [previousvalue], [percentagechange]
FROM [dbo].[fn_WA_GetSummary](@p0, @p1, @p2, @p3, default)
ORDER BY [PropertyName] ASC’
,N’@p0 int,@p1 int,@p2 int,@p3 uniqueidentifier’,@p0=20111213,@p1=20111113,@p2=30,@p3=’9519CAC1-F9AE-3BF8-D261-119A8B0A4F33′

Taking a look inside the function, we found the following T-SQL code being equivalent to the retrieval of unique visitors info:

— Get unique visitors info
DECLARE @CurrentStartDateId int = 20111213
DECLARE @PreviousStartDateId int = 20111113
DECLARE @Duration int = 30
DECLARE @AggregationId [dbo].[AggregationIdDataType] = ‘9519CAC1-F9AE-3BF8-D261-119A8B0A4F33’
DECLARE @IncludeSubSites bit = 1

DECLARE @CurrentEndDateId int, @PreviousEndDateId int
SET @CurrentEndDateId = [dbo].[fn_WA_AddDateId](@CurrentStartDateId, @Duration – 1)
SET @PreviousEndDateId = [dbo].[fn_WA_AddDateId](@PreviousStartDateId, @Duration – 1)
 
SELECT
‘UniqueVisitors’ AS [PropertyName],
(
  SELECT [Frequency]
  FROM [dbo].[fn_WA_GetTotalTrafficVolume]
  (@CurrentStartDateId, @CurrentEndDateId, @AggregationId, @IncludeSubSites, 2)
) AS [CurrentValue],
(
  SELECT [Frequency]
  FROM [dbo].[fn_WA_GetTotalTrafficVolume]
  (@PreviousStartDateId, @PreviousEndDateId, @AggregationId, @IncludeSubSites, 2)
) AS [PreviousValue]

Apparently, the fn_WA_GetTotalTrafficVolume function plays an important role in determining the number of unique visitors. The next T-SQL code determined our current number of unique visitors:

— Current unique visitors from table-valued function
  SELECT [Frequency]
  FROM [dbo].[fn_WA_GetTotalTrafficVolume]
  (@CurrentStartDateId, @CurrentEndDateId, @AggregationId, @IncludeSubSites, 2)
— Current unique visitors from underlying table
SELECT
    ISNULL(SUM([Frequency]), 0) AS [Frequency]
FROM [dbo].[WATrafficAggregationByDate] WITH (NOLOCK)
WHERE
    [AggregationId] = @AggregationId AND
    [IncludeSubSites] = 1 AND
    [MetricType] = 2 AND
    [DateId] BETWEEN @CurrentStartDateId AND @CurrentEndDateId

This function sums all unique users per day, but if you want to see the real number of unique users per day, you do:

— Show unique users per day
SELECT *  
FROM [dbo].[WATrafficAggregationByDate] WITH (NOLOCK)
WHERE
    [AggregationId] = @AggregationId AND
    [IncludeSubSites] = 1 AND
    [MetricType] = 2 AND
    [DateId] BETWEEN @CurrentStartDateId AND @CurrentEndDateId

Using this query, we found 7 entries. This looks a lot more like the Site Collection Web Analytics Reports – Number of Daily Unique Visitors page. So, there was the answer we were looking for.

According to IFABC Global Web Standards, a unique visitor is an IP address plus a further identifier (such as user name, user agent, or a cookie). If you want to know how many daily unique visitors there are, you should go to the Site Collection Web Analytics Reports – Number of Daily Unique Visitors page. The Summary page is nothing more than a summation of all days and all unique visitors.

Is this a bad thing? We feel it is at least a little (okay, more than just a little). It makes sense to provide overviews of the number of unique visitors per day, week, month, year or whatever. By simply summarizing all unique visitors, soon the number of unique visitors on the Summary page goes sky high. It leads to a situation where web analytic reports show that there are 1000 unique visitors, whereas in reality there are only 100. We know it’s a summary page, but we also feel this is a situation where simple math doesn’t apply to this data and leads to confusing results.