SharePoint Dragons

Nikander & Margriet on SharePoint

A case of "easy": Virto SharePoint Bulk Operations Toolkit

Recently, we took a look at Virto’s SharePoint 2013 workflow activities kit and we were happy with what we saw. As a result, we also decided to take a lok at another Virto product, the Virto Bulk Operations Toolkit. This is a bundle of 8 components that all do something useful with bulk operations and is shipped with a nice bulk bundle price saving you over 40%. This is a feature-packed bundle that is definitely capable to handle some of the questions we or our customers have been struggling with. And, oh yeah, it comes with comprehensive and extensive documentation.

Let’s take a look at the 8 components.

1. Bulk Check In and Approve
As the name implies, this component is for SharePoint multiple files checkin and SharePoint multiple files approval. It allows end users to check in bulk groups of documents and has features such as adding comments to the bulk whilst doing a check-in, publishing or drafting versions if

approval is enabled, or doing bulk discards of multiple checked-out files. The user interface for doing this is a lot easier then what SharePoint is offering ootb. The next Figure shows an example (all pictures are taken with permission of Virto):

clip_image002

Bulk Check In and Approve has a comprehensive set of features such as:

  • Search support for check in and approve actions.
  • CAML query support for check in and approve actions.
  • Menus that are aware of current user permissions.
  • Default view type setting.
  • Ability to check-in documents in folders and subfolders.
  • Bulk discard of files.
  • Bulk check-in and approve capabilities.

2. Bulk Data Edit Web Part
This is a component for SharePoint multiple files editing. This is a request we’ve heard time and time again (although you have to be careful with this!): the possibility to do SharePoint bulk edit. In other words, to edit the same metadata field for a bulk of list items. This enables end users to specify metadata for a bulk of items in 1 go (supported in all types of lists and libraries)! The ootb datasheet view alleviates the problem somewhat, but it is nowhere near as easy to use as this component. See the following screenshot to get an impression.

clip_image004

The Bulk Data Edit Web Part has a comprehensive set of features such as:

  • Ability to create new terms.
  • Search support for edit scenarios.
  • CAML query support for edit scenarios.
  • Taxonomy options.
  • Bulk edit for list or site scopes.
  • Editing data of the same field for a group of list items.

3. Bulk File Copy and Move
This is a component for SharePoint multiple file copy and move. The copying (or more often: moving) of large amounts of list items and/or documents is usually something that end users request of administrators, who usually use a high-end migration tool to do the heavy lifting. This component is great in that it places the power of moving larger amounts of misplaced documents/list items in the hands of end users in the form of a separate web part or an additional action in the Action menu. Also check out the Figure below.

clip_image006

Bulk File Copy and Move has features such as:

  • Support for search for copy & move operations.
  • Support for CAML queries for copy & move operations.
  • Filtering possibilities for copy & move operations.
  • Support for using the same SharePoint views as in source libraries.

4. Bulk File Delete Web Part
This is a component fo SharePoint multiple file deletion. This is also a special one. As you would expect, it can delete multiple list items/documents from any type of list. But this component goes a significant step further in that it allows you to specify which

groups of items to delete via an advanced filtering mechanism (that allows you to delete by view, CAML queries, searches, etc). The Figure below shows an example of search support in deletion scenarios:

clip_image008

The Bulk File Delete Web Part has features such as:

  • Search support when deleting files.
  • CAML support when deleting files.
  • Flexible max amount of files that are displayed.
  • Filtering possibilities during file deletion.

5. Bulk File Download Web Part
This is a component for SharePoint multiple files download. This component allows end users to download files from a library and store it locally in a single .zip file. Again, it’s easy to select which files must be downloaded. This component has been around since 2007 and was probably more important then than it is now with all the offline capabilities of SharePoint, although we have seen a customer that periodically bundled documentation that was shipped to external partners, in which case this component would have been quite useful. It also works nice in combination with component nr. 6, the Bulk File Unzip Utility. Currently, we’re exploring these components to quickly fill some sites with relevant test data which seems to work out quite nicely. The next Figure nicely demonstrates what this web part looks like:

clip_image010

The Bulk File Download Web Part has a comprehensive set of features such as:

  • Extensive logging during download.
  • Automatic JavaScript minification.
  • Support for large files.
  • Support for async operations.
  • Choosing specific files for downloading.
  • Download a bulk of files in a single archive.

6. Bulk File Unzip Utility

This is a component for SharePoint multiple files unzipping. This component unpacks zip archives to document libraries, retaining the original structure. For end users migrating to SharePoint from another system, this can be extremely useful. This allows end users to pack their documents, stuff them in SharePoint, and then use the Bulk File Copy and Move and Bulk Data Edit Web Parts to get the documents organized! This is a nice example how the separate components within the bundle seem to enhance each other. The next Figure shows how easy it is: select an archive and click the Unpack button!

clip_image012

The Bulk File Unzip Utility has features such as:

  • Unpacking archives to SharePoint document libraries.
  • Preserving folder structures after unpacking.
  • Ability to retain original file creation/modification datetime! This is a really cool feature which can be very important in certain legal situations.
  • Auto-deletion of archives aftyer unpacking.
  • Unzip settings are scoped at either the library or site level.

7. Bulk File Upload Web Part
This is a component for SharePoint multiple files upload. In the past, we’ve written stuff about uploading bulk files to SharePoint and even have written our own little tool to do it. The bottom line always consists of the following points:

  • The standard SharePoint tools for uploading files in bulks to SharePoint are not advanced and reliable enough.
  • There are community tools that are better, but offer very limited support which is commonly not acceptable in enterprise scenarios.
  • There are high-end migration tools that do a great job at a great cost.

The Bulk File Upload Web Part has a comprehensive set of features such as:

  • Possibility to add field descriptions.
  • Show overwrite option.
  • The possibility to redirect all requests to the standard upload page to the Virto Bulk Upload (via an HTTP module).
  • Support for Content Organizer rules.
  • Possibility to resize jpeg images during upload.
  • Current user defaults for property values.
  • Maximum file size settings.
  • Allowed file type settings.
  • Upload of large files.
  • Partial uploading of operation is cancelled.

It seems that Virto fills a very nice niche here by offering a professional and supported solution at a very reasonable price (at least, that’s what we think). The next Figure shows an impression.

clip_image014

8. HTML 5 Bulk File Upload
Last but not least, this is a component for SharePoint multiple files upload. This component is a nice enhancement to the Bulk File Upload Web Part and is completely written in HTML 5. It is very similar to the Bulk File Upload Web Part except that it’s even better! This component is probably our favorite in the bundle. The next Figure shows what it looks like.

clip_image016

HTML 5 Bulk File Upload has features such as:

  • Uploading of bulk files.
  • Support for custom metadata.
  • Support for uploading files via drag-n-drop leveraging just HTML 5.
  • Ability to overwrite files during upload.
  • Display of file upload progress bar.
  • Restriction of file types that can be selected.
  • Limiting the max file upload size.
  • Support for unlimited amounts of content types and fields.
  • And, as all of Virto’s components, cross browser support for MSIE, FireFox, Chrome and Opera.

So if any of the features of the Virto Bulk Operations toolkit appeal to you, we won’t try to stop you checking it out at http://www.virtosoftware.com/bundles/virto-bulk-operations-toolkit.

jQuery Caret Plugin

Let’s give a positive endorsement for the jQuery Caret Plugin which can be found at https://github.com/acdvorak/jquery.caret This library allowed us to find the current cursor position in a text area with code like this:

targetElement = $(“#myID”);
var currentCursorPosition = targetElement.caret();

Contrary to the other solutions we’ve tried, this one also worked flawlessly in MSIE 8!

Quick link title strangeness in SharePoint 2013: wrong Title value

There is some strangeness surrounding the retrieval of quick launch links in SharePoint 2013 which may very well be a bug in SharePoint. Although we’re the first ones to admit we haven’t been let in on the exact reasoning behind SharePoint’s implementation of the quick launch title, and there may be very good reasoning behind it, we’re inclined to believe that this is another area where multi-language scenarios have been implemented sloppily in SharePoint 2013. In this article we discuss what the problem is and, better yet, how to solve it.

In principle, retrieving the link title of a quick link is easy. The next C# code uses the server-side OM to retrieve the title of every first link of every web site in a site collection.

string siteUrl = “[site collection URL]”;
using (var site = new SPSite(siteUrl))
{
    foreach (SPWeb web in site.AllWebs)
    {
        string title = web.Navigation.QuickLaunch[0].Title;
    }
}

There are situations when the Title does not contain the value you’d expect. There are 2 preconditions that need to be met to experience this strangeness (there may be more scenarios causing it, but we’re describing the one we’ve encountered):

  • We’re editing the default first link of a new team site.
  • The locale of the thread running the code is different from the default language of the SharePoint web site.

To clarify, let’s present an example that runs as expected, then followed by the example that does not behave as we expect.

Example 1: Expected Title

  1. Create a new team site and choose language “English”.
  2. Go to the Navigation link (so the Publishing features need to be enabled in order to do this) and edit the first link and change it’s title to “Test”. Do not change the URL!
  3. Run the code and verify that web.Navigation.QuickLaunch[0].Title indeed returns the value of “Test”, the same value as seen via the UI in the quick launch bar.

Example 2: Unexpected Title

  1. Create a new team site and choose language “Dutch” (or some other language as long as its not English).
  2. Go to the Navigation link (so the Publishing features need to be enabled in order to do this) and edit the first link and change it’s title to “Test”. Do not change the URL as this will ensure the Title functionality to function correctly!
  3. Run the code and verify that web.Navigation.QuickLaunch[0].Title does not return the value of “Test”, but instead returns “Home”, even when the quick launch bar itself displays the link title “Test” correctly.

Inspecting the implementation of the Title property in the Microsoft.SharePoint.Navigation.SPNavigationNode class sheds some light on this issue:

public string Title
{
    get
    {
        if (System.Globalization.CultureInfo.CurrentUICulture.LCID == this.Navigation.Web.UICulture.LCID)
        {
            return this.m_strTitle;
        }
        return this.TitleResource.Value;
    }
}

In our case our code was running under a thread with the LCID of 1033 (English – United States). As long as the default language of the SharePoint web site is set to English, this works fine and the member variable m_strTitle is returned containing the correct quick link title value of “Title”.

But, when the LCID of our thread is 1033 and the default language of the SharePoint web site is set to Dutch, we get this.TitleResource.Value which is “Home” and that is incorrect.

As a remedy, we can use reflection to retrieve the value of private member m_strTitle. In C#, this goes like this:

SPNavigationNode firstLink = web.Navigation.QuickLaunch[0];
FieldInfo fieldInfo = firstLink.GetType().GetField(“m_strTitle”, System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance);
string realLinkTitle = fieldInfo.GetValue(firstLink).ToString();

In PowerShell, the same feat goes like this:

$site = Get-SPSite $siteUrl
foreach ($web in $site.AllWebs)
{
    $firstLink = $web.Navigation.QuickLaunch[0]
    $fieldInfo = $firstLink.GetType().GetField(“m_strTitle”,
        [Reflection.BindingFlags]::NonPublic
        -bor [Reflection.BindingFlags]::Instance)
    $actualLinkTitle = [string]$fieldInfo.GetValue($firstLink)

    $web.Dispose()
}
$site.Dispose()

Now, regardless of the thread your code is running in, you get the actual link title instead of the resource value.

Having fun with Virto software’s SharePoint 2013 workflow activities kit

It is quite common that companies leveraging SharePoint 2013 workflows find there is some functionality lacking. Extending the existing workflow possibilities by creating custom SharePoint workflow activities that can be leveraged by the company’s power users is certainly possible, but can be quite expensive. Typically, the best option is to buy a commercial solution that extends ootb workflow capabilities. 

There are multiple vendors out there delivering such solutions, but recently our interest was peeked by a company called Virtosoftware. Virtosoftware has created over 50 solutions for SharePoint 2007, 2010, 2013 and Office 365, is a Microsoft Gold certified partner and Virto offers a set of valuable workflow activity extensions. You can find more about that over here: SharePoint Workflows 

One of the areas addressed by Virto are SharePoint workflow activities concerned with interacting with Active Directory. Those activities allow you to interact with almost all AD information. This includes retrieving all kinds of AD properties, but also stuff like user and group management. The next Figure shows an example (taken from the Virto web site with permission):

clip_image002

Most often, we find a need to interact with permissions via workflows at either the site or list level. Luckily, such activities are included in the kit. Virto’s workflow activities kit also contains, of course, the ability to work with e-mails and e-mail attachments in an advanced way.  

The kit offers an ideal extension for power users in the form of Manage site activities that allow SharePoint workflows to perform typical management tasks in a controlled and regulated way, such as creating lists, libraries, sites, and even site collections.

Unsurprising, Virto’s workflow kit has extensions for working with lists, but the extension that did surprise us in this area was an activity that allows you to generate reports about a list. Now that is something we can get on board with! 

All in all the SharePoint workflow kit has over 230 workflow activities, so we won’t cover all of them. Let it be noted the activities contain useful additions involving sending Twitter messages, interacting with several instant messaging platforms, sending SMS messages, interacting with HTTP at a low level, and more.

As a final safety net, there are activities that allow you to execute custom VB.NET or C# code, so if what you’re looking for is not there, there is an easy escape route. If that’s not enough, on page SharePoint Workflow Activities Extensions section “Request the missing activity” we found that Virto promises that if you find a specific SharePoint activity that is not there in the Virto SharePoint Workflow Activities Extensions set, you can send a request that will be fulfilled within 3 Business Days (sic!).

Concluding, thumbs up for Virto SharePoint 2013 workflow activities extensions!

Uploading and activating sandbox solutions via CSOM/JSOM

In this post we’re discussing how to upload and activate a sandbox solution via CSOM/JSOM. First, we’ll do it using C#. That’s a little bit easier and we’ve seen several examples discussing that (such as this one: http://blogs.msdn.com/b/frank_marasco/archive/2014/08/10/upload-and-activate-sandbox-solutions-using-csom.aspx , probably the first one written about this topic and without it, we couldn’t have written this post). Then, we’ll do it using JSOM which has some additional challenges. We didn’t find any resources discussing that, so there should be value in that.

First off, create a new sub site using the template team site. Then, save that sub site via Site Settings > Save site as template and go to the Solution Gallery and download it somewhere on your local file system. That way, you’ve created a site template that can be used to upload and activate later on. When finished, remove that solution from the Solution Gallery (otherwise, there’s not much point in uploading it again programmatically, now is there?).

In the C# version, we’re basically doing this:

  1. Establish the correct client context
  2. Upload the solution directly to the Solution gallery.
  3. Read the site template (*.wsp) from the local file system.
  4. Create a new FileCreationInformation object representing the site template that will be uploaded.
  5. Upload the solution somewhere. Please note: somewhere can be the solution gallery, but it can also be any other document library. This is because the DesignPackageInfo class accepts a relative URL of an asset located in SharePoint, takes it, and installs it in the Solution Gallery.
  6. Use the DesignPackageInfo class to install and activate the sandbox solution.

The code to do that looks like this:

using Microsoft.SharePoint.Client;

using Microsoft.SharePoint.Client.Publishing;

using System;

using System.Collections.Generic;

using System.IO;

using System.Linq;

using System.Text;

using System.Threading.Tasks;

using System.Web;

namespace SolutionUploader

{

class Program

{

static void Main(string[] args)

{

try

{

ClientContext context = new ClientContext(“http://%5Burl of site collection] “);

Web web = context.Web;

context.Load(web);

var lib = web.Lists.GetByTitle(“[destination lib]”);

context.Load(lib);

var filePath = @”D:\TestTemplateFolder\TestTemplate.wsp”;

var file = new FileStream(filePath, FileMode.Open);

var fileCI = new FileCreationInformation()

{

ContentStream = file,

Url = “TestSiteTemplate-v1.2.wsp”,

Overwrite = true

};

var uploadedFile = lib.RootFolder.Files.Add(fileCI);

context.Load(uploadedFile);

context.ExecuteQuery();

var wsp = new DesignPackageInfo()

{

// Guid can be empty and is autofilled,

// but specifying it explicitly makes it easier if you want to remove it

// later.

PackageGuid = new Guid(“[GUID]”),

//PackageGuid = Guid.Empty,

MajorVersion = 1,

MinorVersion = 0,

PackageName = “[package name]”

};

string filerelativeurl = “[site relative path of *.wsp] “;

DesignPackage.Install(context, context.Site, wsp, filerelativeurl);

context.ExecuteQuery();

}

catch (Exception ex)

{

Console.WriteLine(ex.Message);

}

}

}

}

You may notice that after installation, the solution is always named like so:

[package name]v[major version].[minor version].wsp

Let’s move on to the JSOM example. In other to write it, there where bits and pieces we’ve copied related to the binary encoding and uploading of files and we’ve tried to give credit where possible, but we’re pretty sure we’re missing an acknowledgement. Rest assured, this wasn’t because of any bad intentions.

We found doing the same thing in JSOM is harder, especially when you’re trying to do it before creating a POC in C#. Part of the reason for this is that the JSOM DesignPackageInfo documentation isn’t really helpful at the time of writing, but it’s here: https://msdn.microsoft.com/en-us/library/office/jj954421.aspx in case you want to take a look.

This example involves an HTML page that uploads the site template and activates it in the Solution Gallery. It goes something like this:

  1. You need a reference to the sp.publishing.js library, because it contains the DesignPackageInfo class.
  2. Include a file control to allow end users to upload the site template.
  3. Use a client-side FileReader object to read the binaries.
  4. Upload the file to a SharePoint library and give it a title. Failing to give the file a title may result in situations where the eventual package name is [a guid consisting of 0’s]v.[major].[minor]. We’ve seen this happen on several occasions and it seems to be a problem that doesn’t happen in C#/CSOM.
  5. Use the DesignPackageInfo class to install and activate the solution.

<!DOCTYPE html>

<html lang=”en” xmlns=”http://www.w3.org/1999/xhtml”&gt;

<head>

<meta charset=”utf-8″ />

<title></title>

<script type=”text/javascript” src=”/_layouts/15/sp.publishing.js”></script>

<script type=”text/javascript” src=”[reference to jquery library]”></script>

</head>

<body>

<h1>Upload en Activeer Solution</h1>

<input id=”inputFile” type=”file” />

<input id=”uploadDocumentButton” type=”Button” value=”Upload Document”/> <p/>

<script>

$(“#uploadDocumentButton”).click(function () {

if (document.getElementById(“inputFile”).files.length === 0) {

alert(“Select a file!”);

return;

}

CreateFile();

});

var file;

var fileCreateInfo;

function CreateFile() {

// Ensure the HTML5 FileReader API is supported

if (window.FileReader) {

input = document.getElementById(“inputFile”);

if (input) {

file = input.files[0];

fr = new FileReader();

fr.onload = receivedBinary;

fr.readAsDataURL(file);

}

}

else {

alert(“The HTML5 FileSystem APIs are not fully supported in this browser.”);

}

}

// Callback function for onload event of FileReader

function receivedBinary() {

var clientContext = SP.ClientContext.get_current();

this.oWebsite = clientContext.get_web();

clientContext.load(this.oWebsite);

var listTitle = “[title of destination library for *.wsp”;

var list = this.oWebsite.get_lists().getByTitle(listTitle);

fileCreateInfo = new SP.FileCreationInformation();

fileCreateInfo.set_url(file.name);

fileCreateInfo.set_overwrite(true);

fileCreateInfo.set_content(new SP.Base64EncodedByteArray());

// Read the binary contents of the base 64 data URL into a Uint8Array

// Append the contents of this array to the SP.FileCreationInformation

var arr = convertDataURIToBinary(this.result);

for (var i = 0; i < arr.length; ++i) {

fileCreateInfo.get_content().append(arr[i]);

}

// Upload the file to the root folder of the document library

this.newFile = list.get_rootFolder().get_files().add(fileCreateInfo);

clientContext.load(this.newFile);

var item = this.newFile.get_listItemAllFields();

item.set_item(“Title”, “[title value]”);

item.update();

clientContext.executeQueryAsync(onUploadSuccess, onUploadFailure);

}

function onUploadSuccess() {

// File successfully uploaded

var clientContext = SP.ClientContext.get_current();

var wsp = new SP.Publishing.DesignPackageInfo();

wsp.set_packageName(“[package name]”);

//wsp.set_packageGuid(“{GUID makes deleting easier}”);

wsp.set_majorVersion(3);

wsp.set_minorVersion(0);

var filerelativeurl = “/[site relative url to *.wsp] “;

SP.Publishing.DesignPackage.install(clientContext,

clientContext.get_site(),

wsp,

filerelativeurl);

clientContext.executeQueryAsync(onActivateSuccess, onActivateFailure);

}

function onUploadFailure() {

// Error occurred

console.log(“Request failed: ” + arguments[1].get_message());

}

function onActivateSuccess() {

alert(“Solution successfully activated!”);

}

function onActivateFailure() {

alert(“Solution activation failed: ” + arguments[1].get_message());

}

// Utility function to remove base64 URL prefix and store base64-encoded string in a

// Uint8Array

// Courtesy: https://gist.github.com/borismus/1032746

function convertDataURIToBinary(dataURI) {

var BASE64_MARKER = ‘;base64,’;

var base64Index = dataURI.indexOf(BASE64_MARKER) + BASE64_MARKER.length;

var base64 = dataURI.substring(base64Index);

var raw = window.atob(base64);

var rawLength = raw.length;

var array = new Uint8Array(new ArrayBuffer(rawLength));

for (i = 0; i < rawLength; i++) {

array[i] = raw.charCodeAt(i);

}

return array;

}

</script>

</body>

</html>

And that’s all folks!

3rd party analytics for SharePoint 2013

In general, ootb analytics do not satisfyingly meet the need of all customers as far as getting insights in the use of sharepoint are concerned. This post provides an overview of vendors that offer additional analytics capabilities for sharepoint or that can be used in combination with sharepoint. The idea behind the article is that it’s a useful starting point for people starting the decisioning process. It also contains a list of useful evaluation criteria that can help to decide which product is right for you. On purpose, we refrain from saying: product x is better than product Y because this depends too much on personal preference and the weight a specific feature is given within your organization.

Here is the list of analytics solutions we have looked at, listed alphabetically.

Adobe Analytics 
Adobe Analytics used to be known as Omniture, but is renamed in 2013. Before Omniture, this product was known as SiteCatalyst and was partly built on a product called Hitbox. According to http://www.slideshare.net/maverickaman/ga-wt-omtr this product is superior to Google Analytics. Adobe Analytics require web analytics data to be stored in the cloud, there is no option to store data on premises. There will certainly be scenarios where this is a deal breaker. Although not a specific sharepoint analytics solution, this tool can be leveraged to provide analytics insights in SharePoint.

Angelfish
Angelfish ( http://analytics.angelfishstats.com), made by Actual Metrics, is an on-premise solution that stores web site traffic in a safe way. Actual Metrics is founded by an ex-employee of Google and supposedly both companies still have strong ties. Angelfish is a generic web analytics solution and replacement of Google Urchin. A big advantage of this solution is that it is able to import Google Analytics data. Although not a specific sharepoint analytics solution, this tool can be leveraged to provide analytics insights in SharePoint.

Cardiolog Analytics Cardiolog Analytics
Cardiolog Analytics Cardiolog Analytics (http://www.intlock.com/intlocksite/productsandservices/cardiolog/cardiolog-analytics.asp) is made by Intlock, a Microsoft Gold Certified partner. This product offers extensive SharePoint web analytics features since 2005 and offers advanced capabilities regarding the analysis of end user interaction.

ControlPoint
ControlPoint (http://www.metalogix.com/Products/ControlPoint.aspx) was originally built by Axceler Inc., then purchased by Metalogix, and seems to be intended for an admin audience, although it does contain a reporting part. In our experience, Metalogics offers extensive product documentation and responds to questions quickly. The ControlPoint product has won several awards (such as Best of TechEd for Productivity and Collaboration) and adheres to several certifications (such as U.S. Navy DADMS Approval).

DocAve Report Center for Microsoft SharePoint Reporting
DocAve Report Center (http://www.avepoint.com/sharepoint-reporting-docave/) of AvePoint provides insights into usage analytics (as well as a lot of other stuff, such as SharePoint performance and security settings). The vendor is well known, although the reports seem to be intended to an admin audience instead of a business audience (because of the nature of the reports, but also because of the amount of privileges required to use report center). One of the interesting features of the product is that it allows admins to detect suspicious activities of end users.

Google Analytics
Google Analytics (http://www.google.nl/intl/nl/analytics/) collects web analytics data but stores them in the cloud. GSA may very well be the de facto web analytics market leader and provides an excellent solution for that, but not every company is willing to hand over their data in custody of Google.

HarePoint Analytics for Microsoft SharePoint 
HarePoint Analytics ( https://www.harepoint.com/Products/HarePointAnalyticsForSharePoint/Default.aspx) is built by a Russian company that has nice looking SharePoint specific reports, extensive documentation, easy installability and a good price/quality ratio.

IBM Digital Analytics Multisite IBM Digital Analytics Multisite
IBM Digital Analytics Multisite IBM Digital Analytics Multisite (http://www-03.ibm.com/software/products/en/enterprise-digital-analytics) is a generic web analytics environment. At the time of writing, the IBM marketing department has not indicated how suitable this product is for SharePoint, but a solution by such a big player is worth mentioning here.

NGAGE
NGAGE Enterprise Analytics (http://www.ngageintelligence.com/products-sharepoint2013.shtml) is an advanced solution built by a relatively small USA based company. What sets NGAGE apart from the rest is that it uses advanced datawarehouse/BI technology to analyze data, fully based on the SharePoint BI suite.

Nintex Analytics
Nintex Analytics (http://en-us.nintex.com/) is a well-known company specialized in building SharePoint products. It supports SharePoint 2010 until July 2015, but does NOT support SharePoint 2013 and according to current plans this product is dead.

Piwik SharePoint Analytics Piwik
Piwik SharePoint Analytics Piwik (http://piwik.org/) is a French initiative and the facto standard for open source web analysis tools. Piwik has been downloaded over 2 million times, contains an impressive number of features and is extremely customizable. Piwik SharePoint Analytics is built on top of Piwik and is a SharePoint specific solution. Piwik relies heavily on the use of cookies and javascript. The fact that Piwik is open source might imply low license costs, but the costs of Piwik SharePoint Analytics (as part of Piwik PRO Enterprise), onsite training, custom development and on site services will be considerable. So far, we have the impression that the effort for installing and configuring the product is considerable, but we may have to experiment further. Another factor to consider is that Piwik leverages technologies that are atypical in the average sharepoint environment (think technologies such as PHP v5.4 or higher, and MySQL v4.1 or higher).

Unilytics Mergence SharePoint Analytics Mergence
Unilytics Mergence SharePoint Analytics Mergence (http://www.unilytics.com) is a product of Unilytics Corporation and a specific analytics solution for SharePoint. It seems that Mergence is not a stand-alone product, but one that integrates with several well-known web analytics products such as WebTrends, Omniture and Google Analytics. At the time of writing, we got an HTTP 404 – Page Not Found error when navigating to Products > Mergence for SharePoint Analytics, so maybe this product is dead.

Site Administrator for SharePoint 
Site Administrator for SharePoint (http://software.dell.com/products/site-administrator-for-sharepoint/) is a SharePoint monitoring and reporting product originally built by Quest and now owned by Dell. This product is aimed towards admins, but offers enterprise-wide discovery and fine grained reports about the entire SharePointnvironment. Site Administrator for SharePoint has won the Gold Community Choice award of 2013 by Windows IT Pro.

WebTrends SharePoint Analytics
WebTrends SharePoint Analytics (https://www.webtrends.com/products-solutions/analytics/sharepoint-analytics-use-cases/ ) is a web analytics pioneer and offers a SharePoint specific analytics solution that is used by more than 300 major enterprises and was developed keeping close contact with the Microsoft engineering and product teams. WebTrends seems to be the market leader as far as SharePoint Analytics go and the product has one several awards and recognitions (such as Industry Leader in Web Analytic volgens Forrester Web Analytics Wave 2014 and Member van de Microsoft SharePoint Partner Advisory Council).

When evaluating 3rd party solutions, we found the following list of criteria useful:
– What are the license costs?
– What are the software requirements and what is the architectural impact on your SharePoint farm?
– Is it possible to store data on premises for privacy reasons?
– Is it possible to view the number of visitors per site collection/site/list item?
– Is it possible to present data anonymously (without displaying user names) or even possible to store analytics data anonymously?
– Is it possible to view the Last page request within a session?
– Is it possible to define which audience sees reports (such as admins, site owners or normal end users)?
– Is it possible to export reports to other formats such as PDF, Excel or extract report data via a web service?
– What’s the status of the vendor in the market?
– Which customers are using the product?
– What are the response times of support and what is the quality of the answers provided?
– What’s the product’s ease of use?
– Is it a SharePoint specific or generic solution?
– Is it good at protecting the analytics data?
– Is it possible to configure fine-grained security permissions for specific reports?
– Does it have advanced BI capabilities (such as trend prediction)?
– Does it offer goal-driven paths functionality (that allows the report admin to specify goals about end user interaction, such as the number of clicks needed to place a comment)?
– Does it offer report drill-down capabilities?
– Does it allow to create custom reports?
– Does it offer real-time reporting capabilities?
– Does it allow filtering during data collection?
– How does it impact the performance of the SharePoint farm?
– Does it provide reports about social behavior within SharePoint?
– Does it integrate with analytics data of social platforms such as Yammer or other sources of analytics data?
– Does it have pruning capabilities (that allows one to remove historic data because it is no longer relevant)?
– Does it support Spike Analysis (to analyze a very specific period of a couple of minutes during which sharepoint was extremely busy or lots of errors occurred)?
– Does it support heat mapping (to analyze which parts of a page are more popular so that the user interface can be improved)?
– Does it support event tracking (to track specific events within a web site, tracking typically occurs via JavaScript)?
– Does it support Visitor segmentation (the possibility to divide visitors in groups based on interests)?
– Does it support Annotations (that allows you to add notes to specific reports)?

SharePoint WPSC is undefined

In a SharePoint 2013 environment we encountered the error “WPSC is undefined”. The WPSC (Web Part Page Services Component) is a JavaScript library that web parts use to communicate with each other client-side. We don’t feel the WPSC used that often nowadays (if that ever was the case), so maybe that explains why we couldn’t find that much info about this problem. The most interesting post was this one: http://sadomovalex.blogspot.nl/2010/02/fix-wpsc-is-undefined-javascript-error.html

But it talks about an issue in combination with FBA, and our problem existed on a page leveraging WPSC for client-side communication when using MSIE 10 or higher (whereas MSIE 9 worked fine). Also, in normal mode the error existed in MSIE 10 and up, in page edit mode it went away. There’s another post worth mentioning, and its this one: http://msharaky.blogspot.nl/2009/06/wpsc-undefined-javascript-error.html

Here, the author defines a dummy object and reports the error goes away. This is true of course, but also kills client-side communication. But, if you’re just looking for a way to suppress an ugly error message, this one does the trick.

To understand more about the problem, it’s worth mentioning that the WPSC object is defined in a JavaScript library called ie55up.js which is included by default. So, if you’ve changed your master pages and its not referenced anymore, then there’s the place where it needs to be corrected.

Another useful thing to know is that if you’re using the SharePoint ScriptLink control to include a JavaScript library into your page, the OnDemand attribute determines if such a library will be included on demand as needed, or immediately. The following line ensures a JS library is loaded on demand:

<SharePoint:ScriptLink ID=”scriptLink1″ runat=”server” Name=”MyScriptLib.js” LoadAfterUI=”true” OnDemand=”true” />

If the js library needs to be loaded immediately, the control just generates a script tag. If the js library needs to be loaded on demand, it generates calls to the SharePoint RegisterSOD() JavaScript function. This is interesting because a SharePoint page in edit mode loads the ie55up.js immediately (and thereby the WPSC), using the following client-side code:

<script type=”text/javascript” src=”/_layouts/15/ie55up.debug.js?rev=pVBnO13dp7gFq%2FZalDmroA%3D%3D”></script>

Remember that the page in edit mode works correctly in conjunction with WPSC.

Note: the script tag refers to a debug version of ie55up.js instead of the minified version. That is because we have configured SharePoint to do this by setting the debug attribute in the web.config file to true.

On the other hand, a page in normal mode that uses the WPSC but fails in MSIE 10 and up renders the ie55up.js library to be loaded on demand, like so:

<script type=”text/javascript”>RegisterSod(“browserScript”, “\u002f_layouts\u002f15\u002fie55up.debug.js?rev=pVBnO13dp7gFq\u00252FZalDmroA\u00253D\u00253D”);RegisterSodDep(“browserScript”, “strings.js”);</script>

Apparently there is some on demand script problem because in which case the ie55up library is registered successfully, but never gets loaded in MSIE 10 and up. Btw, the root cause of this we have yet to discover.

Another thing to note is that the RegisterSod() function is defined in init.js and looks like this:

function RegisterSod(key, url) {
    ULSxSy: ;
    key = NormalizeSodKey(key);
    var sod = new Sod(url);
    _v_dictSod[key] = sod;
}

Later on, we’ll be able to leverage the facts that all keys are stored in the _v_dictSod dictionary and ie55up.js gets registered using the “browserScript” key to our advantage.

In order to experiment with this problem, we needed an easy way to add JavaScript code to a SharePoint page that uses the WPSC. We decided to place a Script Editor web part on a page (page edit mode > ribbon > Insert > Embed Code) that uses JavaScript that uses WPSC to register for a predefined event that happens when a user presses F1 to get help. All that’s needed to determine when and if the WPSC problem is an issue for you is this code:

<script>
WPSC.RegisterForEvent(“urn:schemas-microsoft-com:dhtml”, “onhelp”, myfunc); // respond to predefined event for pressing F1
</script>

Now our approach for dealing with the “WPSC is undefined” problem has 2 sides. If the WPSC object exists (in older browsers and page edit mode for msie 10 and up) we need to do the following:
1 – Establish that the WPSC exists.
2 – Do required communication via WPSC. In this case, we’ll use the WPSC to respond to an F1 key press.

In situations where the WPSC doesn’t exist, our approach is like this:
1 – Establish that the WPSC object does NOT exist.
2 – Find the URL that SharePoint intends to use to load ie55up.js.
3 – Use the SP.SOD library to load ie55up.js on demand.
4 – Establish that ie55up.js is indeed loaded.
5 – Do client-side communication via WPSC. In this case, we’ll use the WPSC to respond to an F1 key press.

This results in code that can be used as a work-around for situations where the WPSC is undefined, making sure its loaded on demand and then do the things you want to do with a fully functioning WPSC. You can paste the following code in a Script Editor web part on a SharePoint page to see it in action (after pressing F1):

<script>
function myfunc()
{
  alert(‘Responding to help!!!’);
}

function WpscLoaded()
{
  execWpscCalls();
}

function execWpscCalls()
{
  WPSC.RegisterForEvent(“urn:schemas-microsoft-com:dhtml”, “onhelp”, myfunc); // respond to predefined event for pressing F1
}

if (typeof(WPSC) == “undefined” )
{
  // default src of js lib defining WPSC
  var url = “\u002f_layouts\u002f15\u002fie55up.js”;

  // get url that sharepoint stores to refer to WPSC lib
  if (_v_dictSod != null && _v_dictSod[“browserScript”] != null)
  {
    url = _v_dictSod[“browserScript”].url;
  }
  SP.SOD.registerSod(‘ie55up.js’, url);
  SP.SOD.executeFunc(‘ie55up.js’, null, function(){});

  // when done, WPSC is available 
  ExecuteOrDelayUntilScriptLoaded(WpscLoaded, “ie55up.js”);
}
else
{
  execWpscCalls();
}
</script>

Emulating browsers and weird CSS problem

Recently, we experienced a weird problem in MSIE 9 where select boxes kept closing themselves when hovering over them, in later browsers we didn’t experience this issue. It turned out that this was caused by multiple CSS selectors pertaining to font families defined in different CSS files which were targeting the same select box. Both CSS files did something like:

select {font-family:’Avenir LT W01 85 Heavy’;}

Removing one of them solved the issue, and allegedly this should do the trick too:

select option {font-family:’Avenir LT W01 85 Heavy’;}

This is described in more detail at: http://stackoverflow.com/questions/5551288/ies-select-boxes-closes-themselves-when-hovering

In the process, we found that MSIE 11 Enterprise Mode causes the browser to run in IE 8 compatibility mode, but this didn’t cause the self-closing select box problem, so this compatibility mode doesn’t seem to be full-fidelity. This can be turned on or off by opening MSIE 11, press ALT > Tools > Enterprise Mode (or: ALT > Extra > Ondernemingsmodus in Dutch). Read more about MSIE 11 Enterprise mode over at: http://www.zdnet.com/article/internet-explorer-11-to-get-new-enterprise-mode/

It also turned out that the problem didn’t appear in MSIE 11 compatibility mode (press F12, next to the Search box choose a compatibility mode such as 8, 9, or 10). As it turned out we were able to use IETester (http://www.my-debugbar.com/wiki/IETester/HomePage) and it was able to simulate MSIE 9 in such a way that we could reproduce the self-closing CSS issue.

Note to Self: Don’t forget to add to indexed properties

If you want to add a property to the spweb property bag, you might want to consider to ad dit to the IndexedPropertyKeys collection as well. This causes the creation of a new crawled property the First time after a full crawl has taken place. The code looks something like this:

using (var site = new SPSite(“http://thesite_coll_url&#8221;))

            {

using (var web = site.OpenWeb())

                {

string propKey = “lctest”;

string propValue = “lcvalue”;

                    web.AllProperties[propKey] = propValue;

if (!web.IndexedPropertyKeys.Contains(propKey))

                    {

                        web.IndexedPropertyKeys.Add(propKey);

                    }

                    web.Update();

                }

            }

SharePoint 2013 IIS Lockdown: Dangerous HTTP methods

Security assessments typically advise to lockdown IIS and limit the number of available HTTP methods that can be used to the minimum.

Tools such as Urlscan (http://learn.iis.net/page.aspx/473/using-urlscan) can help with that, but the big question is if it is a good idea to use such tools in a SharePoint environment.

For a typical web application, support for the HTTP GET and POST methods may be enough, but SharePoint is not aywhere near typical. Forstarters, it supports the WebDAV, REST, and FrontPage Server Extensions protocols.

The following URL contains a list of all known HTTP methods: http://msdn.microsoft.com/en-us/library/aa364664(v=vs.85).aspx

The next URL contains a list of all WebDAV methods: http://msdn.microsoft.com/en-us/library/aa142917(v=EXCHG.65).aspx

Problem is we didn’t find an exhaustive list containing all HTTP methods that are supported within SharePoint. But we did find that SharePoint supports the following HTTP methods (which are actually the majority of all available HTTP methods):

– COPY, DELETE, GET, GETLIB, HEAD, LOCK, MERGE, MKCOL, MOVE, OPTIONS, POST, PROPFIND, PROPPATCH, PUT, UNLOCK.

Further more, the section of the web.config files of sharepoint web applications indicate that most handlers support ALL HTTP methods. Therefore, we consider it to be too risky to exclude dangerous HTTP methods. The only thing that would change our mind would be an exhaustive list with all HTTP methods supported by SharePoint 2013, published by MS themselves.