SharePoint Dragons

Nikander & Margriet on SharePoint

Monthly Archives: June 2015

Uploading and activating sandbox solutions via CSOM/JSOM

In this post we’re discussing how to upload and activate a sandbox solution via CSOM/JSOM. First, we’ll do it using C#. That’s a little bit easier and we’ve seen several examples discussing that (such as this one: http://blogs.msdn.com/b/frank_marasco/archive/2014/08/10/upload-and-activate-sandbox-solutions-using-csom.aspx , probably the first one written about this topic and without it, we couldn’t have written this post). Then, we’ll do it using JSOM which has some additional challenges. We didn’t find any resources discussing that, so there should be value in that.

First off, create a new sub site using the template team site. Then, save that sub site via Site Settings > Save site as template and go to the Solution Gallery and download it somewhere on your local file system. That way, you’ve created a site template that can be used to upload and activate later on. When finished, remove that solution from the Solution Gallery (otherwise, there’s not much point in uploading it again programmatically, now is there?).

In the C# version, we’re basically doing this:

  1. Establish the correct client context
  2. Upload the solution directly to the Solution gallery.
  3. Read the site template (*.wsp) from the local file system.
  4. Create a new FileCreationInformation object representing the site template that will be uploaded.
  5. Upload the solution somewhere. Please note: somewhere can be the solution gallery, but it can also be any other document library. This is because the DesignPackageInfo class accepts a relative URL of an asset located in SharePoint, takes it, and installs it in the Solution Gallery.
  6. Use the DesignPackageInfo class to install and activate the sandbox solution.

The code to do that looks like this:

using Microsoft.SharePoint.Client;

using Microsoft.SharePoint.Client.Publishing;

using System;

using System.Collections.Generic;

using System.IO;

using System.Linq;

using System.Text;

using System.Threading.Tasks;

using System.Web;

namespace SolutionUploader

{

class Program

{

static void Main(string[] args)

{

try

{

ClientContext context = new ClientContext(“http://%5Burl of site collection] “);

Web web = context.Web;

context.Load(web);

var lib = web.Lists.GetByTitle(“[destination lib]”);

context.Load(lib);

var filePath = @”D:\TestTemplateFolder\TestTemplate.wsp”;

var file = new FileStream(filePath, FileMode.Open);

var fileCI = new FileCreationInformation()

{

ContentStream = file,

Url = “TestSiteTemplate-v1.2.wsp”,

Overwrite = true

};

var uploadedFile = lib.RootFolder.Files.Add(fileCI);

context.Load(uploadedFile);

context.ExecuteQuery();

var wsp = new DesignPackageInfo()

{

// Guid can be empty and is autofilled,

// but specifying it explicitly makes it easier if you want to remove it

// later.

PackageGuid = new Guid(“[GUID]”),

//PackageGuid = Guid.Empty,

MajorVersion = 1,

MinorVersion = 0,

PackageName = “[package name]”

};

string filerelativeurl = “[site relative path of *.wsp] “;

DesignPackage.Install(context, context.Site, wsp, filerelativeurl);

context.ExecuteQuery();

}

catch (Exception ex)

{

Console.WriteLine(ex.Message);

}

}

}

}

You may notice that after installation, the solution is always named like so:

[package name]v[major version].[minor version].wsp

Let’s move on to the JSOM example. In other to write it, there where bits and pieces we’ve copied related to the binary encoding and uploading of files and we’ve tried to give credit where possible, but we’re pretty sure we’re missing an acknowledgement. Rest assured, this wasn’t because of any bad intentions.

We found doing the same thing in JSOM is harder, especially when you’re trying to do it before creating a POC in C#. Part of the reason for this is that the JSOM DesignPackageInfo documentation isn’t really helpful at the time of writing, but it’s here: https://msdn.microsoft.com/en-us/library/office/jj954421.aspx in case you want to take a look.

This example involves an HTML page that uploads the site template and activates it in the Solution Gallery. It goes something like this:

  1. You need a reference to the sp.publishing.js library, because it contains the DesignPackageInfo class.
  2. Include a file control to allow end users to upload the site template.
  3. Use a client-side FileReader object to read the binaries.
  4. Upload the file to a SharePoint library and give it a title. Failing to give the file a title may result in situations where the eventual package name is [a guid consisting of 0’s]v.[major].[minor]. We’ve seen this happen on several occasions and it seems to be a problem that doesn’t happen in C#/CSOM.
  5. Use the DesignPackageInfo class to install and activate the solution.

<!DOCTYPE html>

<html lang=”en” xmlns=”http://www.w3.org/1999/xhtml”&gt;

<head>

<meta charset=”utf-8″ />

<title></title>

<script type=”text/javascript” src=”/_layouts/15/sp.publishing.js”></script>

<script type=”text/javascript” src=”[reference to jquery library]”></script>

</head>

<body>

<h1>Upload en Activeer Solution</h1>

<input id=”inputFile” type=”file” />

<input id=”uploadDocumentButton” type=”Button” value=”Upload Document”/> <p/>

<script>

$(“#uploadDocumentButton”).click(function () {

if (document.getElementById(“inputFile”).files.length === 0) {

alert(“Select a file!”);

return;

}

CreateFile();

});

var file;

var fileCreateInfo;

function CreateFile() {

// Ensure the HTML5 FileReader API is supported

if (window.FileReader) {

input = document.getElementById(“inputFile”);

if (input) {

file = input.files[0];

fr = new FileReader();

fr.onload = receivedBinary;

fr.readAsDataURL(file);

}

}

else {

alert(“The HTML5 FileSystem APIs are not fully supported in this browser.”);

}

}

// Callback function for onload event of FileReader

function receivedBinary() {

var clientContext = SP.ClientContext.get_current();

this.oWebsite = clientContext.get_web();

clientContext.load(this.oWebsite);

var listTitle = “[title of destination library for *.wsp”;

var list = this.oWebsite.get_lists().getByTitle(listTitle);

fileCreateInfo = new SP.FileCreationInformation();

fileCreateInfo.set_url(file.name);

fileCreateInfo.set_overwrite(true);

fileCreateInfo.set_content(new SP.Base64EncodedByteArray());

// Read the binary contents of the base 64 data URL into a Uint8Array

// Append the contents of this array to the SP.FileCreationInformation

var arr = convertDataURIToBinary(this.result);

for (var i = 0; i < arr.length; ++i) {

fileCreateInfo.get_content().append(arr[i]);

}

// Upload the file to the root folder of the document library

this.newFile = list.get_rootFolder().get_files().add(fileCreateInfo);

clientContext.load(this.newFile);

var item = this.newFile.get_listItemAllFields();

item.set_item(“Title”, “[title value]”);

item.update();

clientContext.executeQueryAsync(onUploadSuccess, onUploadFailure);

}

function onUploadSuccess() {

// File successfully uploaded

var clientContext = SP.ClientContext.get_current();

var wsp = new SP.Publishing.DesignPackageInfo();

wsp.set_packageName(“[package name]”);

//wsp.set_packageGuid(“{GUID makes deleting easier}”);

wsp.set_majorVersion(3);

wsp.set_minorVersion(0);

var filerelativeurl = “/[site relative url to *.wsp] “;

SP.Publishing.DesignPackage.install(clientContext,

clientContext.get_site(),

wsp,

filerelativeurl);

clientContext.executeQueryAsync(onActivateSuccess, onActivateFailure);

}

function onUploadFailure() {

// Error occurred

console.log(“Request failed: ” + arguments[1].get_message());

}

function onActivateSuccess() {

alert(“Solution successfully activated!”);

}

function onActivateFailure() {

alert(“Solution activation failed: ” + arguments[1].get_message());

}

// Utility function to remove base64 URL prefix and store base64-encoded string in a

// Uint8Array

// Courtesy: https://gist.github.com/borismus/1032746

function convertDataURIToBinary(dataURI) {

var BASE64_MARKER = ‘;base64,’;

var base64Index = dataURI.indexOf(BASE64_MARKER) + BASE64_MARKER.length;

var base64 = dataURI.substring(base64Index);

var raw = window.atob(base64);

var rawLength = raw.length;

var array = new Uint8Array(new ArrayBuffer(rawLength));

for (i = 0; i < rawLength; i++) {

array[i] = raw.charCodeAt(i);

}

return array;

}

</script>

</body>

</html>

And that’s all folks!

3rd party analytics for SharePoint 2013

In general, ootb analytics do not satisfyingly meet the need of all customers as far as getting insights in the use of sharepoint are concerned. This post provides an overview of vendors that offer additional analytics capabilities for sharepoint or that can be used in combination with sharepoint. The idea behind the article is that it’s a useful starting point for people starting the decisioning process. It also contains a list of useful evaluation criteria that can help to decide which product is right for you. On purpose, we refrain from saying: product x is better than product Y because this depends too much on personal preference and the weight a specific feature is given within your organization.

Here is the list of analytics solutions we have looked at, listed alphabetically.

Adobe Analytics 
Adobe Analytics used to be known as Omniture, but is renamed in 2013. Before Omniture, this product was known as SiteCatalyst and was partly built on a product called Hitbox. According to http://www.slideshare.net/maverickaman/ga-wt-omtr this product is superior to Google Analytics. Adobe Analytics require web analytics data to be stored in the cloud, there is no option to store data on premises. There will certainly be scenarios where this is a deal breaker. Although not a specific sharepoint analytics solution, this tool can be leveraged to provide analytics insights in SharePoint.

Angelfish
Angelfish ( http://analytics.angelfishstats.com), made by Actual Metrics, is an on-premise solution that stores web site traffic in a safe way. Actual Metrics is founded by an ex-employee of Google and supposedly both companies still have strong ties. Angelfish is a generic web analytics solution and replacement of Google Urchin. A big advantage of this solution is that it is able to import Google Analytics data. Although not a specific sharepoint analytics solution, this tool can be leveraged to provide analytics insights in SharePoint.

Cardiolog Analytics Cardiolog Analytics
Cardiolog Analytics Cardiolog Analytics (http://www.intlock.com/intlocksite/productsandservices/cardiolog/cardiolog-analytics.asp) is made by Intlock, a Microsoft Gold Certified partner. This product offers extensive SharePoint web analytics features since 2005 and offers advanced capabilities regarding the analysis of end user interaction.

ControlPoint
ControlPoint (http://www.metalogix.com/Products/ControlPoint.aspx) was originally built by Axceler Inc., then purchased by Metalogix, and seems to be intended for an admin audience, although it does contain a reporting part. In our experience, Metalogics offers extensive product documentation and responds to questions quickly. The ControlPoint product has won several awards (such as Best of TechEd for Productivity and Collaboration) and adheres to several certifications (such as U.S. Navy DADMS Approval).

DocAve Report Center for Microsoft SharePoint Reporting
DocAve Report Center (http://www.avepoint.com/sharepoint-reporting-docave/) of AvePoint provides insights into usage analytics (as well as a lot of other stuff, such as SharePoint performance and security settings). The vendor is well known, although the reports seem to be intended to an admin audience instead of a business audience (because of the nature of the reports, but also because of the amount of privileges required to use report center). One of the interesting features of the product is that it allows admins to detect suspicious activities of end users.

Google Analytics
Google Analytics (http://www.google.nl/intl/nl/analytics/) collects web analytics data but stores them in the cloud. GSA may very well be the de facto web analytics market leader and provides an excellent solution for that, but not every company is willing to hand over their data in custody of Google.

HarePoint Analytics for Microsoft SharePoint 
HarePoint Analytics ( https://www.harepoint.com/Products/HarePointAnalyticsForSharePoint/Default.aspx) is built by a Russian company that has nice looking SharePoint specific reports, extensive documentation, easy installability and a good price/quality ratio.

IBM Digital Analytics Multisite IBM Digital Analytics Multisite
IBM Digital Analytics Multisite IBM Digital Analytics Multisite (http://www-03.ibm.com/software/products/en/enterprise-digital-analytics) is a generic web analytics environment. At the time of writing, the IBM marketing department has not indicated how suitable this product is for SharePoint, but a solution by such a big player is worth mentioning here.

NGAGE
NGAGE Enterprise Analytics (http://www.ngageintelligence.com/products-sharepoint2013.shtml) is an advanced solution built by a relatively small USA based company. What sets NGAGE apart from the rest is that it uses advanced datawarehouse/BI technology to analyze data, fully based on the SharePoint BI suite.

Nintex Analytics
Nintex Analytics (http://en-us.nintex.com/) is a well-known company specialized in building SharePoint products. It supports SharePoint 2010 until July 2015, but does NOT support SharePoint 2013 and according to current plans this product is dead.

Piwik SharePoint Analytics Piwik
Piwik SharePoint Analytics Piwik (http://piwik.org/) is a French initiative and the facto standard for open source web analysis tools. Piwik has been downloaded over 2 million times, contains an impressive number of features and is extremely customizable. Piwik SharePoint Analytics is built on top of Piwik and is a SharePoint specific solution. Piwik relies heavily on the use of cookies and javascript. The fact that Piwik is open source might imply low license costs, but the costs of Piwik SharePoint Analytics (as part of Piwik PRO Enterprise), onsite training, custom development and on site services will be considerable. So far, we have the impression that the effort for installing and configuring the product is considerable, but we may have to experiment further. Another factor to consider is that Piwik leverages technologies that are atypical in the average sharepoint environment (think technologies such as PHP v5.4 or higher, and MySQL v4.1 or higher).

Unilytics Mergence SharePoint Analytics Mergence
Unilytics Mergence SharePoint Analytics Mergence (http://www.unilytics.com) is a product of Unilytics Corporation and a specific analytics solution for SharePoint. It seems that Mergence is not a stand-alone product, but one that integrates with several well-known web analytics products such as WebTrends, Omniture and Google Analytics. At the time of writing, we got an HTTP 404 – Page Not Found error when navigating to Products > Mergence for SharePoint Analytics, so maybe this product is dead.

Site Administrator for SharePoint 
Site Administrator for SharePoint (http://software.dell.com/products/site-administrator-for-sharepoint/) is a SharePoint monitoring and reporting product originally built by Quest and now owned by Dell. This product is aimed towards admins, but offers enterprise-wide discovery and fine grained reports about the entire SharePointnvironment. Site Administrator for SharePoint has won the Gold Community Choice award of 2013 by Windows IT Pro.

WebTrends SharePoint Analytics
WebTrends SharePoint Analytics (https://www.webtrends.com/products-solutions/analytics/sharepoint-analytics-use-cases/ ) is a web analytics pioneer and offers a SharePoint specific analytics solution that is used by more than 300 major enterprises and was developed keeping close contact with the Microsoft engineering and product teams. WebTrends seems to be the market leader as far as SharePoint Analytics go and the product has one several awards and recognitions (such as Industry Leader in Web Analytic volgens Forrester Web Analytics Wave 2014 and Member van de Microsoft SharePoint Partner Advisory Council).

When evaluating 3rd party solutions, we found the following list of criteria useful:
– What are the license costs?
– What are the software requirements and what is the architectural impact on your SharePoint farm?
– Is it possible to store data on premises for privacy reasons?
– Is it possible to view the number of visitors per site collection/site/list item?
– Is it possible to present data anonymously (without displaying user names) or even possible to store analytics data anonymously?
– Is it possible to view the Last page request within a session?
– Is it possible to define which audience sees reports (such as admins, site owners or normal end users)?
– Is it possible to export reports to other formats such as PDF, Excel or extract report data via a web service?
– What’s the status of the vendor in the market?
– Which customers are using the product?
– What are the response times of support and what is the quality of the answers provided?
– What’s the product’s ease of use?
– Is it a SharePoint specific or generic solution?
– Is it good at protecting the analytics data?
– Is it possible to configure fine-grained security permissions for specific reports?
– Does it have advanced BI capabilities (such as trend prediction)?
– Does it offer goal-driven paths functionality (that allows the report admin to specify goals about end user interaction, such as the number of clicks needed to place a comment)?
– Does it offer report drill-down capabilities?
– Does it allow to create custom reports?
– Does it offer real-time reporting capabilities?
– Does it allow filtering during data collection?
– How does it impact the performance of the SharePoint farm?
– Does it provide reports about social behavior within SharePoint?
– Does it integrate with analytics data of social platforms such as Yammer or other sources of analytics data?
– Does it have pruning capabilities (that allows one to remove historic data because it is no longer relevant)?
– Does it support Spike Analysis (to analyze a very specific period of a couple of minutes during which sharepoint was extremely busy or lots of errors occurred)?
– Does it support heat mapping (to analyze which parts of a page are more popular so that the user interface can be improved)?
– Does it support event tracking (to track specific events within a web site, tracking typically occurs via JavaScript)?
– Does it support Visitor segmentation (the possibility to divide visitors in groups based on interests)?
– Does it support Annotations (that allows you to add notes to specific reports)?

SharePoint WPSC is undefined

In a SharePoint 2013 environment we encountered the error “WPSC is undefined”. The WPSC (Web Part Page Services Component) is a JavaScript library that web parts use to communicate with each other client-side. We don’t feel the WPSC used that often nowadays (if that ever was the case), so maybe that explains why we couldn’t find that much info about this problem. The most interesting post was this one: http://sadomovalex.blogspot.nl/2010/02/fix-wpsc-is-undefined-javascript-error.html

But it talks about an issue in combination with FBA, and our problem existed on a page leveraging WPSC for client-side communication when using MSIE 10 or higher (whereas MSIE 9 worked fine). Also, in normal mode the error existed in MSIE 10 and up, in page edit mode it went away. There’s another post worth mentioning, and its this one: http://msharaky.blogspot.nl/2009/06/wpsc-undefined-javascript-error.html

Here, the author defines a dummy object and reports the error goes away. This is true of course, but also kills client-side communication. But, if you’re just looking for a way to suppress an ugly error message, this one does the trick.

To understand more about the problem, it’s worth mentioning that the WPSC object is defined in a JavaScript library called ie55up.js which is included by default. So, if you’ve changed your master pages and its not referenced anymore, then there’s the place where it needs to be corrected.

Another useful thing to know is that if you’re using the SharePoint ScriptLink control to include a JavaScript library into your page, the OnDemand attribute determines if such a library will be included on demand as needed, or immediately. The following line ensures a JS library is loaded on demand:

<SharePoint:ScriptLink ID=”scriptLink1″ runat=”server” Name=”MyScriptLib.js” LoadAfterUI=”true” OnDemand=”true” />

If the js library needs to be loaded immediately, the control just generates a script tag. If the js library needs to be loaded on demand, it generates calls to the SharePoint RegisterSOD() JavaScript function. This is interesting because a SharePoint page in edit mode loads the ie55up.js immediately (and thereby the WPSC), using the following client-side code:

<script type=”text/javascript” src=”/_layouts/15/ie55up.debug.js?rev=pVBnO13dp7gFq%2FZalDmroA%3D%3D”></script>

Remember that the page in edit mode works correctly in conjunction with WPSC.

Note: the script tag refers to a debug version of ie55up.js instead of the minified version. That is because we have configured SharePoint to do this by setting the debug attribute in the web.config file to true.

On the other hand, a page in normal mode that uses the WPSC but fails in MSIE 10 and up renders the ie55up.js library to be loaded on demand, like so:

<script type=”text/javascript”>RegisterSod(“browserScript”, “\u002f_layouts\u002f15\u002fie55up.debug.js?rev=pVBnO13dp7gFq\u00252FZalDmroA\u00253D\u00253D”);RegisterSodDep(“browserScript”, “strings.js”);</script>

Apparently there is some on demand script problem because in which case the ie55up library is registered successfully, but never gets loaded in MSIE 10 and up. Btw, the root cause of this we have yet to discover.

Another thing to note is that the RegisterSod() function is defined in init.js and looks like this:

function RegisterSod(key, url) {
    ULSxSy: ;
    key = NormalizeSodKey(key);
    var sod = new Sod(url);
    _v_dictSod[key] = sod;
}

Later on, we’ll be able to leverage the facts that all keys are stored in the _v_dictSod dictionary and ie55up.js gets registered using the “browserScript” key to our advantage.

In order to experiment with this problem, we needed an easy way to add JavaScript code to a SharePoint page that uses the WPSC. We decided to place a Script Editor web part on a page (page edit mode > ribbon > Insert > Embed Code) that uses JavaScript that uses WPSC to register for a predefined event that happens when a user presses F1 to get help. All that’s needed to determine when and if the WPSC problem is an issue for you is this code:

<script>
WPSC.RegisterForEvent(“urn:schemas-microsoft-com:dhtml”, “onhelp”, myfunc); // respond to predefined event for pressing F1
</script>

Now our approach for dealing with the “WPSC is undefined” problem has 2 sides. If the WPSC object exists (in older browsers and page edit mode for msie 10 and up) we need to do the following:
1 – Establish that the WPSC exists.
2 – Do required communication via WPSC. In this case, we’ll use the WPSC to respond to an F1 key press.

In situations where the WPSC doesn’t exist, our approach is like this:
1 – Establish that the WPSC object does NOT exist.
2 – Find the URL that SharePoint intends to use to load ie55up.js.
3 – Use the SP.SOD library to load ie55up.js on demand.
4 – Establish that ie55up.js is indeed loaded.
5 – Do client-side communication via WPSC. In this case, we’ll use the WPSC to respond to an F1 key press.

This results in code that can be used as a work-around for situations where the WPSC is undefined, making sure its loaded on demand and then do the things you want to do with a fully functioning WPSC. You can paste the following code in a Script Editor web part on a SharePoint page to see it in action (after pressing F1):

<script>
function myfunc()
{
  alert(‘Responding to help!!!’);
}

function WpscLoaded()
{
  execWpscCalls();
}

function execWpscCalls()
{
  WPSC.RegisterForEvent(“urn:schemas-microsoft-com:dhtml”, “onhelp”, myfunc); // respond to predefined event for pressing F1
}

if (typeof(WPSC) == “undefined” )
{
  // default src of js lib defining WPSC
  var url = “\u002f_layouts\u002f15\u002fie55up.js”;

  // get url that sharepoint stores to refer to WPSC lib
  if (_v_dictSod != null && _v_dictSod[“browserScript”] != null)
  {
    url = _v_dictSod[“browserScript”].url;
  }
  SP.SOD.registerSod(‘ie55up.js’, url);
  SP.SOD.executeFunc(‘ie55up.js’, null, function(){});

  // when done, WPSC is available 
  ExecuteOrDelayUntilScriptLoaded(WpscLoaded, “ie55up.js”);
}
else
{
  execWpscCalls();
}
</script>