We certainly don’t consider ourselves to be lazy, but we’ll admit we had some hesitations writing this article because of the amount of work it involved. Eventually we went ahead and wrote it, and since we weren’t planning to write an article the size of a book chapter we tried to keep it as succinct as possible. As a down side, this does mean that you, the reader, must possess basic SharePoint 2010 knowledge as well as some WCF knowledge. This article is about using .NET 4 and up (in particular the TPL, the Task Parallel Library) in combination with SharePoint 2010 and it’s object model by invoking several WCF services, so if you’re interested in doing that you should read on.
Laziness
But first this: While we’re on the subject of laziness, for your amusement we took the effort to do some research to find out who in fact are at the top and the bottom of the laziness chain on this earth. As it turns out, the brother of Tammy G is the laziest person in the world because he is 40 years old, has two kids, still lives with his mommy, lets her pay all the bills, and also lets her take him anywhere because he has no drivers license (http://answers.yahoo.com/question/index?qid=20060918224104AAOSnHr). Albeit posthumously, it wouldn’t feel right to give the title of “hardest working person” to anybody else but James Brown.
In SharePoint world, it was a bit easier for us to decide. Sahil Malik (http://blah.winsmarts.com/) has repeatedly stated that he’s the laziest person you’ll ever meet (that is, if you stay away from Tammy G’s brother), so he full well deserves the honorary title of being the laziest person in SharePoint world (that, as well as being one of the most talented guys around when it comes to SharePoint). The hardest working individual may just be Mindsharp’s Todd Bleeker (http://sharepoint.mindsharpblogs.com/todd/default.aspx). For SharePoint 2010, we contributed to some courseware he wrote and in order to get things done in time he adopted a US commando life style for a couple of months that only allowed him 2 hours of sleep every night which enabled him to get more work done. It’s this kind of controlled craziness that makes us nominate him for the honorary title of being the hardest working person in SharePoint world.
On with the story…
To us, the most irritating thing about SharePoint 2010 is the fact that it doesn’t play nice with the .NET 4 framework. The main problem is that you can’t call the object model from a .NET 4 assembly although there are other problems as well, such as the inability to build SharePoint workflows using WF 4. It’s easy to demonstrate the problem that arises when you try to call the SharePoint object model from a .NET 4 assembly:
using (SPSite site = new SPSite(“http://moon/sites/test1″))
{
…
}
This results in the following error message:
“Microsoft SharePoint is not supported with version 4… of the Microsoft .Net Runtime.”
So there you have it. One good thing though, the error message is clear enough, you can’t say the same when you’re building a .NET 3.5 assembly with the wrong (default) target platform of x86. If you run the same code, once you hit the line that opens the site collection, you get the following exception:
The web application at [url] could not be found. Verify that you have typed the URL correctly…
Not the best way of letting you know that the target platform should be x64 instead of x86. In our experience, this is a common mistake for developers new to SharePoint 2010, and although this exception is not really the kind of help one would be looking for, it’s a mistake one probably only makes once.
Please note: As our gentle reader will no doubt understand, we would never make such a foolish mistake, we’ve just heard rumors about other people who did it wrong.
Although we totally appreciate and understand the fact that because of timelines and stuff it was impossible for the SharePoint team to include .NET 4 support in the object model, it’s annoying nonetheless. What’s more, already this has been a point of confusion for many developers. With the developer release of Windows 8 and .NET 4.5, things become even worse. Let’s just take the improvements for parallel programming in .NET 4.5 (see http://blogs.msdn.com/b/pfxteam/archive/2011/09/17/10212961.aspx): .NET technology keeps evolving at a rapid pace and the developer possibilities in SharePoint 2010 already start to fall behind the curve.
I don’t want to be left behind, give me the “Back to the Future” pattern!
.NET 4 has lots of features that, at some point, you may desperately want to use. If you really have a compelling case, there is always the option of building a WCF 4 service and call that from a SharePoint 2010 client. This way, you let the WCF 4 service handle the .NET 4 stuff.
| For fun, let’s call this the “Back to the Future” pattern, based on one of our favorite 80s movies where Doc Brown invents time travel and Marty McFly (Michael J. Fox) gets stuck in the past and has to find a way to get back to the future. This is kind of the situation where SharePoint 2010 forces us in so we thought the name appropriate. |
 |
In the rest of the article we’ll be using .NET 4 parallel programming in a way that based on our performance tests looks to be turning out really well. In this scenario we’ve created a web part that calls a WCF 4 service (which we’ll call the middle tier service) that uses parallel programming to invoke a WCF 3.5 service that runs in the context of SharePoint (we’ll call this one the end tier service).
Please note: You can download Reactive Extensions for .NET 3.5 SP1 from http://msdn.microsoft.com/en-us/devlabs/ee794896.aspx. It contains the System.Threading.dll assembly that contains TPL for .NET 3.5. If you’re happy with that version, you could use that instead, but that’s not really the point of this article. What we need is a way that allows us to leverage all TPL stuff, including the new .NET 4.5 features.
What will we be doing?
Currently we’re working on a very interesting project that requires us to load large amounts of log files into memory, which allows us to perform advanced statistical analysis on this information. For this specific article, we’ve tweaked the requirement a bit and we’ll only be uploading a couple of files to a SharePoint document library. In order to do this in as little time as possible we’ll be leveraging the .NET 4 TPL. To demonstrate our “Back to the Future” pattern, the solution will consist of the following pieces:
- A visual web part that contains a button that triggers the upload of a predefined set of files located on the server.
- A WCF 4.0 service (the middle tier service) that leverages the .NET 4 TPL to make sure all files will be uploaded in a way that is much faster than running sequential code.
- A WCF 3.5 service running in SharePoint context (the end tier service). This is the service that eventually is responsible for uploading a file to a SharePoint document library.
We’ll discuss these list items starting backwards, ending with the visual web part piece.
How to upload and download stuff
Since we’re eventually planning to upload files to a SharePoint doc library, we’ll get that part out of the way by showing a method that can be incorporated in a console application that uploads stuff:
static void Upload()
{
List<string> files = new List<string> { “1.log”, “2.log”, “3.log”, “4.log”, “5.log”, “6.log”, “7.log”, “8.log” };
foreach (string file in files)
{
string path = String.Format(@”c:\temp\temp\{0}”, file);
string url = default(string);
using (SPSite site = new SPSite(“http://moon/sites/test1″))
{
using (SPWeb web = site.OpenWeb())
{
SPList list = web.Lists[“TestLib”];
Stream s = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read);
SPFolder folder = list.RootFolder;
bool overwrite = true;
url = String.Format(“{0}/{1}/{2}”, web.Url, list.RootFolder.Url, folder.Files.Add(file, s, overwrite).Name);
}
}
}
}
Because we’re into symmetry we’ll also be showing code that allows you to download stuff from SharePoint. Try something like this:
public void Download(string url, string fileName)
{
WebClient client = new WebClient();
NetworkCredential creds = new NetworkCredential(“[user name]”, “[password]”, “[domain]”);
client.Credentials = creds;
//client.Credentials = CredentialCache.DefaultCredentials;
string dest = @”c:\temp\download\” + fileName;
client.DownloadFile(url, dest);
}
Building the end tier service
Let’s start with the creation of the end tier service: this is the service that runs in a SharePoint context and is responsible for uploading log files. Basically, this part is a synopsis of MSDN article “Creating a Custom WCF Service in SharePoint Foundation” (http://msdn.microsoft.com/en-us/library/ff521581(d=printer).aspx). We recommend that you read the article and do the following:
- Create an empty SharePoint project and call it ParallelServiceHost instead of RevertService.
- Create the interface for the end tier WCF service running in SharePoint context. This interface will only contain a single upload method that receives the name of a file located on the server that will be uploaded to a SharePoint document library. Add a file called IParallelService.cs that looks like this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.Text;
namespace WcfService
{
[ServiceContract]
public interface IParallelService
{
[OperationContract]
string Upload(string file);
}
}
- Implement the parallel service. Create an Upload method that runs in SharePoint context, and uploads a file from a predefined location on the server. Add a file called ParallelService.cs that looks like this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.Text;
namespace WcfService
{
using Microsoft.SharePoint.Client.Services;
using System.ServiceModel.Activation;
using Microsoft.SharePoint;
using System.IO;
using System.Net;
[BasicHttpBindingServiceMetadataExchangeEndpointAttribute]
[AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Required)]
public class ParallelService : IParallelService
{
public string Upload(string doc)
{
string path = String.Format(@”c:\temp\temp\{0}”, doc);
string url = default(string);
SPList list = SPContext.Current.Web.Lists[“TestLib”];
Stream s = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read);
SPFolder folder = list.RootFolder;
bool overwrite = true;
url = String.Format(“{0}/{1}/{2}”, SPContext.Current.Web.Url, list.RootFolder.Url, folder.Files.Add(doc, s, overwrite).Name);
return url;
}
}
}
- Add a registration file for this service called Parallel.svc to the SharePoint ISAPI folder. Parallel.svc looks like this:
<%@ServiceHost Language=”C#” Debug=”true”
Service=”WcfService.ParallelService, $SharePoint.Project.AssemblyFullName$” Factory=”Microsoft.SharePoint.Client.Services.MultipleBaseAddressBasicHttpBindingServiceHostFactory, Microsoft.SharePoint.Client.ServerRuntime, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c” %>
- Make sure token replacement for .svc files is enabled (step 12 in the article).
- Deploy everything which causes the .svc file to be copied to the ISAPI folder of the SharePoint root folder. It also copies the dll of ParallelService to the GAC.
Please check that the MEX endpoint can be reached at:
http://moon/TestLib/_vti_bin/parallel.svc/mex
If not, it’s back to the drawing board for you. Chances are that your server isn’t named “moon”, so just use your server name or “localhost” instead.
Building the middle tier service
We have the end tier service in place, a service that runs in the context of SharePoint and knows how to upload files to document libraries. Now it’s time to create the middle tier service that leverages the TPL to upload files in parallel.
- Start with creating a WCF 4 service. We’ve just named it Upload4Service to indicate that this is the .NET 4 service.
- Later on, we’ll create an Upload method that receives a single input parameter that contains a collection of all the files we want uploaded. The definition of this input parameter looks like this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.Serialization;
namespace Upload4Service
{
[DataContract(Namespace = “http://schemas.loisandclark.eu/2011/9/Parallel/Upload/”)]
public class UploadRequest
{
[DataMember(IsRequired = true)]
public List<string> Files { get; set; }
}
}
- The aforementioned Upload method returns an UploadResponse containing all new SharePoint locations that looks like this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.Serialization;
namespace Upload4Service
{
[DataContract(Namespace = “http://schemas.loisandclark.eu/2011/9/Parallel/Upload/”)]
public class UploadResponse
{
[DataMember(IsRequired = true)]
public List<string> SharePointLocations { get; set; }
}
}
- If exceptions arise, we’ve created our custom exception that’s suitable for WCF. Please note that because we’re using parallel programming it contains a list of error messages, because multiple tasks might encounter errors. The exception class looks like this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.Serialization;
namespace Upload4Service
{
[DataContract(Namespace = “http://schemas.loisandclark.eu/2011/9/Parallel/”)]
public class FaultInfo
{
[DataMember(IsRequired = true)]
public List<string> Messages { get; set; }
}
}
- The contract of the .NET 4 upload service looks like this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.Text;
namespace Upload4Service
{
[ServiceContract]
public interface IUpload
{
[OperationContract]
[FaultContract(typeof(FaultInfo))]
UploadResponse Upload(UploadRequest request);
}
}
- Our *.svc file, Upload4Service.svc, uses a code-behind file instead of a DLL and looks like this:
<% @ServiceHost Language=”C#” Debug=”true” Service=”Upload4Service.UploadService” CodeBehind=”~/App_Code/UploadService.cs” %>
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Diagnostics;
using System.Net;
using System.Collections.Concurrent;
namespace Upload4Service
{
public class UploadService : IUpload
{
public ConcurrentQueue<string> UploadResults { get; set; }
public UploadResponse Upload(UploadRequest request)
{
try
{
UploadResults = new ConcurrentQueue<string>();
Parallel.ForEach(request.Files, ParallelUpload);
UploadResponse response = new UploadResponse();
response.SharePointLocations = UploadResults.ToList();
return response;
}
catch (AggregateException ae)
{
FaultInfo fi = new FaultInfo();
fi.Messages = new List<string>();
fi.Messages.Add(“Failed to upload files to SharPoint”);
foreach (Exception err in ae.InnerExceptions)
{
fi.Messages.Add(err.Message);
}
throw new FaultException<FaultInfo>(fi);
}
}
private void ParallelUpload(string file)
{
string websiteUrl = “http://moon”;
// Set up proxy.
BasicHttpBinding binding = new BasicHttpBinding();
binding.Security.Mode = BasicHttpSecurityMode.TransportCredentialOnly;
binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.Ntlm;
EndpointAddress endpoint =
new EndpointAddress(websiteUrl + “/_vti_bin/Revert.svc”);
ServiceReference1.RevertClient proxy = new ServiceReference1.RevertClient(binding, endpoint);
proxy.ClientCredentials.Windows.AllowedImpersonationLevel =
System.Security.Principal.TokenImpersonationLevel.Impersonation;
string result = proxy.Upload(file);
UploadResults.Enqueue(result);
}
}
}
As it turns out, adding a new application to a SharePoint web site and assigning it it’s own .NET 4 application pool is just not worth the effort. This results in conflicting web.config settings (with on the one hand, settings inherited from SharePoint, and on the other hand, settings automatically generated by .NET 4). Therefore, we advise to create a new web site dedicated for your WCF 4 services and add a new application to that web site. Then, do the following:
- Place the Upload4Service.svc in the application folder.
- Add a web.config file, this is ours (forget the named pipe stuff as it’s not relevant for this article):
<?xml version=”1.0″ encoding=”utf-8″ ?>
<configuration>
<system.serviceModel>
<trust level=”Full” originUrl=”” />
<services>
<service name=”Upload4Service.UploadService”>
<endpoint address=”” binding=”netNamedPipeBinding”
contract=”Upload4Service.IUpload” />
<endpoint address=”mex” binding=”mexNamedPipeBinding” contract=”IMetadataExchange” />
<host>
<baseAddresses>
<add baseAddress=”net.pipe://moon.lc.corp:8081/MyService/” />
</baseAddresses>
</host>
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior>
<!– To avoid disclosing metadata information,
set the value below to false and remove the metadata endpoint above before deployment –>
<serviceMetadata httpGetEnabled=”True”/>
<!– To receive exception details in faults for debugging purposes,
set the value below to true. Set to false before deployment
to avoid disclosing exception information –>
<serviceDebug includeExceptionDetailInFaults=”True” />
</behavior>
</serviceBehaviors>
</behaviors>
</system.serviceModel>
<system.web>
<customErrors mode=”Off”/>
</system.web>
</configuration>
- Create an App_Code folder and add IUpload.cs and UploadService.cs in it (or create a bin folder and add the DLL to it).
There you have it, a WCF 4 service that leverages the TPL to perform some uploading. At this point, a very important take away is that the end tier service operation that is invoked by the WCF 4 middle tier service runs under the identity of the application pool of the WCF 4 service. In our case, this is lc\administrator, which maps to the special SharePoint System Account (SHAREPOINT\system). Make sure that, whatever account you choose to use, it has sufficient rights to upload files to the document library.
Building the visual web part
Now, we need a SharePoint client that communicates with the middle tier service. We’ve chosen to create a visual web part. That is interesting, because even sandboxed solutions are able to communicate with WCF services. Do the following:
protected void Button1_Click(object sender, EventArgs e)
{
UploadClient upload4 = new UploadClient();
//upload4.ClientCredentials.UserName.UserName = @”lc\administrator”;
//upload4.ClientCredentials.UserName.Password = “mysecret”;
UploadRequest uploadRequest = new UploadRequest()
{
Files = new List<string> { “1.log”, “2.log”, “3.log”, “4.log”, “5.log”, “6.log”, “7.log”, “8.log” }
};
UploadResponse uploadResponse = upload4.Upload(uploadRequest);
foreach (string loc in uploadResponse.SharePointLocations)
{
Label1.Text += loc;
}
Because you’ve added a service reference, the app.config file contains everything you need to know to call the middle tier service.The problem is that SharePoint doesn’t have this info. Therefore, you need to open the web.config file of your SharePoint web application and locate the <system.serviceModel> section. If you haven’t altered it before, it looks like this:
<system.serviceModel>
<serviceHostingEnvironment aspNetCompatibilityEnabled=”true” />
</system.serviceModel>
Add the stuff from your app.config file to the SharePoint web.config file. In our case, this looks like:
<system.serviceModel>
<serviceHostingEnvironment aspNetCompatibilityEnabled=”true” />
<bindings>
<wsHttpBinding>
<binding name=”WSHttpBinding_IUpload” closeTimeout=”00:01:00″ openTimeout=”00:01:00″ receiveTimeout=”00:10:00″ sendTimeout=”00:01:00″ bypassProxyOnLocal=”false” transactionFlow=”false” hostNameComparisonMode=”StrongWildcard” maxBufferPoolSize=”524288″ maxReceivedMessageSize=”65536″ messageEncoding=”Text” textEncoding=”utf-8″ useDefaultWebProxy=”true” allowCookies=”false”>
<readerQuotas maxDepth=”32″ maxStringContentLength=”8192″ maxArrayLength=”16384″ maxBytesPerRead=”4096″ maxNameTableCharCount=”16384″ />
<reliableSession ordered=”true” inactivityTimeout=”00:10:00″ enabled=”false” />
<security mode=”Message”>
<transport clientCredentialType=”Windows” proxyCredentialType=”None” realm=”” />
<message clientCredentialType=”Windows” negotiateServiceCredential=”true” algorithmSuite=”Default” />
</security>
</binding>
</wsHttpBinding>
</bindings>
<client>
<endpoint address=”http://moon.lc.corp:8081/MyService/Upload4Service.svc” binding=”wsHttpBinding” bindingConfiguration=”WSHttpBinding_IUpload” contract=”ServiceReference1.IUpload” name=”WSHttpBinding_IUpload”>
<identity>
<userPrincipalName value=”administrator@lc.corp” />
</identity>
</endpoint>
</client>
</system.serviceModel>
Now you can go ahead and add the visual web part to a page and happily call the middle tier service to do some parallel task processing.
That’s a wrap!
The good news is that if you really need/want to use .NET 4 in combination with SharePoint 2010, you can. The bad news is that it initially requires you to set up an infrastructure to allow for this.
SharePoint Dragons 