SharePoint Dragons

Nikander & Margriet on SharePoint

Tag Archives: sharepoint2010

SharePoint Online or not Online: that’s the question

We liked this blog post that discusses pros and cons of Office 365 vs SharePoint on premises: http://cloudcomputingtopics.com/2012/05/office-365-or-on-premise-which-one-is-right-for-me/

Advertisements

Send me my password

In the forums, a user asked why SharePoint didn’t send the user name and password via e-mail. For a moment, our immediate was: SharePoint doesn’t send out user names and passwords, now does it? After looking it up it turns out that only SharePoint Foundation 2010 in Active Directory Account Creation Mode does: http://dishasharepointworld.blogspot.in/2011/04/active-directory-account-creation-mode.html and http://tristanwatkins.com/index.php/active-directory-account-creation-mode-sharepoint-2010/

Uploading a file using the SharePoint client object model

bitUploading a file using the SharePoint client object model is easy, once you have loaded a reference to the SharePoint list you want to upload the file to. One strange thing though, via the Files collection of the List RootFolder, you can directly upload the file to any subfolder you like. It goes like this:

// FileInfo in System.IO namespace
var fileCreationInformation = new FileCreationInformation();

byte[] bytefile = System.IO.File.ReadAllBytes(“c:\\test\Test2.txt”);
fileCreationInformation.Content = bytefile;
fileCreationInformation.Overwrite = true;
fileCreationInformation.Url = “http://astro/MyLibrary/MyFolder/Test2.txt”;

// CurrentList is a client OM List object
CurrentList.RootFolder.Files.Add(fileCreationInformation);
Context.ExecuteQuery();

 

The great thing about it is that you can use it to upload files in batches. But, that’s not the end of the story. If you do this, you must realize that the SharePoint client OM will use Base64 Encoding. According to WikiPedia (http://en.wikipedia.org/wiki/Base64), the final size of Base64-encoded binary data is equal to 1.37 times the original data size + 814 bytes (for headers). By the way, this is not our experience at all. In our testing, we’ve consistently found the message size to be considerably lower than the equivalent file size on the file system.

Things become more interesting… Check http://support.microsoft.com/kb/2529243 , it describes how to uploading of files larger than 3 MB fails (we did notice that), returning a WebException containing a response property that gives a (400) Bad Request error. This is caused by default internal restrictions on file sizes and timeout settings. Further more, the article says that,unless your files are really huge (like 1 GB) you can catch and ignore it and will notice that the files are uploaded just the same. The last thing doesn’t happen for us, it just fails to upload files larger than 3 MB.

But wait… There seems to be a solution. Article http://blogs.msdn.com/b/sridhara/archive/2010/03/12/uploading-files-using-client-object-model-in-sharepoint-2010.aspx explains how to increase the allowed message batch size using the managed client object model. It does use the server object model to accomplish this though, so be aware of this fact if you’re building a client tool that uploads files using the client OM (and please make sure to target it at x64, otherwise SPWebService.ContentService will return null). See http://msdn.microsoft.com/en-us/library/ff599489.aspx for more info. According to our testing, the SharePoint limit applies to the original file system size, not to the size of the file after Base64 Encoding it. Both articles fail to mention that you need to perform an IISRESET before changes take effect, but after doing so things will be alright.

If you’re interested in seeing this in action, check out the Migration Dragon for SharePoint 2010 at http://gallery.technet.microsoft.com/The-Migration-Dragon-for-628acae0 . It uses the SharePoint client object model to upload file and folder structures from the file system to SharePoint 2010 in batches using techniques discussed here.

Can I leave the lights on?

Somebody wanted to know what would happen if:

  1. You add a user to an AD group
  2. You give SharePoint site access to each member of this group
  3. The end user accesses the SharePoint site
  4. You remove a user from a group
  5. The end user doesn’t log off from his desktop and maintains the token he got from AD
  6. The next day, the end user tries to access the SharePoint site again.

As you would expect, the end user won’t be able to access the SharePoint site. In Windows authentication mode, every time an end user is validated a session cookie is set containing an MSCSAuth ticket ( http://msdn.microsoft.com/en-us/library/ee796739(v=cs.20).aspx ). No explicit expiration data is set for this cookie, and the cookie will be deleted automatically after the session expires. This happens after a period of inactivity. This time out period is 20 minutes by default and is set in the TimeOut property of the SessionStateSection class ( http://msdn.microsoft.com/en-us/library/ms691403(v=vs.90).aspx ).

SharePoint 2010 Best Practices

It’s Friday the 13th. Now is as good as any day to update your SharePoint best practices knowledge, because the TN Wiki SharePoint 2010 Best Practices page is updated again. Check out the latest version at http://social.technet.microsoft.com/wiki/contents/articles/8666.sharepoint-2010-best-practices-en-us.aspx

Setting up a playground

If you want to experiment with SharePoint 2010 and have access to a Windows Server 2008 R2 (64bit) machine, you can download the Information Worker Virtual Machine ( http://www.microsoft.com/en-us/download/details.aspx?id=27417 ). It contains everything you need:

  • SPF 2010 and.or SPS 2010
  • SPD 2010
  • Office 2010
  • Visual Studio.NET 2010
  • .NET framework 4.0
  • Expression Blend
  • SQL Server (Express) 2008

The Migration Dragon for SharePoint 2010 – Migrate from the File System to SharePoint 2010

It is quite common that, during SharePoint implementations, end users want to migrate file and folder structures located on the file system to a SharePoint document library. Out of the box, you can do this by uploading multiple documents or via the Windows Explorer view. Both ways are quite easy to use but have several disadvantages:

  • When migrating larger amounts of data both methods are not really reliable. Failure usually means starting all over again.
  • Both methods don’t offer adequate insight on current progress.

Another approach would be to use third party migration tools. There are very good ones out there, but they have disadvantages too:

  • Obviously, they cost money.
  • They’re usually less easy to use (compared to the previous method).
  • They’re typically used by administrators, which leaves end users wanting to migrate files on the file system themselves in the cold.

Another approach would be to use the Microsoft SyncToy (https://sharepointdragons.com/2012/04/10/synctoy/) tool. SyncToy is a tool that synchronizes files and folders between locations using the Microsoft Sync Framework 2.0. Luckily, by mapping a network drive pointing to a SharePoint document library (http://grounding.co.za/blogs/neil/archive/2008/08/02/using-synctoy-to-synchronize-offline-sharepoint-documents-on-vista.aspx), you can use SyncToy to migrate files from the file system to SharePoint 2010. When doing this, SyncToy leverages the SharePoint Sync Framework API (http://msdn.microsoft.com/en-us/library/ee538641.aspx) to sync the file system to the SharePoint list. Or, to put it in other words, SharePoint 2010 has its own sync provider which can be consumed by clients (such as SyncToy and most notably, SharePoint Workspace 2010): http://www.chakkaradeep.com/post/SharePoint-2010-ListsGetListItemChangesWithKnowledge-Method.aspx. The SyncToy has disadvantages too:

  • It synchronizes files and folders (two-way copy), that’s something quite different from migrating files and folders (one-way copy). For larger structures, this adds considerable overhead and complexity. You should avoid synchronization, unless that’s really what you’re after.
  • It requires a network drive mapping. Most end users won’t know how to do this, and usually in larger organizations end users won’t be allowed to create them anyway. For such organizations it simply won’t be feasible to let admins create network drive mappings whenever end users need to migrate something to a SharePoint library.
  • It’s easy for end users to make a mistake. For example, renaming or deleting files in the SharePoint library may have severe consequences for the original file system after sync’ing both locations anew, without the end user realizing this.

That’s were the Migration Dragon for SharePoint 2010 comes in. Its first and current incarnation is an experimental one and it’s based on a simple idea: what would happen if a tool leverages the SharePoint managed client object model to upload files to SharePoint? The client OM has the unique ability to batch commands, so it’s very possible that the results will be remarkably good.

So, combining that with my current preferences about what a migration tool should do, Migration Dragon for SharePoint 2010 offers the following features:

  • It allows you to specify the batch size of files that are to be uploaded. For example, if you specify a batch size of 20 MB, the tool will create a batch of files up to 20 MB and then upload that batch in one go before proceeding to the next batch.
  • It allows you to specify the max allowed batch size on the server. By default, the max allowed batch size is 3 MB for the client object model which isn’t a very useful amount. If you try to upload a batch of 20 MB of files, while the server only accepts 3 MB at a time, such requests will fail miserably.
  • It’s a tool that can be used by clients. Because it uses the SharePoint client object model, the tool doesn’t need to be executed on the server (except for the part that sets the server max batch size, that functionality is not available on a client because it leverages the SharePoint server object model).
  • The tool tracks which batches failed to upload correctly. After processing every batch, it will retry to upload failed batches. In order to do so, it switches strategy. Instead of uploading the entire failed batch, which will likely fail again, it will upload each file within the failed batch individually.
  • After retrying all failures, the tool provides feedback on the files that were unable to be uploaded, even after retrying.
  • It provides accurate updates about the progression informing the end user about the number of folders that have been processed and the amount of data that has been processed. One of the ways that the tool divulges this info is in the form of progress bars, so it’s made quite clear how much work there is still left to do. This is shown in the next Figure.

    image

  • It translates most of the problematic characters/combinations in file and folder names. Problematic that is, for web environments. This includes the following tricky characters:  ~ # % & * : < > ? / { | }. I say most, because I’m not correcting some faulty combinations (such as a file name starting with dot (.), e.g. .MyFileName.txt).

Let’s take a look at the Migration Dragon for SharePoint 2010:

image

Max Batch Size

Let’s talk about the max batch size some more. The default max batch size in the tool is 10 MB, which means that all files on the file system will be packaged in a batch of up to 10 MB. That is, unless you have files that are larger than the max batch size (otherwise those files could never be uploaded). You can specify this here:

image

Since the default max batch size for the SharePoint client object model is quite small, 3 MB, it’s very likely that you want to change this. That is, if you want to gain any benefit from the batching mechanism. You need to increase this number to:

  • At least the size of the max batch size you want to use.
  • AND At least the size of the largest file you want to upload.

You can change the server max batch size by clicking the Increase button:

image

You can only execute this button when placing the tool on the server, as this part of the code uses the SharePoint server object model. Right now, if you click it on a client, the tool crashes badly. The rest of the tool can be used on any client. It executes the following code:

private void IncreaseMaxReceivedMessageSize()
{
    int increaseSize = Convert.ToInt32(txtBatchSize.Text) * 1024 * 1024;

    SPWebService contentService = SPWebService.ContentService;   
    contentService.ClientRequestServiceSettings.MaxReceivedMessageSize = increaseSize;
    contentService.Update();

}

Please Note You have to restart the web server on the SharePoint WFE(s) in order for the change to take effect.

Client requirements

The tool needs access to the SharePoint client OM. It’s included in the Migration Dragon zip file. The user account under which the tool runs needs to have appropriate permissions to upload all files and folders.

Testing

I did my testing with a corpus of 197 MB of IIS log files. We changed the batch sizes, and after repeating the tests several times, the results on our dev machine were roughly as follows:

Batch Size Processing Time
1 MB 52 secs
3 MB 50 secs
10 MB 50 secs
20 MB 42 secs
30 MB 42 secs
50 MB 60 secs + becomes error prone
100 MB 62 secs + becomes error prone
200 MB 80 secs + becomes error prone

As you can see, varying the batch sizes didn’t result in that many differences, but that’s to be expected when uploading files located on a dev machine to SharePoint on the dev machine. Things will only get interesting when using the tool on various clients.

What now?

It’s an experimental version. I have yet to start to use this tool in various environments. I’d like to get feedback on how well this tool is doing, and how much difference it makes when varying batch size. Feature requests are welcome too. You can download it here: http://gallery.technet.microsoft.com/The-Migration-Dragon-for-628acae0

Nice resource about Wiki pages

We stumbled upon this nice resource about working with SharePoint Wiki pages: http://office.microsoft.com/en-us/sharepoint-server-help/overview-RZ101837218.aspx?section=1

Starting and Stopping a service instance

If you want to start and stop services programmatically, you can use the Provision() method of an SPServiceInstance object to start it, and its Unprovision() method to stop it. You can find details about this at http://msdn.microsoft.com/en-us/library/ee537799.aspx

The following C# code demonstrates, using the server object model run from a console app, how to start and stop the Access service status. You can check the Services on Server page (in our case, http://astro:40000/_admin/Server.aspx ) to see what happens when you run the code.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using Microsoft.SharePoint.Client;
using Microsoft.SharePoint;
using Microsoft.SharePoint.Administration;

namespace ConsoleApplication2
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                SPServiceCollection services = SPFarm.Local.Services;
                foreach (SPService service in services)
                {
                    Console.WriteLine(“SPService {0} status: {1}”, service.Name, service.Status);

                    foreach (SPServiceApplication application in service.Applications)
                    {
                        Console.WriteLine(“SPServiceApplication {0} Status: {1}”, application.Name, application.Status);
                        if (application.Name.ToLower().Contains(“access”))
                        {                           
                            Console.WriteLine(“Toggling Access service instances”);
                            foreach (SPServiceInstance serviceInstance in application.ServiceInstances)
                            {
                                Console.WriteLine(“SPServiceInstance {0} Status: {1}”, serviceInstance.Name, serviceInstance.Status);
                                if (serviceInstance.Status == SPObjectStatus.Online)
                                {
                                    Console.WriteLine(“Stopping service instance”);
                                    serviceInstance.Unprovision();
                                }
                                else if (serviceInstance.Status == SPObjectStatus.Disabled)
                                {
                                    Console.WriteLine(“Starting service instance”);
                                    serviceInstance.Provision();
                                }
                            }
                        }
                    }
                }
            }
            catch (Exception err)
            {
                var errMsg = err.Message;
            }
        }
    }
}

Requests per second –SharePoint Performance Management

It’s well written, entertaining, and very informational: http://www.cleverworkarounds.com/2012/06/09/demystifying-sharepoint-performance-management-part-4making-use-of-rps/ (check out the other parts as well). We liked it so much that we’ve added it to the SharePoint 2010 TN Wiki Best Practices page at http://social.technet.microsoft.com/wiki/contents/articles/8666.sharepoint-2010-best-practices-en-us.aspx. We couldn’t resist to mention our favorite quote:

To that end, I would also be seriously remiss if I did not make you aware of the SharePoint Flavored Weblog Reader tool. It was created by Nikander & Margriet Bruggeman who run the SharePoint Dragons blog – probably the best SharePoint performance related blog out there. This tool was specifically designed to make it easier to analyse IIS logs for SharePoint specific information. It is a command line tool, but much simpler and slicker than the methods I introduced in this post. Instead of specifying a date range you specify the number of items from the logs to process. For example:

sfwr.exe 250,000 “E:\LOGS\IIS_WWW\W3SVC1045333159”