SharePoint Dragons

We were working on a PowerShell script that reads an XML file and uses that as the basis for adding new terms to SharePoint. Some of the term names in XML contained & which eventually gets replaced by the ampersand (&) character. Now, something strange happens, SharePoint converts this to the 65286 character (or 0xFF06 in Hex). This is an ampersand (&) character, but not the same as the normal ampersand (&) 38 character. But, PowerShell converts the ampersand (&) character to the normal 38 character, and then you’re in limbo land. What happens is this:

1. We check if a term exists.
2. If it’s not, we add it.
3. We commit all changes.

Once we were committing, an error occurs stating that we were trying to add a term that already exists. Why is that happening?

1. In PowerShell, we check for a term that contains & (= char 38). It’s not present in the SharePoint term store, there’s just the term containing & (= char 65286).
2. We add a new term containing char 38.
3. SharePoint converts char 38 to char 65286, but that one is already present.
4. An error occurs.

This problem took some time to diagnose, the solution is simpler. Make sure the index name you’re using contains char 65286, instead of char 38. In PowerShell, the code looks like this:

#Please Note:
# This is a tricky piece of code, as SharePoint converts & to a special & sign (char 65286 or 0xff06 in HEX).
# Within PS, this special & sign gets converted and equals the normal & sign (char 38).
# Therefore, a normal equality test of a string in PS NEVER matches the Terms index name in SharePoint,
# because & (value 65286) is not identical to & (value 38)
#
# To solve this issue, & (38) needs to be replaced by & (65286)           

$IndexName = $TermNode.Name.Replace(([char] 38), ([char] 0xff06))
$Term = $Termsetitem.Terms[$IndexName]

We must have won the lottery, because by mistake we also included term names containing two consecutive spaces. SharePoint tries to help out by replacing them with a single space. Now this happens:

1. We check for a term that contains double spaces.
2. The term store only has a term with single spaces, so the new term is added.
3. SharePoint converts double spaces to a single space, adds the term, but that one is already present.
4. An error occurs.

We could do another replace, but decided that this is more of a configuration mistake. So, we’re not handling that one in code, but fixed the XML config file and then things worked like a charm.

The final question is this: is SharePoint right in replacing double consecutive spaces to a single one?  We decided the answer is no. Fixing term spelling makes the mechanism unpredictable and arbitrary (since SharePoint only seems to care about & and double spaces). Besides, it’s not SharePoint that has to decide on the name, it’s the term store administrator/end user who needs to decide. It reminds us of years ago. We were writing a technical design document and correcting the spelling of business terms that were used in a functional document written by the customer. At that time, we had a mentor who said: “Leave it be, you may not agree, but it’s the customer who decides on the business terms”. We thought this was a wise advise.