Introduction

After two years, I think I have all the kinks worked out of the Hybrid Provider, not to mention a new WPF installer that has pretty gradients and spinning animations (still no dancing monkey buttons...sorry) that eliminates the two hours needed (on a good day) to configure the Hybrid Provider on your 2007 SharePoint server.

Read more about version 2.0 here.

So now that I know more than I ever though I'd care to know about SharePoint authentication, I decided to tackle the next logical problem: User Profiles. Once you are successfully authenticated against any server, the only thing known about you is that you exist. Authentication is, philosophically, an existential exercise; in other words, you are, but who are you?

SharePoint, once convinced that you exist, keeps a little object around to represent you. This is essentially a login name, a display name, and an Email. That, everyone, is the sum of your being! So if SharePoint is this great collaboration platform, then it's going to need a community feeling; a place, digitally, where users will feel comfortable and productive being.

And communities are built of people, and people are built of (especially in this day and age) their on line profiles. My friend Michele loves saying: "If it's not on Facebook, it didn't happen." Well, despite the fact that we are 26 years old now, she's right: whenever people spend a lot of time on a "community" site, they will need a way to identify themselves beyond a mere moniker.

This is just as true in the professional world as it is in the MTV / US Weekly / Hannah Montana world...just different properties. On MySpace, you might be concerned (or even lose sleep over) someone's "dating status" as it changes hourly. Tantamountly, on SharePoint, you'll want to keep track of your colleagues, manage your display name and Email, and maintain any number of other properties pertinent to you within the context of your job.

Scenario

Out-of-the-box, SharePoint does this, to a limited extent, for you. In its default configuration (which is using Windows Auth) WSS creates your SharePoint user from your AD account, and then maintains other properties internally. If you install MOSS, you get the User Profile infrastructure, which allows you to configure your profiles to a very granular level and even map one-way properties dumps of AD into MOSS.

However, what happens when AD isn't the only user store? Shouldn't true enterprise collaboration allow heterogeneous sources for your accounts? As we saw more and more instances where SharePoint needed to invite external users to the collaboration party, I came up with the Hybrid Provider.

But that only gets us through authentication. Now, AD users and SQL users can both come into the party, but they are all, to keep with the party metaphor, only wearing underpants and a name tag. And everyone's wearing the same exact white, boring underpants, differing only by what follows "Hello, my name is..." on their name tags.

Now this might sound like a hell of a party, but remember, we're at work, not on Friendster! We need to get everyone clothed and ready to collaborate in style! Right now, people at our SharePoint party are saying, "Hi, hybridprovider:awalker, I'm hybridprovider:cdomino. Nice to meet you. What's your Email address?"

We want administrators to have the ability to dynamically create properties for users' profiles that allow them to, depending on their organization, express themselves as much or as little as they wish. If that means putting a nickname in their Display Name, awesome. If that means keeping an extensive bio up-to-date, awesomer.

This is, unfortunately, quite difficult with heterogeneous user data sources. With authentication, it's easier, because for it to work at all in SharePoint, your membership mechanism needs to construct a user from specific properties: login name, Email, unique identifier, etc. At the end of day, all user tokens will always have the same structure.

But profiles can be anything! Of course, you still need to inherit a class to make sure your mechanism is generically usable via the ASP.NET ProfileManager class. However, profiles can contain dynamic properties. Different user data sources can contain different properties. And of course, things need to be validated. If order to support all of this, we need a profile mechanism that not only implements everything, but also fits nicely into a SharePoint site that's already using custom authentication.

Allow me to introduce the Hybrid Profiler!

The Hybrid Profiler

The Hybrid Profiler takes on the same user paradigm as the Hybrid Provider, only tacking profiles instead of authentication. This paradigm is when we have internal AD users and external SQL users that need to be on the same SharePoint site at the same time, with the ability to put them in the same SharePoint groups and give them access to the same audenices, sites, and content.

The Hybrid Profiler contains a lot more logic than it's authentication counterpart, since there is no out-of-the-box Active Directory profile provider. For authentication, there is both an AD and a SQL provider; all I had to do was wire them together and make it work in SharePoint. For profiles, however, I had to write all the AD code myself. An internal SQL Profile Provider does all the work for the external profiles, leveraging the same ASP.NET user database infrastructure as the Hybrid Provider.

Read more about all of that here.

But like I said, Active Directory has no explicit concept of a profile. Therefore, the AD profiles are really just a listing of selected properties. The way the ASP.NET profile stuff works is that you define all of the properties of a profile in the web.config file, and wire up a class that inherits from ProfileProvider.The above link contains all of the information you need to see how this all comes together.

The one main specialization of the profiles is how these heterogeneous user data sources are mixed. At first, I wanted all users to have the same properties. This way, I wouldn't have to explicitly know which user came from where. The problem is that we didn't want to expose AD to dynamic properties. In fact, our AD guy tried to punch me when I suggested it.

Until now, AD, in terms of SharePoint, has been completely read-only. The fact that AD users can now update the values of properties in their profiles is the first writable aspect of all my "Hybrid" work. That was enough of a battle to win. But I do agree: AD is more or less "sacred" in terms of data stores, since generally only administrators get to touch it. I don't want any of my work to be considered desecratory!

So now, SQL and AD users will have different properties. SharePoint administrators can create any properties they want for external SQL users, and expose any existing AD properties for internal users. This actually ended up working out well, since external users tend to have more of a "presence" in these mixed environments; internal users generally need to only maintain important properties like display name, Email, etc.

Technical Details

The Hybrid Profiler's implementation is based on the ASP.NET profile infrastructure. This entails defining all of the profile properties in the web.config file, and creating a class that inherits from ProfileProvider that does the work. I do recommend reading the two links above to get up to speed on how all that works.

The HybridProfiler class works the same as HybridProvider, where each method implemented from the base class first tries the SQLProfiler, and if that doesn't work, goes to AD. Some of these methods take in the user name as a parameter, and a new static method on the HybridProvider tells us if this is a SQL user, so we don't have to guess first. Other methods, such as GetAllProfiles, simply call both and aggregate the results.

Something you'll notice about ASP.NET's profile infrastructure is that Microsoft was seemingly very careful to make sure that things stayed dynamic. The methods that actually retrieve and save profile property values (for example again, GetAllProfiles) take in a collection of the properties that are to be persisted. To this end, there are four classes you should be aware of:

• SettingsProperty
• SettingsPropertyValue
• SettingsPropertyCollection
• SettingsPropertyValueCollection

What I did was create my own objects that wrap these, and still are able to interact with the profile sub systems. These are:

• HybridProperty
• HybridPropertyCollection

HybridPropertyCollection has a constructor that takes in a SettingsPropertyCollection. It inherits from ObservableCollection to make WPF databinding trivial. So in the code, you'll see a lot of HybridPropertyCollection typed objects hanging around. This class gives us better indexing and some helpers that really reduce a lot of the repetitive profile code you'd otherwise have to write.

This also allows all of the AD code I wrote to not have to take a dependecy on System.Web!

The only remotely "hacky" thing I had to do was come up with a way to declaratively inform the HybridProfiler of some necessary meta data about each profile property. This includes, in the initial release, a Display Name for each property, a flag to mark weather or not it is a SQL property, and another indicator if this property requires a value.

To implement this, the Hybrid Profiler makes extensive use of the "custom provider data" attribute of each property. This is part of the out-of-box schema for ASP.NET profiles. The text of this attribute is automatically deserialized and added to properties of an instance of the HybridProperty Class. Each HybridProperty is then added to a HybridPropertyCollection. The custom provider data attribute's text is a semi-colon delimited list of = pairs, so it can be easily extended. The following properties of a HybridProperty are deserialized out-of-the-box:

• DisplayName - The text that renders as a label on the profile page for each property.  If this is omitted, the Hybrid Profiler takes the Name of the property and converts it from camel case to title case.
• IsSQL - A boolean value that tells the Hybrid Profiler weather or not this property is for SQL users or AD users.  False is assumed if this is not specified.
• IsRequired - Another flag that tells the property if a value is required.  This defaults to true.

Here are the rest of the properties of a HybridProperty:

• Name - The internal name of the property.  This is the identifier of the property, and is part of the XML schema specified by ASP.NET
• TypeName - The fully-qualified name (FQN) of the type of control to render for the property.  I did hijack this one a bit.  It really should be the actual type of the data represented in the property.  For example, it is System.String or System.Boolean in other profile implementations I've done.  But since everything is deserialized as a string, it's not really helpful, and therefore gives us the ability to reuse it and not continually abuse the custom provider data.  See the User Interface section for more details.
• PropertyId - An internal GUID used to uniquely identify the property.  This is mainly to avoid matching on strings for indexing.

Here is a complete profile XML section from a web.config file. (Sorry if the formatting sucks.)

The fully qualified names for these controls is "Catalyst.SharePoint.Controls.[class name from above], Catalyst.SharePoint.Controls, Version=1.0.0.0, Culture=neutral, PublicKeyToken=695a890c7040b3c0" - fill this in for the ellipses above.

Some bullets to point out:

• The first property is the display name for AD users (since IsSQL is false).  Notice that the name of the property is "cn" - common name - from AD.  Therefore a display name of "Display Name" is used.  However, since we completely control SQL properties, we can omit the display name from the custom provider data attribute, since the name of "DisplayName" will be parsed into "Display Name" for us in the second property above.
• In this same regard, we can omit any attributes that default to the value we want.  For example, we never need to put IsRequired=true; in this attribute.
• The "providers" lists the custom providers.  By setting the default provider in the first line to the HybridProfiler, we can use the static ProfileManager ASP.NET class instead of explicitly instantiating an instance of the Hybrid Profiler.  If you add any custom methods to the Hybrid Profiler, you'll have to instantiate it, which seems to be a bit more expensive than using the static ASP.NET methods.  But more importantly, it keeps a nice abstraction between your application (SharePoint in this case) and your profiler.  If you ever need to write an ASP.NET app, for example, that only has SQL users, your presentation code will not need to be modified.
• Use ProfileManager.Provider to get all the methods exposed by ProfileProvider.  Or, cast this to a HybridProfiler to get everything.
• When dealing with the ProfileManager, you'll have to use the standard .NET SettingsProperty and SettingsPropertyValue objects.  But never fear!  Each SettingsPropertyValue has an "Attributes" hash table that is filled with the custom provider data from web.config, so you'll have access to everything.

The source code is included on CodePlex.  I made sure to heavily comment it; you should be able to follow just fine, in case any tweaks need to be made.

User Interface

So how do we see all this wonderful profile stuff in SharePoint? Well, it's not pretty...but it's beautiful. Whenever you click on a "presence" user link in SharePoint, you are taken to a page named "UserDisp.aspx." These "presence" links are everywhere: permissions pages, group membership descriptions, created by / modified by list columns, etc. Also, whenever a user hits the welcome menu drop down and selects "My Settings" they are brought here.

Here are some places where the profile page links to:

This is a static page that is stored in the TEMPALTE\LAYOUTS folder (C:\Program Files\Common Files\microsoft shared\Web Server Extensions\12\TEMPLATE\LAYOUTS by default) on the file system. This means two things to us:

• It is not ghosted, so we can directly mess with the markup.
• It's shared across all web applications, so we had better not F'ing mess with the markup!

Well, if you're a regular reader of my blog, then you'll probably guess that I messed with the markup. But I was really really really careful to do it in such a way that only web apps that have the HybridProfiler installed will be affected. But why hack the HTML? Come on! There's got to be a better way!

There were actually three better ways; not a damn one of them panned out. The first thing I tried, which indeed is the "right" way, is to create a feature that hides the out-of-the-box "My Settings" welcome menu item, and adds a custom one to redirect to a different page. Here is the feature XML of an element that SHOULD do this:

But the "Hide Custom Action" node just doesn't work! You can hide out-of-the-box sections of drop downs and pages all over the place, but the welcome (Standard) menu one just doesn't work! In order to do this manually, you have to modify the HTML of a user control in the controltemplates directory, which just brings the hack full circle. Finally, this doesn't even address clicking on a presence user link; the "right way" won't get us there.

So that's bunk. The next-best way I tried to do this was to first assume that UserDisp.aspx might, somehow, by the grace of GOD be a web part page (which it's not; it's obviously an application page since it's on the file system in this location) and write SharePoint code behind the feature activated event of the Hybrid Profiler that hides the out-of-the-box user information web part and then adds one of our own.

Instead, the meat of UserDisp.aspx is a SharePoint user control. The third idea I had was to again write code behind the feature activated event that modified this page's HTML via File I/O. I wanted to add JavaScript to this page that hides the control, and shows a custom control I would write that interacts with the Hybrid Profiler to visualize the properties.

But what a kludge! Opening up a SHARED page via I/O and modifying it's HTML? Disgusting! I know that the best practices (if I can even mention best practices at this point in our journey) is to deal with full controls and SharePoint objects, not loose HTML...but at what cost? So I thought and thought. The only way to ensure that custom profile functionality could possibly work was to do something with this page, since it's the first place that we are sent when either clicking a user presence link or viewing "My Settings."

The answer finally came to me after a few delicious 312's.  (If you're ever in Chicago, order one at the bar - a wonderful "urban wheat" beer.) I was going to have to modify the page via I/O. There was simply no way around that. But all I was going to do was add one little line of JavaScript that would do a "smart" redirect to a custom application page (UserInfo.aspx) to display the profile data.

By "smart" I mean that this call would only redirect if the current web application had the Hybrid Profiler activated. And the Hybrid Profiler's uninstaller would only remove this line of JavaScript if it was the last web application on the server to have it installed. This way, web apps that didn't use any custom profilers would simply continue loading the standard UserDisp.aspx and be none the wiser; the footprint the installer leaves will be partially or completely removed when necessary.

Here's that magical line (broken up for readability):

The bread and butter here is the call to a static method on the HybridProfiler type. All this method does is spin the though the features of the current web app (using SPContext) and returns true if it was found. This call is made possible by the magical <% %> ASP.NET construct that allows pages to make managed calls at run time from HTML or JavaScript. If this method returns true, I just parse out the URL from the current request (so that all web apps will work) and tack on the query string, while contains the ID of the user who's profile we want to load.

UserInfo.aspx inherits from LayoutsPageBase, making it a standard WSS application page. This means that SharePoint security cuts it some slack, and that it lives, unghosted, on the file system. It's DLL is GACed to ensure that the code behind will run freely in full trust. Finally, it references the application.master master page, which I just copied from the LAYOUTS directory and referenced in my Visual Studio project, creating folders so that the relative paths matched up with how SharePoint does it in IIS.

(And as a side note, don't ever let anyone give you shit for GACing DLLs in SharePoint. Notice that despite all my "SPHacks," NEVER have I changed the trust level in my web app's web.config to anything other than the default wss_minimal. This is a beautiful thing. ONLY code that we specify can therefore unrestrictedly execute.)

Once UserInfo.aspx is loaded, the rest is fairly straight forward. I deserialize the web.config profile information into my objects, and based on the type of each property, dynamically generate the property control. You'll notice that these controls are more than mere textboxes and labels.

They are script controls, which means they will require AJAX. Why take a dependency on AJAX, you ask? Well, for a couple reasons actually:

• Almost every SharePoint user I've worked with has commented that the pages post back too much.  Keep in mind that they don't even know what a post back is!  AJAX script controls don't post back, and provide a more cohesive and responsive user experience.  Please refer to my blog about my AJAX architecture.
• I always advocate the latest technology.  Using AJAX in SharePoint is pretty cutting-edge.
• This will ensure that SharePoint 2007 SP1 is installed (or AJAX won't work).  I haven't heard a single bad thing about the upgrade experience, so I think it's the way to go.
• The Hybrid Profiler actually activates another feature I wrote that installs AJAX on the web application for you!  So all the administrator has to do is make sure service pack 1 is installed.

The Controls

Let's dig deep into these controls. Other than a few labels and a "back" link, (who's redirect is automatically populated with the "Source" querystring variable so that the user will assuredly be redirected to the page they were on when they elected to view a profile) UserInfo.aspx basically contains only one table.

After deserializing the profile properties, the Hybrid Profiler does it's thing to get the values for the specified user, as denoted by their WSS SPUser Id from the querystring. If the current user is not this user, the page will be read only; the edit button will be hidden. Then for each property, I create a new row in the table. The left cell of the row gets a label corresponding to the result of the Display Name. The control matching the type of each property is generated, initialized, and placed in the right cell.

If you are the current user viewing your own profile, the edit button will be displayed. Upon clicking it, all these controls "switch modes" and convert from their read only state (usually a label or a hyperlink) to their write state (textbox, etc.). The edit button is also hidden at this point, and and a save and cancel are shown.

All controls actually use JavaScript inheritance and derive from a base class. This base class defines a method to facilitate this mode switching. The other advantage of using a base class is felt on the server side. Having a common type to reflect against makes dynamic control generation very straightforward. Also, being script controls, we don't have to mess with annoying-as-piss ASP.NET dynamic control state management; all the love is in JavaScript!

Out-of-the-box, or, more accurately, out-of-the-ZIP-file, there are two controls in the Catalyst.SharePoint.Controls namespace: StringControl (text box) and EmailControl (text box with "mailto" hyperlink). Other controls that can (and probably will) be added are BoolControl (checkbox), OptionControl (drop down), and RichTextControl (SilverLight??????).

The only validation I have is to make sure the value isn't null or empty. We really don't have to worry about "bad" data, since SQL doesn't care, and SharePoint and AD will blow up if a required field is set to a null value. Validation is handled by the "IsRequired" custom provider data attribute, which defaults to true. To make a property not required, add "IsRequired=false;" to this attribute in the web.config file.

Speaking of SharePoint, two properties are sort of special cases. Please refer to my rant and rave about the mystifying SPUser object and how the hell it manifests itself around your portal (still in research). These two properties map to the two writeable properties on an SPUser object: Name (display name) and Email. LoginName (account name, for example hybridprovider:cdomino or DOMAIN\cdomino) is read only in SharePoint, and way too scary to even think about changing in AD.

Display Name is special for two reasons. First of all, the Name property on an SPUser object is what is shown on the welcome menu, so that needs to be explicitly set from UserInfo.aspx. Also, I map Display Name to the cn (common name) AD property, which requires special AD code to update.

Email is also special. Not only is it required, again, for SharePoint and AD, but it also a property of a SQL user's membership account. The Hybrid Provider inherits methods that use the Email address to retrieve users for authentication. Therefore, if the user changes their Email address, this needs to propagate down to all sub systems that need it.

So in order to handle these special cases, a special check is made upon saving a profile to see if we have properties named "Display Name" or "Email." If so, the values are used to update the necessary sub systems. Beyond these, all standard string-typed AD properties, along with anything you dream up in SQL, should work.

Here are some screen shots to pull it all together (top is an AD account, bottom is SQL):

The Installer

Finally, I'd like to point out the new features of the Hybrid Provider Installer that support installing and uninstalling the Hybrid Profiler. First of all, it should be noted that the Hybrid Profiler cannot be installed independently of the Hybrid Provider. There is so much infrastructure in the Hybrid Provider that the Hybrid Profiler depends on that splitting the two would be a monstrous, and not necessarily even useful, task.

Conversely, you do not have to install the Hybrid Profiler. There is a new checkbox on the installer that tells it when to add this feeature. In the same regard, you cannot uninstall the Hybrid Profiler without first removing the Hybrid Provider. This is due to that fact that this would add another logical layer of complexity to the installer, when an admin can simply fire up Central Admin and deactivate the feature for that particular web app. Based on the way I "hijacked" the UserDisp.aspx page, you will immediately get default WSS behavior for profiles.

Here's the new installer:

Notice that this checkbox is disabled when the Hybrid Profiler is installed, so that it can't be independently uninstalled. You can click edit at any time to view the properties that have been set at the time of the last installation. Remember, the settings for the installer are persisted independently of the web.config file. This is an installer, not an administrative utility.

(The next version of everything will come with a buttload of SharePoint administrative web parts.)

Here's the new properties screen:

It's pretty straightforward. What you see above gives us Email (SQL and AD), Display Name (SQL and AD) and Favorite Color (SQL only).

That should do it. Please feel free to ask any questions! Have fun!