All Code Deployment
Back in the 2007 days, a lot of your SharePoint deployment mechanisms were built in XML: feature definitions, elements, modules, those goofy ONET site definition files, solution manifests, solution DDL files, (remember those?) and of course your lists, content types, site columns, and so on. Add all that to the host of out-of-the-box XSLT-driven web parts, and you can configure your way into an angle bracket frenzy provisioning your site.
2010, and now 2013, actually tell the same story; the tooling is just better. We have nice UIs for our deployment packages, as well as a folder structure that mimics that of 14/15. But it's still all the same schemas behind the scenes, and even though Visual Studio will help in creating a lot of the project infrastructure and assets, it's still up to you and the angle brackets to create your SharePoint elements.
My favorite third party tools, WSPBuilder (for 2007) and CKS Dev (for 2010...not out for Visual Studio 2012 at the time of this writing), make the job even easier by generating copious amounts of XML; XML that is unavoidable for a site with any extent of complexity. It's the same story with the new list and content type designers built into Visual Studio 2012: out of the box XML machines.
And I think a lot of developers just sort of accepted these as a best practices realities. "If that's the supported way it should be done, and Microsoft has gone through the trouble to providing us tools to facilitate doing it that way, then I guess I'll adopt it too," goes a very valid, rational compliance with standard SharePoint deployment techniques.
If you feel that way as well, or really have an affinity for creating, editing, or consuming XML documents, then this is not the book for you. I am a developer, and therefore not only do I want to be living in Visual Studio for the majority of my life, but I also want to be spending that time writing code as much as possible. XML is great for establishing infinitely scalable dialects for applications and transmitting data among them; I just don't want to have to type it.
I don't want to deal with XML namespaces or XSD schemas or XPath queries or any of that. It's just not fun; it makes configuring SharePoint deployment feel more like I'm filling out a mortgage application than creating a website. So I simply started doing everything in code. The only XML I need is enough to physically call my feature receiver; everything is managed C# from there. I can't guarantee that there won't be an angle bracket or two in your future, but the amount of them you'll need will be greatly reduced.
Now perhaps I'm being an old crotchety architect unwilling to get over himself and deal with a new dialect of XML. Or I'm playing the role of an ultraliberal developer protesting the mainstream and breaking all the rules to do something my own way. Or maybe I'm just being a bitch. But if you ask me, I've simply stumbled upon something that addresses a major concern in our industry and will make all SharePoint projects better.
However, if I'm going to do something in code that can be done via a UI somewhere in Visual Studio or SharePoint Designer, then I had better plead my case pretty well. I know that Visual Studio 2012's integration with SharePoint 2013 addresses the painful deployments from 2007 and fills the holes from 2010. I know that XML is more extensible than managed code and is a better tool for administrators and configurators. I know that hardcoding the structure of a site is very static.
But like I said, this is a development-focused deployment paradigm for applications; SharePoint is really just an example for this strategy. This is a methodology for SharePoint developers to be able to execute more automated, structured deployments. And if we have to bend a few rules and buck a few best practices to get there, then so be it!
The following sections outline what I feel are the major advantages of what I call "All Code" SharePoint deployments.
Brittle Deployment Scripts
First of all, doing things through code is more brittle. If something is "brittle" (especially in the software world) it's generally a negative term; things that are brittle crack easily under pressure. However, this is actually a positive physical characteristic for your deployment mechanisms to have. Remember that this is deployment code; deployment code is different than application code.
Recall my comparison of control developers to application developers. Application code needs to run in different scenarios, under the security context of different users, and while it's at it, it needs to be efficient, scalable, and fast. Deployment code, however, in all honesty, doesn't. It doesn't need to be fast. It doesn't need to be secure (meaning it is designed to only run under the context of an administrator; we still encrypt passwords, are careful with service accounts, etc.). It doesn't even need to be all that clever.
Don't think that we are on easy street, however; deployments must always succeed. Good application logic fails gracefully; errors are anticipated as a natural part of any system. Deployment logic must run front-to-back flawlessly. Every single time. One unhandled exception, wrong step, one wrong URL, one wrong string value and the resulting site, if any of it even gets provisioned at all, is in an inconsistent state and cannot be confidently used.
Deployment scripts, whether they are SharePoint feature receivers, PowerShell, or simple batch files, must be atomic. Think of them as a SQL transaction: either the site is provisioned perfectly in all its glory, or it's as if the script was never run. This is where the brittleness comes in. If the code is brittle, and one little thing snaps it like a twig, we don't have to worry about producing a half-baked site. Since this code can never fail, we want it to burn and die quickly in development so there's as small a chance as possible a bug could escape the QA process before the script is run in production. We want all flaws to be glaringly obvious.
Now with the Visual Studio and all the generated XML, little one-offs can very easily find their way into production. Your XML element files need to be structurally correct in order for your solution to build or deploy at all (which is nice), but they can still be invalid (misspellings, incorrect GUIDs, missing references, etc.) and won't be discovered until the site is in use.
In other words: they won't be discovered until it's too late.
What does it mean for a deployment script to be brittle? We'll get into that when we start looking at some code. In the meantime, the important thing to understand is that although Visual Studio will give us some safeguard against bad XML, doing everything in code puts us in the hands of the C# compiler to make sure all of our I's are dotted and T's are crossed.
As a developer, this, plus some of the techniques we'll discuss to circumvent the cases where the compiler can't hold our hand the whole time, gives me a level of comfort and confidence in my code â€“ a comfort that I'll never feel when all I would otherwise have to preclude erroneous deployment artifacts from flowing into my environment is the result of an XML schema validator.
In All Code deployments, you have the most control over the provisioning of your site: more than any other method. There has never been anything that I've been able to do through the SharePoint UI (or even in Central Administration) that I had to give up on programmatically. I'm not ready to claim that all out-of-the-box SharePoint functionality dogfoods the object model, but so far, the API hasn't let me down.
It's very possible that some remote corner of out-of-the-box SharePoint has been imbued with magic that circumvents some limitation or unsupported aspect of the API. But until that's discovered, and probably even thereafter, I will trust that the SharePoint object model will open more doors for me than any admin UI screen, XML file, or third party control possibly could.
With this assurance, there really is no contest between All Code deployments and other more (or less) conventional ones. If everything else (SharePoint UI screens, Central Admin, Visual Studio, third party controls, etc.) ultimately uses the API to do its bidding, then why not skip that limiting layer and go straight to the source? These other deployment tools make the job easier up front by lessening the decisions you need to make. But it's the abstraction caused by this lessening that makes these deployment paradigms less robust than All Code.
For example, consider the Visual Studio SharePoint solution package designer. It's beautiful in a way I can't describe because it smothers that DDF crap from 2007 in obsolescence. Additionally, it wraps a nice UI around all the common stuff a typical solution deployment would need: drag-and-drop manifest of features and mapped folders, inclusion of assemblies, the option to reset the web servers, and so on.
But what about the other 75% of what solutions can do? Although we don't normally wander into that sector of the solution, my point is that, despite the aforementioned beauty of this deployment mechanism, (that I actually use anyway) it doesn't give us the control that code does. What if we need to update the web.config file? Or add some Bacon Ipsum to our home page? These are not standard tasks, therefore they are not possible to accomplish with a designer.
Since most people use the standard deployment tools, they will just deal with the shortcomings. This is what leads to manual processes that need to be executed in conjunction with your deployment. I think it's kind of ironic to hand clients the deployment package that's supposed to do everything for them, and then hand them a task list of everything else they need to execute.
So instead of automating the easy three quarters of a deployment and then ending up with a smaller mess to deal with at the end, All Code automates things 100% by unleashing the full power of the API, making anything possible. Having this much control over the provisioning of your site allows you to provide a run-once, front-to-back deployment package that does it all without needing any batch files, instructional documents, or to-do lists as sidekicks.
Leverage What's Good
In the above section, I alluded to the fact that All Code still uses all the goodness that Visual Studio 2012's SharePoint integration gives you out of the box. It absolutely leverages the solution deployment paradigm that automates the file copying to all web front end servers. It absolutely leverages the deployment of web parts to the gallery. Reworking those bits would be insane.
Where I draw the line between XML and code is when it comes to provisioning actual SharePoint artifacts. If we're talking about backend image files, DLLs, user controls, CSS, etc. then the standard file deployment stuff is golden. I love how you can right-click-add your way through the solution. However, as soon as I have to start writing XML to provision page layouts, master pages, content types, site columns, etc. â€“ elements that are modeling functional SharePoint components â€“ I start to get anxious.
Since these are going to be the vertebrae that comprise the backbone of our site, I want them to be reference-able in code. Content types will be the base classes for any SPMetal objects we may generate, and those guids that generate them need to be static across environments. Site columns are generally referred to by name, and so I want hard references to those as well.
Does this imply that I am going against like the third thing I ever learned about programming and recommending the hard coding on strings and ids? You betcha! But once again, when we dig into the code, I'll elucidate how these apparent worse-practices actually make our site more sustainable and deployments more durable.
But for now, the idea is to rock the SharePoint deployment boat as little as possible. I'm not coding anything you can drag-and-drop on a designer; I'm only making managed what you'd normally have to do by hand in XML. This follows my general philosophy toward third party software: if it's a slam dunk, drop it on the page and move on. If not, or I have to click more than five times for it to do what I want, I throw it away and code it myself.
Less Error Prone
I didn't sign up to be a developer only to craft a deployment strategy that required data entry. Sometimes it'd unavoidable; I've found myself, for example, feverously mapping crawled properties to managed properties to beat a 6 PM deadline for a search deployment. But "data entry sucks" doesn't provide quite enough perspective on why scripted deployments are far less error prone than manual ones.
Data entry doesn't suck so much because it's boring or mindless; it sucks because it's extremely error prone. It's far too easy to fat finger a column name or click the wrong checkbox or navigate to the wrong sub site to provision something. Whenever you are in the business of typing the same string over and over again or copy-and-pasting Guids (shudder) you need to ask yourself if there's a better way. You will make a mistake, and it will be very difficult to track it down.
All Code deployments are not only less error prone because they don't depend on someone being caffeinated enough to provision SharePoint artifacts manually; they also alleviate the problem of one person (or a small group of people) having intimate knowledge of a site's structure. Far too many times I've heard the following statements:
"Oh yeah, you need to create a settings list in the root site."
"Oh yeah, you need to update the web.config file with the production connection string."
"Oh yeah, you need to activate the publishing feature."
When you have a scripted deployment with explicit dependencies, you won't need to audit your development environment or follow some checklist to know what to do when you move to production. Automation means fewer errors. All you have to do is make sure you get it right the one time when it's hard coded; everything else references that moving forward and you never have to worry about it again.
And then at the end of the project, when it's time to run this script, you'll immediately see the payout from the upfront investment you made in considering deployment first and spending those hours implementing All Code. When there's a week left in the project plan, there will actually be a week left in the project; you won't have to scramble around with stress-inducing, hair-thinning, messy as hell deployments.
Deploy Sample Content Along With Structure
All Code deployments go beyond merely provisioning structure; when we start to look at the deployment code, we'll talk about using a data creator to populate the site with content as well. And this doesn't just mean configuration data or adding web parts to pages. I'm talking about HTML, content editors, publishing pages, list items, full search indexes, etc. A beautifully structured but ultimately empty site is not usable; it's not even testable.
This brings up another interesting problem we'll tackle as well: how to facilitate clients wanting to load content before (or during) development is completed. The bane of following a good iterative dev cycle is getting content authors and project sponsors excited about in-progress sites. Doing things the right way can lull them into a false sense of completeness; they will want to start populating content before we're done building the damn thing!
In my last few projects alone, we've had to creatively deploy final bug fixes and production site structures around new content because the users had already gotten their hands on the portal. Things get tight when your deployment depends on blowing away any existing site collection! This is a challenge for project managers as well as deployment architects. But with a little planning, everyone can be happy.
Our goal is to be able to present iterative deployments of in-progress sites fully populated with Lorum Ipsum (or my preferred Bacon Ipsum) using a SharePoint-specific data creator. Not only will this alleviate awkwardly-blank pages populated with empty, hungry web parts that appear broken, but will also surface a lot of look-and-feel nuances that are often missed by our designers and architects:
- Should longer-than-expected text wrap? Scroll? Be ellipseicized? Or do we need validation around max lengths of columns?
- How does the site handle 16 MB image files?
- Does our caching strategy keep page load times and other target performance metrics in acceptable ranges?
- Do the pages, now complete with web parts and content, actually resemble the wireframes?
Automating the structure alone is a win. But when you can demonstrate what a fully deployed, fully populated site will look like with the activation of a single feature, your clients will not only be impressed, but also hopefully filled with confidence that they will not only actually sleep the night before the site goes live, but sleep well.
Repeatability goes hand-and-hand with robustness against errors, in terms of having a solid process that atomically provisions portals. However, the benefits of the repeatability of All Code deployments needs to be called out explicitly, as they aren't really felt in the (ideally) one and only time your script is run in production.
The repeatability is more felt during development. As new features are cranked out and updated DLLs, user controls, and ASPX pages are crafted and deployed, we need to make sure our environments are agile enough to provide suitable homes to our new components. It is so nice to be able to blow away a site collection and recreate it with a PowerShell script, deploy a new solution through Visual Studio, and then activate an All Code deployment feature through SharePoint whenever you need a fresh environment to test some new code.
An example of this is when new developers roll onto a project. Earlier I mentioned "intra-team" deployments, where a new member can be productive within five minutes of getting access to the project in TFS. Normally this could be more like five hours, but when they can run a single command and have a fully structured site to work with immediately, we can see the value of having a repeatable provisioning process.
Or recall our web part-that-consumes-a-list example. Perhaps in the middle of building the web part, a requirement changes and you need a new field in the list. Instead of manually adding the column, and working it back into the script later, you should update the script first, and rebuild your environment front to back. This is the easiest way to ensure that there will be no regression bugs in your deployment. After you check in, the rest of your team can get the latest, rebuild their environments, and allow the repeatability of All Code deployments to keep everyone almost seamlessly productive.
A byproduct of repeatability is that it keeps all of our environments in sync as well. How often does the client's development server become a wasteland? We treat it with love while we need it (nightly builds, proper service accounts, scheduled Windows updates, etc.) but as soon as we starting giving the client peeks of the infant application on the sparkly new prod hardware, dev is quickly abandoned and becomes overgrown with weeds and bugs and rats.
As the project progresses toward completion from this first push to production, and minor backend or environment tweaks are made, we forgot to retrofit our development environment to keep it fresh. How does this play out? We launch. A critical bug is found. We fix it. We push the fix to dev. But guess what: nothing works in dev because database columns are missing, third party DLLs are missing, and security groups are missing. We end up delivering the bug late because we had to rebuild our test environment.
Having a repeatable deployment process simply makes this all-too-common predicament not happen.
Just Plain Faster
Finally, it's just plain faster, in both a micro and a macro manner. By "micro" I mean that it's sometimes faster to take the time upfront to build something to automate a process than diving in and doing it manually. Forget about repeatability across scenarios, resiliency to errors, and reuse in different environments: on a point-by-point basis, is it faster to perform a certain task through the UI than it is to script it?
If the answer isn't a resounding "No!" then you should consider scripting that task. Whether it's something as big as an All Code deployment, or as menial as tweaking a setting in central admin, sometimes it's simply faster to code than it is to configure. A quick example is adjusting the site setting for the maximum message size of a service call generated from the client object model. I never remember weather that's in the site collection settings or in central admin (or available through the UI at all); either way I find it onerous digging through these admin screens.
It was faster (and a lot more fun) to research and write the following PowerShell script to do it for me than to click-click-click my way through my local development and live production servers:
Code Listing 1: ClientServiceMessageSizeUpdater.ps1
- #ensure sharepoint
- if ((Get-PSSnapin -Name Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue) -eq $null)
- #load snapin
- Add-PSSnapIn Microsoft.SharePoint.PowerShell;
- #get the content service
- $service = [microsoft.sharepoint.administration.spwebservice]::ContentService;
- #allow requests of up to 2GB
- $service.ClientRequestServiceSettings.MaxReceivedMessageSize = [System.Int32]::maxvalue;
At the same time, as good architects, we want to keep a guard up against throwing a beautifully complex solution at a simple problem; we don't want to be using a hammer to get at the fruit of a walnut. Don't script things just because you can. Make sure you have a value-add reason to do so. Maybe something in your deployment script should be part of the initial server build-out instead. Maybe your client has government regulations requiring people with different security clearances to perform certain deployment tasks manually.
Of course it's easy to dream up devilishly-advocated reasons to do or not to do something. All I'm saying is that it's important to have the discipline to know when to when to build a robot to do something, and when you should just do it manually instead. After all, time is money, as they say, so make sure you invest wisely.
Macro-wise, we're talking about the expense of provisioning entire pages or sites. These tasks are time consuming to the extent that SharePoint displays the "Processing..." green-circle-spinning page while the gears are churning behind the scenes. Then multiply that time and effort by how many dozens of sub sites with potentially hundreds of pages; it's not necessary for me to explain that automating this mess is, like I said, just plain faster.
But what if each site is slightly different? We could have different master pages for different sub sites. We could have different landing pages with different layouts and different web parts. Does it make more sense to craft a complex All Code deployment to handle each site essentially uniquely? Or should we let Johnny Intern loose with SharePoint Designer and an Excel page-asset matrix?
We could even do both: an All Code deployment that provisions a portion of the site, and then dumps some site and list templates into the gallery, off of which the more unique aspects of the site can be created manually. This is a great strategy for portals that follow a team site architecture, where they are designed to grow exponentially after going live. Or when the client is anxious to get a quick win "version one" of a site up and "version next" hasn't been defined yet. We just need to get them to the starting line.
Deployments are interesting in the fact that no two are ever identical. This won't sound terribly profound, but whatever your scenario is, do what's best (fastest, smartest, most efficient, most client-pleasing, etc.). It's just that in the vast majority of scenarios I've encountered, scripting has indeed been the fastest, smartest, most efficient, most client-pleasing, and, well, simple the best option for a SharePoint deployment.