The SUESS Lifecycle: Stage 3 - SmoothStreaming

The SUESS Series

Introduction

Stage 1 - Upload

Stage 2 - Encode

Stage 3 - SmoothStreaming

We can now upload media. We can now encode media. Sweet. Now all we need to do is be able to watch it as well! This post covers the final stage of SUESS, which is the SmoothSreaming portion: the actual delivery of content from IIS Media Services to Silverlight. I want to get into a little bit more detail about what's going on both server side and client side to deliver SmoothStreaming content to our users.

There is already a lot out there that describes the basic concepts of SmoothStreaming, and outlines it with stunningly beautiful Visio diagrams. So instead of anteing up into that game, I'll deal my own, giving a more practical overview of the process, and discussing some of the roadblocks I encountered along the way.

So in one sentence: SmoothStreaming, part of IIS Media Services (3.0 at this time this was written), defines a media file format that IIS can adaptively stream to Silverlight clients. Microsoft Expression Encoder allows you to define various "streams" for the media file, containing different bitrates, sizes, and resolutions. The "adaptation" means that IIS will detect your bandwidth and serve up the appropriate version of the file.

Most of the work we need to do to finish off SUESS regards configuring IIS on the media server. Here's all you have to do:

1. Install IIS Media Services on your server. You can get that here.
2. Create a new virtual directory under the Encoder Service's website (let's call it "media") that points to the destination folder that our SUESS Encoder dumps the new sparkly SmoothStreaming files into.
3. Set both EncoderService (website) and media (virtual directory) to only have Anonymous Authentication enabled. A clientaccesspolicy.xml file in the root of EncoderService handles security for us.
4. Make sure the application pool identity and the "Physical Path Credentials" of the virtual directory have sufficient I/O permissions to read from this folder. We don't have to worry about the IIS Double Hop issue here, since Silverlight's call to EncoderService is anonymous. So even if the media virtual directory is on a different machine, that's only one bunny hop. For performance reasons, however, I have Encoder dump its files to the same server that is hosting Media Services.

Let's see where that gets us.

On the left, you can see "EncoderService" and its "media" virtual directory. Expression Encoder, by default, creates a folder with the following naming convention "<computer name> <timestamp>" under the destination folder. This is of course directly configurable in the API. On the right, I have the IIS "Content View" of one of these folders.

Here is what SmoothStreaming formatted media looks like in the light of day. The "ISMV" files are the actual streams that Encoder creates. To ensure again file name collusions, I prepend a Guid to the beginning of each. (This was from a beta version of SUESS that didn't have Encoder create a folder for each media file, which I now like better; the aforementioned "-Guid" is therefore superfluous.) Finally, Encoder affixes an underscore plus the bitrate to the end of each stream file.

Next we have those "ISM" and "ISMC" files hanging around. These are the manifest files that the server uses to define the SmoothStreaming file and tells the requesting clients what to expect. Both are just XML. The "ISMC" one is the "client" manifest file that helps Silverlight dissect the streams into the video and audio components, and exposes the different quality levels available.

The "ISM" file is the "server" manifest that is written in a special dialect of XML called "SMIL." This is more of a file manifest that defines the physical locations of all the constituent pieces that make up the SmoothStream. Finally, the thumbnail is (optionally) created here as well, following the same naming convention. You can specify the size and image format (by way of a codec) in the API, and the appropriate image will be generated.

Something that was way more difficult than it should have been was determining the actual URL of the SmoothStreaming media! For whatever reason, the research I did yielded misleading and conflicting results, so let me explicitly show the construction of the URL that Silverlight needs to have fed to its Uri for the SmoothStreamingMediaElement's SmoothStreamingSource property:

http(s)://<"EncoderService" web site>/<"media" virtual directory>/<folder(s) - if used>/<file name without extension>.ism/manifest

And guess what? That's it. Even though there's a new section called "Media Services" in the "Features View" of each IIS site, you don't need to mess with anything there. Encoder gives us so much control over the format of the SmootheStreaming files that we don't need to tell IIS to do anything else; it just servers the file, and Media Services and Silverlight high-five each other and tag team to show it.

On the client side, Silverlight can play these files by way of the SmoothStreamingMediaElement. This was one of the points that tripped me up: the standard MediaElement doesn't know what to do with these files. Instead, you have to get your hands on Microsoft.Web.Media.SmoothStreaming.dll, which can be found in "C:\Program Files (x86)\Microsoft SDKs\IIS Smooth Streaming Client\v1.0" after you install IIS Media Services.

Drop this DLL into your Media Player Silverlight project, and you can then use the SmoothStreamingMediaElement the same as a MediaElement. The only major different is that this control has a SmoothStreamingSource Uri property in addition to the standard Source property. Just set this to a Uri pointing at your SUESS media server and code up the rest of your player all the same. SmoothStreamingMediaElement also has support for "Live" Encoding, so you can broadcast source from a web cam down to Silverlight clients.

That's really all there is to discuss in terms of SmoothStreaming. Like I said, most of the magic is within the Encoder itself, as it converts raw media into this format. IIS Media Services just sort of knows what to do with it server side, and Silverlight controls with SmoothStreamingMediaElement players just sort of know what to do with it client side.

It just works.

Well, that closes the book on SUESS! Hopefully, as you've trudged through this series with me, there has been some enlightenment not only in terms of creating a Web 2.0 media stack for your clients in general, but also specifically to the new generation of .NET technologies. Silverlight 4, WCF/.Net 4, Microsoft Expression 3.0, and IIS Media Services 3.0 all wrap around the concept of SmoothStreaming and add their own functionality to the paradigm. My goal with SUESS was to bring these separate technologies together in a cohesive, media-centric way.

Have fun!