How We Did It: Podcasting Kit for SharePoint

[LEFT][CODE]http://blogs.msdn.com/sharepoint/archive/2009/01/29/how-did-they-do-it-podcasting-kit-for-sharepoint.aspx[/CODE]


Today’s guest blog post is from 3Sharp, the primary developer of the Podcasting Kit for SharePoint. 3Sharp has very deep technical knowledge of PKS and can effectively leverage it to quickly implement custom social media applications for the enterprise. We hope that today’s blog will help you understand how components of PKS were designed, and how you might leverage these components in your own SharePoint solutions.
Enjoy!
Dave Pae
SharePoint Technical Product Manager
[B]The Podcasting Kit for SharePoint (PKS)[/B]
The PKS is an accelerator for social media, using podcasting and social networks to deliver the next-generation knowledge management solution. Built on top of Office SharePoint Server 2007 and using Silverlight 2, the PKS delivers an integrated experience with a wide variety of podcast-capable devices including computers, iPod mobile digital devices, Zune digital media players, and Windows Mobile-based phones. For more information about how PKS can be used in your company, please see the following resources:

[LIST][*][URL="http://blogs.msdn.com/sharepoint/archive/2008/06/10/announcing-open-source-podcasting-kit-for-sharepoint-the-power-of-social-media-for-the-enterprise.aspx"]Announcing open source Podcasting Kit for SharePoint - the power of social media for the enterprise[/URL][*][URL="http://getsharp.3sharp.com/Pages/pkshomepage.aspx"]Visit GetSharp, 3Sharp’s external implementation of PKS to see a live site in action[/URL][*][URL="http://codeplex.com/pks"]Download PKS for free at Codeplex[/URL][/LIST]
[URL="http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_2.png"][IMG]http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_thumb.png[/IMG][/URL]
Figure 1: GetSharp home page
[B]PKS Functionality Overview[/B]
When developing PKS, we were given a few very specific guidelines. One, the solution needed to pass Microsoft IT’s rigorous internal code reviews and general development guidelines. Two, our development had to be as componentized as possible: ideally, each piece of functionality in the PKS could be broken off and used in other solutions. Microsoft requested this component-based design so that other teams and solutions running within the firewall could take advantage of these pieces of functionality. However, anyone can do the same for their external applications as well.
At its basic level, PKS stores all of its content in a single list derived from the PKS Podcast content type. This content is then shown in a Content by Query (CbQ) Web part, using an XSLT for the fancy grid-view display.
[URL="http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_4.png"][IMG]http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_thumb_1.png[/IMG][/URL]
Figure 2: Screenshot of 3Sharp’s public-facing PKS implementation with the CbQ Web part highlighted
In the screenshot above, you can see the CbQ Web part called out. The remaining controls on the front screen include the Tag Cloud control, which is driven from data in the Tags field of the Podcast Item content type; the filter control, which drives the CbQ query, literally filtering on the Categories field in the PKS Podcast content type; and a few HTML controls.
All of this functionality is built on SharePoint basics, which was a third major goal for this solution. Too frequently when teams develop solutions for SharePoint, they end up developing standalone .NET applications that happen to be running in SharePoint. We wanted to make sure that we took advantage of all that SharePoint has to offer and only added new or different functionality when absolutely necessary.

[B]Our File Storage Solution[/B]
Having just climbed up on our soapbox about pursuing a goal to use out-of-the-box SharePoint functionality, we will now dedicate the rest of this post to describe why we wrote our own file storage mechanism instead of using a SharePoint document library to store our video and audio content.
When considering the nature of the solution, we realized we had a design decision ahead of us regarding where to store the large quantities of relatively sizeable video and audio files. Obviously, SharePoint was designed to store files in lists, along with all of the relevant metadata. If we chose this route, we’d need no custom code and we would meet our directive of using native SharePoint functionality whenever possible. However, we knew that we would also run into problems. First, we had a requirement to store very large files: one of our must-pass tests was uploading a 100 GB file. [[B]Note:[/B] Although this works, we don’t recommend you store a 100 GB file in a PKS site.] Another requirement we had was to allow for “supporting details” to be uploaded with each podcast. These files could be anything from Office documents to XML schemas to source code to other video files. Clearly, bundling all of these files together as a single attachment and storing them with the podcast in the list item could easily overwhelm SharePoint.
Therefore, instead of using the document library approach, we decided to store the files on disk, outside of SharePoint. So, along with our list item, storing all of the metadata about the podcast and its supporting files, we would have [I]n[/I] number of files also stored on disk, independent of SharePoint and its storage mechanism, SQL Server. This gave us the following benefits:

[LIST][*]We’re not overwhelming SharePoint and SQL Server with a lot of very large files.[*]The video files can be backed up independently of the SharePoint SQL backups.[*]It’s easy to store any number of files for a given podcast item.[*]Storing the large files on disk is more efficient than storing them in SQL Server.[*]We’re opening up the possibility of streaming files.[*]We can use technologies like BITS for uploading.[*]Using a Silverlight upload method allows a potential for throttling uploads to more easily shape network performance on the farm.[*]Using a network URI for the local file store (LFS) root allows IIS to be set up in an NLB configuration to take advantage of IIS scale and performance.[/LIST]
However, there are also a few drawbacks.

[LIST][*]The videos files [I]must[/I] be backed up independently of SharePoint SQL backups. That is, we have no referential integrity between our file store and the SharePoint list.[*]Implementers are responsible for making sure the solution works with multiple SharePoint Web front ends.[/LIST]
[B]Our Design[/B]
Even though we chose to store our content outside of the SharePoint list, we still worked hard to use as much native SharePoint functionality as possible. To make the connection between the files on disk and SharePoint, we created a [URL="http://msdn.microsoft.com/en-us/library/ms446361.aspx"]custom field type[/URL], named Podcast Files, and added it to our content type. The custom field type stores some XML that defines the various files corresponding to the podcast item, as well as their location, which one is the “primary” file, and other important attributes.
<ExternalFilesMetadata xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Files>
<ExternalFileInfo IsPrimary="true" IsInLocalFileStore="true">
<FileName></FileName>
<OriginalFileName> </OriginalFileName>
<ServerPath></ServerPath>
<ClientPath></ClientPath>
<ClientLink></ClientLink>
</ExternalFileInfo>
</Files>
<UpdatePrimaryInformation>false</UpdatePrimaryInformation>
</ExternalFilesMetadata>
The custom field type also needs to tell SharePoint what to do when it is rendering the control in the Add/Edit pages as well as the normal display pages. Finally, some of the content stored in the XML is going to be consistent across all files uploaded, namely the ServerPath and the ClientLink. Obviously, we don’t want to ask users to fill these in when they are uploading. If they were to even know this, which is doubtful, they might type in the wrong values. Fortunately, administrators can set this metadata when configuring the podcast list as long as we provide a field editor control in the field type definition.
[URL="http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_6.png"][IMG]http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_thumb_2.png[/IMG][/URL]
Figure 3: Architecture of the Podcast Files field type
[B]Setting the Field’s Properties[/B]
A [URL="http://msdn.microsoft.com/en-us/library/bb802857.aspx"]field editor control[/URL] is used to configure the base directory for the local file store and upload mechanism to upload files. The following properties are defined.

[LIST][*]Upload Location: This property is used to configure the LFS location. Only network and local file system URIs are accepted as valid URIs.[*]Upload Method: The LFS implementation allows for multiple upload mechanisms. This property is a drop-down menu listing the different upload mechanisms available to choose from (HTTP Post, Silverlight).[*]IIS Base URL: This property is the virtual path whose physical path points to the Upload Location URI. This is used by the custom field to generate the URLs to the uploaded files.[*]Filename Size Limit: This helps limit the final full path of a particular uploaded file when combined with the Upload Location URI to within [URL="http://msdn.microsoft.com/en-us/library/aa365247.aspx"]MAX_PATH[/URL] (260 char).[/LIST]
[URL="http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_8.png"][IMG]http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_thumb_3.png[/IMG][/URL]
Figure 4: Custom field properties site column configuration
[B]Displaying the Field[/B]
In order to display our custom field, we had to first inherit from the SPFieldMultiLineText class in the SharePoint API. This class holds the value for our field and also selects which FieldRenderingControl to display based on the custom property, UploadMethod. We wrote a class inheriting from BaseFieldControl for each upload method, HTTP Post and Silverlight, to use as the possible FieldRenderingControls.
[B]Uploading through HTTP Post[/B]
When the UploadMethod property is set to HTTP Post upload, the field will display the HTTP Post FieldRenderingControl in the new/edit page for the podcast. The functionality of this upload method will be familiar to most Internet users, and is similar to the normal SharePoint functionality for adding new items to document libraries. The user simply clicks the Browse button and selects a file from their computer, which is then listed in the text box. They then fill in information on the podcast and click the OK button and the podcast is saved to the list.
[URL="http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_10.png"][IMG]http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_thumb_4.png[/IMG][/URL]

Figure 5: The HTTP Post file upload method displayed on the new/edit page of the podcast list
The HTTP Post FieldRenderingControl uses a group of ASP.NET FileUpload controls to allow the user to select files, and provides a text box for any reference hyperlinks. When the user clicks OK to save a podcast the field control reads the information from these fields and processes it. Any files that are marked to be uploaded are saved to the server using the [I]SaveAs [/I]method of the ASP.NET FileUpload control.
The Upload Location property of the field control determines the base directory for saving the podcast. Inside this directory, nested folders are created using the Web ID, List ID, and Item ID for the podcast. This lets us be sure that the files in a directory are for one particular podcast and lets us use the same base URI for several different PKS sites.
Once the files are uploaded, the data on all the files and any reference links for the item is written to XML and saved in the value of the custom UploadFiles field. Also the value of the FileURL and RssFileURL fields are updated for the podcast. These fields are needed to enable downloading of the primary podcast file from the RSS feed and the home page.
[B]Uploading through Silverlight[/B]
While the HTTP Post method for the Podcasting Kit stays relatively true to native SharePoint functionality, it does have some drawbacks. Specifically, the standard upload methods in ASP.NET don’t have a mechanism for giving feedback to the user on file upload progress, and when uploading large files it is hard to tell if it is working or if the site has stopped responding.
We chose to build a Silverlight uploader to host inside the podcast new upload page [~/listname/NewForm.aspx and EditForm.aspx]. From within the Silverlight uploader, users can add, remove, and edit the item’s files as well as upload them. Because the file upload takes place before the corresponding list item is actually created, this method not only has the relational integrity issues with storing the files outside of SharePoint, but also loses the synchronicity between uploading the files and creating the item. However, it allows for a much more dynamic user experience.
The Silverlight Multi-file Upload custom field control consists of three main parts:
1. The [B]Silverlight control[/B] handles users’ input and breaks the files into chunks that can be sent to the Upload ASPX page.
2. The [B]Upload ASPX page[/B] receives file chunks from the Silverlight control and saves them to the local file store.
3. The [B]SharePoint custom field control[/B] renders the Silverlight control and updates the field’s value with the files that were uploaded.
[URL="http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_12.png"][IMG]http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_thumb_5.png[/IMG][/URL]
Figure 6: The Silverlight control architecture
[B]The Silverlight Control[/B]
The Silverlight control is designed as a stand-alone component that can be used in any Web site, not just the Podcasting Kit for SharePoint. We recognized that this would be a fun control to use in other sites in the future. It expects two parameters on load:

[LIST][*]The location of the Upload ASPX page.[*]The hidden field on the page that will be used for communicating with the UploadFiles custom field.[/LIST]
The UploadFiles custom field uses an HTML hidden field to communicate any currently uploaded files for the podcast to the Silverlight control. If it is a new podcast, it remains blank.
The UI for the Silverlight uploader has two buttons at the top that allow the user to add files from their local system and add hyperlinks to any related Web content. The information on the selected files and hyperlinks is added to a list. Items in the list can be selected as the primary file for the podcast and can also be deleted. Information on the size of the files is also displayed, as well the total file count and size below the list.
[URL="http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_14.png"][IMG]http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_thumb_6.png[/IMG][/URL]
Figure 7: Upload Files Silverlight Control
When a user clicks the Upload button, the files are broken into blocks and sent to the Upload.aspx page that then saves them to the LFS. The control will vary the block size being sent to the server. It starts with 4 KB and can move up to a maximum of 2 MB. This allows the control to respond to the speed of the current network connection.
All the file blocks are sent to the Upload.aspx page as a base 64 encoded string. We decided to use this method so that the data could be put in the page body of the post as a plain string. Base 64 is also a pretty dense encoding, so we also save some space using this method. Along with the file data, the name of the file, the byte offset being written to, whether or not this is the end of the file, and whether or not to overwrite a previously uploaded file is also sent to the Upload.aspx page.
When all the files are uploaded, the Silverlight control reads the metadata for each file and hyperlink and writes it to the hidden field that the UploadFiles field control wrote to the page. When the user finishes entering data for the podcast and clicks the OK button in SharePoint, the UploadFiles field reads this metadata from the hidden field and saves it in the same manner as the HTTP Post method.
It may seem a bit unusual that we are using a regular ASPX page on the server for handling uploads. This came about because of several design restrictions. Silverlight applications exist on the client machine and can’t access the server’s file system directly, so there has to be a Web-exposed mechanism that accepts the data from the client. Normally, one would use a Web service for this task or possibly call-back to the server-side code for the page hosting the Silverlight app. For the PKS project, we had a specific requirement that we could not add Web services to SharePoint, so the first option was out of the question. While we could have used call-backs, we decided against it so that the Silverlight Upload control wouldn’t be tied to the custom field and could be reused in other projects more easily. Using an ASPX page allowed us to get similar usage as an XML Web service, while staying within our design restraints.
[URL="http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_16.png"][IMG]http://blogs.msdn.com/blogfiles/sharepoint/WindowsLiveWriter/HowDidTheyDoItPodcastingKitforSharePoint_D818/image_thumb_7.png[/IMG][/URL]
Figure 8: Upload Files Silverlight control
[B]Conclusion[/B]
Although it is not perfect, the local file store functionality gave us a simple way of storing large files and collections of files that correspond to a SharePoint list. You now know when and why we chose to write custom code rather than using native SharePoint functionality.
Finally, this solution is a free download. It comes as part of the Podcasting Kit for SharePoint, but it can be adapted to use in any SharePoint installation.
[B]Jeremy Campbell [/B]– Developer at 3Sharp
[B]Namita Prakash[/B] – SharePoint Developer
[B]John Peltonen[/B] – Co-founder of 3Sharp
[B]Erica Toelle[/B] – Business Analyst at 3Sharp

[/LEFT]