File Transfer with WCF

August 28, 2009
tags: , ,
by Stefano Ricciardi

WCF streaming is helpful in all those cases when you need to move a lot of data between the client and the service in a way that keeps memory consumption under control (see Large Data and Streaming on MSDN for more details if you are not familiar with the topic).

Streaming does impose some restrictions. Among them is that your operations can only have one input and/or output parameter and they can only be only one of the following:

  • a Stream
  • a Message
  • a class implementing IXmlSerializable

I wanted to expose a very simple service to upload and download files through WCF. And because I wanted to be able to pass both the file content as a stream and at least the file name, I could not just use a Stream but had to resort to using Messages: the headers would convey the file information (such as the file type) and the body the content of the file itself through a stream.

I therefore defined the service as follows:

    [ServiceContract(Namespace = "http://schemas.acme.it/2009/04")]
    public interface IFileTransferService
    {
        [OperationContract(IsOneWay = true)]
        void UploadFile(FileUploadMessage request);
        [OperationContract(IsOneWay = false)]
        FileDownloadReturnMessage DownloadFile(FileDownloadMessage request);
    }

    [MessageContract]
    public class FileUploadMessage
    {
        [MessageHeader(MustUnderstand = true)]
        public FileMetaData Metadata;
        [MessageBodyMember(Order = 1)]
        public Stream FileByteStream;
    }

    [MessageContract]
    public class FileDownloadMessage
    {
        [MessageHeader(MustUnderstand = true)]
        public FileMetaData FileMetaData;
    }

   [MessageContract]
    public class FileDownloadReturnMessage
    {
        public FileDownloadReturnMessage(FileMetaData metaData, Stream stream)
        {
            this.DownloadedFileMetadata = metaData;
            this.FileByteStream = stream;
        }

        [MessageHeader(MustUnderstand = true)]
        public FileMetaData DownloadedFileMetadata;
        [MessageBodyMember(Order = 1)]
        public Stream FileByteStream;
    }

   [DataContract(Namespace = "http://schemas.acme.it/2009/04")]
    public class FileMetaData
    {
        public FileMetaData(
            string localFileName,
            string remoteFileName)
        {
            this.LocalFileName = localFileName;
            this.RemoteFileName = remoteFileName;
            this.FileType = FileTypeEnum.Generic;
        }

        public FileMetaData(
            string localFileName,
            string remoteFileName,
            FileTypeEnum fileType)
        {
            this.LocalFileName = localFileName;
            this.RemoteFileName = remoteFileName;
            this.FileType = fileType;
        }

        [DataMember(Name="FileType", Order=0, IsRequired=true)]
        public FileTypeEnum FileType;</p>
        [DataMember(Name="localFilename", Order=1, IsRequired=false)]
        public string LocalFileName;
        [DataMember(Name = "remoteFilename", Order = 2, IsRequired = false)]
        public string RemoteFileName;
    }

    public class FileTransferService : IFileTransferService
    {
        public void UploadFile(FileUploadMessage request)
        {
             // implementation here
        } 

        public FileDownloadReturnMessage DownloadFile(FileDownloadMessage request)
        {
             // implementation here
        }
    }

I am omitting the details of the implementation, they are pretty straightforward (if you are interested, I will gladly share it. UPDATE: I have put the basic scheleton of the server implementation is a separate post).

And this is how the service was configured on the server side within the web.config.

<system.serviceModel>
    <behaviors>
      <serviceBehaviors>
        <behavior name="serviceBehavior">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceDebug includeExceptionDetailInFaults="true"
                              httpHelpPageEnabled="true" />
          <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
      </serviceBehaviors>
    </behaviors>
    <services>
      <service behaviorConfiguration="serviceBehavior"
                  name="FileTransferService">
        <endpoint address=""
                  name="basicHttpStream"
                  binding="basicHttpBinding"
                  bindingConfiguration="httpLargeMessageStream"
                  contract="IFileTransferService" />
        <endpoint address="mex"
                      binding="mexHttpBinding"
                      contract="IMetadataExchange"/>
      </service>
    </services>
    <bindings>
      <basicHttpBinding>
        <binding name="httpLargeMessageStream"
                    maxReceivedMessageSize="2147483647"
                    transferMode="Streamed"
                    messageEncoding="Mtom" />
      </basicHttpBinding>
    </bindings>
  </system.serviceModel>

This is the base of the plumbing that it’s needed to create the service.

While I was developing it, I have lost quite some time for an error which was really obscure to decipher:
Server Error in ‘/’ Application. Operation ‘UploadFile’ in contract ‘IFileTransferService’ uses a MessageContract that has SOAP headers. SOAP headers are not supported by the None MessageVersion.

Googling and binging around I found almost no relevant information on the topic, so I was pretty on my own.

I thought the problem was in the HTTP binding that I was using (maybe it was not compatible with SOAP headers?). Unfortunately the HTTP binding did not have any way to specify a different MessageVersion. So I ended up creating a custom binding, like the following, but that didn’t help either:

      <customBinding>
        <binding name="customHttpBindingStream">
          <textMessageEncoding messageVersion="Default"/>
          <httpTransport
               transferMode="Streamed"
               maxReceivedMessageSize="2147483647" />
        </binding>
      </customBinding>

I made several other attempts with no luck. I then tried self hosting the service and that did work. Then I was certain that it was something with the web hosting. Tried IIS6, IIS7, etc… Same exception all the time.

To make a long story short, at the end, while reducing my web.config to the bare bones, I found the culprit: it was the name of the service in the service configuration.

Instead of writing:

      <service behaviorConfiguration="serviceBehavior"
                  name="FileTransferService">

I had written by mistake:

      <service behaviorConfiguration="serviceBehavior"
                  name="FileTransfer">

Talk about meaningful error messages!

See more at File Transfer with WCF II

Bookmark and Share

Technorati Tags: , , ,
12 Responses leave one →
  1. September 14, 2009
    Vinay permalink

    I’m very much impressed with your presentation and technical abilities. I request you to share the File Upload service code in detail so that I can implement in one of my services. Thanks in advance.

    • October 2, 2009
      Stefano Ricciardi permalink

      Vinay,
      see my latest post for a rough implementation.

  2. September 16, 2009
    George permalink

    Can you share the implementation based on a class implementing IXmlSerializable? I am new to .net and came recently from java world

    • September 17, 2009
      Stefano Ricciardi permalink

      Hi George, thank you for stepping by. Any particular reason why you need to go through the XMLSerializable route?

      • September 18, 2009
        George permalink

        Hi Stefano,

        I would really like to batch the transfer at times so as far as I understand XMLSerializable gives better control during streaming.
        So I thought of trying that way out.

  3. September 18, 2009
    Stefano Ricciardi permalink

    George,

    there’s nothing that prevents using something along the lines of the solution outlined in this post from being invoked from a batch file. As a matter of fact, the proxy that I created from this service is invoked from legacy C++ code through COM.

    As far as I know, implementing IXMLSerializable can give you more control on the way the objects are serialized. But unless you need to transfer complex metadata describing the files being moved back and forth, default WCF serialization should do quite fine.

  4. September 25, 2009
    Eric permalink

    Hi, Stefano:

    I’ve been looking into a WCF implementation to transfer large binary files (images) around the 250-300 MB range. Much like you, I need the filename. However, my concern is that using the message contract will cause the binary file to be serialized into xml. Is this true or am I way off base here?

    Am I going to trade efficiency of a straight stream sans the filename for a complex and large overhead using the message contract just to have the luxury (ahem – necessity) of including a filename?

    Any insights you can provide from your experiences?

  5. September 25, 2009
    Stefano Ricciardi permalink

    Eric,

    be sure to set the binding message encoding to MTOM (see my config above). MTOM is especially suited to exchange large binary data without using base64 conversion (which increases the size by 1/3).

    Read more about MTOM here

  6. October 1, 2009
    Anita permalink

    Hello Stefano,

    Thank you for sharing this helpful information. Is it possible if you share the source code or at least the implemention of UploadFile and DownloadFile.

    Thank you,

    • October 2, 2009
      Stefano Ricciardi permalink

      Hi Anita,

      unfortunately I cannot share the whole source code without violating the intellectual property of my company. I’ll try to post a scheleton implementation of the UploadFile and Download file as soon as possible.

      • October 2, 2009
        Stefano Ricciardi permalink

        Anita,
        see my latest post for a rough implementation.

  7. October 2, 2009
    Anita permalink

    Excellent! Thank you

Leave a Reply

Note: You can use basic XHTML in your comments. Your email address will never be published.

Subscribe to this comment feed via RSS