Skip to content

File Transfer with WCF

2009 August 28
tags: , ,
by stefanoricciardi

WCF streaming is helpful in all those cases when you need to move a lot of data between the client and the service in a way that keeps memory consumption under control (see Large Data and Streaming on MSDN for more details if you are not familiar with the topic).

Streaming does impose some restrictions. Among them is that your operations can only have one input and/or output parameter and they can only be only one of the following:

  • a Stream
  • a Message
  • a class implementing IXmlSerializable

I wanted to expose a very simple service to upload and download files through WCF. And because I wanted to be able to pass both the file content as a stream and at least the file name, I could not just use a Stream but had to resort to using Messages: the headers would convey the file information (such as the file type) and the body the content of the file itself through a stream.

I therefore defined the service as follows:

    [ServiceContract(Namespace = "http://schemas.acme.it/2009/04")]
    public interface IFileTransferService
    {
        [OperationContract(IsOneWay = true)]
        void UploadFile(FileUploadMessage request);
        [OperationContract(IsOneWay = false)]
        FileDownloadReturnMessage DownloadFile(FileDownloadMessage request);
    }

    [MessageContract]
    public class FileUploadMessage
    {
        [MessageHeader(MustUnderstand = true)]
        public FileMetaData Metadata;
        [MessageBodyMember(Order = 1)]
        public Stream FileByteStream;
    }

    [MessageContract]
    public class FileDownloadMessage
    {
        [MessageHeader(MustUnderstand = true)]
        public FileMetaData FileMetaData;
    }

   [MessageContract]
    public class FileDownloadReturnMessage
    {
        public FileDownloadReturnMessage(FileMetaData metaData, Stream stream)
        {
            this.DownloadedFileMetadata = metaData;
            this.FileByteStream = stream;
        }

        [MessageHeader(MustUnderstand = true)]
        public FileMetaData DownloadedFileMetadata;
        [MessageBodyMember(Order = 1)]
        public Stream FileByteStream;
    }

   [DataContract(Namespace = "http://schemas.acme.it/2009/04")]
    public class FileMetaData
    {
        public FileMetaData(
            string localFileName,
            string remoteFileName)
        {
            this.LocalFileName = localFileName;
            this.RemoteFileName = remoteFileName;
            this.FileType = FileTypeEnum.Generic;
        }

        public FileMetaData(
            string localFileName,
            string remoteFileName,
            FileTypeEnum fileType)
        {
            this.LocalFileName = localFileName;
            this.RemoteFileName = remoteFileName;
            this.FileType = fileType;
        }

        [DataMember(Name="FileType", Order=0, IsRequired=true)]
        public FileTypeEnum FileType;</p>
        [DataMember(Name="localFilename", Order=1, IsRequired=false)]
        public string LocalFileName;
        [DataMember(Name = "remoteFilename", Order = 2, IsRequired = false)]
        public string RemoteFileName;
    }

    public class FileTransferService : IFileTransferService
    {
        public void UploadFile(FileUploadMessage request)
        {
             // implementation here
        }

        public FileDownloadReturnMessage DownloadFile(FileDownloadMessage request)
        {
             // implementation here
        }
    }

I am omitting the details of the implementation, they are pretty straightforward (if you are interested, I will gladly share it. UPDATE: I have put the basic scheleton of the server implementation is a separate post).

And this is how the service was configured on the server side within the web.config.

<system.serviceModel>
    <behaviors>
      <serviceBehaviors>
        <behavior name="serviceBehavior">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceDebug includeExceptionDetailInFaults="true"
                              httpHelpPageEnabled="true" />
          <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
      </serviceBehaviors>
    </behaviors>
    <services>
      <service behaviorConfiguration="serviceBehavior"
                  name="FileTransferService">
        <endpoint address=""
                  name="basicHttpStream"
                  binding="basicHttpBinding"
                  bindingConfiguration="httpLargeMessageStream"
                  contract="IFileTransferService" />
        <endpoint address="mex"
                      binding="mexHttpBinding"
                      contract="IMetadataExchange"/>
      </service>
    </services>
    <bindings>
      <basicHttpBinding>
        <binding name="httpLargeMessageStream"
                    maxReceivedMessageSize="2147483647"
                    transferMode="Streamed"
                    messageEncoding="Mtom" />
      </basicHttpBinding>
    </bindings>
  </system.serviceModel>

This is the base of the plumbing that it’s needed to create the service.

While I was developing it, I have lost quite some time for an error which was really obscure to decipher:
Server Error in ‘/’ Application. Operation ‘UploadFile’ in contract ‘IFileTransferService’ uses a MessageContract that has SOAP headers. SOAP headers are not supported by the None MessageVersion.

Googling and binging around I found almost no relevant information on the topic, so I was pretty on my own.

I thought the problem was in the HTTP binding that I was using (maybe it was not compatible with SOAP headers?). Unfortunately the HTTP binding did not have any way to specify a different MessageVersion. So I ended up creating a custom binding, like the following, but that didn’t help either:

      <customBinding>
        <binding name="customHttpBindingStream">
          <textMessageEncoding messageVersion="Default"/>
          <httpTransport
               transferMode="Streamed"
               maxReceivedMessageSize="2147483647" />
        </binding>
      </customBinding>

I made several other attempts with no luck. I then tried self hosting the service and that did work. Then I was certain that it was something with the web hosting. Tried IIS6, IIS7, etc… Same exception all the time.

To make a long story short, at the end, while reducing my web.config to the bare bones, I found the culprit: it was the name of the service in the service configuration.

Instead of writing:

      <service behaviorConfiguration="serviceBehavior"
                  name="FileTransferService">

I had written by mistake:

      <service behaviorConfiguration="serviceBehavior"
                  name="FileTransfer">

Talk about meaningful error messages!

See more at File Transfer with WCF II

Bookmark and Share

Technorati Tags: , , ,
18 Responses leave one →
  1. Vinay permalink
    September 14, 2009

    I’m very much impressed with your presentation and technical abilities. I request you to share the File Upload service code in detail so that I can implement in one of my services. Thanks in advance.

  2. George permalink
    September 16, 2009

    Can you share the implementation based on a class implementing IXmlSerializable? I am new to .net and came recently from java world

  3. Stefano Ricciardi permalink*
    September 17, 2009

    Hi George, thank you for stepping by. Any particular reason why you need to go through the XMLSerializable route?

  4. George permalink
    September 18, 2009

    Hi Stefano,

    I would really like to batch the transfer at times so as far as I understand XMLSerializable gives better control during streaming.
    So I thought of trying that way out.

  5. Stefano Ricciardi permalink*
    September 18, 2009

    George,

    there’s nothing that prevents using something along the lines of the solution outlined in this post from being invoked from a batch file. As a matter of fact, the proxy that I created from this service is invoked from legacy C++ code through COM.

    As far as I know, implementing IXMLSerializable can give you more control on the way the objects are serialized. But unless you need to transfer complex metadata describing the files being moved back and forth, default WCF serialization should do quite fine.

  6. Eric permalink
    September 25, 2009

    Hi, Stefano:

    I’ve been looking into a WCF implementation to transfer large binary files (images) around the 250-300 MB range. Much like you, I need the filename. However, my concern is that using the message contract will cause the binary file to be serialized into xml. Is this true or am I way off base here?

    Am I going to trade efficiency of a straight stream sans the filename for a complex and large overhead using the message contract just to have the luxury (ahem – necessity) of including a filename?

    Any insights you can provide from your experiences?

  7. Stefano Ricciardi permalink*
    September 25, 2009

    Eric,

    be sure to set the binding message encoding to MTOM (see my config above). MTOM is especially suited to exchange large binary data without using base64 conversion (which increases the size by 1/3).

    Read more about MTOM here

  8. Anita permalink
    October 1, 2009

    Hello Stefano,

    Thank you for sharing this helpful information. Is it possible if you share the source code or at least the implemention of UploadFile and DownloadFile.

    Thank you,

  9. Stefano Ricciardi permalink*
    October 2, 2009

    Hi Anita,

    unfortunately I cannot share the whole source code without violating the intellectual property of my company. I’ll try to post a scheleton implementation of the UploadFile and Download file as soon as possible.

  10. Stefano Ricciardi permalink*
    October 2, 2009

    Anita,
    see my latest post for a rough implementation.

  11. Stefano Ricciardi permalink*
    October 2, 2009

    Vinay,
    see my latest post for a rough implementation.

  12. Anita permalink
    October 2, 2009

    Excellent! Thank you

  13. Pathmini permalink
    March 13, 2010

    Hello Stefano,

    Could you please provide me the code file for File Upload Service. I would like to test it out and see..

    Thanks.

    Pathmini

  14. Stefano Ricciardi permalink*
    March 15, 2010

    Pathmini,

    have you looked at http://stefanoricciardi.com/2009/10/02/file-transfer-with-wcf-part-ii/ ? There you can find a scheleton of the server implementation.

  15. Anton permalink
    April 6, 2010

    Hello, Stefano!

    Thank you for your articale!

    Why I can’t just simply use the following construction to transfer files:

    [DataContract]
    public class UserDataFile
    {

    private FileStream _toStore;
    private int _groupId;

    public UserDataFile(FileStream stream, int groupId)
    {
    ToStore = stream;
    GroupId = groupId;

    }

    [DataMember]
    public FileStream ToStore
    {
    get { return _toStore; }
    set { _toStore = value; }
    }

    [DataMember]
    public int GroupId
    {
    get { return _groupId; }
    set { _groupId = value; }
    }

  16. stefanoricciardi permalink*
    April 6, 2010

    Anton,

    have you tried the approach you outlined? What errors did you get?

    I guess you are not interested in trying streaming since there are limitations on what the DataContract can be in this case, as highlighted at the top of my post.

  17. Anton permalink
    April 7, 2010

    Hello Stefano!
    Thank you for your reply.

    I need to transfer big files from client to service and back. On the service side I need to store it in DB. For this purposes I use ORM that maps varbinary(MAX) to byte[]. So it seems good to transfer files like byte[], but AFAIK, the best way for sending big files is streaming. But when I use stream I can’t get it’s length (throws exception) and as result I can’t convert it to byte[]. Maybe you can suggest something.

    Thanks in advance.

  18. stefanoricciardi permalink*
    April 7, 2010

    Anton,
    the scenario you are describing is similar to mine. I suggest you try the approach I suggested above and see whether it can help you.

    I have been able to use streaming and to pass the parameters I needed once I switched to using Message Contracts. It’s not that much more work to do than plain DataContracts.

Leave a Reply

Note: You can use basic XHTML in your comments. Your email address will never be published.

Subscribe to this comment feed via RSS

8 visitors online now
8 guests, 0 members
Max visitors today: 8 at 12:47 am CEST
This month: 8 at 08-01-2010 12:47 am CEST
This year: 40 at 05-03-2010 04:48 am CEST
All time: 40 at 05-03-2010 04:48 am CEST