File Transfer with WCF: Part III

This is the third post on a small series about transferring large files with WCF using streaming :

I am closing this series of posts by sharing a skeleton for the client side handling of the uploaded and downloaded files. Please note that the following are excerpt from a larger context and so will not compile in your environment as is: they are only intended to give a rough idea of how the files can be handled.

Download

 private void IssueDownloadRequest(string localFile, string serviceUrl, FileDownloadMessage request)
{
    this.service = this.proxy.OpenProxy(serviceUrl);

    try
    {
        using (FileDownloadReturnMessage response = this.service.DownloadFile(request))
        {
            if (response != null && response.FileByteStream != null)
            {
                SaveFile(response.FileByteStream, localFile);
            }
        }

        this.proxy.CloseProxy(this.service);
    }
    catch (Exception e)
    {
        throw new FileTransferProxyException("Error while downloading the file", e);
    }
    finally
    {
        // we expect the stream returned from the server to be closed by the
        // server itself so nothing to be done with it here. Just abort
        // the proxy if needed.
        this.proxy.AbortProxyIfNeeded(this.service);
    }
}

private static void SaveFile(Stream saveFile, string localFilePath)
{
    const int bufferSize = 65536; // 64K

    using (FileStream outfile = new FileStream(localFilePath, FileMode.Create))
    {
        byte[] buffer = new byte[bufferSize];
        int bytesRead = saveFile.Read(buffer, 0, bufferSize);

        while (bytesRead > 0)
        {
            outfile.Write(buffer, 0, bytesRead);
            bytesRead = saveFile.Read(buffer, 0, bufferSize);
        }
    }
}

Upload

public void UploadFile(string localFileName, string serviceUrl, FileTypeEnum fileType)
{
    this.service = this.proxy.OpenProxy(serviceUrl);
    try
    {
        using (Stream fileStream = new FileStream(localFileName, FileMode.Open, FileAccess.Read))
        {
            var request = new FileUploadMessage();
            string remoteFileName = null;
            if (fileType == FileTypeEnum.Generic)
            {
                // we are using the service as a "FTP on WCF"
                // give the remote file the same name as the local one
                remoteFileName = Path.GetFileName(localFileName);
            }

            var fileMetadata = new FileMetaData(localFileName, remoteFileName, fileType);
            request.MetaData = fileMetadata;
            request.FileByteStream = fileStream;

            this.service.UploadFile(request);
            this.proxy.CloseProxy(this.service);
        }
    }
    catch (IOException)
    {
        throw new FileTransferProxyException("Unable to open the file to upload");
    }
    catch (Exception e)
    {
        throw new FileTransferProxyException(e.Message);
    }
    finally
    {
        this.proxy.AbortProxyIfNeeded(this.service);
    }
 }

kick it on DotNetKicks.com

Shout it

This entry was posted in .NET, C#, WCF and tagged , , . Bookmark the permalink.
  • Pingback: DotNetShoutout

  • Tom

    Using this method, is there a way to track the upload progress on the client side?

  • Tom

    Using this method, is there a way to track the upload progress on the client side?

  • solomon

    Excellent stuff. But i am having one issue.

    Please look at the web.config (server)

    Below is the configuration on the client.

    It seems that i have set everything correctly on my server configuration but when you look the client config file it says maxReceivedMessageSize=”65536″

    and i am gettting the error

    The maximum message size quota for incoming messages (65536) has been exceeded. To increase the quota, use the MaxReceivedMessageSize property on the appropriate binding element.

    Please suggest me a solution and it is very urgent!

    Thanks

  • solomon

    Excellent stuff. But i am having one issue.

    Please look at the web.config (server)

    Below is the configuration on the client.

    It seems that i have set everything correctly on my server configuration but when you look the client config file it says maxReceivedMessageSize=”65536″

    and i am gettting the error

    The maximum message size quota for incoming messages (65536) has been exceeded. To increase the quota, use the MaxReceivedMessageSize property on the appropriate binding element.

    Please suggest me a solution and it is very urgent!

    Thanks

  • stefanoricciardi

    @Tom: I think you might want to have a look at this class: http://www.koders.com/csharp/fid50D325D3316A9E46FDE7DCF81300AD82847BAA23.aspx?s=file#L10

    @Solomon: WCF default settings are pretty conservative (to avoid denial of service attacks, etc…). Have a look to this discussion for more details: http://social.msdn.microsoft.com/forums/en-US/wcf/thread/d2090703-eb60-46f4-adf7-31127338a0a0

  • stefanoricciardi

    @Tom: I think you might want to have a look at this class: http://www.koders.com/csharp/fid50D325D3316A9E46FDE7DCF81300AD82847BAA23.aspx?s=file#L10

    @Solomon: WCF default settings are pretty conservative (to avoid denial of service attacks, etc…). Have a look to this discussion for more details: http://social.msdn.microsoft.com/forums/en-US/wcf/thread/d2090703-eb60-46f4-adf7-31127338a0a0

  • Ali

    Thank you for posting this great article. Would you mind sharing the full project for this article? thanks

  • Ali

    Thank you for posting this great article. Would you mind sharing the full project for this article? thanks

  • Ali

    Never mind. I figured it out

  • Ali

    Never mind. I figured it out

  • solomon

    Okay now i have changed the app.config manually similar to the server config and now i get the exception “The remote server returned an unexpected response: (400) Bad Request.” where as this works good when the transfermode is set to buffered and it does not work if change this to streamed. Can u pls help me out to figure this?

    Thanks

  • solomon

    Okay now i have changed the app.config manually similar to the server config and now i get the exception “The remote server returned an unexpected response: (400) Bad Request.” where as this works good when the transfermode is set to buffered and it does not work if change this to streamed. Can u pls help me out to figure this?

    Thanks

  • Allan Bredahl

    Extremely helpful articles.

    But I can’t get the upload to run. It’s as if the FileByteStream in the UploadRequest message is empty when it gets to the server.

    This line always returns 0:

    request.FileByteStream.Read(buffer, 0, bufferSize)

    I have checked the stream on the client proxy and it looks as it should.

    Any ideas ?

    The dowload is running though its quite slow: 300KBs FTP runs 1500KBs from the same server. Any suggestions to how to tweak WCF speed ?

  • Anonymous

    Allan,

    I’m glad you’ve found my posts useful.

    It’s difficult to understand what might be the reason that you are not able to upload the file. Have you tried using tools like Fiddler to see what’s being passed back and forth?

    As for the performances, FTP runs on raw TCP sockets. The solution that I have sketched is based on HTTP where the stress is based on interoperability more than performance.

    You might try creating a netTcp endpoint and see whether that gives you better performances (it should).

  • Allan Bredahl

    Hi

    Must say that I’m running the test on localhost, should remove most problems, but still no file upload.

    I’m using a netTCP channel

  • Anonymous

    Then I’d suggest you try simpler streaming scenarios (without using message headers) and see whether those works.

  • Allan Bredahl

    Changed the Upload Method to just having a stream as parameter, but still no luck :(

  • Anonymous

    Then there’s something wrong with your code and/or config files. I suggest to have a look at Microsoft WCF information on MSDN, it might help you to sort this out. Good luck!

  • Allan Bredahl

    hmmm

  • Allan Bredahl

    It was me being stupid and chasing ghosts.

    Tried to call the Upload methods directly and could see thats the position of the FileStream was set to its end position. So setting position = 0 before calling the Upload methods solved the problem :)

  • Anonymous

    Great!

    One of these days I should take the time and extract the relevant pieces of
    the whole thing and post it as a solution (right now it’s embedded within
    some other business logic which is not generally relevant).

  • Allan Bredahl

    Just a small Conceptual question about WCF and file streaming. Perhaps you have an opinion about this.

    Say I want to upload eg. 10 files using these upload methods. One way would be to call the upload method 10 times. But would it be cleverer to add the 10files to the same stream before uploading or perhaps, utilizing the same stream for all 10 files but one at a time ??

  • Anonymous

    One of the advantages of streaming versus the default buffered mode is that
    even when you have to transfer megabytes of data the amount of memory that
    needs to be allocated is always controlled. Therefore, in theory there would
    not be any major risk in concatenating several files into one.

    However, I wonder whether that would really be worth it. Granted, you would
    extablish a single connection instead of ten. However, you’d still have some
    post-processing to do on those files to extract the original content back
    into the single files.

    I think only a test can say with your tipical file size can give a
    definitive answer.

  • Joakim

    What I want to know is how you implemented IDisposable on FileDownloadReturnMessage,
    this because the problem I got is that when the client downloads a file it will work the first time, but the second time it would throw an exception since the stream is never disposed of on the server.

    The way I implemented it right now is that the Dispose method does another call to the service with a key that identifies the stream to be disposed of (in other words all streams are saved waiting for this call).

  • Joakim

    What I want to know is how you implemented IDisposable on FileDownloadReturnMessage,
    this because the problem I got is that when the client downloads a file it will work the first time, but the second time it would throw an exception since the stream is never disposed of on the server.

    The way I implemented it right now is that the Dispose method does another call to the service with a key that identifies the stream to be disposed of (in other words all streams are saved waiting for this call).

  • Joakim

    Ok, the first time I did Dispose on FileByteStream it did this on a local copy of it, when I went inside the proxy I changed the method to directly output the response from the wcf service, after this when I did a Dispose on FileByteStream it got disposed on the service as well.

  • Anonymous

    Joakim,
    since I wrote this post I have changed my job therefore I don’t have the original code in front of me anymore.

    There’s a comment in the Part I post of this series from Razzi that might help.

    In the scenarios where I have used my code I have never encountered the problem you mentioned.

    Hope this help.
    Stefano

  • Joakim

    Well, must say this has been a great deal of help, I actually tried a different setup than Razzi on FileShare.ReadWrite, i used FileShare.Read which would enable multiple users to download the file but it never closed on the server itself which is weird, I’ll try with FileShare.ReadWrite and see if it fixes it.
    Many thanks for the blog-posts they were extremely helpful together with msdn, stackoverflow and the rest of the blogs I’ve read over the past few days.