IOException when uploading to Blob Storage in published app

Question

I've developed a simple app, which just have to upload files from a folder to Azure Blob Storage, I runs fine when I run in from VS, but in the published app I get this error once in a while:

ved System.IO.__Error.WinIOError(Int32 errorCode, String, maybeFullPath) ved System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) ved System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access) ved Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob.UploadFromFile(String path, FileMode mode, AccessCondition accessCondition, BlobRequestOptions options, OperationsContext operationContext) ved Program.MainWindow.Process(object sender, NotifyCollectionChangedEventArgs e)

My code for uploading looks like this:

private void Process(object sender, NotifyCollectionChangedEventArgs e)
{
    if (paths.Count > 0){
        var currentPath = paths.Dequeue();
        CloudStorageAccount storageAccount = CloudStorageAccount.Parse(UserSettings.Instance.getConnectionString());
        CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
        CloudBlobContainer blobContainer = blobClient.GetContainerReference(UserSettings.Instance.getContainer());
        CloudBlockBlob b = blobContainer.GetBlockBlobReference(System.IO.Path.GetFileName(currentPath));
        try
        {
           b.UploadFromFile(currentPath, FileMode.Open);
        }
        catch (StorageException s)
        {
            throw new System.InvalidOperationException("Could not connect to the specified storage account. Please check the configuration.");
        }
        catch (IOException exc)
        {
            throw new System.InvalidOperationException(exc.StackTrace);
        }
    }
}

It is the IOException in the catch that gets hit once in a while, any idea how to fix this?

If I go through the docs, I just get informed that the exception occour if a storage service error occoured. Any idea how to investigate the further?

https://learn.microsoft.com/en-us/java/api/com.microsoft.azure.storage.blob._cloud_blob.uploadfromfile?view=azure-java-legacy#com_microsoft_azure_storage_blob__cloud_blob_uploadFromFile_final_String_

Answer 1

I observed that the error only occours when i copy the files into the monitored folder, if I drag them, it works just fine?

It looks like your app is trying to read local file(s) that are still being written to the disk (incomplete) or locked by other processes. Dragging the files are somehow "atomic" operations (ie very fast) thus considerably reducing the chance of getting this error.

Try implementing the method from this answer to test if the file isn't locked prior to calling UploadFromFile(). Now depending on your code logic, you will need to implement some form of "retries" if the file is locked. Here's an example:

    protected virtual bool IsFileLocked(FileInfo file)
    {
        FileStream stream = null;

        try
        {
            stream = file.Open(FileMode.Open, FileAccess.Read, FileShare.None);
        }
        catch (IOException)
        {
            //the file is unavailable because it is:
            //still being written to
            //or being processed by another thread
            //or does not exist (has already been processed)
            return true;
        }
        finally
        {
            if (stream != null)
                stream.Close();
        }

        //file is not locked
        return false;
    }

    private void Process(object sender, NotifyCollectionChangedEventArgs e)
    {
        if (paths.Count > 0)
        {
            var currentPath = paths.Dequeue();
            CloudStorageAccount storageAccount = CloudStorageAccount.Parse(UserSettings.Instance.getConnectionString());
            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
            CloudBlobContainer blobContainer = blobClient.GetContainerReference(UserSettings.Instance.getContainer());
            CloudBlockBlob b = blobContainer.GetBlockBlobReference(System.IO.Path.GetFileName(currentPath));
            try
            {
                FileInfo fi = new FileInfo(currentPath);
                while (IsFileLocked(fi))
                    Thread.Sleep(5000); // Wait 5 seconds before checking again
                b.UploadFromFile(currentPath, FileMode.Open);
            }
            catch (StorageException s)
            {
                throw new System.InvalidOperationException("Could not connect to the specified storage account. Please check the configuration.");
            }
            catch (IOException exc)
            {
                throw new System.InvalidOperationException(exc.StackTrace);
            }
        }
    }

Note that in no ways this will guarantee the file won't be locked again by another process between the IsFileLocked() call and the b.UploadFromFile() call.