Java creates HUGE file when writing via Windows Remote Desktop (tsclient)

Question

A client of ours reported a very weird issue when our Swing application is writing a file to the users local machine via Windows Remote Desktop (the application is hosted on a terminal server where users connects).

The flow is:

• Users logon and run the application via remote desktop (with their C:\ included as a "Local resource")
• While working they export data from the database into files
• The user chooses what data to export
• The user selects a destination file on their local computer like \\tsclient\C\Temp\TestFile.txt
• Files could be big so 1000 rows are fetched from database and written to file per batch
• On the second batch, when Java opens the file and write to it again, something really weird starts to happen!
  • The file increases rapidly in size and stops at around 2 GB
  • Then data continues to be written to the file

I'm not sure if this is a problem in the core Java libraries, the Remote Desktop implementation or a combination. Our application is also hosted via Citrix which works fine, and writing to local disk or UNC network paths works fine as well.

I've created a SSCCE demonstrating the problem, connect to a computer with Remote Desktop (make sure C:\ is a "local resource") and run the program to see some really strange behavior! I'm using JDK-7u45.

import static java.nio.file.StandardOpenOption.APPEND;
import static java.nio.file.StandardOpenOption.CREATE;
import static java.nio.file.StandardOpenOption.TRUNCATE_EXISTING;
import static java.nio.file.StandardOpenOption.WRITE;

import java.io.BufferedWriter;
import java.io.File;
import java.io.IOException;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.nio.charset.Charset;
import java.nio.charset.CharsetEncoder;
import java.nio.file.Files;
import java.nio.file.OpenOption;
import java.util.Collections;

/**
 * Demonstrates weird issue when writing (appending) to a file over TsClient (Microsoft Remote Desktop).
 * 
 * @author Martin
 */
public class WriteOverTsClientDemo
{
    private static final File FILE_TO_WRITE = new File("\\\\tsclient\\C\\Temp\\TestFile.txt");
    //private static final File FILE_TO_WRITE = new File("C:\\Temp\\TestFile.txt");

    private static final String ROW_DATA = "111111111122222222223333333333444444444555555555566666666667777777777888888888899999999990000000000";

    public static void main(String[] args) throws IOException
    {
        if (!FILE_TO_WRITE.getParentFile().exists())
        {
            throw new RuntimeException("\nPlease create directory C:\\Temp\\ on your local machine and run this application via RemoteDesktop with C:\\ as a 'Local resource'.");
        }
        FILE_TO_WRITE.delete();
        new WriteOverTsClientDemo().execute();
    }

    private void execute() throws IOException
    {
        System.out.println("Writing to file: " + FILE_TO_WRITE);
        System.out.println();

        for (int i = 1; i <= 10; i++)
        {
            System.out.println("Writing batch " + i + "...");
            writeDataToFile(i);
            System.out.println("Size of file after batch " + i + ": " + FILE_TO_WRITE.length());
            System.out.println();
        }
        System.out.println("Done!");
    }

    private void writeDataToFile(int batch) throws IOException
    {
        Charset charset = Charset.forName("UTF-8");
        CharsetEncoder encoder = charset.newEncoder();

        try(OutputStream out = Files.newOutputStream(FILE_TO_WRITE.toPath(), CREATE, WRITE, getTruncateOrAppendOption(batch));
            BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(out, encoder)))
        {
            writeData(batch, writer);
        }
    }

    private void writeData(int batch, BufferedWriter writer) throws IOException
    {
        for (String data : createData())
        {
            writer.append(Integer.toString(batch));
            writer.append(" ");
            writer.append(data);
            writer.append("\n");
        }
    }

    private Iterable<String> createData()
    {
        return Collections.nCopies(100, ROW_DATA);
    }

    /**
     * @return option to write from the beginning or from the end of the file
     */
    private OpenOption getTruncateOrAppendOption(int batch)
    {
        return batch == 1 ? TRUNCATE_EXISTING : APPEND;
    }
}

Answer 1

I do not have a setup (No Windows) to verify this effect :( so just thoughts:

2GB sounds like Filesystem related max file size. 32bit Windows Operating system at your client side?

The behaviour sounds like clever Filesystem caching on bad block FS: rapid file write access of big blocks remotely tries to cleverly preoccupy the file in an attempt to fasten future writes to the file having blocks together. Try a different FS to verify? Tried FreeRDP?

Keep the file open. Re-opening for write of huge blocks could hint clever systems to cache.

Update:

FileChannelImpl.java:248

// in append-mode then position is advanced to end before writing
p = (append) ? nd.size(fd) : position0(fd, -1);

leads finally to FileDispatcherImpl:136

static native long More ...size0(FileDescriptor fd) throws IOException;

what as being native can hold any bug. When it comes to protocolls inbetween. I would file this rather as bug in nio/Windows, as They might not have foreseen any funny thing with RDP underneath.

It looks like the returned size is Integer.MAX_VALUE and the file pointer is moved there…

Alternate implementation java.io.FileWriter and no encoding to reduce lines of code:

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.util.Collections;

/**
 * Demonstrates weird issue when writing (appending) to a file over TsClient (Microsoft Remote Desktop).
 *
 * @author Martin
 */
public class WriteOverTsClientDemo
{
   // private static final File FILE_TO_WRITE = new File("\\\\tsclient\\C\\Temp\\TestFile.txt");
   private static final File FILE_TO_WRITE = new File("/tmp/TestFile.txt");

   private static final String ROW_DATA = "111111111122222222223333333333444444444555555555566666666667777777777888888888899999999990000000000";

   public static void main(final String[] args) throws IOException
   {
      if (!FILE_TO_WRITE.getParentFile().exists())
      {
         throw new RuntimeException("\nPlease create directory C:\\Temp\\ on your local machine and run this application via RemoteDesktop with C:\\ as a 'Local resource'.");
      }
      FILE_TO_WRITE.delete();
      new WriteOverTsClientDemo().execute();
   }

   private void execute() throws IOException
   {
      System.out.println("Writing to file: " + FILE_TO_WRITE);
      System.out.println();

      for (int i = 1; i <= 20; i++)
      {
         System.out.println("Writing batch " + i + "...");
         writeDataToFile(i);
         System.out.println("Size of file after batch " + i + ": " + FILE_TO_WRITE.length());
         System.out.println();
      }
      System.out.println("Done!");
   }

   private void writeDataToFile(final int batch) throws IOException
   {
      try (BufferedWriter writer = new BufferedWriter(new FileWriter(FILE_TO_WRITE, batch > 1)))
      {
         writeData(batch, writer);
      }
   }

   private void writeData(final int batch, final BufferedWriter writer) throws IOException
   {
      for (final String data : createData())
      {
         writer.append(Integer.toString(batch));
         writer.append(" ");
         writer.append(data);
         writer.append("\n");
      }
   }

   private Iterable<String> createData()
   {
      return Collections.nCopies(100, ROW_DATA);
   }

}