Creating a file using Perl on Windows in a directory the length of whose name exceeds 220 characters

Question

I'm experiencing a problem where I cannot create a file under a directory the length of whose name exceeds 220 characters.

The following is a test script that reproduces the observed behavior, at least on my machine:

use warnings;
use strict;

use Win32::LongPath;

print system ('rmdir /s /q test');
mkdirL('test');

for my $i (200 .. 255) {

  my $dir_name = 'test/' . sprintf("%04d", $i) . ('a' x ($i-4));
  mkdirL($dir_name);

  openL(\my $fh, '>', "$dir_name/" . ('_' x 200) . '.txt') or die "$^E";
  print $fh 'Hello!';
# closeL $fh;
}

This script will create a _________.....___.txt file under 0220aaaa...aaa/ but not under 0221aaaa...aaa/.

Is there a reason for this problem and how can I change the script so that the *.txt file is created in all directories?

Update:

The script, when run, doesn't die or produce any error messages.

Update 2:

This question does not already have an answer in Why does the 260 character path length limit exist in Windows? As the other question, and its answers, clearly show, this path length limit is a Windows issue, not an NTFS issue. NTFS allows paths with a length of up to 32K characters.

In fact, my test script is able to create directories up to 255 characters (as expected from the NTFS specification), it just seems impossible to store files within a directory when the directory name is longer than 220 characters.

Answer 1

TL;DR

The root of your problem is the method by which you are trying to verify that the files have been created. For more information, read the detailed explanation below.

Here are the facts:

1. perl does not give any errors when using mkdirL or openL calls.
2. perl can open files created, and read the contents, using openL.

Therefore the problem is due to the fact that whatever tool you are using, is either using ANSI versions of Windows API calls, or specifying relative paths, or a combination of both, and therefore their paths are restricted to 260 characters.

To test this, I ran the script under D:\t. Lo and behold, GVim failed to open a file when $i = 250:

D:\t is four characters, \test is another five. Therefore, 250 + 9 = 259, which hits 260 the moment you add another \.

Using shortpathL

Try this:

#!/usr/bin/env perl

use strict;
use warnings;

use Win32::LongPath;

`cmd /c rd /s /q test`;

mkdirL('test') or die "$^E";

my $dir_length = 255;
my $dir_name = 'test/'
             . sprintf("%04d", $dir_length)
             . ('a' x ($dir_length - 4))
;

mkdirL($dir_name) or die "$^E";

my $file_name = "$dir_name/" . ('z' x 200) . '.txt';

printf "% 3d\n", length $file_name;

openL(\my $fh, '>', $file_name) or die "$^E";

print $fh "Hello!\n" or die "$^E";

close $fh or die "$^E";

system 'notepad.exe', shortpathL($file_name);

You will get:

Therefore, give the short path to any external programs which you cannot rely on to use the Unicode interface.

Long winded explanation

Now that I have had a chance to actually try this on a 64-bit Windows 8.1 system, I cannot replicate the problem.

Here is the code I used:

#!/usr/bin/env perl

use strict;
use warnings;

use Win32::LongPath;

`cmd /c rd /s /q test`;

mkdirL('test')
    or die "$^E";

for my $i (200 .. 255) {
    my $dir_name = 'test/' . sprintf("%04d", $i) . ('a' x ($i-4));
    mkdirL($dir_name) or die "$^E";

    my $file_name = "$dir_name/" . ('_' x 200) . '.txt';

    printf "% 3d\n", length $file_name;

    openL(\my $fh, '>', $file_name)
        or die "$^E";

    print $fh 'Hello!' or die "$^E";

    close $fh or die "$^E";
}

Here is the output:

C:\…\Temp> perl tt.pl          
 410                                                   
 411                                                   
 412                                                   
 413                                                   
 414                                                   
 415                                                   
…
 460    
 461    
 462    
 463    
 464    
 465

I am also attaching a couple of screenshots:

Now, I can report that Explorer has trouble navigating into any directory after C:\...\Temp\test\0220aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

and GVim cannot open files in subsequent directories. That is, system 'c:/.../gvim.exe', $file_name; issued from within the script results in

Presumably GVim runs into the following from Naming Files, Paths, and Namespaces

Because you cannot use the "\\?\" prefix with a relative path, relative paths are always limited to a total of MAX_PATH characters.

By coincidence, the length of the path of my %TEMP% directory happens to be 33 characters. Add to that 5 (length of \test), and we have 38. Add that to 221, and we get 259. Now, the moment you add a directory separator to that string, you hit 260.

Which brings me to, what is the length of the full path of the working directory under which you are creating test?

In better news, perl is able to read back everything written:

#!/usr/bin/env perl

use strict;
use warnings;

use Win32::LongPath;

`cmd /c rd /s /q test`;

mkdirL('test')
    or die "$^E";

for my $i (220 .. 255) {
    my $dir_name = 'test/' . sprintf("%04d", $i) . ('a' x ($i-4));
    mkdirL($dir_name) or die "$^E";

    my $file_name = "$dir_name/" . ('_' x 200) . '.txt';

    printf "% 3d\n", length $file_name;

    openL(\my $fh, '>', $file_name)
        or die "$^E";

    print $fh 'Hello!' or die "$^E";

    close $fh or die "$^E";

    openL(\my $in, '<', $file_name)
        or die "$^E";

    print <$in>, "\n" or die "$^E";

    close $in or die "$^E";
}

outputs:

…
 459  
Hello!
 460  
Hello!
 461  
Hello!
 462  
Hello!
 463  
Hello!
 464  
Hello!
 465  
Hello!

Because Win32::LongPath internally normalizes paths so they follow the "To specify an extended-length path, use the "\\?\" prefix" recommendation, and then uses the Unicode version of API calls, e.g. CreateFileW, openL does not run into such issues.

In the ANSI version of this function, the name is limited to MAX_PATH characters. To extend this limit to 32,767 wide characters, call the Unicode version of the function and prepend "\\?\" to the path. For more information, see Naming Files, Paths, and Namespaces.

How are you verifying if the files have been correctly created?

Also, see "Why is Perl system call failing to invoke internal Windows command?" for the explanation of qx{cmd /c rd /s /q test}