new File::Path failing in win32 smokes

Post by Jarkko Hietaniemi
It seems that the new File::Path
http://perl5.git.perl.org/perl.git/commitdiff/139271cd3e9cb3cf25072ebdd400e52275c61b96
is failing in George Greer's smokes (though not in Jenkins win32), e.g.
http://www.nntp.perl.org/group/perl.daily-build.reports/2015/08/msg180930.html

Test #68 (after which it dies) is something involving symlinks that
should be getting skipped on MSWin32:

https://github.com/rpcme/File-Path/blob/master/t/Path.t#L391

There are also a bunch of new failures on VMS. I was testing a patch
against blead when I discovered upstream has even more failures, which
I haven't gotten to yet.

bulk 88

2015-08-06 19:52:16 UTC

----------------------------------------

Date: Thu, 6 Aug 2015 12:51:07 -0500
Subject: Re: new File::Path failing in win32 smokes

Test #68 (after which it dies) is something involving symlinks that
https://github.com/rpcme/File-Path/blob/master/t/Path.t#L391
There are also a bunch of new failures on VMS. I was testing a patch
against blead when I discovered upstream has even more failures, which
I haven't gotten to yet.

FAT32 vs NTFS disk? Its been about a year or 2 since I last did a FAT32 smoke and submitted patches for it.

Jarkko Hietaniemi

2015-08-06 20:06:42 UTC

Post by bulk 88
----------------------------------------

Date: Thu, 6 Aug 2015 12:51:07 -0500
Subject: Re: new File::Path failing in win32 smokes

FAT32 vs NTFS disk? Its been about a year or 2 since I last did a FAT32 smoke and submitted patches for it.

--
There is this special biologist word we use for 'stable'. It is
'dead'. -- Jack Cohen

Jarkko Hietaniemi

2015-08-06 20:07:51 UTC

Post by bulk 88
FAT32 vs NTFS disk? Its been about a year or 2 since I last did a FAT32 smoke and submitted patches for it.

Whatever George Greer's win32 smoker has.

--
There is this special biologist word we use for 'stable'. It is
'dead'. -- Jack Cohen

Craig A. Berry

2015-08-06 21:26:28 UTC

Post by bulk 88
FAT32 vs NTFS disk? Its been about a year or 2 since I last did a FAT32 smoke and submitted patches for it.

Whatever George Greer's win32 smoker has.

George's log file (linked in all of his smoke reports) says:

../cpan/File-Fetch/t/null_subclass.t .............................. ok
Can't locate Carp.pm in @INC (you may need to install the Carp module)
(@INC contains: ../../lib ../../t .) at ../../lib/File/Path.pm line
45.

# Failed test '... now chdir '
# at t/Path.t line 835.
Can't locate Carp.pm in @INC (you may need to install the Carp module)
(@INC contains: ../../lib ../../t .) at ../../lib/File/Path.pm line
45.
END failed--call queue aborted at t/Path.t line 45.
# Looks like you planned 159 tests but ran 68.
# Looks like you failed 1 test of 68 run.
# Looks like your test exited with 2 just after 68.
../cpan/File-Path/t/Path.t ........................................
Dubious, test returned 2 (wstat 512, 0x200)
Failed 92/159 subtests
(less 2 skipped subtests: 65 okay)
../cpan/File-Path/t/taint.t ....................................... ok

which was briefly mentioned in Message-ID:
<***@drei.m-l.org> but I didn't see any
resolution.

The code in Path.t where the trouble is looks like:

my $p = getcwd();
my $x = "x$$";
my $xx = $x . "x";

# setup
ok(mkpath($xx), "make $xx");
ok(chdir($xx), "... and chdir $xx");
END {
ok(chdir($p), "... now chdir $p");
ok(rmtree($xx), "... and finally rmtree $xx");
}

and it fails in chdir inside the END block. Since the log just says
"... now chdir" with nothing after it, it looks like $p must be an
empty string, which I believe only recently started causing chdir to
fail. But we've just done C<my $p = getcwd();> a few lines up. So
under what conditions does getcwd() return an empty string on Windows?

George Greer

2015-08-07 14:10:34 UTC

Post by bulk 88
FAT32 vs NTFS disk? Its been about a year or 2 since I last did a FAT32 smoke and submitted patches for it.

Whatever George Greer's win32 smoker has.

../cpan/File-Fetch/t/null_subclass.t .............................. ok
45.
# Failed test '... now chdir '
# at t/Path.t line 835.
45.
END failed--call queue aborted at t/Path.t line 45.
# Looks like you planned 159 tests but ran 68.
# Looks like you failed 1 test of 68 run.
# Looks like your test exited with 2 just after 68.
../cpan/File-Path/t/Path.t ........................................
Dubious, test returned 2 (wstat 512, 0x200)
Failed 92/159 subtests
(less 2 skipped subtests: 65 okay)
../cpan/File-Path/t/taint.t ....................................... ok
resolution.
my $p = getcwd();
my $x = "x$$";
my $xx = $x . "x";
# setup
ok(mkpath($xx), "make $xx");
ok(chdir($xx), "... and chdir $xx");
END {
ok(chdir($p), "... now chdir $p");
ok(rmtree($xx), "... and finally rmtree $xx");
}
and it fails in chdir inside the END block. Since the log just says
"... now chdir" with nothing after it, it looks like $p must be an
empty string, which I believe only recently started causing chdir to
fail. But we've just done C<my $p = getcwd();> a few lines up. So
under what conditions does getcwd() return an empty string on Windows?

The "END" block runs even if the script died half-way through the tests,
so there is no guarantee that any of those previous lines actually ran.

Remember it says "Looks like you planned 159 tests but ran 68." so the
Carp messages might be a symptom of the real problem.

--
George Greer

George Greer

2015-08-07 14:12:53 UTC

Post by bulk 88
FAT32 vs NTFS disk? Its been about a year or 2 since I last did a FAT32 smoke and submitted patches for it.

Whatever George Greer's win32 smoker has.

Almost positive it is NTFS but can't check at the moment.

--
George Greer

Craig A. Berry

2015-08-07 19:40:14 UTC

Post by George Greer

Post by bulk 88
FAT32 vs NTFS disk? Its been about a year or 2 since I last did a FAT32
smoke and submitted patches for it.

Whatever George Greer's win32 smoker has.

Almost positive it is NTFS but can't check at the moment.

Is the working directory on a network drive? One of the things that's
new in the test looks like this:

my $UNC_path = catdir(getcwd(), $tmp_base, 'uncdir');
#dont compute a SMB path with $ENV{COMPUTERNAME}, since SMB may be turned off
#firewalled, disabled, blocked, or no NICs are on and there the PC has no
#working TCPIP stack, \\?\ will always work
$UNC_path = '\\\\?\\'.$UNC_path;

If getcwd() returns something with "\\" in it, won't $UNC_path end up
with two double backslashes in it, so that what gets passed to mkpath
and rmtree would look like:

\\?\\someserver\somedir\test-123\uncdir

?

As far as I can tell from
<https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx>,
that would have to be specified as:

\\?\UNC\someserver\somedir\test-123\uncdir

to be valid.

OK, that was my third guess, so if this one is wrong too I'll just shut up.

George Greer

2015-08-11 15:37:48 UTC

Post by George Greer

Post by bulk 88
FAT32 vs NTFS disk? Its been about a year or 2 since I last did a FAT32
smoke and submitted patches for it.

Whatever George Greer's win32 smoker has.

Almost positive it is NTFS but can't check at the moment.

Is the working directory on a network drive?

No, plain old D: and it is NTFS.

Post by Craig A. Berry
One of the things that's
my $UNC_path = catdir(getcwd(), $tmp_base, 'uncdir');
#dont compute a SMB path with $ENV{COMPUTERNAME}, since SMB may be turned off
#firewalled, disabled, blocked, or no NICs are on and there the PC has no
#working TCPIP stack, \\?\ will always work
$UNC_path = '\\\\?\\'.$UNC_path;
If getcwd() returns something with "\\" in it, won't $UNC_path end up
with two double backslashes in it, so that what gets passed to mkpath
\\?\\someserver\somedir\test-123\uncdir
?
As far as I can tell from
<https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx>,
\\?\UNC\someserver\somedir\test-123\uncdir
to be valid.
OK, that was my third guess, so if this one is wrong too I'll just shut up.

I can't get any form of "\\?\..." to work from the explorer or
command-line. At what Windows version was this syntax added?

--
George Greer

Craig A. Berry

2015-08-11 15:56:02 UTC

I can't get any form of "\\?\..." to work from the explorer or command-line.
At what Windows version was this syntax added?

I don't know but it seems to work in XP.

Ivan Pozdeev

2015-08-11 17:41:21 UTC

I can't get any form of "\\?\..." to work from the explorer or command-line.
At what Windows version was this syntax added?

I don't know but it seems to work in XP.

open(r"\\?\C:\test.txt","w")

<open file '\\\\?\\C:\\test.txt', mode 'w' at 0x014EE4F0>

In console, this is preferrable:

C:\>dir \\.\C:\
Volume in drive \\.\C: has no label.
Volume Serial Number is 2C85-AFE2

Directory of \\.\C:
<...>

The finishing slash is requires for the root dir.

This is actually the native path specification for NT.
It references objects in the kernel's Object Manager directory.
The prefix \\?\ or \\.\ references its \?? directory.
http://stackoverflow.com/questions/21194530/what-does-mean-when-prepended-to-a-file-path

For a programmer, the main concern about such paths is that various
limitations don't apply to them and no preprocessing is done on them.
(this is covered in detail by the article linked from the quesiton).

--
Regards,
Ivan Pozdeev

James E Keenan

2015-08-06 22:39:04 UTC

We would very much appreciate your reporting these test failures in
File-Path's bug queue at:
https://rt.cpan.org//Dist/Display.html?Queue=File-Path

Thank you very much.
Jim Keenan

Tony Cook

2015-08-06 22:51:47 UTC

The strange thing is it's only failing on non-threaded builds.

I tried to reproduce it locally but couldn't.

Tony

bulk 88

2015-08-06 23:16:06 UTC