[perl #125710] ext/POSIX/t/math.t fails on cygwin

[Tony Cook (via RT)]

# New Ticket Created by Tony Cook
# Please include the string: [perl #125710]
# in the subject line of all future correspondence about this issue.
# <URL: https://rt.perl.org/Ticket/Display.html?id=125710 >


Test Summary Report
-------------------
../ext/POSIX/t/math.t (Wstat: 256 Tests: 136 Failed: 1)
Failed test: 123
Non-zero exit status: 1
Files=1, Tests=136, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.05 cusr 0.11 csys = 0.17 CPU)
Result: FAIL

I enabled NV_PAYLOAD_DEBUG and later added some extra debugging code to the issignalling() XS code, for the failing test this produced:

a[0] = 0x12345
set p[ 0] = 45 (i = 0, m = ff, s = 0, b = 45, u = 00000045)
set p[ 1] = 23 (i = 1, m = ff, s = 8, b = 23, u = 00002300)
set p[ 2] = 01 (i = 2, m = ff, s = 16, b = 01, u = 00010000)
set p[ 3] = 00 (i = 3, m = ff, s = 24, b = 00, u = 00000000)
set p[ 4] = 00 (i = 4, m = ff, s = 32, b = 00, u = 00000000)
set p[ 5] = 00 (i = 5, m = ff, s = 40, b = 00, u = 00000000)
set p[ 6] = f8 (i = 6, m = 07, s = 48, b = 00, u = 00000000)
45 23 01 00 00 00 f0 ff
a[0] = 12345
45 23 01 00 00 00 f8 ff
issig: 45 23 01 00 00 00 f8 ff
offset 6 shift 3 bit 0x8

# Failed test 'setpayloadsig + issignaling'
# at t/math.t line 176.

where the extra code is:

@@ -2710,6 +2710,15 @@ issignaling(nv)
NV nv
CODE:
RETVAL = Perl_isnan(nv) && NV_NAN_IS_SIGNALING(&nv);
+ { unsigned i;
+ unsigned char *p = (unsigned char *)&nv;
+ PerlIO_printf(PerlIO_stderr(), "issig:");
+ for (i = 0; i < sizeof(NV); ++i)
+ PerlIO_printf(PerlIO_stderr(), " %02x", p[i]);
+ PerlIO_printf(PerlIO_stderr(), "\n");
+ PerlIO_printf(PerlIO_stderr(), "offset %d shift %d bit %#02x\n",
+ (int)NV_NAN_QS_BYTE_OFFSET, (int)NV_NAN_QS_BIT_SHIFT, (unsigned)NV_NAN_QS_BIT);
+ }
OUTPUT:
RETVAL

This fails at all optimization levels from -O0 through -O3.

--
Summary of my perl5 (revision 5 version 23 subversion 2) configuration:
Derived from: d218c3b757387fb6067599e2d3991945b37b0eac
Platform:
osname=cygwin, osvers=2.1.0(0.28753), archname=cygwin-64int
uname='cygwin_nt-6.1-wow saturn 2.1.0(0.28753) 2015-07-14 21:26 i686 cygwin '
config_args='-des -Dusedevel -Uusethreads -Doptimize=-O2'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=undef, uselongdouble=undef
usemymalloc=y, bincompat5005=undef
Compiler:
cc='gcc', ccflags ='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -D_FORTIFY_SOURCE=2',
optimize='-O2',
cppflags='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong'
ccversion='', gccversion='4.9.3', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678, doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12, longdblkind=3
ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='g++', ldflags =' -Wl,--enable-auto-import -Wl,--export-all-symbols -Wl,--enable-auto-image-base -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/lib/gcc/i686-pc-cygwin/4.9.3/include-fixed /usr/lib /usr/lib/gcc/i686-pc-cygwin/4.9.3/../../../../include/w32api /usr/local/lib /lib
libs=-lpthread -ldl -lcrypt
perllibs=-lpthread -ldl -lcrypt
libc=/usr/lib/libc.a, so=dll, useshrplib=true, libperl=cygperl5_23_2.dll
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags=' --shared -Wl,--enable-auto-import -Wl,--export-all-symbols -Wl,--enable-auto-image-base -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl):
Compile-time options: HAS_TIMES MYMALLOC PERLIO_LAYERS PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
PERL_PRESERVE_IVUV PERL_USE_DEVEL
PERL_USE_SAFE_PUTENV USE_64_BIT_INT USE_LARGE_FILES
USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
USE_PERL_ATOF
Locally applied patches:
uncommitted-changes
Built under cygwin
Compiled at Jul 29 2015 10:23:46
@INC:
lib
/usr/local/lib/perl5/site_perl/5.23.2/cygwin-64int
/usr/local/lib/perl5/site_perl/5.23.2
/usr/local/lib/perl5/5.23.2/cygwin-64int
/usr/local/lib/perl5/5.23.2
/usr/local/lib/perl5/site_perl
.

[Jarkko Hietaniemi via RT]

grep nanbytes config.sh

[Jarkko Hietaniemi via RT]

Also, what is Perl_isnan() ending up as? "make POSIX.i" should work in Cygwin, I think.

[Tony Cook via RT]

Post by Tony Cook (via RT)
Test Summary Report
-------------------
../ext/POSIX/t/math.t (Wstat: 256 Tests: 136 Failed: 1)
Failed test: 123
Non-zero exit status: 1

The same test is failing on 32-bit Solaris.

Tony

---
via perlbug: queue: perl5 status: new
https://rt.perl.org/Ticket/Display.html?id=125710

[Tony Cook via RT]

Post by Jarkko Hietaniemi via RT
grep nanbytes config.sh

$ grep nanbytes config.sh
doublenanbytes='0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf8, 0xff'
longdblnanbytes='0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0xff, 0xff, 0x2f, 0x61'

Post by Jarkko Hietaniemi via RT
Also, what is Perl_isnan() ending up as? "make POSIX.i" should work
in Cygwin, I think.

I think this is what you were after:

static void XS_POSIX_issignaling(PerlInterpreter* my_perl __attribute__((unused)), CV* cv);
static void XS_POSIX_issignaling(PerlInterpreter* my_perl __attribute__((unused)), CV* cv)
{
extern int Perl___notused(void); SV **sp = (my_perl->Istack_sp); I32 ax = (*(my_perl->Imarkstack_ptr)--); SV **mark = (my_perl->Istack_base) + ax++; I32 items = (I32)(sp - mark);
if (items != 1)
Perl_croak_xs_usage(cv, "nv");
{
NV nv = (NV)(((((my_perl->Istack_base)[ax + (0)])->sv_flags & (0x00000200|0x00200000)) == 0x00000200) ? ((XPVNV*) ((my_perl->Istack_base)[ax + (0)])->sv_any)->xnv_u.xnv_nv : Perl_sv_2nv_flags(my_perl, (my_perl->Istack_base)[ax + (0)],2))
;
int RETVAL;
SV * const targ = (((my_perl->Iop)->op_private & 0x04) ? ((my_perl->Icurpad)[(my_perl->Iop)->op_targ]) : Perl_sv_newmortal(my_perl));
# 2712 "POSIX.xs"
RETVAL = (((sizeof(nv) == sizeof(float)) ? __fpclassifyf(nv) : __fpclassifyd(nv)) == 0) && (((((U8*)(&nv))[6]) & (1 << ((((52 - 1)) % 8)))) == ((((((U8*)(PL_nan.u8))[6]) & (1 << ((((52 - 1)) % 8)))) == (1 << ((((52 - 1)) % 8)))) ? 0 : (1 << ((((52 - 1)) % 8)))));
# 3329 "POSIX.c"
(sp = (my_perl->Istack_base) + ax - 1); (void)( { Perl_sv_setiv(my_perl, targ,(IV)((IV)RETVAL)); (void)( { (void)( { if (__builtin_expect(((((targ)->sv_flags & 0x00400000)) ? (_Bool)1 : (_Bool)0),(0))) Perl_mg_set(my_perl, targ); } ); (*++sp = (targ)); } ); } );
}
(void)( { const IV tmpXSoff = (1); (my_perl->Istack_sp) = (my_perl->Istack_base) + ax + (tmpXSoff - 1); return; } );
}

isnan() appears to be producing reasonable values (or other tests would fail):

***@saturn ~/dev/perl/git/perl
$ ./perl -Ilib -MPOSIX=setpayloadsig,issignaling,isnan -le 'my $x = 0; setpayloadsig($x, 0x100); print isnan($x); print issignaling($x)'
1
0

***@saturn ~/dev/perl/git/perl
$ ./perl -Ilib -MPOSIX=setpayload,issignaling,isnan -le 'my $x = 0; setpayload($x, 0x100); print isnan($x); print issignaling($x)'
1
0

Tony


---
via perlbug: queue: perl5 status: new
https://rt.perl.org/Ticket/Display.html?id=125710

[Tony Cook via RT]

Post by Tony Cook via RT

Post by Tony Cook (via RT)
Test Summary Report
-------------------
../ext/POSIX/t/math.t (Wstat: 256 Tests: 136 Failed: 1)
Failed test: 123
Non-zero exit status: 1

The same test is failing on 32-bit Solaris.

Also fails on 32-bit FreeBSD 10, FreeBSD 11, Fedora 24 (Rawhide).

Passes on ARM (RPi2).

So it looks specific to 32-bit x86.

Tony

---
via perlbug: queue: perl5 status: new
https://rt.perl.org/Ticket/Display.html?id=125710

[Jarkko Hietaniemi via RT]

Post by Tony Cook via RT

Post by Tony Cook via RT

Post by Tony Cook (via RT)
Test Summary Report
-------------------
../ext/POSIX/t/math.t (Wstat: 256 Tests: 136 Failed: 1)
Failed test: 123
Non-zero exit status: 1

The same test is failing on 32-bit Solaris.

Also fails on 32-bit FreeBSD 10, FreeBSD 11, Fedora 24 (Rawhide).
Passes on ARM (RPi2).
So it looks specific to 32-bit x86.

Thanks for the find. Now I'm able to repro it.



---
via perlbug: queue: perl5 status: new
https://rt.perl.org/Ticket/Display.html?id=125710

[Jarkko Hietaniemi via RT]

Post by Jarkko Hietaniemi via RT

Post by Tony Cook via RT
So it looks specific to 32-bit x86.

Thanks for the find. Now I'm able to repro it.

The first explorations look ... argh. The signalingness (the bit 3 of the [6]'th byte of the double) is first set correctly (the bit is cleared) on the NV by S_setpayload(), but then the signalingness vanishes (the bit is set again) by the time it gets to the returned SV.

I'll do some more comparative research in other 32-bit places I can find.







---
via perlbug: queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=125710

[Tony Cook]

Post by Jarkko Hietaniemi via RT

Post by Jarkko Hietaniemi via RT

Post by Tony Cook via RT
So it looks specific to 32-bit x86.

Thanks for the find. Now I'm able to repro it.

The first explorations look ... argh. The signalingness (the bit 3 of the [6]'th byte of the double) is first set correctly (the bit is cleared) on the NV by S_setpayload(), but then the signalingness vanishes (the bit is set again) by the time it gets to the returned SV.
I'll do some more comparative research in other 32-bit places I can find.

http://stackoverflow.com/questions/22816095/signalling-nan-was-corrupted-when-returning-from-x86-function-flds-fstps-of-x87

In particular, quoted from
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57484#c12:


The ABI is just wrong for the underlying x87 hardware as far as
NaNs are concerned.

This issue is unfortunately unfixable. x87 and x86-32 ABI are just
not designed to handle all details of IEEE 754 standard.

The 32-bit x86 ABI returns floating point results in the x87
registers, and according to
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57484#c11 it's the
conversion from 32- or 64-bit float values to the internal format that
causes the loss of the signalling bit (or the gain of the quiet bit,
given the implementation).

The 64-bit (amd64) ABI returns float and double types in the SSE registers, avoiding the conversion.

Based on that comment, I expected long double builds to succeed, and
so it did, a build with -Duselongdouble passes our failing test.

So I suspect the fix for this case is to skip the test on 32-bit x86
unless uselongdouble is defined.

Tony

[Jarkko Hietaniemi via RT]

Post by Tony Cook
Based on that comment, I expected long double builds to succeed, and
so it did, a build with -Duselongdouble passes our failing test.
So I suspect the fix for this case is to skip the test on 32-bit x86
unless uselongdouble is defined.

Nice sleuthing! I'll amend the test, and document profusely.


---
via perlbug: queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=125710

[Jarkko Hietaniemi via RT]

In sparc-solaris and irix which both have ivsize=4 (but both byteorder=4321, not 1234), the test passes.

[Jarkko Hietaniemi via RT]

As of 8384a6d6 this should be fixed. (Tthe actually *needed* change that skips a test that cannot work in 32-bit x86 is a262b72a.)

---
via perlbug: queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=125710