Discussion:
[PATCH] MakeMaker, XS and C++
Fandiño
2003-11-20 20:43:08 UTC
Permalink
Nick Ing-Simmons
2003-11-21 09:25:56 UTC
Permalink
Salvador "Fandiño <***@yahoo.com> writes:
>Hi,
>
>Now that 5.8.2 has been released, Michael G Schwern has accepted to
>consider my patch to add support for C++ modules for inclusion on
>MakeMaker but he has requested me to get it reviewed by somebody on
>here first so...
>
>A new version of the patch is available from here:
> http://www.nondoc.org/perl/cxx-0.10.patch.gz
>
>C++ modules are expected to contain a .xss file instead of the usual
>.xs one. That file is preprocessed by xsubpp to a .cpp file and then
>compiled by the C++ compiler.

Good patch in general - a few questions/comments:

xsubpp has a -C++ option - is the some reason that you don't use
that when processing a .xss file ?
(I forget what it does in detail but I have found it useful in the past.)

+ # Use some heuristics to find the C++ compiler and linker:
+
+ if ( $Config{ccname} =~ /gcc/i) {
+ my($vol, $dir, $file) = File::Spec->splitpath($self->{CC});
+ $self->{CXX} ||= $dir eq '' ? 'c++' : File::Spec->join($vol, $dir, 'c++');
+ $self->{CXXLD} ||= $self->{CXX};
+ } else {
+ $self->{CXX} ||= $self->{CC};
+ $self->{CXXLD} ||= $self->{LD};
+ }
+

If $Config{cc} is 'gcc' you should probaly try/use 'g++' - there may be
a system c++ which does not match the gcc.


+This target is stubbed out. Not sure why.
+
+=cut
+
+sub xss_o {
+ return ''
+}
+

I think because you get less re-re-builds if the rules stack.
i.e. one rule
.xss -> .cpp
and then one
.cpp -> .o

and let make figure out to chain the two rules.
If you do a direct .xss -> .o rule as well make can get confused as
to which one to use, it may depend if an old .cpp is about from
an prior run and the guts of rules must match.

+=item CXXLD
+
+Program to be used to link C++ libraries for dynamic loading.
+
+Defaults to $Config{ld}.

I think it should default to being the compiler you found i.e. CXXCMD.


>
>Two new variables can be used on Makefile.PL: CXX to set the C++
>compiler and CXXLD to set the linker to be used for C++ modules. As
>there isn't any information available from Config.pm about the C++
>compiler, linker, libs, etc., I had to use some heuristics to find
>default values for those variables from the C config. Also, new hints
>should be added for compilers others than gcc and MSVC and support
>for other vars (CXXFLAGS, CXXLIBS?) could be needed.
>
>The user can also set CXX and CXXLD himself when running Makefile.PL
>from the command line (or even set them on CPAN.pm config).
>
>Deducting C++ compiler config from info on Config.pm is not bullet
>proof but the only alternative would be to run some kind of
>auto-config process, that doesn't seem easy to implement... any
>volunteer?

It isn't too hard to have/write a tiny C++ file and "try" the c++ command
you find.

open(my $fh,">test.cpp") || die;
print $fh <<'END';
#include <iostream>
int main(int argc,char *argv[])
{
std::cout << "It works\n";
return 0;
}
END
close($fh);
if (system($cpp_to_try,"test.cpp") == 0)
{
# that will do
}
Salvador Fandiño
2003-11-21 10:29:52 UTC
Permalink
Nick Ing-Simmons wrote:

>
> Good patch in general - a few questions/comments:
>
> xsubpp has a -C++ option - is the some reason that you don't use
> that when processing a .xss file ?
> (I forget what it does in detail but I have found it useful in the past.)

It seems that in newer perls -C++ is a no-op and so I forgot
about it but I think that you are right, because older
perl/xsubpp will require it.


> + # Use some heuristics to find the C++ compiler and linker:
> +
> + if ( $Config{ccname} =~ /gcc/i) {
> + my($vol, $dir, $file) = File::Spec->splitpath($self->{CC});
> + $self->{CXX} ||= $dir eq '' ? 'c++' : File::Spec->join($vol, $dir, 'c++');
> + $self->{CXXLD} ||= $self->{CXX};
> + } else {
> + $self->{CXX} ||= $self->{CC};
> + $self->{CXXLD} ||= $self->{LD};
> + }
> +
>
> If $Config{cc} is 'gcc' you should probaly try/use 'g++' - there may be
> a system c++ which does not match the gcc.

right


> +This target is stubbed out. Not sure why.
> +
> +=cut
> +
> +sub xss_o {
> + return ''
> +}
> +

I just copied the comment from the xs_o block :-)


> +=item CXXLD
> +
> +Program to be used to link C++ libraries for dynamic loading.
> +
> +Defaults to $Config{ld}.

oops, it should say...

"Its default value is inferred from the C compiler configuration
stored in %Config"


> I think it should default to being the compiler you found i.e. CXXCMD.

It's not always CXXCMD, i.e., for MSVC it is 'link'.


>>Deducting C++ compiler config from info on Config.pm is not bullet
>>proof but the only alternative would be to run some kind of
>>auto-config process, that doesn't seem easy to implement... any
>>volunteer?
>
>
> It isn't too hard to have/write a tiny C++ file and "try" the c++ command
> you find.
>
> open(my $fh,">test.cpp") || die;
> print $fh <<'END';
> #include <iostream>
> int main(int argc,char *argv[])
> {
> std::cout << "It works\n";
> return 0;
> }
> END
> close($fh);
> if (system($cpp_to_try,"test.cpp") == 0)
> {
> # that will do
> }

but testing that a C++ program compiles is not enough, i.e., gcc
(the command) would pass this test and still create unusable
modules because of the missing stdc++ lib.

The only reliable solution would be to include a C++ test module
and try to build and check it with different configurations until
one succeeds.


Bye,

- Salva
Nick Ing-Simmons
2003-11-21 10:56:19 UTC
Permalink
=?UTF-8?B?U2FsdmFkb3IgRmFuZGnDsW8=?= <***@yahoo.com> writes:
>> It isn't too hard to have/write a tiny C++ file and "try" the c++ command
>> you find.
>>
>> open(my $fh,">test.cpp") || die;
>> print $fh <<'END';
>> #include <iostream>
>> int main(int argc,char *argv[])
>> {
>> std::cout << "It works\n";
>> return 0;
>> }
>> END
>> close($fh);
>> if (system($cpp_to_try,"test.cpp") == 0)
>> {
>> # that will do
>> }
>
>but testing that a C++ program compiles is not enough, i.e., gcc
>(the command) would pass this test and still create unusable
>modules because of the missing stdc++ lib.

We are not building a module at this point, it is an executable.
The above will either produce 'a.out' (or test.exe) or fail to link.
It is not too much harder to

my $name = 'test'.$Config{exe_ext};
if (system($cpp_to_try,-o => $name, "test.cpp") == 0)
{
my $line = `$name`;
if ($? == 0 && defined($line) && $line eq '"It works\n")
{
# it ran as well
}
}

"We" could do with some Configure-oid modules to do this kind of thing.

Here is the C version that Tk uses (for library probing), it isn't bomb proof
but it does what I need.
(From Tk804.025/Tk/MMtry.pm)

my $stderr_too = ($^O eq 'MSWin32') ? '' : '2>&1';

sub try_run
{
my ($file,$inc,$lib) = @_;
$inc = [] unless $inc;
$lib = [] unless $lib;
my $out = basename($file,'.c').$Config{'exe_ext'};
warn "Test Compile/Run $file\n";
my $msgs = `$Config{'cc'} -o $out $Config{'ccflags'} @$inc $file @$lib $stderr_too`;
my $ok = ($? == 0);
# warn "$Config{'cc'} -o $out $Config{'ccflags'} @$inc $file @$lib:\n$msgs" if $msgs;
if ($ok)
{
my $path = File::Spec->rel2abs($out);
$msgs = `$path $stderr_too`;
$ok = ($? == 0);
# warn "$path:$msgs" if $msgs;
}
unlink($out) if (-f $out);
return $ok;
}


>
>The only reliable solution would be to include a C++ test module
>and try to build and check it with different configurations until
>one succeeds.

Certainly an option - perl has a XS test extension, we could
add a C++ test extension to MakeMaker - it does not have to be huge
after all!

>
>
>Bye,
>
> - Salva
Michael G Schwern
2003-11-21 23:48:54 UTC
Permalink
On Fri, Nov 21, 2003 at 10:56:19AM +0000, Nick Ing-Simmons wrote:
> >The only reliable solution would be to include a C++ test module
> >and try to build and check it with different configurations until
> >one succeeds.
>
> Certainly an option - perl has a XS test extension, we could
> add a C++ test extension to MakeMaker - it does not have to be huge
> after all!

The major reason I've been avoiding putting any XS tests into MakeMaker
is being able to reliably determine if the user has a working compiler
and what that compiler is. Perl has it easy, the user obviously has a
C compiler, the one they're using to compile perl!

Only if the probing for a compiler can be done without too many false
negatives might I consider it. Module::Build has something that seems to
be working, it might be worth looking into.


--
Michael G Schwern ***@pobox.com http://www.pobox.com/~schwern/
"May you eat three balanced meals a day and have healthful, regular bowel
movements."
-- Gimlet, son of Groin, "Bored of the Rings"
Rafael Garcia-Suarez
2003-11-22 08:18:40 UTC
Permalink
Michael G Schwern wrote:
> The major reason I've been avoiding putting any XS tests into MakeMaker
> is being able to reliably determine if the user has a working compiler
> and what that compiler is. Perl has it easy, the user obviously has a
> C compiler, the one they're using to compile perl!

That's of course an easy assumption, but it's not true, esp. if you
use most commercial unixes, which ship with perl but without a compiler
by default.
Nick Ing-Simmons
2003-11-22 10:25:27 UTC
Permalink
Rafael Garcia-Suarez <***@free.fr> writes:
>Michael G Schwern wrote:
>> The major reason I've been avoiding putting any XS tests into MakeMaker
>> is being able to reliably determine if the user has a working compiler
>> and what that compiler is. Perl has it easy, the user obviously has a
>> C compiler, the one they're using to compile perl!
>
>That's of course an easy assumption, but it's not true, esp. if you
>use most commercial unixes, which ship with perl but without a compiler
>by default.

I think you are agreeing with Michael!
His point that when building _perl_ it is appropriate to build an XS test
as we know we have the compiler we are building perl with.
However, for MakeMaker in the wild as you point out the compiler may not
be present. So install of MakerMaker cannot always pass XS building tests.

However, if someone is trying to build an XS extension they had better
have a compiler. So MakeMaker could have code to probe for compiler
and find one that works and then pass that on to XS rules for C, C++.
If it can't find one then process can fail at Makefile.PL time
Salvador Fandiño
2003-11-26 19:40:57 UTC
Permalink
Hi,

A new version of the "C++ support for MakeMaker" patch is ready.

It's available from here:
http://www.nondoc.org/perl/mm-cxx-0.11.patch.gz

And the full patched ExtUtil::MakeMaker package is here:
http://www.nondoc.org/perl/ExtUtils-MakeMaker-6.22_02.tar.gz

I have included most of the changes suggested by Nick Ing-Simmons
and corrected some problems from the previous version.

It's tested on Linux+gcc and Windows+MSVC and only support for
these platforms is really fully implemented.

I have added two new modules to ExtUtils:
- ExtUtils::Cxx that looks for the C++ compiler and checks that
it works
- ExtUtils::Which that looks for an executable file on the path.

To add support for different OS/compiler combination
ExtUtils::Cxx internal data has to be updated.

MakeMaker looks for the C++ compiler only when C++ sources are
detected. For C or pure Perl modules MakeMaker does the same as
before.

Only old C++ modules with hard coded compiler settings should be
affected or broken by this patch.

Bye,

- Salvador.
Michael G Schwern
2003-11-27 08:48:47 UTC
Permalink
On Wed, Nov 26, 2003 at 07:40:57PM +0000, Salvador Fandiño wrote:
> A new version of the "C++ support for MakeMaker" patch is ready.

The more I think about this the more nervous I get at putting all this new
functionality into MakeMaker in a part that's not very well understood
(XS module building) while I'm trying to wind down development.

Sorry to ask this so late in the process, but is there a good reason this
couldn't be done with Module::Build instead? And "because lots of people
use MakeMaker" isn't one of them.


--
Michael G Schwern ***@pobox.com http://www.pobox.com/~schwern/
Playstation? Of course Perl runs on Playstation.
-- Jarkko Hietaniemi
Fandiño
2003-11-27 10:04:56 UTC
Permalink
Ed Avis
2003-11-28 19:10:38 UTC
Permalink
On Thu, 27 Nov 2003, Salvador Fandi=F1o wrote:

>Actually, most C/C++ developers should feel more comfortable using
>and customizing ExtUtils::MakeMaker/make than Module::Build and not
>because EU::MM has been there forever and M::B is new but because
>they can look at the generated Makefile and understand it.

LOL!

--=20
Ed Avis <***@membled.com>
Michael G Schwern
2003-12-03 05:24:58 UTC
Permalink
On Fri, Nov 28, 2003 at 07:10:38PM +0000, Ed Avis wrote:
> On Thu, 27 Nov 2003, Salvador Fandiño wrote:
>
> >Actually, most C/C++ developers should feel more comfortable using
> >and customizing ExtUtils::MakeMaker/make than Module::Build and not
> >because EU::MM has been there forever and M::B is new but because
> >they can look at the generated Makefile and understand it.
>
> LOL!

*chuckle* Yeah, my sentiments exactly. Like any crufty automated code
generation tool, MM's output is a tad obfuscated.

Anyhow, if we do our job right nobody should have to look at the Makefile.


--
Michael G Schwern ***@pobox.com http://www.pobox.com/~schwern/
Stupid am I? Stupid like a fox!
Nicholas Clark
2003-11-29 18:51:22 UTC
Permalink
On Thu, Nov 27, 2003 at 02:04:56AM -0800, Salvador Fandiño wrote:

> well, Module::Build biggest strength is that it?s a pure Perl module
> that doesn't depend on an external tool like make but for C/C++
> modules you will need a development environment anyway and using make
> is not a problem at all.

I don't think that "using make is not a problem at all" is a valid argument.
ExtUtils::MakeMaker already jumps through a lot of hoops to (try to) make
portable makefiles. For example, we had a load of fun trying to work out
why SDBMFile broke on a MakeMaker change. It worked on Linux, but not
FreeBSD. Not the two most distant OSes in the world.

The perl5 core doesn't have a single Makefile for all of Unix and Win32.
In fact, it doesn't have a single Makefile for Win32. Every make is
different. Sometimes those differences bite.

More practical things like "nested modules" and the level of CPAN support
are reasonable reasons. But they need to be addressed in Module::Build,
rather than continuing to add blocks to the top of the MM tower.

Nicholas Clark
Salvador Fandiño
2003-12-01 22:07:05 UTC
Permalink
Nicholas Clark wrote:

> I don't think that "using make is not a problem at all" is a valid argument.
> ExtUtils::MakeMaker already jumps through a lot of hoops to (try to) make
> portable makefiles. For example, we had a load of fun trying to work out
> why SDBMFile broke on a MakeMaker change. It worked on Linux, but not
> FreeBSD. Not the two most distant OSes in the world.
>
> The perl5 core doesn't have a single Makefile for all of Unix and Win32.
> In fact, it doesn't have a single Makefile for Win32. Every make is
> different. Sometimes those differences bite.

ok, make has more problems than just "existential" ones, but that
does not invalidate my point: using make also has some benefits.

I'm not against Module::Build. I just think that both builders
can coexist. M::B will be better in some cases and EU::MM in
others. Let the programmer decide which one he prefers to use.

Perl motto is TMTOWTDI, isn't it?

- Salva.
Michael G Schwern
2003-12-03 05:38:35 UTC
Permalink
On Thu, Nov 27, 2003 at 02:04:56AM -0800, Salvador Fandiño wrote:
> well, Module::Build biggest strength is that it?s a pure Perl module
> that doesn't depend on an external tool like make but for C/C++
> modules you will need a development environment anyway and using make
> is not a problem at all.

In this context, Module::Build's strength is that its not a crufty
monster that's being prepped for mothballing. MM is. I really have no
desire to add or maintain new functionality to MM. Especially functionality
that's missing from Module::Build. Double especially new functionality that
has to do with compilers, the most frightening, least documented and least
tested part of MM. Unless you want to debug C++ builds on VMS for me? :)

I really don't want to open this maintenance can of worms without a really
good reason. I'd suggest you have a look at putting the functionality into
MB instead. Its really quite a clean architecture. I've found it
infinately easier to work with than MM.


> Actually, most C/C++ developers should feel more comfortable using
> and customizing ExtUtils::MakeMaker/make than Module::Build and not
> because EU::MM has been there forever and M::B is new but because
> they can look at the generated Makefile and understand it.

I've never found the "people can understand Makefiles" argument terribly
compelling, even if true. Besides, plain Makefiles aren't portable.


--
Michael G Schwern ***@pobox.com http://www.pobox.com/~schwern/
Do not try comedy at home! Milk & Cheese are advanced experts! Attempts at
comedy can be dangerously unfunny!
Nick Ing-Simmons
2003-11-27 15:58:13 UTC
Permalink
Michael G Schwern <***@pobox.com> writes:
>On Wed, Nov 26, 2003 at 07:40:57PM +0000, Salvador Fandiño wrote:
>> A new version of the "C++ support for MakeMaker" patch is ready.
>
>The more I think about this the more nervous I get at putting all this new
>functionality into MakeMaker in a part that's not very well understood
>(XS module building)

Some of us understand it ;-)

>while I'm trying to wind down development.
>
>Sorry to ask this so late in the process, but is there a good reason this
>couldn't be done with Module::Build instead?

I don't think so.

Does Module::Build do nested modules yet?

>And "because lots of people
>use MakeMaker" isn't one of them.
Paul Marquess
2003-11-30 10:24:17 UTC
Permalink
From: Nick Ing-Simmons [mailto:nick.ing-***@elixent.com]

>
>
> Michael G Schwern <***@pobox.com> writes:
> >On Wed, Nov 26, 2003 at 07:40:57PM +0000, Salvador Fandiño wrote:
> >> A new version of the "C++ support for MakeMaker" patch is ready.
> >
> >The more I think about this the more nervous I get at putting
> all this new
> >functionality into MakeMaker in a part that's not very well understood
> >(XS module building)
>
> Some of us understand it ;-)

Indeed. The interface provided by the -C++ option in xsubpp does make it dead easy to map a C++ class to the Perl equivalent.

The only issue I had was the lack of generic C++ exception to Perl exception mapping code. In the end I had to wrap all my XSUBs in something like this


void
SomeClass::SomeMethod()
INIT:
try {
CLEANUP:
}
catch (err & e)
{
// map C++ exception to Perl exception
}
catch (...)
{
// map C++ exception to Perl exception
}


This works fine, but it doesn't half bloat the code.

Paul
Michael G Schwern
2003-12-03 05:26:10 UTC
Permalink
On Thu, Nov 27, 2003 at 03:58:13PM +0000, Nick Ing-Simmons wrote:
> Does Module::Build do nested modules yet?

Dunno. Why is this relevant?


--
Michael G Schwern ***@pobox.com http://www.pobox.com/~schwern/
The eye opening delightful morning taste of expired cheese bits in sour milk!
Nick Ing-Simmons
2003-12-03 10:50:34 UTC
Permalink
Michael G Schwern <***@pobox.com> writes:
>On Thu, Nov 27, 2003 at 03:58:13PM +0000, Nick Ing-Simmons wrote:
>> Does Module::Build do nested modules yet?
>
>Dunno. Why is this relevant?

Because Tk:: and Audio:: (my two major XS modules on CPAN) are
both structured that way.

When Module::Build has a way to handle nested modules then
I as the de-facto XS expert can lead the heard down the M::B route.
Michael G Schwern
2003-12-03 17:57:49 UTC
Permalink
On Wed, Dec 03, 2003 at 10:50:34AM +0000, Nick Ing-Simmons wrote:
> Michael G Schwern <***@pobox.com> writes:
> >On Thu, Nov 27, 2003 at 03:58:13PM +0000, Nick Ing-Simmons wrote:
> >> Does Module::Build do nested modules yet?
> >
> >Dunno. Why is this relevant?
>
> Because Tk:: and Audio:: (my two major XS modules on CPAN) are
> both structured that way.
>
> When Module::Build has a way to handle nested modules then
> I as the de-facto XS expert can lead the heard down the M::B route.

Then you'd best get a-patchin'! ;)


--
Michael G Schwern ***@pobox.com http://www.pobox.com/~schwern/
What's a classy place like this doing around a low-life like you?
Loading...