GithubHelp home page GithubHelp logo

module-compile-pm's Introduction

Name

Module::Compile - Perl Module Compilation

Version

This document describes Module::Compile version 0.38.

Synopsis

package Foo;
use Module::Compile -base;

sub pmc_compile {
    my ($class, $source) = @_;
    # Convert $source into (most likely Perl 5) $compiled_output
    return $compiled_output;
}

In Bar.pm:

package Bar;

use Foo;
...
no Foo

or (implied "no Foo;"):

package Bar;

{
    use Foo;
    ...
}

To compile Bar.pm into Bar.pmc:

perl -c Bar.pm

Description

This module provides a system for writing modules that compile other Perl modules.

Modules that use these compilation modules get compiled into some altered form the first time they are run. The result is cached into .pmc files.

Perl has native support for .pmc files. It always checks for them, before loading a .pm file.

Example

You can declare a v6.pm compiler with:

package v6;
use Module::Compile -base;

sub pmc_compile {
    my ($class, $source) = @_;
    # ... some way to invoke pugs and give p5 code back ...
}

and use it like:

# MyModule.pm
use v6-pugs;
module MyModule;
# ...some p6 code here...
no v6;
# ...back to p5 land...

On the first time this module is loaded, it will compile Perl 6 blocks into Perl 5 (as soon as the no v6 line is seen), and merge it with the Perl 5 blocks, saving the result into a MyModule.pmc file.

The next time around, Perl 5 will automatically load MyModule.pmc when someone says use MyModule. On the other hand, Perl 6 can run MyModule.pm s a Perl 6 module just fine, as use v6-pugs and no v6 both works in a Perl 6 setting.

The v6.pm module will also check if MyModule.pmc is up to date. If it is, then it will touch its timestamp so the .pmc is loaded on the next time.

Benefits

Module::Compile compilers gives you the following benefits:

  • Ability to mix many source filterish modules in a much more sane manner. Module::Compile controls the compilation process, calling each compiler at the right time with the right data.

  • Ability to ship precompiled modules without shipping Module::Compile and the compiler modules themselves.

  • Easier debugging of compiled/filtered code. The .pmc has the real code you want to see.

  • Zero additional runtime penalty after compilation, because perl has already been doing the .pmc check on every module load since 1999!

Parsing and Dispatch

NOTE: *** NOT FULLY IMPLEMENTED YET ***

Module::Compile attempts to make source filtering a sane process, by parsing up your module's source code into various blocks; so that by the time a compiler is called it only gets the source code that it should be looking at.

This section describes the rather complex algorithm that Module::Compile uses.

First, the source module is preprocessed to hide heredocs, since the content inside heredocs can possibly confuse further parsing.

Next, the source module is divided into a shallow tree of blocks:

PREAMBLE:
  (SUBROUTINE | BAREBLOCK | POD | PLAIN)S
PACKAGES:
  PREFACE
  (SUBROUTINE | BAREBLOCK | POD | PLAIN)S
DATA

All of these blocks begin and end on line boundaries. They are described as follows:

PREAMBLE

Lines before the first package statement.

PACKAGES

Lines beginning with a `package statement and continuing

until the next `package` or `DATA` section.
DATA

The DATA section. Begins with the line __DATA__ or

__END__.

SUBROUTINE

A top level (not nested) subroutine. Ending '}' must be

on its own line in the first column.

BAREBLOCK

A top level (not nested) code block. Ending '}' must be

on its own line in the first column.

POD

Pod sections beginning with ^=\w+ and ending with =cut.

PLAIN

Lines not in SUBROUTINE, BAREBLOCK or POD.

PREFACE

Lines before the first block in a package.

Next, all the blocks are scanned for lines like:

use Foo qw'x y z';
no Foo;

Where Foo is a Module::Compile subclass.

The lines within a given block between a use and no statement are marked to be passed to that compiler. The end of an inner block effectively acts as a no statement for any compile sections in that block. use statements in a PREFACE apply to all the code in a PACKAGE. use statements in a PREAMBLE apply to all the code in all PACKAGES.

After all the code has been parsed into blocks and the blocks have been marked for various compilers, Module::Compile dispatches the code blocks to the

compilers. It does so in a most specific to most general order.  So inner
blocks get compiled first, then outer blocks.

A compiler may choose to declare that its result not be recompiled by some other containing parser. In this case the result of the compilation is replaced by a single line containing the hexadecimal digest of the result in double quotes followed by a semicolon. Like:

"f1d2d2f924e986ac86fdf7b36c94bcdf32beec15";

The rationale of this is that random strings are usually left alone by compilers. After all the compilers have finished, the digest lines will be expanded again.

Every bit of the default process described above is overridable by various methods.

Distribution Support

Module::Install makes it terribly easy to prepare a module distribution with compiled .pmc files. See Module::Install::PMC. All you need to do is add this line to your Makefile.PL:

pmc_support;

Any of your distrbution's modules that use Module::Compile based modules will automatically be compiled into .pmc files and shipped with your distribtution precompiled. This means that people who install your module distribtution do not need to have the compilers installed themselves. So you don't need to make the compiler modules be prerequisites.

See Also

Authors

Copyright and License

Copyright 2006-2019. Ingy döt Net.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See http://www.perl.com/perl/misc/Artistic.html

module-compile-pm's People

Contributors

ingydotnet avatar knowledgejunkie avatar mohawk2 avatar xenu avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar

module-compile-pm's Issues

Failed test '.pmc load works same' on Windows

Dear maintainers,

when trying to install M::C on Strawberry Perl (tried 5.26, 5.28 and 5.30), I run into the following test failure.
I found this when installing PDL, which depends on M::C.

Best,
Simon

Building and testing Module-Compile-0.37 ... cp lib/Module/Compile.pod blib\lib\Module\Compile.pod
cp lib/Module/Compile/Opt.pm blib\lib\Module\Compile\Opt.pm
cp lib/Module/Compile.pm blib\lib\Module\Compile.pm
cp lib/Module/Optimize.pod blib\lib\Module\Optimize.pod
cp lib/Module/Optimize.pm blib\lib\Module\Optimize.pm
cp lib/Module/Compile/Opt.pod blib\lib\Module\Compile\Opt.pod
"C:\strawberry\perl\bin\perl.exe" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib\lib', 'blib\arch')" t/*.t
t/author-pod-syntax.t .. skipped: these tests are for testing by the author
t/compile.t ............ ok
#   Failed test '.pmc load works same'
#   at t/data1.t line 29.
# 
#          got: '1'
#     expected: '0'
# 
# Output was: t/data2.t.subrun .. 
# No subtests run 
# 
# Test Summary Report
# -------------------
# t/data2.t.subrun (Wstat: 0 Tests: 0 Failed: 0)
#   Parse errors: No plan found in TAP output
# Files=1, Tests=0,  0 wallclock secs ( 0.05 usr +  0.00 sys =  0.05 CPU)
# Result: FAIL
# Error was: 
# Looks like you failed 1 test of 4.
t/data1.t .............. 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/4 subtests 
t/fold_blocks.t ........ ok
t/lexical.t ............ ok
t/orz.t ................ ok
t/parse.t .............. ok
t/pm.t ................. skipped: In progress...
t/pmc.t ................ ok
Test Summary Report
-------------------
t/data1.t            (Wstat: 256 Tests: 4 Failed: 1)
  Failed test:  4
  Non-zero exit status: 1
Files=9, Tests=23,  3 wallclock secs ( 0.03 usr +  0.01 sys =  0.05 CPU)
Result: FAIL
Failed 1/9 test programs. 1/23 subtests failed.
gmake: *** [Makefile:858: test_dynamic] Error 255
! Installing Module::Compile failed. See C:\Users\appveyor\.cpanm\work\1568317892.1504\build.log for details. Retry with --force to force install it.
FAIL

t/000-compile-modules.t Error: can't open (eval 6) for input

On a Mac OS X VM (10.10.1), using the system Perl (5.18.2), installation of Module::Compile (via CPAN and manually) fails because t/000-compile-modules.t fails.

The failure is exactly the same as in, e.g., http://www.cpantesters.org/cpan/report/b3617498-ded7-11e4-aa8d-adc4dc153cf8

#   Failed test 'use Module::Optimize;'
#   at t/000-compile-modules.t line 9.
#     Tried to use 'Module::Optimize'.
#     Error:  Can't open (eval 6) for input:
# No such file or directory at <snip>/blib/lib/Module/Compile.pm line 201.
# BEGIN failed--compilation aborted at (eval 6) line 2.

#   Failed test 'use Module::Compile;'
#   at t/000-compile-modules.t line 9.
#     Tried to use 'Module::Compile'.
#     Error:  Can't open (eval 7) for input:
# No such file or directory at <snip>/blib/lib/Module/Compile.pm line 201.
# BEGIN failed--compilation aborted at (eval 7) line 2.
# Looks like you failed 2 tests of 3.
t/000-compile-modules.t .. 
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/3 subtests 

Module::Compile fails to differentiate between left shift and heredoc

Given this code:

package X;
use Module::Compile;

sub pmc_compile {
        my $x = "
        sub foo {
                return shift << 14;
        }
1;
";
}

1;

instead of leaving the left shift unmodified, it creates a pmc file like this

# Generated by Module::Compile 0.37 (Module::Compile 0.37) - do not edit!
################((( 32-bit Checksum Validator III )))################
#line 1
BEGIN { use 5.006; local (*F, $/); ($F = __FILE__) =~ s!c$!!; open(F)
or die "Cannot open $F: $!"; binmode(F, ':crlf'); if (unpack('%32N*',
$F=readline(*F)) != 0xF3A041E1) { use Filter::Util::Call; my $f = $F;
filter_add(sub { filter_del(); 1 while &filter_read; $_ = $f; 1; })}}
#line 1
package X;

sub pmc_compile {
        my $x = "         
        sub foo {
                return shift <<14;
6aad8405895e8b8edb5f83273d2252c12783c6d2

1;


`

can not install module

Hey, I cannot install the module because the distribution is missing the Base.pm. Here is the top of the report:

cpanm (App::cpanminus) 1.7001 on perl 5.016003 built for x86_64-linux-thread-multi
Work directory is /ws/steshriv-rtp/perl_build/cpanm/work/1411424375.16733
You have make /usr/bin/make
You have LWP 6.08
You have /bin/tar: tar (GNU tar) 1.15.1
You have /usr/cisco/packages/dbdoracle/current/bin/unzip
Checking if you have ExtUtils::MakeMaker 6.31 ... Yes (6.98)
Checking if you have ExtUtils::Install 1.46 ... Yes (1.58)
Searching Module::Compile on mirror file:///ws/steshriv-rtp/CPAN ...
Downloading index file file:///ws/steshriv-rtp/CPAN/modules/02packages.details.txt.gz ...
Uncompressing index file...
--> Working on Module::Compile
Fetching file:///ws/steshriv-rtp/CPAN/authors/id/I/IN/INGY/Module-Compile-0.34.tar.gz
-> OK
Unpacking Module-Compile-0.34.tar.gz
Entering Module-Compile-0.34
Checking configure dependencies from META.json
Checking if you have ExtUtils::MakeMaker 6.30 ... Yes (6.98)
Configuring Module-Compile-0.34
Running Makefile.PL
-> OK
Checking dependencies from MYMETA.json ...
Checking if you have Digest::SHA1 2.13 ... Yes (2.13)
Checking if you have ExtUtils::MakeMaker 0 ... Yes (6.98)
Building and testing Module-Compile-0.34
cp lib/Module/Compile.pm blib/lib/Module/Compile.pm
cp lib/Module/Optimize.pm blib/lib/Module/Optimize.pm
cp lib/Module/Compile.pod blib/lib/Module/Compile.pod
cp lib/Module/Compile/Opt.pm blib/lib/Module/Compile/Opt.pm
cp lib/Module/Compile/Optimize.pod blib/lib/Module/Compile/Optimize.pod
cp lib/Module/Install/Admin/PMC.pm blib/lib/Module/Install/Admin/PMC.pm
cp lib/Module/Compile/Opt.pod blib/lib/Module/Compile/Opt.pod
cp lib/Module/Install/PMC.pm blib/lib/Module/Install/PMC.pm
PERL_DL_NONLAZY=1 /auto/usrcisco-linux-rhel5.0-x86-64-hard/packages/perl-testing-NO-ENTRY/perl-5.16.3/bin/perl "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/.t

Failed test 'use Module::Install::PMC;'

at t/000-compile-modules.t line 11.

Tried to use 'Module::Install::PMC'.

Error: Can't locate Module/Install/Base.pm in @inc (@inc contains: /ws/steshriv-rtp/perl_build/cpanm/work/1411424375.16733/Module-Compile-0.34/blib/lib /ws/steshriv-rtp/perl_build/cpanm/work/1411424375.16733/Module-Compile-0.34/blib/arch

---snip---

hope this helps.
steve

Depends on Module::Install?

I tried to install Module::Compile on a fresh perlbrew of Perl v5.20. The process failed, stating that it couldn't find Module::Install::Base. Installing Module::Install fixed the problem. Is this a dependency that is not listed?

Module::Install::PMC no longer indexed

as on #toolchain:

19:42 <@ether> ingy: why did you remove Module::Install::PMC from the latest release of Module::Compile? there was no explanation given, and the github repo has no commits between releases
19:42 <@ether> it remains indexed in the previous distribution, but this could change at any time if you happen to delete that tarball
19:43 <@ether> do I need to create a Dist::Zilla::Plugin::ModuleCompile to generate .pmc files?

There are still some dists out there shipping .pmc files.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.