Proc-FastSpawn-1.2/0000755000000000000000000000000012155240702012657 5ustar rootrootProc-FastSpawn-1.2/FastSpawn.pm0000644000000000000000000001141612155240674015136 0ustar rootroot=head1 NAME Proc::FastSpawn - fork+exec, or spawn, a subprocess as quickly as possible =head1 SYNOPSIS use Proc::FastSpawn; # simple use my $pid = spawn "/bin/echo", ["echo", "hello, world"]; ... waitpid $pid, 0; # with environment my $pid = spawn "/bin/echo", ["echo", "hello, world"], ["PATH=/bin", "HOME=/tmp"]; # inheriting file descriptors pipe R, W or die; fd_inherit fileno W; my $pid = spawn "/bin/sh", ["sh", "-c", "echo a pipe >&" . fileno W]; close W; print ; =head1 DESCRIPTION The purpose of this small (in scope and footprint) module is simple: spawn a subprocess asynchronously as efficiently and/or fast as possible. Basically the same as calling fork+exec (on POSIX), but hopefully faster than those two syscalls. Apart from fork overhead, this module also allows you to fork+exec programs when otherwise you couldn't - for example, when you use POSIX threads in your perl process then it generally isn't safe to call fork from perl, but it is safe to use this module to execute external processes. If neither of these are problems for you, you can safely ignore this module. So when is fork+exec not fast enough, how can you do it faster, and why would it matter? Forking a process requires making a complete copy of a process. Even thought almost every implementation only copies page tables and not the memory itself, this is still not free. For example, on my 3.6GHz amd64 box, I can fork a 5GB process only twenty times a second. For a real-time process that must meet stricter deadlines, this is too slow. For a busy and big web server, starting CGI scripts might mean unacceptable overhead. A workaround is to use C - this function isn't very portable, but it avoids the memory copy that C has to do. Some systems have an optimised implementation of C, and some systems have nothing. This module tries to abstract these differences away. As for what improvements to expect - on the 3.6GHz amd64 box that this module was originally developed on, a 3MB perl process (basically just perl + Proc::FastSpawn) takes 3.6s to run /bin/true 10000 times using fork+exec, and only 2.6s when using vfork+exec. In a 22MB process, the difference is already 5.0s vs 2.6s, and so on. =head1 FUNCTIONS All the following functions are currently exported by default. =over 4 =cut package Proc::FastSpawn; # only used on WIN32 - maddeningly complex and doesn't even work sub _quote { $_[0] = [@{ $_[0] }]; # make copy for (@{ $_[0] }) { if (/[\x01-\x20"]/) { # some sources say only space, "\t\n\v need to be escaped, microsoft says space and tab s/(\\*)"/$1$1\\"/g; # double + extra escape before " s/(\\+)$/$1$1/; # just double at end $_ = '"' . $_ . '"'; } } } BEGIN { $VERSION = '1.2'; our @ISA = qw(Exporter); our @EXPORT = qw(spawn spawnp fd_inherit); require Exporter; require XSLoader; XSLoader::load (__PACKAGE__, $VERSION); } =item $pid = spawn $path, \@argv[, \@envp] Creates a new process and tries to make it execute C<$path>, with the given arguments and optionally the given environment variables, similar to calling fork + execv, or execve. Returns the PID of the new process if successful. On any error, C is currently returned. Failure to execution might or might not be reported as C, or via a subprocess exit status of C<127>. =item $pid = spawnp $file, \@argv[, \@envp] Like C, but searches C<$file> in C<$ENV{PATH}> like the shell would do. =item fd_inherit $fileno[, $on] File descriptors can be inherited by the spawned processes or not. This is decided on a per file descriptor basis. This module does nothing to any preexisting handles, but with this call, you can change the state of a single file descriptor to either be inherited (C<$on> is true or missing) or not C<$on> is false). Free portability pro-tip: it seems native win32 perls ignore $^F and set all file handles to be inherited by default - but this function can switch it off. =back =head1 PORTABILITY NOTES On POSIX systems, this module currently calls vfork+exec, spawn, or fork+exec, depending on the platform. If your platform has a good vfork or spawn but is misdetected and falls back to slow fork+exec, drop me a note. On win32, the C<_spawn> family of functions is used, and the module tries hard to patch the new process into perl's internal pid table, so the pid returned should work with other Perl functions such as waitpid. Also, win32 doesn't have a meaningful way to quote arguments containing "special" characters, so this module tries it's best to quote those strings itself. Other typical platform limitations (such as being able to only have 64 or so subprocesses) are not worked around. =head1 AUTHOR Marc Lehmann http://home.schmorp.de/ =cut 1 Proc-FastSpawn-1.2/t/0000755000000000000000000000000012155240702013122 5ustar rootrootProc-FastSpawn-1.2/t/00_load.t0000644000000000000000000000017512125314007014525 0ustar rootrootBEGIN { $| = 1; print "1..1\n"; } END {print "not ok 1\n" unless $loaded;} use Proc::FastSpawn; $loaded = 1; print "ok 1\n"; Proc-FastSpawn-1.2/t/01_basic.t0000644000000000000000000000073512137070774014707 0ustar rootrootBEGIN { $| = 1; print "1..9\n"; } use Proc::FastSpawn; print "ok 1\n"; my $pid = spawn $^X, ["perl", "-e", "exit 5"]; print $pid ? "" : "not ", "ok 2\n"; print +($pid == waitpid $pid, 0) ? "" : "not ", "ok 3\n"; print $? == 0x0500 ? "" : "not ", "ok 4\n"; print "ok 5\n"; $pid = spawnp $^X, ["perl", "-e", "exit 6"]; print $pid ? "" : "not ", "ok 6\n"; print +($pid == waitpid $pid, 0) ? "" : "not ", "ok 7\n"; print $? == 0x0600 ? "" : "not ", "ok 8\n"; print "ok 9\n"; Proc-FastSpawn-1.2/t/02_inherit.t0000644000000000000000000000130512130145315015246 0ustar rootrootBEGIN { $| = 1; print "1..6\n"; } use Proc::FastSpawn; print +(pipe R, W) ? "" : "not ", "ok 1\n"; fd_inherit fileno R, 0; fd_inherit fileno W, 1; # OpenBSD has a corrupted $^X when linking aaginst -lpthread # so use Config instead. use Config; my $pid = spawn $Config{perlpath}, [ qw(perl -e), ' my $gr = (open my $r, "<&" . $ARGV[0]) ? 1 : 0; my $gw = (open my $w, ">&" . $ARGV[1]) ? 1 : 0; syswrite $w, "$gr$gw"; ', fileno R, fileno W ]; print $pid ? "" : "not ", "ok 2\n"; close W; my $grw = ; print +($pid == waitpid $pid, 0) ? "" : "not ", "ok 3\n"; print $? == 0x0000 ? "" : "not ", "ok 4\n"; print $grw eq "01" ? "" : "not ", "ok 5 # $grw\n"; print "ok 6\n"; Proc-FastSpawn-1.2/Changes0000644000000000000000000000154512155240670014163 0ustar rootrootRevision history for Perl extension Proc::FastSpawn 1.2 Mon Jun 10 05:02:10 CEST 2013 - do not use vfork with spawnp, to avoid running into lots of problems (execvp might call malloc, better thread safety etc.), at the cost of using fork instead of vfork. - fix a (rather theoretical) memory corruption bug. 1.1 Sun Apr 28 02:47:31 CEST 2013 - add spawnp function for convenience. 1.0 Sun Apr 7 03:32:54 CEST 2013 - work around corrupted $^X on openbsd, when linked against libpthread (only affects a testcase). 0.2 Thu Apr 4 08:24:32 CEST 2013 - define _WIN32_WINNT to windows xp, to make sure we get the GetProcessId prototype. - add synopsis section with some examples. 0.1 Sat Mar 30 02:16:48 CET 2013 - intiial CPAN release. 0.01 Fri Mar 29 14:32:28 CET 2013 - first rough idea. Proc-FastSpawn-1.2/FastSpawn.xs0000644000000000000000000000640712155236637015164 0ustar rootroot/* GetProcessId is XP and up, which means in all supported versions */ /* but older SDK's might need this */ #define _WIN32_WINNT NTDDI_WINXP #include "EXTERN.h" #include "perl.h" #include "XSUB.h" #include #ifdef WIN32 /* perl probably did this already */ #include #else #include #include #include /* openbsd seems to have a buggy vfork (what would you expect), */ /* while others might implement vfork as fork in older versions, which is fine */ #if __linux || __FreeBSD__ || __NetBSD__ || __sun #define USE_VFORK 1 #endif #if !USE_VFORK #if _POSIX_SPAWN >= 200809L #define USE_SPAWN 1 #include #else #define vfork() fork() #endif #endif #endif static char *const * array_to_cvec (SV *sv) { AV *av; int n, i; char **cvec; if (!SvROK (sv) || SvTYPE (SvRV (sv)) != SVt_PVAV) croak ("expected a reference to an array of argument/environment strings"); av = (AV *)SvRV (sv); n = av_len (av) + 1; cvec = (char **)SvPVX (sv_2mortal (NEWSV (0, sizeof (char *) * (n + 1)))); for (i = 0; i < n; ++i) cvec [i] = SvPVbyte_nolen (*av_fetch (av, i, 1)); cvec [n] = 0; return cvec; } MODULE = Proc::FastSpawn PACKAGE = Proc::FastSpawn PROTOTYPES: ENABLE BOOT: #ifndef WIN32 cv_undef (get_cv ("Proc::FastSpawn::_quote", 0)); #endif long spawn (const char *path, SV *argv, SV *envp = &PL_sv_undef) ALIAS: spawnp = 1 INIT: { #ifdef WIN32 if (w32_num_children >= MAXIMUM_WAIT_OBJECTS) { errno = EAGAIN; XSRETURN_UNDEF; } argv = sv_2mortal (newSVsv (argv)); PUSHMARK (SP); XPUSHs (argv); PUTBACK; call_pv ("Proc::FastSpawn::_quote", G_VOID | G_DISCARD); SPAGAIN; #endif } CODE: { extern char **environ; char *const *cargv = array_to_cvec (argv); char *const *cenvp = SvOK (envp) ? array_to_cvec (envp) : environ; intptr_t pid; fflush (0); #ifdef WIN32 pid = (ix ? _spawnvpe : _spawnve) (_P_NOWAIT, path, cargv, cenvp); if (pid == -1) XSRETURN_UNDEF; /* do it like perl, dadadoop dadadoop */ w32_child_handles [w32_num_children] = (HANDLE)pid; pid = GetProcessId ((HANDLE)pid); /* get the real pid, unfortunately, requires wxp or newer */ w32_child_pids [w32_num_children] = pid; ++w32_num_children; #elif USE_SPAWN { pid_t xpid; errno = (ix ? posix_spawnp : posix_spawn) (&xpid, path, 0, 0, cargv, cenvp); if (errno) XSRETURN_UNDEF; pid = xpid; } #else pid = (ix ? fork : vfork) (); if (pid < 0) XSRETURN_UNDEF; if (pid == 0) { if (ix) { environ = (char **)cenvp; execvp (path, cargv); } else execve (path, cargv, cenvp); _exit (127); } #endif RETVAL = pid; } OUTPUT: RETVAL void fd_inherit (int fd, int on = 1) CODE: #ifdef WIN32 SetHandleInformation ((HANDLE)_get_osfhandle (fd), HANDLE_FLAG_INHERIT, on ? HANDLE_FLAG_INHERIT : 0); #else fcntl (fd, F_SETFD, on ? 0 : FD_CLOEXEC); #endif Proc-FastSpawn-1.2/META.json0000644000000000000000000000142712155240702014304 0ustar rootroot{ "abstract" : "unknown", "author" : [ "unknown" ], "dynamic_config" : 1, "generated_by" : "ExtUtils::MakeMaker version 6.64, CPAN::Meta::Converter version 2.120630", "license" : [ "unknown" ], "meta-spec" : { "url" : "http://search.cpan.org/perldoc?CPAN::Meta::Spec", "version" : "2" }, "name" : "Proc-FastSpawn", "no_index" : { "directory" : [ "t", "inc" ] }, "prereqs" : { "build" : { "requires" : { "ExtUtils::MakeMaker" : "0" } }, "configure" : { "requires" : { "ExtUtils::MakeMaker" : "0" } }, "runtime" : { "requires" : {} } }, "release_status" : "stable", "version" : "1.2" } Proc-FastSpawn-1.2/META.yml0000644000000000000000000000065512155240702014136 0ustar rootroot--- abstract: unknown author: - unknown build_requires: ExtUtils::MakeMaker: 0 configure_requires: ExtUtils::MakeMaker: 0 dynamic_config: 1 generated_by: 'ExtUtils::MakeMaker version 6.64, CPAN::Meta::Converter version 2.120630' license: unknown meta-spec: url: http://module-build.sourceforge.net/META-spec-v1.4.html version: 1.4 name: Proc-FastSpawn no_index: directory: - t - inc requires: {} version: 1.2 Proc-FastSpawn-1.2/README0000644000000000000000000001061612155240702013543 0ustar rootrootNAME Proc::FastSpawn - fork+exec, or spawn, a subprocess as quickly as possible SYNOPSIS use Proc::FastSpawn; # simple use my $pid = spawn "/bin/echo", ["echo", "hello, world"]; ... waitpid $pid, 0; # with environment my $pid = spawn "/bin/echo", ["echo", "hello, world"], ["PATH=/bin", "HOME=/tmp"]; # inheriting file descriptors pipe R, W or die; fd_inherit fileno W; my $pid = spawn "/bin/sh", ["sh", "-c", "echo a pipe >&" . fileno W]; close W; print ; DESCRIPTION The purpose of this small (in scope and footprint) module is simple: spawn a subprocess asynchronously as efficiently and/or fast as possible. Basically the same as calling fork+exec (on POSIX), but hopefully faster than those two syscalls. Apart from fork overhead, this module also allows you to fork+exec programs when otherwise you couldn't - for example, when you use POSIX threads in your perl process then it generally isn't safe to call fork from perl, but it is safe to use this module to execute external processes. If neither of these are problems for you, you can safely ignore this module. So when is fork+exec not fast enough, how can you do it faster, and why would it matter? Forking a process requires making a complete copy of a process. Even thought almost every implementation only copies page tables and not the memory itself, this is still not free. For example, on my 3.6GHz amd64 box, I can fork a 5GB process only twenty times a second. For a real-time process that must meet stricter deadlines, this is too slow. For a busy and big web server, starting CGI scripts might mean unacceptable overhead. A workaround is to use "vfork" - this function isn't very portable, but it avoids the memory copy that "fork" has to do. Some systems have an optimised implementation of "spawn", and some systems have nothing. This module tries to abstract these differences away. As for what improvements to expect - on the 3.6GHz amd64 box that this module was originally developed on, a 3MB perl process (basically just perl + Proc::FastSpawn) takes 3.6s to run /bin/true 10000 times using fork+exec, and only 2.6s when using vfork+exec. In a 22MB process, the difference is already 5.0s vs 2.6s, and so on. FUNCTIONS All the following functions are currently exported by default. $pid = spawn $path, \@argv[, \@envp] Creates a new process and tries to make it execute $path, with the given arguments and optionally the given environment variables, similar to calling fork + execv, or execve. Returns the PID of the new process if successful. On any error, "undef" is currently returned. Failure to execution might or might not be reported as "undef", or via a subprocess exit status of 127. $pid = spawnp $file, \@argv[, \@envp] Like "spawn", but searches $file in $ENV{PATH} like the shell would do. fd_inherit $fileno[, $on] File descriptors can be inherited by the spawned processes or not. This is decided on a per file descriptor basis. This module does nothing to any preexisting handles, but with this call, you can change the state of a single file descriptor to either be inherited ($on is true or missing) or not $on is false). Free portability pro-tip: it seems native win32 perls ignore $^F and set all file handles to be inherited by default - but this function can switch it off. PORTABILITY NOTES On POSIX systems, this module currently calls vfork+exec, spawn, or fork+exec, depending on the platform. If your platform has a good vfork or spawn but is misdetected and falls back to slow fork+exec, drop me a note. On win32, the "_spawn" family of functions is used, and the module tries hard to patch the new process into perl's internal pid table, so the pid returned should work with other Perl functions such as waitpid. Also, win32 doesn't have a meaningful way to quote arguments containing "special" characters, so this module tries it's best to quote those strings itself. Other typical platform limitations (such as being able to only have 64 or so subprocesses) are not worked around. AUTHOR Marc Lehmann http://home.schmorp.de/ Proc-FastSpawn-1.2/Makefile.PL0000644000000000000000000000050712133751531014636 0ustar rootrootuse ExtUtils::MakeMaker; use 5.008; WriteMakefile( dist => { PREOP => 'pod2text FastSpawn.pm | tee README >$(DISTVNAME)/README; chmod -R u=rwX,go=rX . ;', COMPRESS => 'gzip -9v', SUFFIX => '.gz', }, NAME => "Proc::FastSpawn", VERSION_FROM => "FastSpawn.pm", ); Proc-FastSpawn-1.2/COPYING0000644000000000000000000000007612125313575013723 0ustar rootrootThis module is licensed under the same terms as perl itself. Proc-FastSpawn-1.2/MANIFEST0000644000000000000000000000042612155240702014012 0ustar rootrootREADME Changes MANIFEST COPYING Makefile.PL FastSpawn.pm FastSpawn.xs t/00_load.t t/01_basic.t t/02_inherit.t META.yml Module YAML meta-data (added by MakeMaker) META.json Module JSON meta-data (added by MakeMaker)