What could cause Perl system calls to start failing?

Question

A small request: I read Stack Overflow's Perl questions every day, and answer/contribute where I can; today I need the community's help!

Perl setup: I'm running Active Perl 5.8.8 on Windows. The installation is on our department server's local drive, which is also shared to the network. All department users run Perl on their own PCs by pointing to this network-installed Perl. This has worked for years, and isn't causing a problem, but it's a piece of info needed to understand the problem.

The server in question is also our "cron" (Scheduled Task) server, handling a variety of automation tasks. Suddenly last week, system calls in Perl scripts (on the server) started failing (details below). At first, I suspected a corrupt Perl installation, but all of the client PCs can still run the same Perl scripts without any issues, leading me to think it's a server issue. I've rebooted the server twice, and the problem is persistent, thus I need help!

Here are some examples of the various way that system calls are failing, boiled down to Perl one-liners:

% perl -e "system('dir')"

That should print a "dir" listing, but instead it opens a sub-shell. If I type "exit", I can exit the sub-shell, and I'm back in the original shell (confirmed by examining the shell history using the UP arrow key).

% perl -e "print `dir`"

This actually hangs. Nothing happens at all. If I Ctrl-C to kill the process, I get the message "Terminating on signal SIGINT(2)" and the DOS prompt comes back. But, any future commands in the DOS prompt (even just hitting ENTER) cause the error "The process tried to write to a nonexistent pipe.". You have to exit the DOS prompt as it's effectively useless.

Last example:

% perl -e "system('Z:/Scripts/rebuild.pl')"

'ebuild.pl' is not recognized as an internal or external command, operable program or batch file.

In this case, Perl switches the forward-slashes (/) to DOS/Windows back-slashes (), which it has done just fine for years. But, Perl is interpreting the "\r" at the beginning of the "rebuild.pl" filename as a carriage-return (I think) and looking for the remaining "ebuild.pl". Calls to other scriptnames whose characters can't be misinterpreted like that result in the above hangs (if you use backticks) of sub-shells being opened (for system() calls).

I'm not just puzzled by this - I'm desperate! Our department server's "cron" jobs are useless right now since we use a lot of system calls.

Again, I don't think this is a corrupt Perl install, since the network users can run fine. So, what could happen on an individual machine (not tied to the Perl install itself) that could cause Perl's system calls to fail like this?

Environment settings, as requested:

ALLUSERSPROFILE=C:\Documents and Settings\All Users
APPDATA=C:\Documents and Settings\engmodem\Application Data
CDSROOT=Z:\Cadence\SPB_16.5
CDS_CONCEPT_NOSPLASH=TRUE
CDS_LIC_ONLY=1
CDS_SITE=Z:\Cadence\Sites\16.5
CHDL_LIB_INST_DIR=%CDSROOT%
CLIENTNAME=USENTUTTLJL3C
ClusterLog=C:\WINDOWS\Cluster\cluster.log
CommonProgramFiles=C:\Program Files\Common Files
COMPUTERNAME=CORPUSAPP5
ComSpec=C:\WINDOWS\system32\cmd.exe
CONCEPT_INST_DIR=%CDSROOT%
FP_NO_HOST_CHECK=NO
HOMEDRIVE=H:
HOMEPATH=\
HOMESHARE=\\PF1\HOME
ICMHOME=Z:\Software\PTC\INTERC~1
INSTDIR=%CDSROOT%
LOGONSERVER=\\ENGMAHO5
LSF_BINDIR=Z:\Software\LSF\bin
LSF_ENVDIR=\\hwc151\LSF_6.2\etc
MESSAGE=BROADCAST
NUMBER_OF_PROCESSORS=2
OA_PLUGIN_PATH=%CDSROOT%\Share\oaPlugIns
OS=Windows_NT
Path=C:\Program Files\Legato\nsr\bin;Z:\oracle\ora92\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Windows Resource Kits\Tools\;Z:\Software\Perl\5.8.8\bin;C:\Program Files\Oracle\jre\1.3.1\bin;C:\Program Files\Oracle\jre\1.1.8\bin;C:\Program Files\Support Tools\;Z:\Software\LSF\bin;C:\Program Files\PHP\;C:\Program Files\Microsoft SQL Server\90\Tools\binn\;C:\Program Files\EMC RepliStor;C:\GitStack\python;C:\GitStack\python\Scripts;C:\GitStack\git\cmd;Z:\Scripts;Z:\bin;Z:\Cadence\SPB_16.5\tools\bin;Z:\Cadence\SPB_16.5\tools\fet\bin;Z:\Cadence\SPB_16.5\tools\pcb\bin;Z:\Cadence\SPB_16.5\OpenAccess\bin\win32\opt
PATHEXT=.COM;.EXE;.BAT;.PL;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.VBS
PCB_LIBRARY=16
PERL5SHELL=cmd
PHPRC=C:\Program Files\PHP\
PROCESSOR_ARCHITECTURE=x86
PROCESSOR_IDENTIFIER=x86 Family 6 Model 29 Stepping 1, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=1d01
ProgramFiles=C:\Program Files
PROMPT=$P$G
PULLUP_DIFF_PAIRS=TRUE
SESSIONNAME=RDP-Tcp#1
SystemDrive=C:
SystemRoot=C:\WINDOWS
TZ=EST5EDT
VISUALSVN_SERVER=C:\Program Files\VisualSVN Server\
WF_RESOURCES=Z:\oracle\ora92\WF\RES\WFus.RES
windir=C:\WINDOWS

Answer 1

It turned out the reason of this weird behavior was incorrectly defined PERL5SHELL variable: cmd.exe (the shell interpreter in Windows) should be called with some parameters for proper processing - there parameters went missing after some updates. )

By the way, in The Doc it's said that Perl usually assumes the 'cmd.exe /x /c' line as a shell executable anyway if PERL5SHELL environment variable is not defined at all.

P.S. I really like this thread: it clearly shows the purpose of comments. )