Fetching plugin data or config via telnet yields no output

Rlated issue regarding the munin-master componenent as mentioned in the report: #847

Ok, I seem to be closing in into the problem: I recompiled the RPM (using the exact same sources as on the SLE10) un my openSUSE 11.1 box, and everything worked.

So, the million bucks prize question is: what is different from a Perl 5.8.8 (on SLE10) to a Perl 5.10.0 (on a openSUSE 11.x/SLE11) that _this_ above behaviour appears. And: can it be fixed apart from upgrading the whole perl (we might install a /usr/local or /opt perl 5.10.x)

I have exact same problem on cca 50 servers with no option to upgrade Perl, these are SLES10 SP2/ OES2 SP1 servers. Upgrading to SLES10 SP3 change nothing Perl is same 5.8.8.

David

I analyzed this a bit further, as it was blocking the rollout of Munin 1.4.x on our infrastructure. It seems that SLES10's perl swallows (or misassigns) STDOUT when exec()ing an external program - something I haven't been able to verify via Google or Novell's Knowledge Base.

Using some print statements in Munin/Node/OS.pm and Munin/Node/Service.pm and using Data::Dumper to analyze the result of the structure returned from fork_service I get the following output in Debug mode:

2010/04/26-14:00:53 [30416] DEBUG: > # munin node at localhost
2010/04/26-14:00:55 [30416] DEBUG: < fetch apache_accesses
2010/04/26-14:00:55 [30416] DEBUG: Running command "fetch apache_accesses^M".
Atempting to read from plugins stderr
Read 200 bytes from plugin stderr
Atempting to read from plugins stdout
Read 0 bytes from plugin stdout
2010/04/26-14:00:55 [30416] DEBUG: In _run_service: res = $VAR1 = {\n          '
timed_out' => '',\n          'stderr' => [\n                        '# Set /rgid
/ruid/egid/euid/ to /10001/10001/10001 10001 /10001/',\n                        
'# Setting up environment',\n                        '# Environment url = http:/
/127.0.0.1:%d/server-status?auto',\n                        '# About to run \'/e
tc/munin/plugins/apache_accesses\''\n                      ],\n          'retval
' => '0',\n          'stdout' => []\n        };
2010/04/26-14:00:55 [30416] Error output from apache_accesses:
2010/04/26-14:00:55 [30416]     # Set /rgid/ruid/egid/euid/ to /10001/10001/1000
1 10001 /10001/
2010/04/26-14:00:55 [30416]     # Setting up environment
2010/04/26-14:00:55 [30416]     # Environment url = http://127.0.0.1:%d/server-s
tatus?auto
2010/04/26-14:00:55 [30416]     # About to run '/etc/munin/plugins/apache_access
es'
2010/04/26-14:00:55 [30416] DEBUG: In _print_service: started
2010/04/26-14:00:55 [30416] DEBUG: > .
2010/04/26-14:00:55 [30416] DEBUG: In _print_service: finished
2010/04/26-14:01:05 [30416] Connection timed out

Obviously, the exec() writes to STDERR, but nothing to STDOUT.

A quick and dirty solution is to not use exec(), which can be done with the following patch:

--- node/lib/Munin/Node/Service.pm.orig 2010-04-26 12:33:56.217889267 +0000
+++ node/lib/Munin/Node/Service.pm      2010-04-26 14:07:09.951259792 +0000
@@ -212,7 +212,14 @@
     print STDERR "# About to run '", join (' ', @command), "'\n" 
        if $config->{DEBUG};
 
-    exec @command;
+    # work around exec bug in SLES10's Perl
+    # exec @command;
+    my $res = `@command`;
+    print STDERR "# '", join (' ', @command), "' returned $res\n"
+        if $config->{DEBUG};
+    print STDOUT $res;
+    exit;
+
 } 

I haven't found a way of properly integrating this into the code such that it is only used on SLES10.

Please execute proper care when using this patch - it is, after all, only a workaround.

Thorsten

I just found out that SLES 9 (Update 4) seems to have the same problem, and that the above patch solves it here, too.

Thorsten