diskstats error: Can't use an undefined value as a HASH reference at /etc/munin/plugins/diskstats line 120

[blueyed]

I am getting the following error from the diskstats plugin:

2011/03/12-01:26:45 [20111] Error output from diskstats:
2011/03/12-01:26:45 [20111]     Can't use an undefined value as a HASH reference at /etc/munin/plugins/diskstats line 120.
2011/03/12-01:26:45 [20111] Service 'diskstats' exited with status 255/0.

This happens probably since a long time already, but I have not noticed it since the multigraphing has been broken anyway.

The line in question appears to be: http://munin-monitoring.org/browser/branches/1.4-stable/plugins/node.d.linux/diskstats.in#L120

Unfortunately I cannot decrypt this Perl really.

The plugin runs on a hardware node (OpenVZ). It has returned some values successfully (months) ago.

[fish]

We ran into the same problem.
My investigations showed, that restore_state() in Munin::Plugin stops reading after 4k.
This happens not if run from munin-run, but from within the munin-node.
I have no clue why this happens, setting the line delimiter to undef should allow to slurp the whole file at ones. Probably were got hit by a perl bug.

To check if you run into the same issue, check if the file is complete in @state (Plugin.pm, line 305). In some cases the graph rendering still works, because diskstats selects the first (and still complete) data set from the state file.

All other plugins are affected too, but without any impact because they tend to store only a few bytes of data.

I don't really understand why you unset the line delimiter to read the whole file at once and afterwards split it? Why not stick to the default delimiter and read the file line by line? This avoids the issue (for lines <4k of course).
Here is a patch which simplifies the section and works around that problem:

Index: plugins/lib/Munin/Plugin.pm
===================================================================
--- plugins/lib/Munin/Plugin.pm	(revision 4276)
+++ plugins/lib/Munin/Plugin.pm	(working copy)
@@ -297,11 +297,10 @@
 
 sub restore_state {
     # Read a state vector from a plugin appropriate state file
-    local $/;
-
     open my $STATE, '<', $statefile or return;
 
-    my @state = split(/\n/, <$STATE>);
+    my @state = <$STATE>;
+    chomp @state;
 
     my $filemagic = shift(@state);
 

[lgosselin]

We hit the same issue with Munin 2.0.2.

We ran out of disk space one night and diskstats stopped reporting any metric, even after some disk space was freed. The patch suggested in the previous comment did not help in our case. We finally fixed the problem by removing the plugin state:

• stop munin-node
• move away /var/lib/munin-node/plugin-state/nobody/diskstats-::ffff:127.0.0.1
• start munin-node
• wait for a few measurements to be taken