Frequent false positive UNKNOWNs for rarely changing values

[schamane]

For details see also the mailing list thread "UNKNOWN and unexplained":
​http://sourceforge.net/mailarchive/message.php?msg_id=28805027

Since I upgraded an installation of Munin 1.4.5 to 2.0 (from Debian Experimental, i.e. 1.999.4603-1), I receive apparently bogus UNKNOWNs for 1 plugin of 1 node.

Relevant parts of munin-run spamassassin_min config:

  graph_args --base 1000 --lower-limit 0
  graph_period minute
  graph_scale no
  graph_total total
  spam.label spam
  spam.type DERIVE
  spam.min 0
  spam.draw AREA

$ munin-run spamassassin_min
spam.value 85
ham.value 190

Bogus UNKNOWNs messages:

  UNKNOWNs: spam is unknown.
  OKs: ham is 0.01.

Cleared later with e.g.

  OKs: spam is 0.01, ham is 0.01.

Today, at 6:30, that is 5 minutes after log rotation, I received the following message:

  OKs: spam is -0.43, ham is -0.62.

[schamane]

It is mentioned in the cited mailing list thread however I wanted to add here
that I apparently missed one relevant part of the plugin config:

spam.warning 20
spam.critical 40
ham.warning 20
ham.critical 40

See also the following comment about LimitsOld.pm

[schamane]

Warning ahead: I am not a programmer and I don't know much Perl. Still, I tried to dig a bit deeper:

The following minimal pseudo plugin should trigger UNKNOWNs after about 25 minutes:

#!/bin/bash
# Minimal Munin plugin to test a possible bug in Munin 2.0
# http://www.munin-monitoring.org/ticket/1203

[[ $1 = 'config' ]] && cat <<-EOM
	graph_title Munin Test Ticket 1203
	graph_args --base 1000 --lower-limit 0
	graph_category Munin
	value.label value
	value.type DERIVE
	value.min 0
	value.warning 20
	value.critical 40
EOM

VALARR=(10 11 11 11 11 11 11 12 13 13 13 14)
# let's iterate through $VALARR every 5 minutes (+%M)
echo value.value ${VALARR[$(($(date +%M)*2/10))]}

Apparently, lines 331-333 of the 2.0RC6 version of
/usr/share/perl5/Munin/Master/LimitsOld.pm play some role:

  } elsif (! defined $current_updated_value || ! defined $previous_updated_value 
             || $current_updated_value eq $previous_updated_value) {
    # No derive computing possible. Report unknown.
    $value = "U";

Could someone with a better understanding explain why we here check for

  $current_updated_value eq $previous_updated_value

I haven't had much chance of testing yet but for now it seems that removing this check helps to resolve my problem.

HTH.

[snide]

Replying to schamane:

Could someone with a better understanding explain why we here check for

  $current_updated_value eq $previous_updated_value

Oooooh. That's exactly the issue here. We shouldn't compare VALUE, but EPOCH.

Fixed in r4869.

Thx !

[schamane]

Thanks for fixing this!

Unfortunately, using the posted test plugin, with r4869, it seems Munin does not report any UNKNOWNs at all. I changed the last line to only echo value.value for more than an hour but no UNKNOWNs were reported :/

[schamane]

Again, I tried to track this down a bit further. It might well be that the original problem reported here is solved and that we are confronted with a new problem:

Even if my test plugin does not return a value, in LimitsOld.pm the 2 variables $current_updated_value and $previous_updated_value are defined (holding an old value), so LimitsOld.pm is actually calculating a derived value while the graph shows -nan. Hence, it never reports an UKNOWN state.

[schamane]

For the new issue of LimitsOld.pm not reporting UNKNOWNs I created a new ticket: #1221.

I suggest that the status of #1203 is changed to closed.