wiki:protocol-multifetch

RFC multifetch protocol

This text follows RFC-like language. Refer to RFC2119.

Design goals:

  • Simple ability for plugins to send sub-5min sampled values
  • Storing protocol for spoolfetch repository
  • Simplicity & transparency: storing by append-only

Existing model

Actually each plugin gives only one value.

Advantages

  • Lightweight plugin
  • Ability to have a streaming plugin

Disadvantages

  • For very high sampling rates (<< 1m) the overhead is noticable.

Proposed new model

A plugin MAY just send many timed-values at once. It MUST then prefix the values with an epoch. This format is mostly used for an efficient spoolfetch storage : the format is plain text and append-only. The different values MAY be separated with a blank line for easier parsing.

The epoch MUST be only increasing.

Streaming plugins

This enables having a streaming plugin : it never ends when asked for values.

It would be very useful for the cpu plugin that would open a streaming vmstat, massage its output, and restream the output. The advantage would be to have a very little overhead and then avoiding regular monitoring spikes.

Protocol example

field1.value 1234567890:1
field2.value 1234567890:2
field3.value 1234567890:3

field1.value 1234567891:10
field2.value 1234567891:20
field3.value 1234567891:30

field1.value 1234567900:10
field2.value 1234567900:60
field3.value 1234567900:7.3423
Last modified at 2010-06-23T09:13:02+02:00 Last modified on 2010-06-23T09:13:02+02:00