Concepts and Data model
To support such flexible deployments and make custom monitoring easy, we have devised a simple model for your monitoring setup. All our monitoring configuration builds upon a few simple concepts
The core piece of infrastructure, a host, refers to a physical server or virtual machine on your network. This traditionally where your different Services would reside. We track details about a Host (e.g. networks, location and running processes) as well as resource usage (e.g. CPU, memory, I/O, and disk space)
With the introduction of Containers and serverless, there is now an additional layer of infrastructure. Containers provide a means of deployment and isolation for your Services. For this reason, we track their details and resource usage. If you are having a problem with a service you can track which container it is running on and the host for that container (provided we can access the underlying Host).
Service Check (or Check)
Service checks are used, as the name indicates, to check the status and the performance of a service. Checks are made up of:
- An executable command, which outputs data to STDOUT or STDERR, and returns a Nagios exit code to indicate the state.
- Some variables that are passed to the command as ENVIRONMENT VARIABLES when run.
- selectors to specify the hosts/containers to target with the check.
- and a format (or handler) to tell outlyer how to handle the data returned from the command
The command can be any executable, but usually, this would be a call to a plugin. One of the formats supported is Nagios, so you can take any existing Nagios plugin and use that.
Plugins are a generic way of collecting metrics from a particular type of service. These plugins should be configurable via command line flags or environment variables, then output performance metrics via STDOUT or STDERR. Plugins can then be called from different Checks simple by setting the appropriate config options.
Plugins dictate how to collect the data, and Checks dictate where from. For example, you might create a MySQL plugin, which knows how to obtain performance stats from MySQL. You would then create a per MySQL cluster in your environment, and call your generic MySQL plugin with options (e.g. URL and port) appropriate for the particular cluster being checked.