Monitoring your deployed Applications

Checking App for Life

Chuck Hill

We usually just use a script that runs as a cron job.  It hits a direct action on the app that returns a known "all is OK" response.  If that does not get returned, alarms ring.

Pascal Robert

We use Nagios with the checkhttp plugin.  This plugin actually make a HTTP (or HTTPS, if the plugin was compiled with openssl support) to our apps and check a DirectAction to see if it's answer "1".  If not, Nagios send a trigger.

Ray Kiddy

We use nagios in several ways. We use checkping to make sure the machines are alive. We use checkhttp to check that the front-end of our cluster is responding on port 80 and we have another service that checks WebDAV directories that we use. These last need to check that authentication is working, as well as access. We also have a DirectAction that checks the whole chain, from the web server to the apps and then to the databases. We send pages only from this last service.

We check that wotaskd is running with a script that acts as a plugin. It would be nice if we could use checkhttp but there is no single string which tells us what the wotaskd is doing. For example, it would be helpful if there was, somewhere, a single line in the wotaskd port 1085 output which said something like "host: am1.bigserver.com, applications expected 200, applications running 200". Then we could just use the s parameter on checkhttp instead of doing a bunch of extra stuff ourselves.

I prefer many small checks to help diagnose problems, but sending a page in the middle of the night is only from the one service that checks the whole chain of dependencies. We now have 176 services, but I add at least a few a week. There are always more things trying to break our deployments....