My friend and coworker Jesse Becker has penned (err, typed) an article for Linux Magazine on using open source tools to monitor a High Performance Computing cluster from collecting data, parsing it, displaying it, and using it. Here's a brief excerpt, but check out the whole article.

The amount of data collected in even a small cluster can quickly become unwieldy, and making sense of it is the other half of "monitoring" (and indeed, the whole point!).