Archive for August 2010

Monitoring Hadoop

August 24th, 2010 — 7:53pm

As a side project this summer, I implemented a simple visual interface for HOP, an extended version of Hadoop. This was used by the HOP creators in their demo of HOP at this summer’s SIGMOD.

Screenshot of hop visual interface

Hop visual interface

The graphical elements were produced using Protovis since I needed an excuse to play around with it. We ran into minor performance problems using Protovis for so many plots in a single page. In a production system it would be wiser to generate and cache the plots on the server side.

Update: The screenshot shows a task scheduling imbalance bug that we found in HOP using the visual interface.

Comment » | visualization

Back to top