Data mining middleware for wide-area high-perfomance networks

Abstract

In this paper, we describe two distibuted, data intensive applications that were demonstrated at iGrid2005. One involves transporting astronomical data from the SDSS and the other involves computing histograms from multiple high-volume data streams. Both rely on newly developed data transport and data mining middleware. Specifically, we describe a new version of the UDT network protocol called Composible-UDT, a file transfer utility based upon UDT called UDT-Gateway, and an application for building histogram on high-volume data flows called BESH. For both demonstrations, we include a summary of the experimental studies performed at iGrid2005.