download0 view153
twitter facebook

공공누리This item is licensed Korea Open Government License

Title
On the role of message broker middleware for many-task computing
Author(s)
NGUYEN NGOC CAO김직수이재환황순욱
Publication Year
2018-03-31
Abstract
We have designed and implemented a new data processing framework called ‘‘Many-task computing On HAdoop’’
(MOHA) which aims to effectively support fine-grained many-task applications that can show another type of dataintensive
workloads in the YARN-based Hadoop 2.0 platform. MOHA is developed as one of Hadoop YARN applications
so that it can transparently co-host existing many-task computing (MTC) applications with other data processing workflows
such as MapReduce in a single Hadoop cluster. In this paper, we investigate main characteristics of two well-known opensource
message broker middleware systems (Apache ActiveMQ and Kafka) and their implications on a many-task management
scheme in our MOHA framework. Through our extensive experiments with a real MTC application, we
demonstrate and discuss trade-offs between parallelism and load balancing of data access patterns in message broker
middleware systems for Many-Task Computing on Hadoop.
Keyword
Many-task computing; Message broker middleware; Hadoop; YARN; ActiveMQ; Kafka; MOHA
Journal Title
Cluster Computing
ISSN
1386-7857
Files in This Item:
There are no files associated with this item.
Appears in Collections:
7. KISTI 연구성과 > 학술지 발표논문
URI
https://repository.kisti.re.kr/handle/10580/14774
Export
RIS (EndNote)
XLS (Excel)
XML

Browse