download0 view140
twitter facebook

공공누리This item is licensed Korea Open Government License

Title
Enhancing network IO performance for a virtualized Hadoop cluster
Author(s)
정진규조희승최동훈
Publication Year
2016-10-14
Abstract
A MapReduce programming model is proposed to process big data using Hadoop, one of the major cloud computing frameworks. With the increasing adoption of cloud computing, runninga Hadoop framework on a virtualized cluster is a compelling approach to reducing costs and increasing efficiency. In this paper, we measure the performance of a virtualized network andanalyze the impact of network performance on Hadoop workloads running on a virtualized cluster. Then, we propose a virtualized network I/O architecture as a novel optimization for avirtualized Hadoop cluster for a public/private cloud provider. The proposed network architecture combines traditional network configurations and achieves better performance for Hadoopworkloads. We also show a better way to utilize the rack awareness feature of the Hadoop framework in the proposed computing environment. The evaluation demonstrates that the proposednetwork architecture and mechanisms improve performance by up to 4.1 times compared with a bridge network architecture. This novel architecture can even virtually match the performanceof the expensive, hardware‐based single root I/O virtualization network architecture.
Keyword
Hadoop; performance; virtualization
Journal Title
Concurrency and Computation: Practice and Experience
ISSN
1532-0626
Files in This Item:
There are no files associated with this item.
Appears in Collections:
7. KISTI 연구성과 > 학술지 발표논문
URI
https://repository.kisti.re.kr/handle/10580/14534
Export
RIS (EndNote)
XLS (Excel)
XML

Browse