download0 view63
twitter facebook

공공누리This item is licensed Korea Open Government License

dc.contributor.author
PREETHIKA KASU
dc.contributor.author
김영재
dc.contributor.author
SCOTT ATCHLEY
dc.contributor.author
박경석
dc.contributor.author
엄정호
dc.contributor.author
김태욱
dc.date.accessioned
2022-01-12T06:29:50Z
dc.date.available
2022-01-12T06:29:50Z
dc.date.issued
2019-03-21
dc.identifier.issn
2169-3536
dc.identifier.uri
https://repository.kisti.re.kr/handle/10580/16257
dc.description.abstract
The layout-aware data scheduling (LADS) data movement framework optimizes congestion for end-to-end data transfers. During data transfer, LADS can avoid congested storage elements by exploiting the underlying storage layout at each endpoint. This improves the I/O bandwidth and hence the data transfer rate across high-speed networks. However, the absence of fault tolerance (FT) in LADS results in data retransmission overhead and may lead to possible data integrity issues upon faults. In this paper, we propose object-logging FT mechanisms to avoid transmitting the objects that are successfully written into the parallel file system (PFS) at the sink end. Depending on the number of log files created for the whole dataset, we have classified our FT mechanisms into three different categories: file logger, transaction logger, and universal logger. Also, to address the space overhead, we have proposed different methods of populating the log files with the information of the successfully transferred objects. We have evaluated the data transfer performance and recovery time overhead of the proposed object-logging-based FT mechanisms on the LADS data transfer framework. Our experimental results show that FT mechanisms exhibit negligible overhead (<; 1%) with respect to the data transfer time. However, the fault recovery time is 10% higher than the total data transfer time at any fault point.
dc.language.iso
eng
dc.publisher
IEEE
dc.relation.ispartofseries
IEEE Access;
dc.title
FTLADS: Object-Logging Based Fault-Tolerant Big Data Transfer System Using Layout Aware Data Scheduling
dc.identifier.doi
10.1109/access.2019.2905158
dc.citation.volume
7
dc.contributor.approver
KOAR, ADMIN
dc.date.dateaccepted
2022-01-12T06:29:50Z
dc.date.datesubmitted
2022-01-12T06:29:50Z
dc.identifier.bibliographicCitation
vol. 7
dc.subject.keyword
Big data
dc.subject.keyword
geo-distributed data centers
dc.subject.keyword
fault tolerance
dc.subject.keyword
parallel system
Appears in Collections:
7. KISTI 연구성과 > 학술지 발표논문
Files in This Item:
There are no files associated with this item.

Browse