Data transfer is now an essential function for science discoveries, particularly within big data environments. To
support data transfer for big data science, there is a need for high performance, scalable, end-to-end, and programmable
networks that enable science applications to use the network most efficiently. The existing network
paradigm that support big data science consists of three major components: terabit networks that provide high
network bandwidths, Data Transfer Nodes (DTNs) and Science DMZ architecture that bypasses the performance
hotspots in typical campus networks, and on-demand secure circuits/paths reservation systems, such as ESNet
OSCARS and Internet2 AL2S, which provides automated, guaranteed bandwidth service in WAN. This network
paradigm has proven to be very successful. However, to reach its full potentials, we claim that existing network
paradigm for big data science must address three major problems: the last mile problem, the scalability problem,
and the programmability problem. To address these problems, we proposed a solution called AmoebaNet.
AmoebaNet applies Software Defined Networking (SDN) technology to provide “QoS-guaranteed” network
services in campus or local area networks. AmoebaNet complements existing network paradigm for big data
science: it allows application to program networks at run-time for optimum performance; and, in conjunction
with WAN circuits/paths reservation system such as ESNet OSCARS and Internet2 AL2S; it solves the last mile
problem and the scalability problem.
Network as a service; QoS; Data science; Big data; End-to-end path