Elsevier

Computer Networks

Volume 112, 15 January 2017, Pages 279-293
Computer Networks

Review article
A survey: Control plane scalability issues and approaches in Software-Defined Networking (SDN)

https://doi.org/10.1016/j.comnet.2016.11.017Get rights and content

Abstract

Software-Defined Networking (SDN) architecture has emerged in response to limitations of traditional networking architectures in satisfying today’s complex networking needs. In particular, SDN allows network administrators to manage network services through abstraction of lower-level functionality. However, SDN is a logically centralized technology. Therefore, scalability, and especially the control plane (i.e. controller) scalability in SDN is one of the problems that needs more attention. In this survey paper, we first discuss the scalability problems of controller(s) in an SDN architecture. We then comprehensively survey and summarize the characterizations and taxonomy of state-of-the-art studies in SDN control plane scalability. We organize the discussion on control plane scalability into two broad approaches: Topology-related approaches and Mechanisms-related approaches. In Topology-related approaches, we study the relation between topology of architectures and scalability issues. It has sub-categories of Centralized (Single) Controller Designs and Distributed approaches. Distributed approaches, in turn, have also sub-categories: Distributed (Flat) Controller Designs, Hierarchical Controller Designs, and Hybrid Designs. In Mechanisms-related approaches, we review the relation between various mechanisms used to optimize controllers and scalability issues. It has sub-categories of Parallelism-based Optimization and Control Plane Routing Scheme-based Optimization. Furthermore, we outline the potential challenges and open problems that need to be addressed further for more scalable SDN control planes.

Introduction

Increasing cloud services, server virtualization, sharp growth of mobility and content-like video have led researchers to rethink today’s network architectures. In traditional architectures, network devices and appliances are complex and difficult for (re)configuration and (re)installation since they require highly skilled personnel. Adding or moving a device from a network requires extra costs. It is also time-consuming because IT people need to deal with multiple switches, routers, etc. and update ACLs, VLANs and other mechanisms [1]. Furthermore, as business demands or user needs increase day by day, application developers, carriers, and enterprises delve into evolving new services and facilities. However, vendor dependency is an obstacle deterring them from developing new networking applications and services for their networks due to slow equipment product cycle, application testing and deployment. Therefore, today’s data centers, carriers, and campuses need more dynamic architectures.

Software Defined Networking (SDN) [2], [3], [4], [5], [6] architecture has emerged in response to the aforementioned limitations of traditional networking architectures. SDN aims to decouple the control plane and data plane. This separation provides network operators/administrators with efficient use of network resources and eases provisioning of resources. Also, SDN brings ease of programmability in changing the characteristics of whole networks. This simplifies the management of the network, since it is decoupled from the data plane. Therefore, network operators can easily and quickly manage, configure, and optimize network resources with dynamic, automated and proprietary-free programs in SDN architecture [7]. Google’s datacenter WAN, B4 [8], is one of the examples for SDN adopted in a large-scale network with the aforementioned purposes. In addition, since the network is logically centralized in SDN, controllers have a global visibility of the whole network unlike conventional networking. Hence, they can dynamically optimize flow-management and resources.

Despite the advantages of centralized control in SDN architectures, SDN faces some issues challenging its nature (i.e. centralized control) due to day by day increasing network demands. Although network operators enhance the performance of the network controllers, it still cannot be enough to meet the high network demands such as flow request and monitoring network statistics. For example, one of the earlier SDN controllers, NOX [9], can serve only 30K flow requests per second with a response time less than 10 ms. This insufficiency appears more in large-scale networks or data centers compared to small networks. Kandula et al. [10] report that a cluster of 1500 servers receives 100K flows per second on average. Also, Erickson [11] states that a network with 100 switches can result in 10 million flow arrivals per second in the worst case. These numbers indicate that the control plane in an SDN architecture is prone to suffer from scalability issues due to its centralized nature. Furthermore, Sezer et al. [1] state that one of the main challenges in SDN is the scalability issue, which especially needs more attention by researchers. Therefore, understanding and improving the scalability of the SDN control plane (i.e. controller) is a critical problem for successful adoption of SDN for large scale networks or networks with many flows.

In this paper, we survey scalability problems of the control plane (i.e. controllers) in SDN architectures as opposed to other general SDN surveys. We discuss the main causes that make the control plane suffer from scalability issues in an SDN architecture. We also present characterizations and classifications of proposals based on the primary concepts exploited to alleviate the controller scalability issues. In addition, we point out the main challenges along with existing proposals in controller scalability.

We note that data plane scalability in SDN is not a part of this paper’s scope. However, as a brief note, data plane scalability in SDN is mostly dominated by (1) processing power, (2) capacity of memory/buffer, and (3) software implementation of data plane devices. For a more detailed and comprehensive discussion on data plane scalability in SDN, we would like the readers to direct following studies: [12], [13], [14], [15], [16], [17], [18], [19].

In the remaining sections of the paper, Section 2 gives a light-weight overview of the SDN framework with OpenFlow protocol. In Section 3, we discuss the Scalability concept regarding its meaning and present some scalability metrics proposed in the literature to quantitatively measure the scalability of systems both in general and SDN context as well as contributors to scalability issues in SDN. Section 4 presents our organization of the studies over control plane scalability in SDN. Section 5 outlines the relation between topology of architectures and scalability issues while Section 6 discusses the relation between other mechanisms used to optimize the controller performance and scalability issues. Section 7 presents a comparative discussion over control plane scalability proposals. In Section 8, we outline the potential challenges and open issues that need to be addressed further for fully scalable SDN control planes in the future in a nutshell. Finally, Section 9 wraps the paper up with concluding remarks.

Section snippets

An overview of SDN architecture and openflow protocol

SDN architecture with OpenFlow protocol enables network operators to treat flows in a finer-granular way compared to the traditional networks by means of controllers. In a traditional network, flows (or packets) are mainly treated based on a single or a few attribute combinations of packet headers, such as longest destination IP prefixes, destination MAC addresses, or a combination of IP addresses and TCP/UDP port numbers etc. SDN allows to manage flows based on more attributes of packet

Scalability and its causes in SDN

Scalability is a frequently-claimed attribute of various systems. It is a multi-dimensional topic. While the basic notion is intuitive, the term scalability does not evoke the same concept to everybody. Therefore, there is no general precise agreement on neither its definition nor content. While some people may refer to scalability as optimization of processing power to CPUs, others may define it as a measure of parallelization of applications across different machines. However, regardless of

Classification of control plane scalability proposals

As discussed in Section 3, there is no consensus on the definition of scalability. Therefore, it is not easy to present an unified classification for scalability solutions. The organization that we present in this paper reflects our own point of view over the proposed studies in SDN control plane scalability.

As shown in Fig. 2, we organize the discussion on control plane scalability into two broad approaches. The first approach is Topology-related Approaches with sub-categories of Centralized

Topology-related approaches

In this approach, we review the relation between topology of architectures and scalability issues. The proposals that use different topology models, illustrated in Fig. 3, can be classified in four prevalent architectures: Centralized (Single) Controller Designs, Distributed (Flat) Controller Designs, Hierarchical Controller Designs, and Hybrid Designs. These designs have their own intrinsic advantages and disadvantages with respect to control plane scalability. We explain these architectures

Mechanisms-related approaches

In Section 5, we have discussed topology-related approaches. In this section, we discuss other mechanisms used to optimize controller(s). Mechanisms-related approaches primarily exploit various optimization techniques in order to alleviate the foregoing scalability issues in SDN networks. They aim to empower the controller performance so that it can handle more packet flows per second (i.e. throughput), improve the latency, and reduce overhead. One way to increase throughput and improve latency

Comparison of control plane scalability proposals

Controllers are the main entities in decision-making processes in SDN networks. They perform crucial tasks affecting performance of the whole network. Currently, there exist more than 35 different publicly-available and proprietary SDN OpenFlow controllers created by different research groups, vendors, and organizations from both academia and industry, written in different languages, and having different performances. This rapid growing of controllers has raised questions regarding performance

Challenges and existing proposals in SDN control plane

While SDN is becoming a mature technology, the control plane scalability issues deserve more research efforts from both academia and industry. In this section, we discuss the general problems in an SDN control plane. However, each of these problems affects the scalability of the control plane in SDN. Therefore, these problems need to be taken care of by network operators while designing/operating their SDN networks. In the following, we state the main SDN control plane challenges along with

Conclusions

Software Defined Networking is a promising emerging architecture for many networking environments such as data centers, enterprise networks, campus networks, cloud networks, and WAN. The major advantages of SDN are its programmability and agility. However, the scalability issues in the control plane is one major problem in SDN that needs more research attention. In the paper, we have firstly given an overview of the SDN architecture and OpenFlow protocol along with its support mechanisms for

Acknowledgments

This work was partially supported by National Science Foundation under Grant No. 1547411.

Murat Karakus received the BS Degree in Mathematics from Suleyman Demirel University, Turkey, in 2009, and the MS Degree in Computer Science and Information Systems from the University of Michigan-Flint, in 2013. He is currently working through his PhD in the Department of Computer and Information Science at Indiana University Purdue University - Indianapolis. He is the recipient of the Best Paper Award at ACM SIGITE 2011 conference. His current research interests include new network

References (110)

  • S. Sezer et al.

    Are we ready for sdn? implementation challenges for software-defined networks

    Commun. Mag., IEEE

    (2013)
  • Software-Defined Networking: The New Norm for Networks, Technical Report. Open Networking Foundation (ONF),...
  • K. Kirkpatrick

    Software-defined networking

    Commun. ACM

    (2013)
  • F. de Oliveira Silva et al.

    Enabling future internet architecture research and experimentation by using software defined networking

    Software Defined Networking (EWSDN), 2012 European Workshop on

    (2012)
  • G. Goth

    Software-defined networking could shake up more than packets

    Internet Comput., IEEE

    (2011)
  • W. Xia et al.

    A survey on software-defined networking

    Commun. Surv. Tut., IEEE

    (2014)
  • K. Bakshi

    Considerations for software defined networking (sdn): approaches and use cases

    Aerospace Conference, 2013 IEEE

    (2013)
  • S. Jain et al.

    B4: experience with a globally-deployed software defined wan

    Proceedings of the ACM SIGCOMM 2013 Conference on SIGCOMM

    (2013)
  • N. Gude et al.

    Nox: towards an operating system for networks

    SIGCOMM Comput. Commun. Rev.

    (2008)
  • S. Kandula et al.

    The nature of data center traffic: Measurements & analysis

    Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement Conference

    (2009)
  • D. Erickson

    The beacon openflow controller

    Proceedings of the Second ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking

    (2013)
  • Y. Luo et al.

    Accelerating openflow switching with network processors

    Proceedings of the 5th ACM/IEEE Symposium on Architectures for Networking and Communications Systems

    (2009)
  • V. Tanyingyong et al.

    Using hardware classification to improve pc-based openflow switching

    2011 IEEE 12th International Conference on High Performance Switching and Routing

    (2011)
  • R. Narayanan et al.

    Macroflows and microflows: Enabling rapid network innovation through a split sdn data plane

    2012 European Workshop on Software Defined Networking

    (2012)
  • G. Lu et al.

    Using cpu as a traffic co-processing unit in commodity switches

    Proceedings of the First Workshop on Hot Topics in Software Defined Networks

    (2012)
  • K. Kannan, S. Banerjee, Compact TCAM: Flow Entry Compaction in TCAM for Power Aware SDN, Springer Berlin Heidelberg,...
  • N. Kang et al.

    Optimizing the “one big switch” abstraction in software-defined networks

    Proceedings of the Ninth ACM Conference on Emerging Networking Experiments and Technologies

    (2013)
  • N. Katta et al.

    Infinite cacheflow in software-defined networks

    Proceedings of the Third Workshop on Hot Topics in Software Defined Networking

    (2014)
  • Q. Zuo et al.

    On generality of the data plane and scalability of the control plane in software-defined networking

    China Commun.

    (2014)
  • N. McKeown et al.

    Openflow: enabling innovation in campus networks

    SIGCOMM Comput. Commun. Rev.

    (2008)
  • F. Hu et al.

    A survey on software defined networking (sdn) and openflow: from concept to implementation

    Commun. Surv. Tut., IEEE

    (2014)
  • S.J. Vaughan-Nichols

    Openflow: the next generation of the network?

    IEEE Comput.

    (2011)
  • OpenFlow Switch Specification, 1.5.1, 2015. URL...
  • SDN architecture. Technical Report. Open Networking Foundation (ONF)....
  • P. Lin et al.

    A west-east bridge based sdn inter-domain testbed

    Commun. Mag., IEEE

    (2015)
  • M. Fernandez

    Comparing OpenFlow controller paradigms scalability: reactive and proactive

    Advanced Information Networking and Applications (AINA), 2013 IEEE 27th International Conference on

    (2013)
  • OpenFlow Switch Specification, 1.1.0, 2011. URL...
  • OpenFlow Switch Specification, 1.2.0, 2011. URL...
  • B.J. van Asten et al.

    Scalability and resilience of software-defined networking: an overview

    CoRR

    (2014)
  • K. Hwang et al.

    Scalable Parallel Computing: Technology, Architecture, Programming

    (1998)
  • X.H. Sun et al.

    Scalability of parallel algorithm-machine combinations

    IEEE Trans. Parallel Distrib. Syst.

    (1994)
  • V. Kumar et al.

    Introduction to Parallel Computing: Design and Analysis of Algorithms

    (1994)
  • A.Y. Grama et al.

    Isoefficiency: measuring the scalability of parallel algorithms and architectures

    IEEE Concurrency

    (1993)
  • L. Pastor et al.

    An efficiency and scalability model for heterogeneous clusters

    Cluster Computing, 2001. Proceedings. 2001 IEEE International Conference on

    (2001)
  • X.H. Sun et al.

    Scalability of heterogeneous computing

    Proceedings of the 2005 International Conference on Parallel Processing

    (2005)
  • P. Jogalekar et al.

    Evaluating the scalability of distributed systems

    IEEE Trans. Parallel Distrib. Syst.

    (2000)
  • A. Tootoonchian et al.

    On controller performance in software-defined networks

    Proceedings of the 2Nd USENIX Conference on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services

    (2012)
  • S.A. Shah et al.

    An architectural evaluation of sdn controllers

    2013 IEEE International Conference on Communications (ICC)

    (2013)
  • F. Benamrane et al.

    Performances of openflow-based software-defined networks: an overview

    Journal of Networks

    (2015)
  • K. He et al.

    Latency in software defined networks: Measurements and mitigation techniques

    Proceedings of the 2015 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems

    (2015)
  • P. Isaia et al.

    Performance benchmarking of sdn experimental platforms

    2016 IEEE NetSoft Conference and Workshops (NetSoft)

    (2016)
  • J. Hu et al.

    Scalability of control planes for software defined networks: Modeling and evaluation

    Quality of Service (IWQoS), 2014 IEEE 22nd International Symposium of

    (2014)
  • M. Karakus et al.

    A scalability metric for control planes in Software Defined Networks (SDNs)

    2016 IEEE 30th International Conference on Advanced Information Networking and Applications (AINA)

    (2016)
  • S. Yeganeh et al.

    On scalability of software-defined networking

    Commun. Mag., IEEE

    (2013)
  • A. Shalimov et al.

    Advanced study of sdn/openflow controllers

    Proceedings of the 9th Central & Eastern European Software Engineering Conference in Russia

    (2013)
  • T. Benson et al.

    Network traffic characteristics of data centers in the wild

    Proceedings of the 10th ACM SIGCOMM Conference on Internet Measurement

    (2010)
  • B. Heller et al.

    The controller placement problem

    Proceedings of the first workshop on Hot topics in software defined networks

    (2012)
  • S. Azodolmolky et al.

    Performance evaluation of a scalable software-defined networking deployment

    Software Defined Networks (EWSDN), 2013 Second European Workshop on

    (2013)
  • R. Hartert et al.

    A declarative and expressive approach to control forwarding paths in carrier-Grade networks

    SIGCOMM Comput. Commun. Rev.

    (2015)
  • A.R. Curtis et al.

    Devoflow: scaling flow management for high-performance networks

    SIGCOMM Comput. Commun. Rev.

    (2011)
  • Cited by (302)

    View all citing articles on Scopus

    Murat Karakus received the BS Degree in Mathematics from Suleyman Demirel University, Turkey, in 2009, and the MS Degree in Computer Science and Information Systems from the University of Michigan-Flint, in 2013. He is currently working through his PhD in the Department of Computer and Information Science at Indiana University Purdue University - Indianapolis. He is the recipient of the Best Paper Award at ACM SIGITE 2011 conference. His current research interests include new network architectures (particularly Software-Defined Networking (SDN)), scalability, Quality of Service (QoS), routing, and introducing programming to non-CS majors.

    Arjan Durresi is a Professor of Computer Science at Indiana University Purdue University in Indianapolis, Indiana. In the past, he held positions at LSU, and The Ohio State University. His research interest include network architectures, security and trust management. He has published over eighty papers in journals and over 200 papers in conference proceedings, and seven book chapters. He also has over thirty contributions to standardization organizations such as IETF, ATM Forum, ITU, ANSI and TIA. His research has been funded by NSF, the States of Ohio and Louisiana, as well as university and industry sources.

    View full text