packetC Programming

The first question most developers ask when they hear about packetC is, “Why do we need yet another language?” The premise is simply to enhance the pace with which applications that live within the network can be developed and deployed. While that may seem overly simple, the issue is that building applications, or solutions, for today’s networks isn’t easy. What is meant by

within the network

? Solutions that are within the network are not generally considered client or server solutions. In the simplest cases, they are switches and routers. In more complex cases, they include components such as VoIP session border controllers, per subscriber broadband policy management, and core network infrastructure protection employing capabilities such as DDoS mitigation and DNS defense. Solutions such as these must be highly scalable, secure, and often require certification or accreditation. The requirements drive the need for leveraging massively-parallel systems and highly secure coding practices, while also representing networking protocols and transactions in the simplest manner possible. Finding that no existing language addressed the breadth of these requirements, we concluded that a new language was required. The introduction of packetC facilitates development of applications for this massively-parallel, highly secure, network-oriented world efficiently from concept to deployment.

The primary objective in the packetC design is to define a language that will allow software developers to use familiar, high-level language constructs to express coding solutions for packet processing applications for general purpose, and for CloudShield-enabled platforms in particular.

This chapter covers packetC coding-style recommendations based on common C++ development community practices. The following recommendations establish the guidelines that CloudShield-developed software is expected to follow within packetC. As always with style guidelines, these are suggestions and individual third-party developers may choose to follow their own style guidelines and packetC compilers shall not be your jury.

packetC is designed to be used with parallel-processing runtime environments, although an implementation may be compliant without providing this. The language was designed for environments in which a packetC program will typically be executing with large numbers of simultaneous contexts. In packetC, similar to interrupt service routines, the arrival of a packet is an event that triggers processing by a context. The packetC program is considered re-entrant and is designed such that more than one copy is executing at any point in time. Parallel processing is based upon each context running the entire application from receipt of a packet to completion of processing for the assigned packet. The packetC language specification does not prescribe how actual or apparent parallelism must be implemented. Hence, each copy of a packet module or shared module is termed a

context

, since the terms

process, task

, or

thread

imply how parallel execution is to be implemented and imply characteristics that may vary from one operating system to another.

C programmers will find that packetC provides a variety of familiar data types that follow C99 conventions. The use of identifiers, literals, and basic scalar data types is almost identical to what you would expect from C. In a few instances, such as enumeration types and structure bit-fields, there are subtle differences introduced to clean up ambiguities in C99 or to address some specific issues related to network structures required by packetC. Those differences don’t appear visually or grammatically as much as they do in the implementation expectations and meaning. Where deviations occur, they will be highlighted to help the C programmer note the important changes.

Many questions start to appear when coding moves from defining a few variables into actually doing something. For example, understanding the role and rules of initializing variables with regard to compile time and run-time instantiation is crucial. Constants and their initialization expressions require values which can be determined at compile time, not run time. In packetC, operators follow what you would expect from C99, although there are many special data types that interact in a few interesting ways with the standard operators.

Just like C, a packetC

function

is a discrete subprogram that can be invoked from multiple places in the program text and that returns a value (which may be of type

void

). The

function call

or

function enable

can associate a set of values with a corresponding set of variables declared as the function’s

parameters

, which act as variables local to the called function. C programmers will find packetC functions to play the same role and operate in a familiar manner. In addition, functions play new roles in packetC in support of parallel processing and scoping for the purposes of data hiding.

When it comes to working with data types and changing some rules from C99, packetC has done its best to treat types consistently and yet advance language practice by refining how packet data is defined and manipulated. As packet processing is all about analyzing byte streams of packet data into protocol and payload information, appropriate, domain-specific data types are central to packetC. Informally, complex data types are those other than the basic scalar types such as byte, short, int, and long. packetC complex types include traditional C types such as enumerations, arrays, structures, unions, and our extended types such as databases, records, searchsets, descriptors, and references, which are built on familiar type principles and practices.

Working with data types in packetC introduces some interesting dynamics that will initially require care and extra thought by C developers from time to time given some of the restrictions, mostly due to strong type enforcement. The notion of casting and the strong casting rules is imperative to ensuring code works as expected. Packets are modeled as arrays of bytes and working with portions of packets is essential to making sense of the data. In packetC, the notion of an array slice was introduced to allow for direct access to portions of byte arrays without the need for pointers and for keeping them in line with strong type enforcement. This also applies to complex structures and unions that can be copied or, better yet, cast back and forth to byte arrays, providing multiple ways to view data elements, depending on what is most convenient for the programmer.

The packet can be seen as either one of the simplest data elements in the language or one of the most complex, and hopefully both in a good way. In its simplest form, the packet is an array of bytes, as defined in

cloudshield.ph

, it simply looks similar to the statement below:

While many programs can perform quite a lot in a straight-line list of algorithms operating on data, at some point, making some conditional branches in the logic becomes the only way to really progress from a simple formula into a program. Control flow statements fall into the conditional branching category as well as into looping. This chapter discusses the minor variations between packetC and C conditional branching while subsequent chapters cover looping.

Control statements direct the program to perform a set of operations repeatedly until some condition exists that terminates the repetition. packetC provides all of the C control statements. This includes forloops, while-do, do-while, and do-until, as well as jump controls such as goto, break, continue, return, and exit. There are no surprises with regard to how packetC implements these control statements.

packetC utilizes a try-catch-throw exception-handling system similar to many modern languages, such as C++ and Java. In packetC, all exceptions must be caught whether they are system-defined or application-defined. That means that all applications must be wrapped within an error-handler to accommodate catching any errors that may be thrown by the functionality utilized. Any possible errors thrown by an application that are not caught will be flagged by the compiler and compilation will not be allowed to proceed.

In packetC, several extended data types are present that do not appear in standard C but do provide significant support for packet-processing applications. These data types are extensions of familiar C types with special characteristics, including the fact that they define objects with methods that operate on their data. These types are databases, descriptors, and searchsets.

As was described in the chapter covering databases, packetC introduces several extended-data types that do not appear in standard C to provide significant support for packet-processing applications. These data types are extensions of familiar C types with special characteristics including the familiar C++ syntax as objects with methods that operate on their data. These types are databases, descriptors, and searchsets.

After reaching the halfway point in this book, have you finally hit the wall on still trying to figure out how the language gets away without pointers? Array slices, databases that can insert and delete entries provide some clever ways to avoid them, but at some point some of the data structures will need to have a pointer to another structure. But how would that work with the strict typing? Some complex data types have a real need for selecting between multiple other complex data types, such as building a relational database or even trying to determine which table of strings to scan based upon a match protocol field in header analysis. Welcome to the need for references.

The complexities of parallel processing introduce the problem that generally leads to complex interprocess communications. With packetC, the notion of global data is ever-present and even though access to data elements is atomic, programs represent information in more complex structures than basic scalar types. As such, a mechanism for communication with other contexts is required in order to notify them to not touch portions of global memory that are in use, or more important, being manipulated by another context. Locking of memory can grind a parallel system to a halt. So, instead, packetC introduces a method of semaphores where any global

int

data element can be used as a semaphore representing whatever the application desires. It is a cooperative inter-process memory protection model that trades off some security in return for significant performance. Lock and unlock methods are applicable to global data elements which provide an atomic test-and-set, storing a magic number in the data element itself. Should the lock be successful, the application can proceed to work on other data elements safely, knowing it is not in competition.

Unlike C, packetC is based upon the presumption of underlying capabilities being provided by the operating system or generated by the compiler for a target platform. These include the management of packet handling functionality including receipt, buffering, and transmission as well as an initial level of decoding and manipulation on transmission. In addition, parallel processing management and a base set of control plane functionality must be present for packetC applications to interact with. These functions may differ from one system to another; however, a base set of functionality must remain consistent. To provide a common interface, the language specification for packetC mandates a predefined base set of types that are passed as parameters to

main()

as well as a set of built-in methods and operators.

packetC provides data types that do not appear in standard C but do provide significant support for packet-processing applications. These data types are often extensions of familiar C types. The extended data type described in this chapter is descriptors. This chapter is divided into two different approaches. The first part of this chapter is focused on simply covering examples of descriptors and the packetC standard include file

protocols.ph

. The second part of this chapter covers an in-depth view into the background of the descriptors and how they operate under the hood as these are new to packetC.

At a conceptual level, packetC capabilities can be categorized as

data plane

capabilities, which involve examining, changing, and routing packets and

control plane

capabilities, which involve displaying system-level values and messages. In some systems, the two planes may be on separate boards or on distant components in a far-flung system. In others, such as a PC-based emulation system, the two sets of capabilities might execute on the same hardware. Thus, packetC specifies control plane capabilities in a general way, while allowing considerable latitude in how they are implemented

As you develop a large program, and often even in a small program, you’ll want to leverage more than just the current packet module source code file. As you desire to include text from other sources, such as packet descriptors pre-crafted and provided with your development environment, or a set of function prototypes for a library of interest, you are leaving packetC and delving into the pre-processor. As packetC is very similar to C, the use of the C pre-processor was possible without much change. The following section describes the available pre-processor features and some suggestions on effective usage in packetC applications.

Pragma clauses

are compiler directives, which have the general form shown below:

Developing large applications in packetC is quite similar to that of developing large applications in languages such as C++ from code organization and development team collaboration points of view. There are several grammar changes that impact standards that one would follow as well as new areas of concern regarding performance, security, and networking aspects not often present in other application domains. What becomes the single most important planning aspect of developing large applications in packetC is the planning itself.

Based upon the compiler and development environment that you are using with packetC, the tool-chain and your course of action to create a packetC application may vary. As of the time of writing, there is only one available tool-chain that supports the various vendor execution platforms. The following discussion provides some insight into the tool-chain and the methodology behind building a packetC application. Refer to the PacketWorks IDE User Guide for a complete walk-through of the development environment and the associated options available for installation, development, and debugging of an application.

This chapter highlights some of the key principles of descriptors through highlighting examples in the standard libraries. Many layer 2 through 4 descriptors are provided with packetC development environments while upper-layer protocols and custom packet techniques will require handcrafted descriptors tailored to an application. Descriptors may be as simple as the one for Ethernet II provided below:

In developing data-plane network applications for modern high-speed networks, there exists a trade-off between the limits of available processing power and the requirements for increased analytics within applications. With packetC, this battle rages on and must remain at the heart of any application design. Network applications must ensure low latency and minimization of processing in every algorithm possible in order to maximize the quantity of transactions that may be evaluated. Evaluating an application not only for valid logic but also for efficiency must become core to every packetC development project.

To get a sense of the impact standard libraries have had on C developers, try a little test. Ask a few C programmers to list ten C language commands. Use a generic word like commands that does not send them automatically down the path of operators, control statements, or some other specific mechanism in C. Some may get rolling quickly with for, if, while, and so on, but generally as the list nears eight the casual assessment will often skip operators like +, ++, *, or sizeof, and will run quickly into things such as

atoi

. The point is that it is hard for many modern C programmers to distinguish what is C and what are really fundamental features that are provided by standard libraries.

Increasingly, applications that process network packets, especially those that inspect packet

payload

content, need to run at speeds in the range of 1–40 Gigabits per second. These requirements encourage exploiting specialized hardware. Thus, typical approaches involve programming a specific network processor in assembly language or a C dialect that exposes machine-specific particulars so developers can closely control task switching and machinespecific resources. This paper describes CloudShield Technology’s alternative approach, which combines a parallel model, the packetC language and heterogeneous multiprocessor implementations. The parallel packet processing model uses coarse-grain, SPMD parallelism to free users from thread management and it requires the host system to locate

protocol headers

in the packet before a parallel copy of the program executes. The packetC language abstracts and encapsulates familiar packet processing data sets and operations into new aggregate data types and operators, e.g., for

packets

,

databases

and

searchsets

. These language constructs can be implemented by ordinary arrays, strings, instructions, etc. However, they are especially amenable to being implemented with specialized hardware without requiring any specific hardware. Our current implementation combines FPGAs, microcoded NPUs (Network Processing Units), content addressable memories and other specialized chips.

Network packet processing applications increasingly execute at speeds of 1–40 gigabits per second, often running on multi-core chips that contain multithreaded network processing units (NPUs) and a general-purpose processor core. Such applications are typically programmed in a language that exposes NPU specifics needed to optimize lowlevel thread control and resources. This facilitates optimization at the cost of increased software complexity and reduced portability. In contrast, our approach provides portability by combining coarse-grained parallelism with programming in the packetC® (CloudShield Technologies) language’s high-level constructs. This paper focuses on searching packet contents for

packet protocol headers

. We require the host system to locate headers for layers 2, 3 and 4, and to encode their offsets in a

packet information block

(PIB). packetC provides

descriptors

, C-style structures superimposed on the packet array at runtime-calculable, user or PIB-supplied offsets. We deliver state-of-the-practice performance via an FPGA for locating layer offsets and via micro-coded interpretation that treats PIB layer offsets as a special addressing mode.

A significant part of packet processing consists of detecting whether certain standard

protocol headers

are present, where they are located and whether they include optional information. Packet processing programs are on tight time budgets, especially to handle speeds in the gigabits per second (gbps) range. Thus, high-speed mechanisms for finding and accessing headers are critical. Our approach lets users define headers as C-style

structures

in a high-level language, packetC [1], and specify header locations in terms of offsets from the start of the current packet, which is treated as an array of unsigned bytes. These offsets can be expressed in terms of network layer offsets, constant values, runtime-calculated variables and combinations of all of these. This paper focuses on the principal forms these offset expressions can take and on how our FPGAs (Field Programmable Gate Arrays), compiler and interpreter collectively handle them at runtime. For simple and complex header offset scenarios we provide users with intuitive, high-level ways to describe offsets and provide effective runtime mechanisms to access header fields.

Network packet processing’s increasing speeds and volume create an incentive to use parallel processing. Such processing often involves comparing selected packet data to the contents of large tables (e.g., for routing packets or controlling system access). Thus, commercial systems often use multiple network processors [1] to provide parallel processing in general and use associative memory chips to provide parallel table operations in particular. Parallel network programming is usually done in a C dialect with machine-specific extensions. The associative memory capabilities are often provided by ternary content addressable memory (TCAM) chips in order to supply the fast, masking-based searches needed in this domain. TCAM use is normally controlled by vendor software, rather than by the application developer. Thus, an application is typically restricted to a small number of predefined templates and mediated by vendor system software. Thus, application developers cannot use high-level languages to express network table operations in an intuitive, portable way, nor exploit parallel devices like TCAMs in a flexible manner. This paper presents CloudShield’s packetC® language [2], a C dialect that hides most host-machine specifics, supports coarse-grain parallelism and supplies high-level data type and operator extensions for packet processing. We describe packetC’s

database

and

record

constructs that support network application table operations, including masked matching. We show how our implementation of packetC with network processors, FPGAs and TCAMs lets the user enjoy parallel performance benefits without the usual vendor constraints or reliance on hardware-specific programming.

Increasing speeds and volumes push network packet applications to use parallel processing to boost performance. Examining the packet

payload

(message content) is a key aspect of packet processing. Applications search payloads to find strings that match a pattern described by

regular expressions

(

regex

). Searching for multiple strings that may start anywhere in the payload is a major obstacle to performance. Commercial systems often employ multiple network processors to provide parallel processing in general and use regex software engines or special regex processors to speed up searching performance via parallelism. Typically, regex rules are prepared separately from the application program and compiled into a binary image to be read by a regex processor or software engine. Our approach integrates specifying search rules with specifying network application code written in packetC, a C dialect that hides host-machine specifics, supports coarse-grain parallelism and supplies high-level data type and operator extensions for packet processing. packetC provides a

searchset

data type, as well as

match

and

find

operations, to support payload searching. We show that our searchset operator implementation, using associative memory and regex processors, lets users enjoy the performance benefits of parallel regex technology without learning hardwarespecifics or using a separate regex toolchain’s use.

Network packet processing often involves having the results of a database match or packet content search select the next program operation. Our packetC[1] language provides specialized aggregate data types,

databases

[2] for masked matching and

searchsets

[3] for packet payload search terms. It is common to chain operations on these types, so that the result of one operation selects the specific aggregate to use for the next operation. packetC offers a reference data type to support this kind of chaining. A packetC

reference

provides an abstract way to refer to one of the extended data type aggregates in source code. At runtime the reference construct’s value will indicate a particular aggregate of the relevant type. Thus, this construct is a classic

reference

in the computer science sense of a mechanism for indirect variable access. Unlike a C++ reference our construct must be assignable to support chaining that is driven by runtime results. Unlike a C-style pointer our reference must have strong typing, since runtime recovery from an illegal reference value (indicating a nonexistent or inappropriate aggregate) is not practical in this unforgiving, high-speed domain. The paper illustrates practical applications and shows that compressed code from using references does not cause performance penalties.

Network packets place some

protocol

data in

bit fields

that are smaller than typical processor operand sizes. C language

structures

can represent such protocols but the uncertain layout and endian-specific nature of C’s bit fields cause problems. Research has ranged from alternative bit field constructs, through specialized bit registers to using analytic techniques to identify programs’ implicit

subword

usage. This paper describes the packetC language’s two-fold approach to handling protocols in a portable way. The language addresses bit field layout and operation uncertainties as language design matters that can be overcome with a

container

-oriented approach and unambiguous layout rules. It tackles the problems of endianness and of packet bit field processing by two means. On the language design level packetC imposes big endian byte allocation order for structure and packet array storage. Second, the language is built around a packet processing model that involves triggering a parallel copy of a program after the host system assembles the entire packet in a byte array, locates standard protocols within that packet and saves protocol location information. By providing both portable protocol representation and protocol

layer offset

calculation, packetC frees engineering resources to pursue other packet processing tasks.

C and C-based languages give implementations substantial freedom in choosing schemes for bit and byte allocation. The ensuing variability can make it difficult to port programs that depend on particular structure and bit-field layouts. With these problems in mind, packetC prescribes that implementations use big endian byte allocation order and little endian bit allocation order. This paper reviews the related issues of structure field, byte and bit allocation order and describes an algorithmic approach for compensating when the implementation executes on a processor that uses little endian byte allocation order.

The centrality of networked communications to effective military action is a truism of this age. Against this background heterogeneous parallel architectures have been successfully applied to cyber domains as diverse as

exascale

computing, multimedia applications and network processing. High-speed communications are a natural match for such architectures, since they exhibit a high degree of parallelism and are amenable to multiple kinds of specialized processing. This paper reports our novel approach to developing communications applications for heterogeneous processors. Key aspects of the approach include extending high-level languages with domain-specific data types and operators, encapsulating the extensions within a virtual machine and implementing the operators with specialized processors. Expressing the virtual machine in microcode provides an effective way to orchestrate communications with the disparate kinds of processors used - and to replace them with new kinds of processors. Benefits include speed to meet real-time demands, portable high-level language code and an extensible virtual machine.