OS X and iOS Kernel Programming

The role of an operating system is to provide an environment in which the user is able to run application software. The applications that users run rely on services provided by the operating system to perform tasks while they execute, in many cases without the user—or even the programmer—giving much thought to them. For an application to read a file from disk, for example, the programmer simply needs to call a function that the operating system provides. The operating system handles the specific steps required to perform that read. This frees the application programmer from having to worry about the differences between reading a file that resides on the computer’s internal hard disk or a file on an external USB flash drive; the operating system takes care of such matters.

Mac OS X is a modern Unix-based operating system developed by Apple Inc for their Macintosh computer series. OS X is the tenth incarnation of Mac OS.

Apple has a good track record of taking care of its developers and providing them with intuitive, user-friendly tools and APIs to develop for the Mac and iOS platforms. Anyone who has written application software for the Mac or iPhone will be familiar with the object-oriented Cocoa framework, which provides a rich set of interfaces to support graphical user interfaces and other services required by user applications. Likewise, kernel developers are provided with APIs that are designed to help with the tasks performed by a kernel extension. For driver development, Apple provides the I/O Kit, which is an object-oriented framework for interfacing with hardware. The following chapter discusses the tools and frameworks you will need to get started with kernel development and includes a tutorial for building and installing a simple kernel extension.

Device drivers for Mac OS X are written using a framework known as the I/O Kit. The I/O Kit consists of header files and libraries that provide the services required by drivers, as well as header files and libraries that are used by user space code to locate a kernel driver and interact with it. There are two main parts of the I/O Kit:

In the previous chapter, we learnt about I/O Kit drivers, which live in the kernel. On the other hand, the applications that users interact with live in user space. So, if the user is going to use the services provided by your driver, the kernel/user space boundary needs to be crossed.

Memory management in the kernel is significantly more complex than it is for a user space program. A user space program typically deals with a flat linear address space and can allocate memory in more or less arbitrary blocks without worrying about the source or arrangement of this memory. It has simple interfaces that typically take a size in bytes as an argument and deliver a yay or nay result, depending on the availability of the requested memory. At worst, the consequence of a failed allocation or misuse of memory is the termination of the offending process. However, things are not as straightforward in the kernel. The kernel has to deal with multiple memory spaces, including its own, as well as the mapping of memory between those memory spaces and physical memory. While user space programs deal with virtual memory, where the underlying physical arrangement is irrelevant, the kernel often needs to know whether the memory is contiguous and where it is located. This is because some hardware devices are unable to read from certain memory addresses or have specific requirements regarding the alignment of the memory, for example, because it can read only from memory that has been aligned to a 16-byte boundary or because it cannot read from addresses higher than 32-bit. However, the most obvious challenge of kernel memory management is to use as little as possible because it is a scarce resource, especially for embedded devices such as the iPhone or iPad. Incorrect use of memory in the kernel can lead to subtle and not so subtle consequences.

As we have seen throughout this book, the role of a driver is to make the functionality that is provided by a hardware device available to the operating system and to user applications. This means that the code inside a driver may be called from any number of running applications at any time, depending on when an application wishes to request the services of the hardware device. In handling these requests, the driver runs in the thread context of the application that made the control call. In addition to these requests, the hardware itself can require servicing and may generate interrupts at arbitrary times that the driver must respond to. The end result for the driver developer is that driver code runs in a complex multithreaded environment, even without the driver creating any additional threads of its own.

The Universal Serial Bus (USB) is a ubiquitous technology found in a wide variety of products, notably computer peripherals, including mice, keyboards, hard drives, and printers, as well as almost any other type of device or equipment that can be connected to a computer. The USB is a specification that defines the communication between a device, such as a printer or mobile phone, and a host controlled by a computer device, such as your Mac or iPad. The USB specification was developed in 1996 by a consortium of companies, including Compaq, DEC, IBM, Intel, Microsoft, NEC, and Nortel. The motivation was to replace a series of connectors with a universal connector, making it easier to connect external devices to personal computers. The USB specification is currently at version 3.0. Support for 3.0 is still emerging, and support for version 2.0 is by far the most ubiquitous at this time. Apple has yet to release hardware capable of supporting the latest USB 3.0 specification, but Apple computers have shipped with USB support since before OS X. The iOS series of devices are themselves USB devices, but they can also act as USB hosts. An example of this is the iPad, which can act as a host for USB devices, such as digital cameras.

PCI (Peripheral Component Interconnect) is a high-speed bus developed by Intel, in the early nineties, to replace various older and slower bus technologies such as EISA, ISA, MCA, and VESA. The term PCI is often used to describe the family of technologies based on the original PCI specification. Throughout this chapter, when we refer to PCI, we refer to commonalities found in the PCI-based technologies; namely, PCI Express, Thunderbolt, and to a lesser extent ExpressCard. Most people associate PCI with expansion boards plugged into a computer, but it is worth noting that PCI is fundamental to many computer systems—even those without PCI slots, such as iMacs— that have internal PCI buses that connect the CPU to USB, Firewire, and SATA controllers. Recent PCI-based advancements (like Thunderbolt) allow the PCI bus to be extended outside of the computer, much in the same way as USB and Firewire.

Power management has become a fundamental feature across all computing devices. Every platform that runs Mac OS X can be put into a lower power mode, and so power management is just as important for a desktop computer that is always connected to a power supply as it is for a laptop or an iPhone that is running from a battery.

A serial port provides a basic communications interface for the purpose of getting low bandwidth data into and out of a computer. Although modern interfaces such as USB and FireWire have replaced many applications in which the traditional serial port was once used, the serial port driver is still well supported in modern day operating systems, including Mac OS X, despite the fact that Apple has not released hardware with a built-in serial port for over a decade.

Audio devices are among the most common peripherals attached to a computer apart from storage devices. They are used for everything from voice recorders to MP3 players, headsets with microphones, security systems, and DJ and professional recording systems. Many video devices also have audio capabilities and come with their own audio drivers that allow you to use the audio features of the device independently or together with the video features, for example, a web camera with a built-in microphone. The microphone will have its own audio driver, representing the microphone in the system as an audio device that can be used independently of the camera.

Network support in the kernel is implemented primarily in the BSD layer. The BSD flavors of UNIX are renowned for their robust and secure networking support. Consequently, code from the BSD networking stack has made its way into a wide variety of operating systems, including Mac OS X and iOS. While the networking support is primarily in the BSD layer, it has hooks into I/O Kit, which provides the interface for building hardware-based network drivers. A conceptual view of the kernel network architecture is shown in Figure 13-1.

Storage devices encompass many types of devices, including hard disk drives, CDs and DVDs, USB flash drives, FireWire-based hard disks, and a file-based disk image that has been mounted as a virtual drive. For the user, a storage device appears as a volume on their desktop that they can read files from and write files to, but what the user doesn’t see is the multiple drivers that work together in the kernel to make this possible.

From a user’s perspective, an application that requires a kernel driver detracts from the user experience. To begin with, driver installation involves writing to the “Extensions” directory, which requires administrative privileges. Therefore, the user needs to run an installer and enter the password of an administrative account, and then possibly restart before they can begin using the application. If, on the other hand, the application doesn’t require a kernel driver, the installation procedure can be as simple as downloading an application from the Mac App Store.

Debugging is part of the development process and the ongoing maintenance of a kernel level driver or extension. Therefore, having the skills and knowledge to debug the kernel effectively is an important part of a kernel engineer’s job description. Although great care is taken during the development and quality assurance process, bugs are often unavoidable. This is partly because, once released, your driver is likely to run against hardware/software combinations that haven’t been as well tested, if at all. For example, your driver may run on a faster or slower CPU than was tested initially, thus uncovering timing issues.

This chapter covers miscellaneous topics that are of interest to more advanced kernel programmers. We’ll discuss how Streaming SIMD Extensions (SSE) and floating point can be used in the kernel. (SIMD is short for Single Input Multiple Data.) We will also examine strategies for dealing with drivers for multifunction devices, and discuss the implementation of I/O Kit families. We’ll cover the kernel control KPI that can be used for user space communication with KEXTs such as Network Kernel Extensions (NKE) that does not use the I/O Kit. We also show how to work with and manipulate processes from the kernel, such as getting the process identifier (PID) of a process and sending signals to the process. Some drivers may need additional resources loaded from the file system, such as firmware images. This chapter provides a discussion of how these resources can be loaded using the OSKextRequestResource() function. The chapter concludes with a discussion of how a driver can send messages to a user space daemon using notifications.

Thus far, we have provided background information and looked at the practical implementation of several types of drivers and kernel extensions. This chapter focuses on how we prepare our work for end-user delivery. Apple is known for providing user-friendly software (and hardware) solutions; both Apple and its customers have come to expect the same level of customer experience from third-party vendors. Frustrating customers or users with complex installation procedures is a good way to lose business to competitors on any platform. Deploying a piece of software like a kernel extension may seem easy at first, but there are a multitude of issues to consider, such as how to accommodate a wide range of different hardware and operating system versions. Many customers may be reluctant to upgrade. This is especially true for larger business or government installations— so, you may be required to support bleeding edge, as well as legacy operating system versions— all of which may have different features that require special handling. Besides the external factors, your software’s distribution may be complicated. Rarely will you distribute only the KEXT itself; it often requires additional bundled software. For example, a computer graphics card may be delivered with a system preferences pane, a framework used to access the device’s special features, applications for upgrading firmware, and perhaps bundled applications that show off the card’s capabilities like games. You will also need to handle the possibility that a customer will upgrade or downgrade your software distribution.