main-content

We are visual animals. But before we can see the world in its true splendor, our brains, just like our computers, have to sort and organize raw data, and then transform that data to produce new images of the world. Beginning Python Visualization: Crafting Visual Transformation Scripts, Second Edition discusses turning many types of data sources, big and small, into useful visual data. And, you will learn Python as part of the bargain.

In this second edition you’ll learn about Spyder, which is a Python IDE with MATLAB® -like features. Here and throughout the book, you’ll get detailed exposure to the growing IPython project for interactive visualization. In addition, you'll learn about the changes in NumPy and Scipy that have occurred since the first edition. Along the way, you'll get many pointers and a few visual examples.

As part of this update, you’ll learn about matplotlib in detail; this includes creating 3D graphs and using the basemap package that allows you to render geographical maps. Finally, you'll learn about image processing, annotating, and filtering, as well as how to make movies using Python. This includes learning how to edit/open video files and how to create your own movie, all with Python scripts.

Today's big data and computational scientists, financial analysts/engineers and web developers – like you - will find this updated book very relevant.

### Chapter 1. Navigating the World of Data Visualization

A Case Study
Abstract
As an engineer, I work with data all the time. I parse log files, analyze data, estimate values, and compare the results with theory. Things don’t always add up. So I double-check my analysis, perform more calculations, or run simulations to understand the results better. I refer to previous work because the ideas are similar or sometimes because they’re dissimilar. I look at the graphs and realize I’m missing some crucial information. So I add the missing data, but it’s noisy and needs filtering. Eventually, I realize my implementation of the algorithm is poor or that there is a better algorithm with better results, and so it’s back to square one. It’s an iterative process: tweak, test, and tweak again until I’m satisfied with the results.
Shai Vaingast

### Chapter 2. The Environment

Abstract
Chapter 1 demonstrated Python at work in a case study involving the collection, analysis, and visualization of GPS data. To put Python to work yourself, you first need to build your own development environment. This chapter will walk you through the various software components you need and help you weigh your installation options. Unless you’re already familiar with Python and have the packages we used in Chapter 1 installed, read on.
Shai Vaingast

### Chapter 3. Python for Programmers

Abstract
Python is a very readable language. Assuming you’ve had some previous experience in programming, you should be able to read and understand the code presented in the book without much trouble. That said, this book would be incomplete without some coverage of the Python programming language.
Shai Vaingast

### Chapter 4. Data Organization

Abstract
A preliminary step to designing and programming an algorithm is gathering data and sorting it. When you first test a thesis or write code to analyze network traffic, only part of the information is readily available; some of the data is still unknown. You make initial estimations based on the first set of data files. As data is gathered, you discover new insights and understandings, resulting in possible changes to the processing script and data gathering application, such as adding a previously unlogged parameter and graphing it over time. Some changes may include data gathering over substantially longer time periods than originally anticipated. Consequently, to accommodate for manageable data files, you reduce the sampling rate, implemented by logging only every nth value. Another plausible scenario is that of parsing log files, where the generating application (e.g., a web server) recently went through a software upgrade that altered the file format and the file name scheme.
Shai Vaingast

### Chapter 5. Processing Text Files

Abstract
A considerable amount of the data we process is text-based. Viewed from a simplistic approach, text files are files that contain characters. The Python scripts we write are text files. The HTML files our web browser receives are text files. The e-mail messages we read are text files. They’re simply everywhere. Because of the abundance of text files, you’re likely to analyze data that comes in some form of a text file.
Shai Vaingast

### Chapter 6. Graphs and Plots

Abstract
Graphs and plots are efficient methods to present data. Done properly, a graph can convey an idea better than an entire article.
Shai Vaingast

### Chapter 7. Math Games

Abstract
Math is a fundamental tool in data visualization. Python provides outstanding math support, which makes it an ideal development environment for performing analysis prior to visualization. Python’s interactive nature makes manipulating data and observing intermediate results easy. Python also makes it easy to modify results and quickly plot them. Another reason I like using Python for data visualization: there are a wide range of popular, freely available, mature numerical packages. Lastly, Python is structured, allowing the development of production-level code for generating quality reports.
Shai Vaingast

### Chapter 8. Science and Visualization

Abstract
I’ve covered many topics associated with data analysis and visualization: reading and writing files, text processing and converting text to numerical data, plotting and graphing, writing scripts, and implementing algorithms. It’s time to take a deeper dive and analyze numerical data.
Shai Vaingast

### Chapter 9. Image Processing

Abstract
Up to this point we’ve mostly dealt with one-dimensional data. That is, we’ve covered graphs and data that are essentially composed of a series of values. We’ve plotted the data, analyzed it, and created an image that was later saved to file or displayed to screen.
Shai Vaingast

### Chapter 10. Advanced File Processing

Abstract
A common task of programmers is to work with files. Programmers do more than read and write files; they also organize them, move them around, delete them, compress them, archive them, and more. To accomplish these tasks, I often find myself borrowing code from my previous projects, especially code that deals with reading and parsing files, typically via copy and paste. But that seems such a waste—why not come up with a library of functions that addresses these common needs?
Shai Vaingast