Skip to main content
main-content

Über dieses Buch

Provide evidence-based answers that can be measured and relied upon by your business. Database administrators will be able to make sound architectural decisions in a fast-changing landscape of virtualized servers and container-based solutions based on the empirical method presented in this book for answering “what if” questions about database performance.
Today’s database administrators face numerous questions such as: What if we consolidate databases using multitenant features?
What if we virtualize database servers as Docker containers?
What if we deploy the latest in NVMe flash disks to speed up IO access?
Do features such as compression, partitioning, and in-memory OLTP earn back their price?
What if we move our databases to the cloud?
As an administrator, do you know the answers or even how to test the assumptions?
Database Benchmarking and Stress Testing introduces you to database benchmarking using industry-standard test suites such as the TCP series of benchmarks, which are the same benchmarks that vendors rely upon. You’ll learn to run these industry-standard benchmarks and collect results to use in answering questions about the performance impact of architectural changes, technology changes, and even down to the brand of database software. You’ll learn to measure performance and predict the specific impact of changes to your environment. You’ll know the limitations of the benchmarks and the crucial difference between benchmarking and workload capture/reply.

This book teaches you how to create empirical evidence in support of business and technology decisions. It’s about not guessing when you should be measuring. Empirical testing is scientific testing that delivers measurable results. Begin with a hypothesis about the impact of a possible architecture or technology change. Then run the appropriate benchmarks to gather data and predict whether the change you’re exploring will be beneficial, and by what order of magnitude. Stop guessing. Start measuring. Let Database Benchmarking and Stress Testing show the way.

What You'll LearnUnderstand the industry-standard database benchmarks, and when each is best used
Prepare for a database benchmarking effort so reliable results can be achieved
Perform database benchmarking for consolidation, virtualization, and cloud projects
Recognize and avoid common mistakes in benchmarking database performanceMeasure and interpret results in a rational, concise manner for reliable comparisonsChoose and provide advice on benchmarking tools based on their pros and cons

Who This Book Is For

Database administrators and professionals responsible for advising on architectural decisions such as whether to use cloud-based services, whether to consolidate and containerize, and who must make recommendations on storage or any other technology that impacts database performance

Inhaltsverzeichnis

Frontmatter

Chapter 1. Benchmarking Basics

This book intends to enlighten readers on all things related to database benchmarking, stress testing, and workload capture/replay. At this moment you may feel that these are one and the same topic, or simply that what differentiates them is inconsequential. Moreover you may consider these topics as somewhat esoteric and thus more suited for academia and theoreticians, but not really relevant in the real world where you work. Finally, you might simply believe that the time and effort for performing such testing activities is unwarranted, or at least very difficult to justify. As a result, many people are skeptical about the true value of these activities. This book endeavors to transform that mistaken opinion or at least enlighten the reader as to the genuine value of these topics.
Bert Scalzo

Chapter 2. Industry Standard Benchmarks

In Chapter 1 we differentiated between a database benchmark, database stress test, and workload capture/replay. By understanding these differing approaches, we formed the foundation for a common understanding and vocabulary. Plus we learned why these different techniques are useful and when to best utilize them. To recap, our basic definitions were:
Bert Scalzo

Chapter 3. Benchmarking Tools

In Chapter 1 we differentiated between a database benchmark, database stress test, and workload capture/replay. Then in Chapter 2, we covered all the relevant database benchmarks, both current and past (i.e., obsolete). So much like a teen learning to drive a car, you’ve now studied the rules of the road and passed the written test to obtain a learner’s permit. Now it’s time to exercise that knowledge and to do some actual driving (long before ever taking the final road test). For that, the anxious and excited teen would need the use of a car and a legal driver as a passenger (note that there are other limitations such as no other teens and no driving at night). So similarly for database benchmarking, you are going to need some tools and to do some test driving with limits until you get sufficient experience before doing it for real. Thus, here in Chapter 3, we’ll cover some of the database benchmarking tools that are available and how to best use them. Furthermore, we’ll cover some other tools that you may consider a benchmarking tool but which better fit under the category of database stress test or workload capture/replay tools.
Bert Scalzo

Chapter 4. Benchmarking Preparation

At this point you may believe yourself fully ready to start your database benchmarking efforts, armed with just the information provided thus far: the database benchmarking basics, an overview of the various industry-standard database benchmarks, and an elucidation of the IO subsystem and database benchmarking tools available. Therefore, some of you might feel like this chapter is merely filler or fluff. But I’m going to impress that this chapter is, in fact, the most critical one in the entire book, and should be read with the most attention. In fact, I will further state that failure to prepare as described in this chapter is arguably the single largest contributor to the next chapter: missteps and failure.
Bert Scalzo

Chapter 5. Benchmarking Mistakes

As the prior chapter stressed, sufficient preparation or lack thereof can make or break a database benchmarking effort. For example, not just anyone has all the requisite background and skills to successfully run a database benchmark. They must know a lot about databases, especially proper configurations, performance monitoring, and tuning techniques. Likewise you need to have scheduled ample time and other resources for the effort. But assuming you’ve done all that, there are still many things that can go wrong during a database benchmarking effort. This chapter will highlight some of the major and most common missteps that plague many database benchmarking projects once they are underway.
Bert Scalzo

Chapter 6. Benchmarking Hardware Options

The first five chapters have educated and prepared you for performing database benchmarking. Now it’s time to leverage that knowledge. This chapter focuses on issue if you’re working on physical servers or “raw iron, and not working with virtualization or the cloud. While both of those paradigms are both new and hot, nonetheless there still remains the need on occasion to work with on-premise database servers. Your company may not be quite ready for database deployments on a hypervisor or the cloud, plus you may need to benchmark on premises as a baseline in order to compare to virtualization or the cloud. There are many other reasons as well, so let’s now look at what one should do when benchmarking on premise.
Bert Scalzo

Chapter 7. Benchmarking Software Options

The last chapter covered some hardware options important to review and consider changing when performing database benchmarking projects. While hardware advances the past decade have been nothing short of amazing (e.g., I just built a new desktop PC where the CPU offers 16 logical cores or threads), we cannot and should not always automatically look to hardware for solutions to performance issues. In reality, most benchmarking efforts can see the largest improvements to performance scores by leveraging appropriate new database features. The database vendors have likewise made significant improvements, often in response to leverage many new hardware technologies. This chapter will focus on both normal database features to consider as well as the newer ones that leverage these new hardware technologies.
Bert Scalzo

Chapter 8. Benchmarking for Consolidation

In this chapter we’re going to review benchmarking considerations for when you are testing scenarios in order to consolidate database instances. Database consolidation occurs when you attempt to reduce the total number of database licenses in order to reduce costs. With the rising cost of database licenses and the business considering a database as just a commodity, building-block object required for an application, your management might be required to undertake consolidation efforts. Moreover with today’s servers being so powerful, with multi-core CPUs and lots of cheap memory, we no longer must intentionally segregate databases using the traditional “silo” approach of one database per server. DBAs can now realistically look to co-locate all the data from multiple databases instances under a single database instance. Sometimes such efforts may coincide with a major database version upgrade or a licensing “true up” effort. One thing is for sure, with all the “database sprawl” that has occurred over the past 20 years, we all have more database instances than anyone could have reasonably imagined. So database consolidation efforts are more common than you might think.
Bert Scalzo

Chapter 9. Benchmarking for Virtualization

In this chapter we’re going to review benchmarking considerations for when you are benchmarking scenarios in order to virtualize database instances and their databases. In many respects the prior chapter on database consolidation is a prerequisite for this chapter. That chapter’s benchmarking efforts to identify which databases can share an instance in order to reduce the instance count, and then how many servers are required to host those instances are much the same. As such, we’ll therefore consider the prior chapter the first past part of the virtualization effort. This chapter will then deal with the second part: are their issues pertinent and specific virtualization that can affect database benchmarking efforts and their results.
Bert Scalzo

Chapter 10. Benchmarking for Public Cloud

In this chapter we’re going to review benchmarking considerations for when you’re considering or moving database instances and their databases to the public cloud. This chapter is purposefully kept short in order to focus on just the main issues, because the cloud has so many variables plus it is evolving every day. A whole book covering just the topic of deploying databases in the cloud would fill a volume all by itself. So here we must concentrate on the big-ticket items that are of the most value.
Bert Scalzo

Chapter 11. Workload Capture and Replay

In this chapter we're going to review database workload capture and replay. This approach to stress testing or database benchmarking has some fairly obvious advantages. First, it's your application and database from which the transactional activity is derived. Therefore it has a direct correlation and relevance to your actual performance situation and issues at hand. Second, you already are intimately familiar with both the database design and the general business requirements being handled. So there are no specs to read and learn. Third, it's easier to research and propose potential solutions due to this familiarity. Plus it's easy to bounce ideas off of peers who also have such familiarity. For reasons such as these (and several others), some people might well consider this the ideal approach to stress testing or database benchmarking. It's hard to argue against such logic. I see their point. While Chapter 8 may have shown a way to create a mixture of industry-standard benchmarks in an attempt to approximate any system of interest, it nonetheless remains an approximation at best. We could easily forget some aspect that could radically skew the achievable results. You simply cannot be more accurate than running the actual workload.
Bert Scalzo

Chapter 12. Database Stress Testing

In this final chapter, we’re going to discuss simple database stress testing, plus present a free scripting tool that I’ve written for people to utilize or alter to fit their specific needs. A stress test is much like an industry-standard benchmark in that it attempts to press the database. It’s quite different as it applies simple, atomic transactions to merely stress the underlying subsystems, especially the IO bandwidth. I think of stress testing as being more like “gunning the engine” of an automobile, where the car is in neutral and you just want to see how high the RPM’s can go before redlining. That does not mean that such testing is any less valuable, just that it basically serves a different testing purpose. For many people this is sufficient for what they might call database benchmarking.
Bert Scalzo

Backmatter

Weitere Informationen

Premium Partner

BranchenIndex Online

Die B2B-Firmensuche für Industrie und Wirtschaft: Kostenfrei in Firmenprofilen nach Lieferanten, Herstellern, Dienstleistern und Händlern recherchieren.

Whitepaper

- ANZEIGE -

Best Practices für die Mitarbeiter-Partizipation in der Produktentwicklung

Unternehmen haben das Innovationspotenzial der eigenen Mitarbeiter auch außerhalb der F&E-Abteilung erkannt. Viele Initiativen zur Partizipation scheitern in der Praxis jedoch häufig. Lesen Sie hier  - basierend auf einer qualitativ-explorativen Expertenstudie - mehr über die wesentlichen Problemfelder der mitarbeiterzentrierten Produktentwicklung und profitieren Sie von konkreten Handlungsempfehlungen aus der Praxis.
Jetzt gratis downloaden!

Bildnachweise