Skip to main content

2018 | Buch

Website Scraping with Python

Using BeautifulSoup and Scrapy

insite
SUCHEN

Über dieses Buch

Closely examine website scraping and data processing: the technique of extracting data from websites in a format suitable for further analysis. You'll review which tools to use, and compare their features and efficiency. Focusing on BeautifulSoup4 and Scrapy, this concise, focused book highlights common problems and suggests solutions that readers can implement on their own.
Website Scraping with Python starts by introducing and installing the scraping tools and explaining the features of the full application that readers will build throughout the book. You'll see how to use BeautifulSoup4 and Scrapy individually or together to achieve the desired results. Because many sites use JavaScript, you'll also employ Selenium with a browser emulator to render these sites and make them ready for scraping.
By the end of this book, you'll have a complete scraping application to use and rewrite to suit your needs. As a bonus, the author shows you options of how to deploy your spiders into the Cloud to leverage your computer from long-running scraping tasks.
What You'll LearnInstall and implement scraping tools individually and together
Run spiders to crawl websites for data from the cloud
Work with emulators and drivers to extract data from scripted sites

Who This Book Is For
Readers with some previous Python and software development experience, and an interest in website scraping.

Inhaltsverzeichnis

Frontmatter
Chapter 1. Getting Started
Abstract
Instead of installation instructions, which follow later for each library, we will dive right into deep water: this chapter introduces website scraping in general and the requirements we will implement throughout this book.
Gábor László Hajba
Chapter 2. Enter the Requirements
Abstract
After the introductory chapter, it is time to get you started with a real scraping project.
Gábor László Hajba
Chapter 3. Using Beautiful Soup
Abstract
In this chapter, you will learn how to use Beautiful Soup, a lightweight Python library, to extract and navigate HTML content easily and forget overly complex regular expressions and text parsing.
Gábor László Hajba
Chapter 4. Using Scrapy
Abstract
After a lengthy introduction to Beautiful Soup and custom scrapers, it’s time to look at Scrapy: the website scraping tool for Python.
Gábor László Hajba
Chapter 5. Handling JavaScript
Abstract
This chapter is all about handling websites that utilize JavaScript to render information dynamically.
Gábor László Hajba
Chapter 6. Website Scraping in the Cloud
Abstract
Running website scraping locally is fine for do-once tasks and small amounts of data, where you can easily trigger the crawl manually.
Gábor László Hajba
Backmatter
Metadaten
Titel
Website Scraping with Python
verfasst von
Gábor László Hajba
Copyright-Jahr
2018
Verlag
Apress
Electronic ISBN
978-1-4842-3925-4
Print ISBN
978-1-4842-3924-7
DOI
https://doi.org/10.1007/978-1-4842-3925-4