Python Web Scraping

Description

Successfully scrape data from any website with the power of Python 3.x About This Book A hands-on guide to web scraping using Python with solutions to real-world problems Create a number of different web scrapers in Python to extract information This book includes practical examples on using the popular and well-maintained libraries in Python for your web scraping needs Who This Book Is For This book is aimed at developers who want to use web scraping for legitimate purposes. Prior programming experience with Python would be useful but not essential. Anyone with general knowledge of programming languages should be able to pick up the book and understand the principals involved. What You Will Learn Extract data from web pages with simple Python programming Build a concurrent crawler to process web pages in parallel Follow links to crawl a website Extract features from the HTML Cache downloaded HTML for reuse Compare concurrent models to determine the fastest crawler Find out how to parse JavaScript-dependent websites Interact with forms and sessions In Detail The Internet contains the most useful set of data ever assembled, most of which is publicly accessible for free. However, this data is not easily usable. It is embedded within the structure and style of websites and needs to be carefully extracted. Web scraping is becoming increasingly useful as a means to gather and make sense of the wealth of information available online. This book is the ultimate guide to using the latest features of Python 3.x to scrape data from websites. In the early chapters, you'll see how to extract data from static web pages. You'll learn to use caching with databases and files to save time and manage the load on servers. After covering the basics, you'll get hands-on practice building a more sophisticated crawler using browsers, crawlers, and concurrent scrapers. You'll determine when and how to scrape data from a JavaScript-dependent website using PyQt and Selenium. You'll get a better understanding of how to submit forms on complex websites protected by CAPTCHA. You'll find out how to automate these actions with Python packages such as mechanize. You'll also learn how to create class-based scrapers with Scrapy libraries and implement your learning on real websites. By the end of the book, you will have explored testing websites with scrapers, remote scraping, best practices, working with images, and many other relevant topics. Style and approach This hands-on guide is full of real-life examples and solutions starting simple and then progressively becoming more complex. Each chapter in this book introduces a problem and then provides one or more possible solutions.

Get the book

Amazon.com

Search the book

eBay.com

Search the book

Similar books

• 

  Family History Digital Libraries

  By William Sims Bainbridge

  One named Sara and Timberlake had 11 male workers, 1 female worker, and 4 children workers, so it might have employed the Minor family.

• 

  Foundation Dreamweaver MX

  By Craig Grannell, Jerome Turner, Matt Stephens

  So here's what we need to do to arrive at our layout: s Create the main table to hold all the page elements. s Deal with the navigation area which is ...

• 

  Cisco CCNA Certification, 2 Volume Set: Exam 200-301

  By Todd Lammle

  This inclusive, two-book set provides what you need to know to succeed on the new CCNA exam. The set includes Understanding Cisco Networking Technologies: Volume 1 and the CCNA Certification Study Guide: Volume 2.

• 

  CompTIA Network+ Study Guide: Exam N10-006

  By Todd Lammle

  ... you can use: –a –A –c –n –r –R –S –s All nbtstat switches are case sensitive. Generally speaking, lowercase switches deal with NetBIOS names of hosts, ...

• 

  CompTIA Network+ Study Guide with Online Labs: N10-007 Exam

  By Todd Lammle, Jon Buhagiar

  ... you can use: –a –A –c –n –r –R –S –s All nbtstat switches are case sensitive. Generally speaking, lowercase switches deal with NetBIOS names of hosts, ...

• 

  CCNA: Cisco Certified Network Associate FastPass

  By Todd Lammle

  S The S reference point defines the point between the customer router and an ... with the letter E deal with using ISDN on the existing telephone network.

• 

  Stranger in the Chat Room

  By Todd Hafer, Jedd Hafer

  A sequel to In the Chat Room With God finds a group of teens contacted by a mysterious and increasingly malevolent character who claims to know about their encounters with the Almighty and challenges their beliefs. Original.

• 

  Error Correction Coding: Mathematical Methods and Algorithms

  By Todd K. Moon

  M M−1∑ k=0 −∞ ∞ k=0 The average energy per signal E s ∫ can be related to the ... we will deal primarily with additive white Gaussian noise (AWGN), ...

• 

  Security+ Training Guide

  By Todd King

  ... to deal with most , but unfortunately not all , of these potential threats . ... The S / MIME standard implements encryption for message content using ...

• 

  CCDA: Cisco Certified Design Associate Study Guide: Exam 640-861

  By Todd Lammle, Andy Barkl

  S reference point The S reference point defines the reference point between ... with the letter E deal with using ISDN on the existing telephone network.