Overview of Web Scraping with Python’s BeautifulSoup and requests library

pip install beautifulsoup4
pip install requests
from bs4 import BeautifulSoup
import requests
url = requests.get('http://quotes.toscrape.com/')
url_text = url.text
soup = BeautifulSoup(url_text, 'lxml')
web page that we are going to scrape
for quote in soup.find_all('span', class_='text'):
print(quote.text)
from bs4 import BeautifulSoup
import requests
url = requests.get('http://quotes.toscrape.com/')
url_text = url.text
soup = BeautifulSoup(url_text, 'lxml')for quote in soup.find_all('span', class_='text'):
print(quote.text)
“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”
“It is our choices, Harry, that show what we truly are, far more than our abilities.”
“There are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.”
“The person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.”
“Imperfection is beauty, madness is genius and it's better to be absolutely ridiculous than absolutely boring.”
“Try not to become a man of success. Rather become a man of value.”
“It is better to be hated for what you are than to be loved for what you are not.”
“I have not failed. I've just found 10,000 ways that won't work.”
“A woman is like a tea bag; you never know how strong it is until it's in hot water.”
“A day without sunshine is like, you know, night.”

--

--

--

pretending to teach is one of the best way to learn.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

How to Keep Windows From Sleeping

Announcing AWS SQS Connector for Apache Pulsar

DevOps for SAP Integration Suite (CPI) with ReleaseOwl

Head First Java — Chapter 01 (Breaking the Surface)

Solana Blockchain data scraping

FastAI TfmdLists

Make the Gameobjects Move in Timeline- Game Dev Series 64

Announcing Nrwl Nx 1.0

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Shounak Lohokare

Shounak Lohokare

pretending to teach is one of the best way to learn.

More from Medium

Get channel name from YouTube link using Python

Selenium basics to automate whatever you want

Which Is The Best Method To Extract The Data — Web Scraping Vs. API?

Amazing Image Processing in Python — In a Simple Web Application [Part-2]