Overview of Web Scraping with Python’s BeautifulSoup and requests library

pip install beautifulsoup4
pip install requests
from bs4 import BeautifulSoup
import requests
url = requests.get('http://quotes.toscrape.com/')
url_text = url.text
soup = BeautifulSoup(url_text, 'lxml')
web page that we are going to scrape
for quote in soup.find_all('span', class_='text'):
print(quote.text)
from bs4 import BeautifulSoup
import requests
url = requests.get('http://quotes.toscrape.com/')
url_text = url.text
soup = BeautifulSoup(url_text, 'lxml')for quote in soup.find_all('span', class_='text'):
print(quote.text)
“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”
“It is our choices, Harry, that show what we truly are, far more than our abilities.”
“There are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.”
“The person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.”
“Imperfection is beauty, madness is genius and it's better to be absolutely ridiculous than absolutely boring.”
“Try not to become a man of success. Rather become a man of value.”
“It is better to be hated for what you are than to be loved for what you are not.”
“I have not failed. I've just found 10,000 ways that won't work.”
“A woman is like a tea bag; you never know how strong it is until it's in hot water.”
“A day without sunshine is like, you know, night.”

--

--

--

pretending to teach is one of the best way to learn.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Comparing API Gateway Performances: NGINX vs. ZUUL vs. Spring Cloud Gateway vs. Linkerd

Find the Height of the Tree, Find the Diameter of the Tree

13 Big Web Development Trends to Rule 2021

CQRS — another “buzzword” or game changer?

Batch Renaming Files Using Go

GCP Task-1

Mobile App Development Tips for Your Business

CS373 Fall 2021: Sungwook Kim (Week of 25 Oct — 31 Oct)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Shounak Lohokare

Shounak Lohokare

pretending to teach is one of the best way to learn.

More from Medium

Web Scraping ebay with selenium and export data to csv

Python In Business

Scraping User Information On GitHub Using PyGithub

How To Scrape Amazon Customer Review Data Using Python?