How to scrape tables with BeautifulSoup?

We can parse a table's content with BeautifulSoup by finding all <tr> elements, and finding their <td> or <th> children.

Here is an example on how to parse this demo table using BeautifulSoup:

import requests
from bs4 import BeautifulSoup

response = requests.get("https://demo.scrapingbee.com/table_content.html")
soup = BeautifulSoup(response.content, 'html.parser')

data = []
table = soup.find('table')
table_body = table.find('tbody')

rows = table.find_all('tr')
for row in rows:
    cols = row.find_all(['td', 'th'])
    cols = [ele.text.strip() for ele in cols]
    data.append([ele for ele in cols if ele])
print(data)

 

Related BeautifulSoup web scraping questions: