How to find elements by XPath in Selenium?

You can find elements by XPath selectors in Selenium by utilizing the find_element and find_elements methods and the By.XPATH argument.

find_element returns the first occurence of the XPath selector being used, while find_elements returns all elements of the page that match the selector. And By.XPATH simply tells Selenium to use the XPATH selector matching method.

Tip: // in XPath matches an element wherever it is on the page whereas / matches a direct child element.

Here is some example code which navigates to ScrapingBee website and extracts some data using XPath:

from selenium import webdriver
from selenium.webdriver.common.by import By

DRIVER_PATH = '/path/to/chromedriver'
driver = webdriver.Chrome(executable_path=DRIVER_PATH)

# Open Scrapingbee's website
driver.get("http://www.scrapingbee.com")

# Match all img tags on the page
all_img = driver.find_elements(By.XPATH, "//img")

# Match all a tags that contain the class of btn
all_btn = driver.find_elements(By.XPATH, "//a[contains(@class, 'btn')]")

# Match the first h1 tag on the page
first_h1 = driver.find_element(By.XPATH, "//h1")

# Get text of h1 tag
first_h1_text = first_h1.text

# Get count of all_img and all_btn
all_img_count = len(all_img)
all_btn_count = len(all_btn)

print(first_h1_text)
print(all_img_count)
print(all_btn_count)

Related Selenium web scraping questions: