Playwright vs Selenium: Which is the best Headless Browser

For years Selenium has reigned as the undisputed champion of web automation, dominating the ring with its vast capabilities and developer loyalty. But now a formidable rival has risen, Playwright. This battle of the titans is set to determine which tool truly deserves the crown of web automation champion. Each contender brings its own unique strengths and strategies to the arena, but which will emerge victorious in the fight for web automation supremacy?

In this article, we dive deep into this epic showdown, comparing the strengths, weaknesses, and hidden advantages of both Selenium and Playwright. Which one will prove to be the ultimate tool for your automation needs?

What Is Playwright and Selenium?

Let's first learn the official description of Playwright and Selenium.

Playwright is a relatively new open-source automation library introduced by Microsoft at the start of 2020. By design, it enables testing across all modern web browsers that rely heavily on JavaScript. Additionally, it supports most modern web browsers, such as Chromium, Firefox, and WebKit, through a single API.

On the other hand, Selenium is an established open-source automation framework that has existed since the early 2000s. It offers support for automation across various browsers and platforms. Selenium WebDriver, a component of the Selenium Project, directly interacts with the web browser and allows for complex end-to-end testing.

Although these tools have overlapping features and capabilities, they also have unique strengths and limitations that can help us choose the right tool for our web automation needs.

Advantage of using Playwright over Selenium

Let's now see what the advantages of using Playwright are.

First, as the new kid on the block, Playwright has a more modern API that integrates well with modern browsers. In fact, Playwright provides a unified API for different browsers, making it easier to write code that can run across multiple browsers without changing the code. On the other hand, Selenium requires us to install drivers for each browser we want to interact with.

Second, Playwright runs faster and more efficiently than Selenium by design. Also, it uses a much simpler syntax. Unlike Selenium, whose API can be verbose, Playwright reduces boilerplate code to a minimum, which can help us set up automation tests more quickly. Additionally, it supports asynchronous testing out of the box and simplifies handling complex scenarios involving JavaScript-based applications even further. Selenium can also run asynchronous tests, but again, we are required to install additional tools or frameworks like AsyncWebDriver.

Third, tests written with Playwright can run in parallel across different browsers. Parallel testing can significantly reduce the overall test execution time and allow us to create fast, continuous integration pipelines where fast feedback is crucial. In contrast, Selenium can be laggy and flaky as it has an older architecture that can sometimes struggle with modern web applications.

Advantage of using Selenium over Playwright

Selenium is not going anywhere, and it has its advantages over Playwright.

First, Selenium has a vast and active community with abundant resources, tutorials, and support forums. This superabundance of resources can be a huge advantage when you're stuck with an issue or need help with a specific problem. Playwright, being a newer tool, has a growing community, but it might still need to have the same breadth of resources.

Second, Selenium is a mature tool that has been around for a long time, which means it has a robust ecosystem with a wide range of plugins and integrations. If you're looking for a tool that can be easily integrated with other testing frameworks or CI/CD tools, Selenium might be the better choice.

Third, Selenium supports multiple programming languages, such as Java, Python, C#, Ruby, and JavaScript. Having more freedom in selecting the programming language makes it a versatile tool across different tech stacks. Playwright, on the other hand, primarily supports JavaScript and TypeScript, with beta support for Python, C#, and Java.

Finally, Selenium is compatible with legacy systems and tools, making it a reliable choice for testing older applications. If you need to support a wide range of browsers, including legacy ones like Internet Explorer, Selenium is the only choice.

Overview key features of Playwright and Selenium

So, selecting between Playwright and Selenium depends on various factors, such as the browsers you need to support, the programming languages your team uses, the complexity of your web application, and your existing testing infrastructure.

We can summarize the key differences between Playwright and Selenium across various dimensions:

Using Playwright to Scrape Weather Information

Now, let's see how to use Playwright to scrape weather information from a website.

In this example, we'll extract the current temperature for a given location from the National Weather Service website. We'll use Playwright to navigate to the website, locate the temperature element, and extract the temperature information.

Let's start by installing Playwright using npm:

npm install playwright

Now, we'll create a JavaScript file to scrape the weather information from the Google search widget.

First, we need to import the playwright module we just installed:

const playwright = require('playwright');
// More code

Now, we want to implement a function that fetches the weather information for a given city:

const browser = await chromium.launch({headless: false});
console.log('Launched new Chromium browser instance');

For our example, we are using the Chromium browser with the headless mode set to false. So this will allow us to see the browser window as the script runs. When we are confident that the script is working as expected, we can set the headless mode to true or remove the options as the default value is true to run the script in the background. Also, we'll add logging statements so we can track the progress of our script. Next, we need to open a new page within the browser, ready to navigate to the Google search page and begin our weather information quest.

const page = await browser.newPage();
console.log('Created new page');
await page.goto('https://www.google.com', { timeout: 60000 });
console.log('Navigated to Google search page');

When opening a new page, we can set a timeout value to wait for the page to load. In this case, we set it to 60 seconds. This will ensure that the page has enough time to load before we proceed. Next, we'll want to type the city name into the search text area and press the Enter key to trigger the search for weather information. But before we do that, we need to check for and reject the privacy notice that may appear on the Google search page, ensuring a smooth user experience.

if (await page.$('"Reject all"') !== null) {
  await page.click('"Reject all"');
 console.log('Rejected the privacy notice');
}

Now, we can type the city name into the search text area and press the Enter key, triggering the search for weather information.

await page.fill('textarea[name="q"]', `weather in ${city}`, { timeout: 60000 });
console.log(`Typed "${city}" in the search textarea`);
await page.press('textarea[name="q"]', 'Enter');
console.log('Pressed Enter key');

Loading the page might take a few seconds, so we can add a wait statement to ensure that the weather information is visible on the page before we print it to the console.

await page.waitForSelector('.wob_t', { timeout: 60000 });
console.log('Waiting for weather information to appear on the page');

const temperature = await page.textContent('.wob_t[id="wob_tm"]');
const weatherCondition = await page.textContent('.wob_dcp #wob_dc');
const humidity = await page.textContent('#wob_hm');
const windSpeed = await page.textContent('#wob_ws');
console.log('Extracted weather information:');
console.log(`Temperature: ${temperature}`);
console.log(`Condition: ${weatherCondition}`);
console.log(`Humidity: ${humidity}`);
console.log(`Wind Speed: ${windSpeed}`);

Now that we have all the information we need, we can close the browser instance.

await browser.close();
console.log('Closed the Chromium browser instance');

In this example, we saw how easy it is to use Playwright to scrape weather information from a website.

For further reading, check out our in depth guide on Scraping the web with Playwright , Playwight for Python tutorial and Crawlee Tutorial which is a crawling package that uses Playwright as the headless browser.

Using Selenium to Scrape Weather Information

Similarly, we can use Selenium to scrape weather information from a website. To make the comparison fair, we'll use the same website, the same information we used in the Playwright example, and the same language, JavaScript.

Let's first install the Selenium WebDriver using npm:

npm install selenium-webdriver

Now, we'll create a JavaScript file to scrape the weather information from the Google search widget. We start by importing the necessary tools from the selenium-web driver module along with the browser of our choice:

const {Builder, By, Key, until} = require('selenium-webdriver');
const chrome = require('selenium-webdriver/chrome');

We'll use Builder to initialize the Chrome driver and By to locate elements on the page. We'll also use Key to simulate keyboard actions and until to wait for elements to appear on the page. Next, we'll initialize the Chrome driver and launch a new browser instance:

const options = new chrome.Options();
let driver = await new Builder().forBrowser(Browser.CHROME).setChromeOptions(options.addArguments("--disable-search-engine-choice-screen")).build();
console.log('Launched new Chrome browser instance');

We bootstrapped the browser and turned off the search engine choice prompt that might appear when launching Chrome. Keep in mind that the browser will run in headed mode. Next, we'll navigate to the Google search page:

await driver.get('https://www.google.com');
console.log('Navigated to Google search page');

As done with Playwright, we'll check for and reject the privacy notice that may appear on the Google search page:

if (await driver.findElement(By.xpath('//*[text()="Reject all"]')).isDisplayed()) {
    await driver.findElement(By.xpath('//*[text()="Reject all"]')).click();
 console.log('Rejected the privacy notice');
}

Now, with the privacy notice dismissed, we can type the city name into the search text and press the Enter key:

let searchBox = await driver.findElement(By.name('q'));
await searchBox.sendKeys(`weather in ${city}`);
console.log(`Typed "${city}" in the search textarea`);
await searchBox.sendKeys(Key.RETURN);
console.log('Pressed Enter key');

Next, we'll wait for the weather information to load on the page and finally extract the temperature, weather conditions, humidity, and wind speed:

await driver.wait(until.elementLocated(By.css('.wob_t')), 60000);
console.log('Waiting for weather information to appear on the page');

let temperature = await driver.findElement(By.css('.wob_t[id="wob_tm"]')).getText();
let weatherCondition = await driver.findElement(By.css('.wob_dcp #wob_dc')).getText();
let humidity = await driver.findElement(By.css('#wob_hm')).getText();
let windSpeed = await driver.findElement(By.css('#wob_ws')).getText();

console.log('Extracted weather information:');
console.log(`Temperature: ${temperature}`);
console.log(`Condition: ${weatherCondition}`);
console.log(`Humidity: ${humidity}`);
console.log(`Wind Speed: ${windSpeed}`);

At last, we'll close the browser instance:

await driver.quit();
console.log('Closed the Chrome browser instance');

Now, we can appreciate the similarities and differences between Playwright and Selenium in action.

For further reading, check out our guides on How to use Selenium with Python , How to set up a rotating proxy in Selenium with Python and How to use undetected_chromedriver which is a fortified version of Selenium.

Conclusion

In conclusion, Playwright and Selenium are both powerful tools for automating web browsers and testing web applications. While Selenium has been the industry standard for over a decade, Playwright is a newer entrant gaining popularity for its powerful features and ease of use.

We started this article by exploring the key differences between Playwright and Selenium, first exploring the official descriptions of the two tools. We then discussed the advantages of using Playwright over Selenium and vice versa. Finally, to better understand how to use these tools, we provided a practical example of scraping weather information from a website using both Playwright and Selenium.