What is the difference between testing on Safari vs Webkit?

Question

I'm looking at writing some E2E tests using Playwright. I can see that the library allows tests to be run against Chromium, Firefox and Webkit. My understanding is that Webkit is the underlying engine of Safari, and Chromium is the base of Chrome (and other browsers).

When it comes to testing against Webkit/Chromium, how close is it to testing against the user facing browsers which use the underlying engines? I'd imagine that there would be some quirks which the user can run in to whilst using Safari and it wouldn't be caught when running the tests on Webkit.

Thanks

Answer 1

Stock browsers like Google Chrome, Apple Safari embed rendering engines (Chromium, WebKit) and add stuff on top of them. In particular, they add proprietary media codecs, inject browser extensions, etc. They also add surrounding interfaces such as bookmarks sync. But they reuse the underlying web platform implementation.

Chromium

Chromium is the open source web platform implementation used by Google Chrome, Opera, Microsoft Edge and other browsers. It implements web specs, renders content, works with network, etc. Playwright uses a stock Chromium build that can be automated with the Playwright API for e2e testing.

For Google Chrome things are simple: Chromium is a safe target to test, modulo proprietary media codecs and DRM. You can point Playwright against stock Chrome Canary or Edge Canary to use proprietary media codecs.

WebKit

WebKit is the open source web platform implementation used by Apple Safari and Epiphany. As of June 2020, Playwright provides a WebKit build that can be automated with the Playwright API for e2e testing. Playwright WebKit works across all platforms (macOS, Linux, Windows), in both headless and headful modes.

When WebKit runs on macOS, it is a safe target to test Safari. WebKit on Linux and Windows differs from Apple Safari in the following ways: it uses a non-macOS network stack, uses non-Core Animation to composite scene and produce image raster. This means that screenshots on Linux and Windows will not perfectly match screenshots from macOS. In terms of the web platform, the same WebKit code will layout the page and run JavaScript—it will match how WebKit works in Safari.

To conclude, we consider the browsers provided by Playwright to be the best of what you can get for e2e testing. Running WebKit on a Linux CI would use a different compositing pipeline than macOS, but it can be used for e2e testing of web applications in WebKit-powered browsers.