-
-
Notifications
You must be signed in to change notification settings - Fork 60
Description
Environment (please complete the following information):
Node.js version: 22.21.1
WebdriverIO version: 9.23.3
@wdio/visual-service version: 9.1.3
Config of WebdriverIO + @wdio/visual-service
alwaysSaveActualImage: false
Describe the bug
If a comparison result sits somewhere between 0 and the threshold that I have added, an actual image is still generated. Let's say that I have toBeLessThanOrEqual(0.1) and the comparison results in a 0.05 difference, the test will be marked as passed AND an actual image is created.
This bug is only present for browser.checkElement/Screen/FullPageScreen.
toMatchScreen/Element/FullPageSnapshot works as expected, no actual image is generated
To Reproduce
Steps to reproduce the behavior:
it('checkscreen test', async function () {
await browser.url('https://webdriver.io/docs/api');
await expect(
await browser.checkScreen('A TEST CHECKSCREEN', {
})
).toBeLessThanOrEqual(0.9);
});
Make some adjustments to the baseline image. The adjustments should not be greater than 0.9 so make sure that the test still pass. Once executed with the test passing, you should still have an actual image despite alwaysSaveActualImage: false
Expected behavior
browser.checkElement/Screen/FullPageScreen should work the same way as toMatchScreen/Element/FullPageSnapshot if alwaysSaveActualImage: false if a test is marked as passed.
Log
n/a
Additional context
n/a