Skip to content

alwaysSaveActualImage: false does not respect browser.checkElement/Screen/FullPageScreen #1115

@BosseKarat

Description

@BosseKarat

Environment (please complete the following information):

Node.js version: 22.21.1
WebdriverIO version: 9.23.3
@wdio/visual-service version: 9.1.3

Config of WebdriverIO + @wdio/visual-service
alwaysSaveActualImage: false

Describe the bug
If a comparison result sits somewhere between 0 and the threshold that I have added, an actual image is still generated. Let's say that I have toBeLessThanOrEqual(0.1) and the comparison results in a 0.05 difference, the test will be marked as passed AND an actual image is created.

This bug is only present for browser.checkElement/Screen/FullPageScreen.

toMatchScreen/Element/FullPageSnapshot works as expected, no actual image is generated

To Reproduce
Steps to reproduce the behavior:

    it('checkscreen test', async function () {
        await browser.url('https://webdriver.io/docs/api');

        await expect(
            await browser.checkScreen('A TEST CHECKSCREEN', {
            })
        ).toBeLessThanOrEqual(0.9);
    });

Make some adjustments to the baseline image. The adjustments should not be greater than 0.9 so make sure that the test still pass. Once executed with the test passing, you should still have an actual image despite alwaysSaveActualImage: false

Expected behavior
browser.checkElement/Screen/FullPageScreen should work the same way as toMatchScreen/Element/FullPageSnapshot if alwaysSaveActualImage: false if a test is marked as passed.

Log
n/a

Additional context
n/a

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions