Cleanly handling file deletes during MODIFY operations

[Markzipan]

We now handle missing files during update loops by 'escalating' them as delete operations (rather than throwing). IDEs sometimes create/delete interstitial files, and this handles their automatic deletes more gracefully.

Fixes dart-lang/webdev#2795

[github-actions]

PR Health

Changelog Entry ✔️

Package	Changed Files

Changes to files need to be accounted for in their respective changelogs.

This check can be disabled by tagging the PR with skip-changelog-check.

[davidmorgan]

Thanks for the PR, this is a fun problem that I have ignored so far :)

I think a more robust approach is needed, as I think this still leaves a race between the second time existence is checked and when it's actually read.

First I suggest adding an e2e test that makes lots of file adds+deletes, hopefully it's easy enough to repro the basic failure and then maybe also it will still show a failure with this PR. It can test "watch" as that's simpler than "serve" and should show the same issue.

Then I think the right fix is to move updating+hashing+reading to cache to a single place, so we actually do get one snapshot of the filesystem for the build. I would consider removing the "low resources" option at the same time, or we just treat it as broken for this issue.

That's a bit more of a refactor, I can help or take over as you like :) ... it's a cleanup I've been meaning to do for a while, the scattered nature of where hashing happens is brittle and actually cost me half a day wasted this week :)

[Markzipan]

@davidmorgan Hmm, I'll need a more complicated repro for sure. Based on what you're saying, I'm thinking of registering a digestsCache early (maybe in the reader_writer or even earlier) and always consulting that instead of all the readAsBytes calls. So extraneous reads of these files would hit our cache, and we'd eventually get the proper Remove signals (from the watcher or equivalent).

I'm unfortunately going to be out for at least a few weeks, so I'm fine with you taking over this issue. Or I can revisit when I'm back.

[davidmorgan]

Taking a look at this now...

[davidmorgan]

I think #4865 should do it, still need to take another look before sending it for review.

[Markzipan]

Superceded by #4865