Resize down fails if the empty space is not in the volume below

[npelov]

Current Behavior

In the image below I tried to resize vol03 down. it worked until vol02 was filled. then it won't transfer to vol01a

Expected Behavior

To be able to size down volume when there is empty space in any volume

Steps to Reproduce

1. create 3 volmes and number them in order
2. when 1 and 2 are filled and 3 starts filling add little space to 1 and 2
3. size down 3 to move storage to 1 and 2
4. will only move to 2, not to 1

Version

v2.4.1

Anything else?

No response

[n8mgr]

Can you get the logs for the migration?

[npelov]

2025-10-16T21:14:45Z	ERROR	sqlite3.migrate	failed to migrate sector	{"startIndex": 143051, "error": "sector corrupt: 888b301b42d484bd21b9787c41c245dd534c68cbf7cbdea19ab0de932199715b != d7bdee7ddce10cdf425c2bfbb65b739c117d61f3a852273e33aeaa69395a7236", "index": 150379, "root": "888b301b42d484bd21b9787c41c245dd534c68cbf7cbdea19ab0de932199715b"}
2025-10-16T21:14:46Z	ERROR	sqlite3.migrate	failed to migrate sector	{"startIndex": 143051, "error": "sector corrupt: 3f8b509e30fb2c485cdbff877d2f95a9396fa7bdd88e5234cd599fab686889c5 != 8801ba8dc8107e1ed4734f3eb8ccf44b923011f25fa91c3c36bff80fdf25bcf3", "index": 152222, "root": "3f8b509e30fb2c485cdbff877d2f95a9396fa7bdd88e5234cd599fab686889c5"}
2025-10-16T21:14:46Z	ERROR	sqlite3.migrate	failed to migrate sector	{"startIndex": 143051, "error": "sector corrupt: 3fa1e0526e83cec988abc69bf5de64875b92d4a90a8f987e965bb78f10a7d0da != 99c3fd58ea471596882192f0317f7d1200b4bdb85cda412267a8864251592545", "index": 152037, "root": "3fa1e0526e83cec988abc69bf5de64875b92d4a90a8f987e965bb78f10a7d0da"}
2025-10-16T21:14:46Z	ERROR	sqlite3.migrate	failed to migrate sector	{"startIndex": 143051, "error": "sector corrupt: 8b4875b104df31fd4d4c28d1e9b50bb45c0d3a19cecbe3d90f8ad7087d5071f0 != 22aade5a08c931b60e515925b6ec8abc8bfaed9f46d741dc1af356b4261f83fa", "index": 151995, "root": "8b4875b104df31fd4d4c28d1e9b50bb45c0d3a19cecbe3d90f8ad7087d5071f0"}
2025-10-16T21:14:46Z	ERROR	sqlite3.migrate	failed to migrate sector	{"startIndex": 143051, "error": "sector corrupt: 8159fa592c0b838aa4091045d19ada0e72a161eb22a68190ea7d34112ef4a104 != ced488c4da610635a3ee6a5a692c8641f2bbf9a1ef3a87b9f41495aa46e3f859", "index": 151864, "root": "8159fa592c0b838aa4091045d19ada0e72a161eb22a68190ea7d34112ef4a104"}
2025-10-16T21:14:46Z	ERROR	sqlite3.migrate	failed to migrate sector	{"startIndex": 143051, "error": "sector corrupt: ccc9189c0919db517f8f06f6dba88bbb42e1754dbfa34aaecf229b6d2c119129 != 180cb0398420a498d47cbc95443914ce591b7c9e17903764af0897abc730b4c6", "index": 151857, "root": "ccc9189c0919db517f8f06f6dba88bbb42e1754dbfa34aaecf229b6d2c119129"}
2025-10-16T21:14:46Z	ERROR	sqlite3.migrate	failed to migrate sector	{"startIndex": 143051, "error": "sector corrupt: 07f0977ff59cb737e308acdb8296008071eb58350a3e0f02115969e884d4e470 != f353b59534bff8150c303e9d74a30d7081ab4e7e625626879863031d6e3124ba", "index": 151672, "root": "07f0977ff59cb737e308acdb8296008071eb58350a3e0f02115969e884d4e470"}
2025-10-16T21:14:45Z	ERROR	sqlite3.migrate	failed to migrate sector	{"startIndex": 143051, "error": "sector corrupt: 70e96b99c595a0fc47e6449c1e415097169b88e4e656b70c929514fbfcabab72 != 79a814b99cee6467ad2aeeb4a3be1648f1de2d8c6139d364c640240c24567a27", "index": 151134, "root": "70e96b99c595a0fc47e6449c1e415097169b88e4e656b70c929514fbfcabab72"}
2025-10-16T21:14:45Z	ERROR	sqlite3.migrate	failed to migrate sector	{"startIndex": 143051, "error": "sector corrupt: e1a0c6ca02d25446e51f4bad8e920d235ba120d561bd8fb34b2d100fb10ff2b8 != dc985f80d298d71c489d7b1b472b2719279d45b86c19f7fa10bd19c1c65f2031", "index": 150863, "root": "e1a0c6ca02d25446e51f4bad8e920d235ba120d561bd8fb34b2d100fb10ff2b8"}
2025-10-16T21:14:45Z	ERROR	sqlite3.migrate	failed to migrate sector	{"startIndex": 143051, "error": "sector corrupt: 63827283559cab2d17ea9977407c61020a37608d5cadba8d73e0f40173863a22 != 263ed78c6162b313ea2beec8ab86e2d6a1b14cb9914370f431c68974aff5372f", "index": 150598, "root": "63827283559cab2d17ea9977407c61020a37608d5cadba8d73e0f40173863a22"}
2025-10-16T21:14:45Z	ERROR	sqlite3.migrate	failed to migrate sector	{"startIndex": 143051, "error": "sector corrupt: 133dfe34db6059962bf3a2253b736f71313ccce670b25c28c004a5c84e33b888 != b7f06a6d86cf6195e57b22485346ec3f01eb400c9447fdbeabb61c549ec56d43", "index": 150472, "root": "133dfe34db6059962bf3a2253b736f71313ccce670b25c28c004a5c84e33b888"}
2025-10-16T21:14:45Z	ERROR	sqlite3.migrate	failed to migrate sector	{"startIndex": 143051, "error": "sector corrupt: 5256d3a37936200971d114f8c4d7dc5d8a16785e61495a1cb6df1e23c9923c40 != 93a752c11ebfea780d2cd104d6e8266c7d8c42c4e9c0edb5f7f834d5a8648ccb", "index": 150574, "root": "5256d3a37936200971d114f8c4d7dc5d8a16785e61495a1cb6df1e23c9923c40"}

[npelov]

Why is a sector corrupt? I've done nothing to break it. I have zfs with raid-5. fs can't be corrupted.

[npelov]

I think I might have run out of space on the read-write volumes while the volumes with free space were read-only once. That happened few weeks ago. I noticed I have lots of bandwidth, but no storage. That was for about 20 hours. Can you double check if this causes broken sectors?

[npelov]

I've done integrity check on all active contracts with non-zero data. No errors.

[npelov]

I think I'm getting these after canceled resize. Then next resize fails for some reason. I keep getting this error:
2025-10-19T23:05:31Z ERROR sqlite3.migrate failed to migrate sector {"startIndex": 479221, "error": "sector corrupt: 39b789e007876fb9883014e3f3fe0039d3494981e0560d39c2d1dad22bb67ab3 != 6be910b4c06f71756278f3a13b4a3d9ed021d2a79be929a781796f5d46c3a532", "index": 480145, "root": "39b789e007876fb9883014e3f3fe0039d3494981e0560d39c2d1dad22bb67ab3"}

on other hosts too. The host I first saw this error no longer gives it. I don't know what fixed it. I tried with very small resize of 1 gb. Not sure if this fixed it or integrity check or a restart.

[npelov]

For the second host I tried:

1. resize up + resize down again
2. resize down by small amount - 1gb
3. restart
4. integrity check for all active contracts - passed

None of these worked. I still can't resize.

[npelov]

I think I got past the error by locking all other volumes so data is moved within the resized volume. So I resized it down and it worked. The resize was slow, so data was moved. After I got past the error I enabled read/write for the other volume and it was able to move data to the other volume.
I don't have enough information to be sure, but I think this error only occurs when moving data from one volume to another. Also the volume was fragmented. the data is at the end and the free space was somewhere in the middle (a contract finished).

P.S. I tried this again when another error happened - resizing the volume with the others locked, but it didn't work this time. I get the error no matter what I do

[npelov]

Hint to repilcate: try to make holes by contracts expiring, then fill those holes by sizing down other volumes

[npelov]

Another guess - just tried it. I have following situation:
volume 1 - 1.99/2.00 TB
volume 2 - 450/600GB

I resize volume 2 down to 550GB - it was empty space, so it happened fast. Then I resized down to 500gb. it started filling volume 1. and when volume 1 got full I got the error. So it may be happening when there is not enough space in the other volume and it has to switch from volume 1 to moving data within the same volume.

[ChrisSchinnerl]

Your migration appears to be failing due to corrupted sectors. Those won't be moved for safety and if more than one are encountered, the migration fails overall. Sectors that are not corrupted will be moved though.

That said I'm not sure why your sectors are corrupting. The fact that the integrity checks pass for all contracts might indicate that the corrupted sectors are temporary sectors. Those should expire after a few days and be pruned.