Handle VMFS5's double indirect pointer for files > 256G

[mlsorensen]

https://github.com/glandium/vmfs-tools/issues/12

This provides read support for files > 256G, due to vSphere 5
adding double indirect block pointers. It uses a double indirect
lookup if the file has a blocksize of 1M and is over the VMFS size
threshold for using double indirect blocks. Perhaps there's a cleaner
way of determining the use of double indirect from the inode.

We may also want to implement a block pointer cache like VMware introduced
with this feature, however given the use cases of this software it may
not be necessary.

[antonu17]

Thank you for patch. You saved my day! 600Gb VMDK file was copied successfully.
Is it planned to be released some day?

[mlsorensen]

No idea! The pull request has been here awhile.

[grogthegreat]

@mlsorensen Thanks so much for the large file support! I have ScaleIO for my VM storage. With your fix I can mount the ScaleIO volume on CentOS, have linux read the file system, and backup the VMs with Borg! Just FYI: I couldn't get your fork to Make with GCC 4.8.5 on CentOS7.2. Using GCC 4.4 worked.

[skandragon]

Is there a reason this hasn't been merged? Disks are much larger these days, and I've got several large files I need to copy off. Unfortunately for me, it's also off a 4 G disk, which will require more development work on my part, I fear.

[skandragon]

According to the book "Storage Design and Implementation in vSphere 6", <= 8k uses "sub-blocks", >8k && <= 256M uses file blocks, > 256M && <= 256GB uses pointer blocks, and > 256 GB && < 64 TB uses pointer blocks + secondary pointer blocks, so I think this patch works as expected. It seems the size is the only factor, nothing magic in the inode.

[knackko]

Fix tested on a 400GB vmdk file and it works fine !

[daedalus]

please merge

[gilles504]

Tested to rescue a 500GB image with (g)ddrescue (disk had some bad sectors) : works great. I think this patch should be merged in linux distros (tested with debian 11 with a manual merge from debian sources).

[phkhh]

Dear mlsorensen, Your pool request was published about 8 years ago, but it is still relevant! The new Ubuntu and Debian distributions contain an old version of vmfs-tools, which did not include your code. You helped me solve the problem with a large vmdk file, thank you so much!

[cmruffin]

This is a nine year old useful patch. It enabled me to recover files from a crashed / inop ESXI instance. PR was submitted nine years ago. @mlsorensen @glandium Request to merge.