Connected components stack out dtype argument

[MattTheCuber]

It appears the out_dtype argument in connected_components_stack is not hooked up to do anything.

Again, this is a note to self, as I would love to hook this up in the near future 👍

[william-silversmith]

This could be a useful change. However, it might be tricky to determine ahead of time whether the dtype is appropriate, so errors will happen after a lot of computation.

[MattTheCuber]

Oh, I thought it was going to be as easy as calling .astype() at the end. Maybe the argument should just be removed for now?

[william-silversmith]

It can be that simple. It's just if it's too small, you might have done minutes or hours of computation before you see the error at the end. This function is designed for huge data that is fed in a chunk at a time.

The nice thing about the CrackleArray is that it can be cheap to change the dtype. It will incur (usually) a copy of the compressed binary, but that will ordinarily be hundreds or thousands of times smaller than the numpy representation. It could be even cheaper (in-place modification of one or two bits), but it would violate the immutability of a bytes object.

The reason for the out_dtype argument in connected_components is that if you allocate e.g. a np.uint64 from the jump, you won't have to do:

x:np.ndarray[np.uint32] = cc3d.connected_components(...)
y = x.astype(np.uint64) # memory usage = 1.5x (uint64 + uint32)
del x

Instead you just have:

# 1x memory usage
x:np.ndarray[np.uint64] = cc3d.connected_components(..., out_dtype=np.uint64)

So maybe you are right, perhaps the argument should just be removed. I think I initially put it in for compatibility with the function signature of connected_components, but it's not really needed.