16 Bit Image Support?

[arvinmoses]

I'm not sure if I'm just missing it somewhere, but I'm using a 16 bit grayscale (height map) png as the input, but still getting an 8 bit rgb output. Am I just missing something, or is it not capable of this yet? I thought I saw some terrain tests, and I was assuming those were 16 bit

[Jake-Shadle]

Yes, currently all operations are done in RGB space, though I think you might be able to get what you want using the repeat transform feature? https://github.com/EmbarkStudios/texture-synthesis#api---08_repeat_transform

[arvinmoses]

Gonna try it! So sorry for delayed response.

You're saying to try if repeat transform will operate on 16 bits?

I can do it straight on the normal maps, but the way the images are blending aren't really proper.
I think... ideally ... the images would transform with a slight fall off and height blend as they draw in.

Thank you Jake!

[virtualritz]

+1. It would be great if the channel depth could be made a parameter of the model (using generics). Common depths are u8 u16, f16 (see half) and f32.

[arvinmoses]

Got the chance to test it with repeat transform - so -

It works amazing with for 8 bit maps. I did it with an albedo map and repeat transformed a normal map and it produced really nice results. Some of it was a littile "tiley" and the pattern seemed to grab a pretty basic noise that didn't necessarily catch some of the flow of the albedo. (tree trunk)
Overall though the results were very nice.

However repeat still produced an 8 bit map for the height map. :(

[arvinmoses]

I've been thinking about this a little more... Is there a reasonable way we could parse the .xform file?

If it is simply a coordinate system defining the loocation of the original pixels, we could use a simply image manipulator like pil or imagemagick to transform pixels of a new image.

Assuming it's something like
[5,1] //Copy pixel at 5,1 to 0,0
[400,30] //Copy pixel at 400,30 to 1,0
[50,24] //Copy pixel at 50,24 to 2,0
etc.

It would be pretty simple to write something to match the transforms. Potentially we could even use a pixel process in Substance Designer.

Is there any documentation on how we can read an xform file?

[materialjan]

Or simply save a 2d transformation map in float(stmap in nuke). But that may represent some extra work if there isn't any code using float at the moment.

[arvinmoses]

Decided to dig into the code a little bit... Is it correct to assume that most of the actual processing is happening at 32bits, and just normalizing out to an 8 bit image at the end?

[materialjan]

I respond because nobody on this but I think the computation, neighbours etc is done at 8bit to save ressources.

[arvinmoses]

Thanks jan - appreciate it. I'll dig into it some more. Would be pretty easy if most of it was done and just normalized at the end.

[materialjan]

Sorry but is there anything plan for this?
-without float or 16bit output our best hope would be to use the the .xform file which certainly contain ids for the maps and their pixel position but its structure is probably custom and I can't write code for that, especially in rust... arvin did you made progress on this?
thanks.

[arvinmoses]

Unfortunately I hadn't gotten a chance to mess with this.. Sorry jan. :)

I found a simple solution in the mean time - I used affinity designer which lets you do an operation on multiple layers at the same time. Similar to painter - working at the material level, but able to use all ps tools - clone, heal etc.