utilizing edges and contours

[PaulaScharf]

opencv provides a function called Canny to detect edges. maybe this could be used to improve the contours of the hand

[PaulaScharf]

This is the result after calculating the edges and then getting the contours:

I think it has some potential I just need to figure out a way to use it. Biggest problem is that the contours are not perfect (eg often they do not close completly).

[PaulaScharf]

There is a machine learning approach called graph-cut that does semantic segmentation using edge detection but I am not sure if it is fast enough.

[PaulaScharf]

Interesting aproach to filling the mask until edges here. It will probably be too slow though...

[PaulaScharf]

I added a mode just now where only the edges of a hand are visualized:

The procedure is: first calculate edges for the entire frame, then create the hand mask as usual but heavily dilate it and lastly use the mask to cut out the edges.

I think the contours of these edges are often better than the mask. The outline is more realistic. However quality of the result is still very dependent on the mask. For example if the mask is missing a finger this will also be cut out of the edges.

A big draw back is the low visibility of the edges. Maybe I can figure out an intelligent way to set the color of the edges.

[PaulaScharf]

I tried filling the edges with the watershed function. The results are fine, but not better than what we currently have.

Edit:
I improved the watershed again today. I would say that the results look promising. I especially like how the fingers are segmented. The framerate is awful but the code is also really messy at the moment so there is probably room for improvements.