DenseDepth VS fast-depth

For a long time, the state-of-the-art models for monocular depth estimation such as DORN and DenseDepth were built with convolutional neural networks. Recently, however, both transformer-based models such as DPT and GLPN, and diffusion-based models like Marigold have achieved remarkable results!

Instead of just doing grayscale for depth, you should consider using “monocular depth estimation” that actually tries to reconstruct a depth map from an RGB image. There are a variety of open-source libs available like this one.

After that I have to do the same for Python I think, and then I have to find out how to figure out how to use a library like https://github.com/ialhashim/DenseDepth or https://github.com/nianticlabs/monodepth2 for that GStreamer plugin (or element, still trying to grasp the terminology here)

Others model implemented like github.com/ialhashim/DenseDepth have constraints on input (I think only 4:3 is one of them, if I remember correctly)

💡For this walkthrough, we will only use the NYU depth v2 portions. NYU depth v2 is permissively licensed for commercial use (MIT), and can be downloaded from Hugging Face directly.