【2018/ECCV】Weakly-supervised 3D Hand Pose Estimation from Monocular RGB Images

文章链接：http://openaccess.thecvf.com/content_ECCV_2018/papers/Yujun_Cai_Weakly-supervised_3D_Hand_ECCV_2018_paper.pdf

TL;DR

在3D hand pose estmation任务中，由于标注的准确性和数量限制，目前方法3D还原效果一般。文中提出了一种RGB和depth image结合的弱监督模型，能够取得不错的效果。

Introduce the weakly supervised problem of leveraging low-cost depth maps during training for 3D hand pose estimation from RGB images.
Propose an end-to-end learning based 3D hand pose estimation model for weakly-supervised adaptation from fully-annotated synthetic images to weakly-labeled real-world images.
The weakly-supervised approach compares favorably with existing works and the proposed fully-supervised method outperforms all the state-of-the-art methods.

模型如下所示：

主要的创新点在于将深度图的特征加入模型当中，训练的过程如下所示：

For weakly-supervised learning, model is pretrained on RHD and then adapted to STB by fusing the training of both datasets.
For fully-supervised learning, the two datasets are used independently in the training and evaluation processes.

其中depth regularizer结构如下所示：