The unconstrained head pose estimation results of our SemiUHPE on wild challenging heads (e.g., heavy blur, extreme illumination, severe occlusion, atypical pose, and invisible face).

Short Phone Video 1	Short Phone Video 2	Short Phone Video 3

Abstract

Existing research on unconstrained in-the-wild head pose estimation suffers from the flaws of its datasets, which consist of either numerous samples by non-realistic synthesis or constrained collection, or small-scale natural images yet with plausible manual annotations. To alleviate it, we propose the first semi-supervised unconstrained head pose estimation method SemiUHPE, which can leverage abundant easily available unlabeled head images. Technically, we choose semi-supervised rotation regression and adapt it to the error-sensitive and label-scarce problem of unconstrained head pose. Our method is based on the observation that the aspect-ratio invariant cropping of wild heads is superior to the previous landmark-based affine alignment given that landmarks of unconstrained human heads are usually unavailable, especially for less-explored non-frontal heads. Instead of using an empirically fixed threshold to filter out pseudo labeled heads, we propose dynamic entropy based filtering to adaptively remove unlabeled outliers as training progresses by updating the threshold in multiple stages. We then revisit the design of weak-strong augmentations and improve it by devising two novel head-oriented strong augmentations, termed pose-irrelevant cut-occlusion and pose-altering rotation consistency respectively. Extensive experiments and ablation studies show that SemiUHPE outperforms existing methods greatly on public benchmarks under both the front-range and full-range settings.

SemiUHPE vs. previous SOTAs

Visualizations of front-range or full-range head pose estimation (HPE). Our method SemiUHPE is more robust to severe occlusion, atypical pose and invisible face.

Qualitative results of our method (3rd line) and DAD-3DNet (2nd line) on heads from DAD-3DHeads test-set (1st line), which never appeared during SSL training.

More Quantitative Results of SemiUHPE

Real Classrooms.

Recorded Vlogs.

Many in-the-wild images with multiple persons.

Citation

Acknowledgements

We acknowledge the effort from authors of human-related datasets including CrowdHuman, COCOHumanParts and DAD-3DHeads. These datasets make researches and downstream applications about Semi-Supervised Unconstrained Head Pose Estimation in the Wild possible.

The website template was borrowed from Jon Barron and Zip-NeRF.