任务

输入一张包含人脸图像（通过人脸检测模型crop出来或者手动crop），进行人脸2D关键点检测，输出人脸68个关键点的2D坐标。

68点人脸关键点定义

模型描述

该模型主要用于人脸2D关键点检测，主要基于STAR loss(CVPR2023)网络，通过扩充训练数据集，优化数据增强部分，调整训练策略方式, 在300W等人脸关键点公开数据集上指标上达到SOTA, 能很好地应用于真人人脸关键点检测场景。

训练数据

该模型主要使用学界300W，300VW, FaceSynthetics等包含68个人脸关键点标注信息的手动标注数据及渲染数据作为训练数据。

指标及可视化效果对比

指标对比

主要在300W数据集 (https://ibug.doc.ic.ac.uk/resources/300-W/) 上进行对比。

可视化效果对比

与目前学界较好的算法进行可视化效果对比，如下图所示，提供的模型特别在大角度(如半侧脸，侧脸)2D关键点检测更为精确。

安装及使用方式

您可以通过如下git clone命令，或者ModelScope SDK来下载模型

SDK下载

# 安装ModelScope
pip install modelscope

# SDK模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('Damo_XR_Lab/cv_human_68-facial-landmark-detection')

Git下载

# Git模型下载
git clone https://www.modelscope.cn/Damo_XR_Lab/cv_human_68-facial-landmark-detection.git

引用

@inproceedings{Zhou_2023_CVPR,
    author    = {Zhou, Zhenglin and Li, Huaxia and Liu, Hong and Wang, Nanyang and Yu, Gang and Ji, Rongrong},
    title     = {STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2023},
    pages     = {15475-15484}
}
@inproceedings{Prados-Torreblanca_2022_BMVC,
  author    = {Andrés  Prados-Torreblanca and José M Buenaposada and Luis Baumela},
  title     = {Shape Preserving Facial Landmarks with Graph Attention Networks},
  booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
  publisher = {{BMVA} Press},
  year      = {2022},
  url       = {https://bmvc2022.mpi-inf.mpg.de/0155.pdf}
}
@article{JLS21,
  title={Pixel-in-Pixel Net: Towards Efficient Facial Landmark Detection in the Wild},
  author={Haibo Jin and Shengcai Liao and Ling Shao},
  journal={International Journal of Computer Vision},
  publisher={Springer Science and Business Media LLC},
  ISSN={1573-1405},
  url={http://dx.doi.org/10.1007/s11263-021-01521-4},
  DOI={10.1007/s11263-021-01521-4},
  year={2021},
  month={Sep}
}