File size: 2,649 Bytes
4112525 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
---
license: apache-2.0
language:
- en
---
# MP_FaceMesh_V2
## Model Description
MP_FaceMesh_V2 is a pytorch port of tensorfolow [FaceMeshV2](https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker/index) model from Google's [mediapipe](https://github.com/google-ai-edge/mediapipe) library.
The model takes a cropped 2D face with 25% margin on each side resized to 256 x 256 pixels and outputs a dense 473 landmark coordinates in a 3D (x,y,z) coordinate space.
The original tensorflow model was ported to ONNX and then to pytorch using [onnx2torch](https://github.com/ENOT-AutoDL/onnx2torch). Currently, we are serializing the converted model, which requires onnx2torch as a dependency.
See the mediapipe [model card](https://storage.googleapis.com/mediapipe-assets/Model%20Card%20Blendshape%20V2.pdf) for more details.
## Model Details
- **Model Type**: Convolutional Neural Network (MobileNetV2-like)
- **Framework**: pytorch
## Model Sources
- **Repository**: [GitHub Repository](https://github.com/cosanlab/py-feat)
- **Model Card**: [Attention Mesh: High-fidelity Face Mesh Prediction in Real-time](https://storage.googleapis.com/mediapipe-assets/Model%20Card%20MediaPipe%20Face%20Mesh%20V2.pdf)
- **Paper**: [Mediapipe FaceMesh model card](https://arxiv.org/abs/2006.10962)
## Citation
If you use the mp_facemesh_v2 model in your research or application, please cite the following paper:
Grishchenko, I., Ablavatski, A., Kartynnik, Y., Raveendran, K., & Grundmann, M. (2020). Attention mesh: High-fidelity face mesh prediction in real-time. arXiv preprint arXiv:2006.10962.
```
@misc{grishchenko2020attentionmeshhighfidelityface,
title={Attention Mesh: High-fidelity Face Mesh Prediction in Real-time},
author={Ivan Grishchenko and Artsiom Ablavatski and Yury Kartynnik and Karthik Raveendran and Matthias Grundmann},
year={2020},
eprint={2006.10962},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2006.10962},
}
```
## Example Useage
```python
import torch
from huggingface_hub import hf_hub_download
device = 'cpu'
# Load model and weights
landmark_model_file = hf_hub_download(repo_id='py-feat/mp_facemesh_v2', filename="face_landmarks_detector_Nx3x256x256_onnx.pth")
landmark_detector = torch.load(landmark_model_file, map_location=device, weights_only=False)
landmark_detector.eval()
landmark_detector.to(device)
# Test model
face_image = "path/to/your/test_image.jpg" # Replace with your extracted face image that is [224, 224]
# Extract Landmarks
landmark_results = landmark_detector(torch.tensor(face_image).to(device))
``` |