FaceScore
FaceScore: Benchmarking and Enhancing Face Quality in Human Generation
Traditional facial quality assessment focuses on whether a face is suitable for recognition, while image aesthetic scorers emphasize overall aesthetics rather than details. FaceScore is the first reward model that focuses on faces in text-to-image models, designed to score the faces generated in images. It is fine-tuned on positive and negative sample pairs generated using an inpainting pipeline based on real face images and surpasses previous models in predicting human preferences for generated faces.
LoRA based on SDXL
We leverage FaceScore to filter data and perform direct preference optimization on SDXL. The LoRA weight is here. Here we provide a quick example:
from diffusers import StableDiffusionXLPipeline, UNet2DConditionModel
import torch
# load pipeline
inference_dtype = torch.float16
pipe = StableDiffusionXLPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
torch_dtype=inference_dtype,
)
vae = AutoencoderKL.from_pretrained(
'madebyollin/sdxl-vae-fp16-fix',
torch_dtype=inference_dtype,
)
pipe.vae = vae
# You can load it locally
pipe.load_lora_weights("AIGCer-OPPO/FaceScore-dpo-SDXL-LoRA")
pipe.to('cuda')
generator=torch.Generator(device='cuda').manual_seed(42)
image = pipe(
prompt='A woman in a costume standing in the desert',
guidance_scale=5.0,
generator=generator,
output_type='pil',
).images[0]
image.save('A woman in a costume standing in the desert.png')
We provide some examples generated by ours (right) and compare with the original SDXL (left) below.
A woman in a costume standing in the desert.
A woman wearing a blue jacket and scarf.
A young woman in a blue dress performing on stage.
A woman with black hair and a striped shirt.
A woman with white hair and white armor is holding a sword.
A woman with long black hair and a white shirt.
Citation
@misc{liao2024facescorebenchmarkingenhancingface,
title={FaceScore: Benchmarking and Enhancing Face Quality in Human Generation},
author={Zhenyi Liao and Qingsong Xie and Chen Chen and Hannan Lu and Zhijie Deng},
year={2024},
eprint={2406.17100},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2406.17100},
}