UniEM-Gen

🖼️ Example Outputs

📘 Model Summary

This is the text-to-image diffusion model trained on the complete UniEM-3M dataset.
It is designed for electron microscopy (EM)-style image generation, enabling:

• Scientific data augmentation
• Proxy generation for microstructural distributions
• Multimodal research in materials science

---

🚀 Usage Example

Using diffusers

from diffusers import StableDiffusionPipeline
import torch

# Load model from Hugging Face
model_id = "NNNan/UniEM-Gen"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

# Example prompt, sampled from UniEM-3M.
prompt = "SEM of Ceramic Powder: A mix of plate-like and rod-shaped particles. nanostructured. high density. densely packed and agglomerated. Multilayer. Wide range of particle sizes. Grayscale particles on a dark background."

# Generate image
image = pipe(prompt).images[0]

# Save or display
image.save("generated_em.png")
image.show()

---

🔄 Flexible Prompt Composition

You can generate structured scientific descriptions by randomly sampling one term from each of the nine attribute categories defined in the UniEM-3M. This repository includes attribute_values.json, which contains all observed values for these attributes, enabling reproducible and diverse prompt generation.

Use the following template to assemble the sampled terms into a coherent description:

<microscopy_type> of <subject>: <morphology>. <surface_texture>. <particle_density>. <distribution>. <layering>. <pixel_size_profile>. <color_profile>.
"💬 Replace each <...> placeholder with a real value from the corresponding category."

# Sampled attributes from UniEM-3M
from huggingface_hub import hf_hub_download
import json
import random

# Download the attribute_values.json file from the repo
json_path = hf_hub_download(
    repo_id="NNNan/UniEM-Gen",
    filename="attribute_values.json"
)

# Load the attribute values
with open(json_path, "r", encoding="utf-8") as f:
    attribute_values = json.load(f)

# Generate a random description
template = "{microscopy_type} of {subject}: {morphology}. {surface_texture}. {particle_density}. {distribution}. {layering}. {pixel_size_profile}. {color_profile}."
sampled = {k: random.choice(list(v)) for k, v in attribute_values.items()}
prompt = template.format(**sampled)
print(prompt)

---

📖 Citation

If you use this dataset, please cite:

@misc{wang2025uniem3muniversalelectronmicrograph,
      title={UniEM-3M: A Universal Electron Micrograph Dataset for Microstructural Segmentation and Generation}, 
      author={Nan wang and Zhiyi Xia and Yiming Li and Shi Tang and Zuxin Fan and Xi Fang and Haoyi Tao and Xiaochen Cai and Guolin Ke and Linfeng Zhang and Yanhui Hong},
      year={2025},
      eprint={2508.16239},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2508.16239}, 
}