recoilme commited on
Commit
b121468
·
1 Parent(s): 3b12f1b
Files changed (1) hide show
  1. README.md +104 -1
README.md CHANGED
@@ -1,3 +1,106 @@
1
  ---
2
- license: mit
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - text-to-image
6
+ - stable-diffusion
7
+ pipeline_tag: text-to-image
8
+
9
  ---
10
+
11
+ # ColorfulXL-Lightning
12
+
13
+ ![promo](promo.png)
14
+
15
+
16
+ ## Model Details
17
+
18
+ ColorfulXL merged with lightning loras (2,8 steps) from bytedance, for fast inference (3-6 steps).
19
+
20
+ High range of resolutions supported (576 - 1280), 576*832 example:
21
+
22
+ ![steps](5.png)
23
+
24
+ Due to training LORA's on the base version of SDXL, there are problems with hands and faces:
25
+
26
+ ![problems](6.jpg)
27
+
28
+
29
+
30
+ ## Usage
31
+
32
+ ```python
33
+
34
+ from diffusers import DiffusionPipeline
35
+ from diffusers import EulerDiscreteScheduler
36
+ import torch
37
+
38
+
39
+ pipeline = DiffusionPipeline.from_pretrained("recoilme/ColorfulXL-Lightning", torch_dtype=torch.float16,variant="fp16", use_safetensors=True).to("cuda")
40
+ pipeline.scheduler = EulerDiscreteScheduler.from_config(pipeline.scheduler.config, timestep_spacing="trailing")
41
+
42
+ prompt = "girl sitting on a small hill looking at night sky, fflix_dmatter, back view, distant exploding moon, nights darkness, intricate circuits and sensors, photographic realism style, detailed textures, peacefulness, mysterious."
43
+ height = 1024
44
+ width = 1024
45
+ steps = 3
46
+ scale = 0
47
+ seed = 2139965163
48
+ generator = torch.Generator(device="cpu").manual_seed(seed)
49
+
50
+ image = pipeline(
51
+ prompt = prompt,
52
+ height=height,
53
+ width=width,
54
+ guidance_scale=scale,
55
+ num_inference_steps=steps,
56
+ generator=generator,
57
+ ).images[0]
58
+ image.show()
59
+ image.save("girl.png")
60
+ ```
61
+
62
+ ## Model Details
63
+
64
+ * **Developed by**: [AiArtLab](https://aiartlab.org/)
65
+ * **Model type**: Diffusion-based text-to-image generative model
66
+ * **Model Description**: This model is a fine-tuned model based on [colorfulxl](https://civitai.com/models/185258/colorfulxl).
67
+ * **License**: This model is not permitted to be used behind API services. Please contact [email protected] for business inquires, commercial licensing, custom models, and consultation.
68
+
69
+ ## Uses
70
+
71
+ ### Direct Use
72
+
73
+
74
+ Research: possible research areas/tasks include:
75
+
76
+ - Generation of artworks and use in design and other artistic processes.
77
+ - Applications in educational or creative tools.
78
+ - Research on generative models.
79
+ - Safe deployment of models which have the potential to generate harmful content.
80
+ - Probing and understanding the limitations and biases of generative models.
81
+
82
+ Excluded uses are described below.
83
+
84
+ ### Out-of-Scope Use
85
+
86
+ The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
87
+
88
+ ## Limitations and Bias
89
+
90
+ ### Limitations
91
+
92
+ - The model does not achieve perfect photorealism
93
+ - The model cannot render legible text
94
+ - The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to “A red cube on top of a blue sphere”
95
+ - Faces and people in general may not be generated properly.
96
+ - The autoencoding part of the model is lossy.
97
+
98
+ ### Bias
99
+ While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.
100
+
101
+
102
+ ## Contact
103
+ * For questions and comments about the model, please join [https://aiartlab.org/](https://aiartlab.org/).
104
+ * For future announcements / information about AiArtLab AI models, research, and events, please follow [Discord](https://discord.com/invite/gsvhQEfKQ5).
105
+ * For business and partnership inquiries, please contact https://t.me/recoilme
106
+