Hugging Face's logo

Spaces:
ujin-song
/
ortha
Running

App Files Community
ujin-song commited on
Commit
3ce2981
•
1 Parent(s): cff622c

added Hiro weights

Browse files
Files changed (3) hide show
  1. experiments/single-concept/hiro/7657_hiro_ortho.yml +97 -0
  2. experiments/single-concept/hiro/models/edlora_model-latest.pth +3 -0
  3. experiments/single-concept/hiro/train_7657_hiro_ortho_20240615_141549.log +192 -0
experiments/single-concept/hiro/7657_hiro_ortho.yml ADDED
@@ -0,0 +1,97 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # GENERATE TIME: Sat Jun 15 14:15:49 2024
2
+ # CMD:
3
+ # train_edlora.py -opt single-concept/train_configs/7657_hiro_ortho.yml
4
+
5
+ name: 7657_hiro_ortho
6
+ manual_seed: 7657
7
+ mixed_precision: fp16
8
+ gradient_accumulation_steps: 1
9
+ datasets:
10
+ train:
11
+ name: LoraDataset
12
+ concept_list: single-concept/data_configs/hiro.json
13
+ use_caption: true
14
+ use_mask: true
15
+ instance_transform:
16
+ - type: HumanResizeCropFinalV3
17
+ size: 512
18
+ crop_p: 0.5
19
+ - type: ToTensor
20
+ - type: Normalize
21
+ mean:
22
+ - 0.5
23
+ std:
24
+ - 0.5
25
+ - type: ShuffleCaption
26
+ keep_token_num: 1
27
+ - type: EnhanceText
28
+ enhance_type: human
29
+ replace_mapping:
30
+ <TOK>: <hiro1> <hiro2>
31
+ batch_size_per_gpu: 2
32
+ dataset_enlarge_ratio: 500
33
+ val_vis:
34
+ name: PromptDataset
35
+ prompts: single-concept/validation_prompts/characters/test_boy_disney.txt
36
+ num_samples_per_prompt: 8
37
+ latent_size:
38
+ - 4
39
+ - 64
40
+ - 64
41
+ replace_mapping:
42
+ <TOK>: <hiro1> <hiro2>
43
+ batch_size_per_gpu: 4
44
+ models:
45
+ pretrained_path: nitrosocke/mo-di-diffusion
46
+ enable_edlora: true
47
+ finetune_cfg:
48
+ text_embedding:
49
+ enable_tuning: true
50
+ lr: 0.001
51
+ text_encoder:
52
+ enable_tuning: true
53
+ lora_cfg:
54
+ rank: 5
55
+ alpha: 1.0
56
+ where: CLIPAttention
57
+ lr: 1.0e-05
58
+ unet:
59
+ enable_tuning: true
60
+ lora_cfg:
61
+ rank: 5
62
+ alpha: 1.0
63
+ where: Attention
64
+ lr: 0.0001
65
+ new_concept_token: <hiro1>+<hiro2>
66
+ initializer_token: <rand-0.013>+boy
67
+ noise_offset: 0.01
68
+ attn_reg_weight: 0.01
69
+ reg_full_identity: false
70
+ use_mask_loss: true
71
+ gradient_checkpoint: false
72
+ enable_xformers: true
73
+ path:
74
+ pretrain_network: null
75
+ train:
76
+ optim_g:
77
+ type: AdamW
78
+ lr: 0.0
79
+ weight_decay: 0.01
80
+ betas:
81
+ - 0.9
82
+ - 0.999
83
+ unet_kv_drop_rate: 0
84
+ scheduler: linear
85
+ emb_norm_threshold: 0.55
86
+ val:
87
+ val_during_save: false
88
+ compose_visualize: false
89
+ alpha_list:
90
+ - 0.7
91
+ - 1.0
92
+ sample:
93
+ num_inference_steps: 50
94
+ guidance_scale: 7.5
95
+ logger:
96
+ print_freq: 10
97
+ save_checkpoint_freq: 10000.0
experiments/single-concept/hiro/models/edlora_model-latest.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:970bce4fe8d01a407986a143ca863b6d93f4745165f8b2cda075ba4aeba10a72
3
+ size 35173046
experiments/single-concept/hiro/train_7657_hiro_ortho_20240615_141549.log ADDED
@@ -0,0 +1,192 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2024-06-15 14:15:49,238 INFO: Distributed environment: MULTI_GPU Backend: nccl
2
+ Num processes: 2
3
+ Process index: 0
4
+ Local process index: 0
5
+ Device: cuda:0
6
+
7
+ Mixed precision type: fp16
8
+
9
+ 2024-06-15 14:15:49,239 INFO:
10
+ name: 7657_hiro_ortho
11
+ manual_seed: 7657
12
+ mixed_precision: fp16
13
+ gradient_accumulation_steps: 1
14
+ datasets:[
15
+ train:[
16
+ name: LoraDataset
17
+ concept_list: single-concept/data_configs/hiro.json
18
+ use_caption: True
19
+ use_mask: True
20
+ instance_transform: [{'type': 'HumanResizeCropFinalV3', 'size': 512, 'crop_p': 0.5}, {'type': 'ToTensor'}, {'type': 'Normalize', 'mean': [0.5], 'std': [0.5]}, {'type': 'ShuffleCaption', 'keep_token_num': 1}, {'type': 'EnhanceText', 'enhance_type': 'human'}]
21
+ replace_mapping:[
22
+ <TOK>: <hiro1> <hiro2>
23
+ ]
24
+ batch_size_per_gpu: 2
25
+ dataset_enlarge_ratio: 500
26
+ ]
27
+ val_vis:[
28
+ name: PromptDataset
29
+ prompts: single-concept/validation_prompts/characters/test_boy_disney.txt
30
+ num_samples_per_prompt: 8
31
+ latent_size: [4, 64, 64]
32
+ replace_mapping:[
33
+ <TOK>: <hiro1> <hiro2>
34
+ ]
35
+ batch_size_per_gpu: 4
36
+ ]
37
+ ]
38
+ models:[
39
+ pretrained_path: nitrosocke/mo-di-diffusion
40
+ enable_edlora: True
41
+ finetune_cfg:[
42
+ text_embedding:[
43
+ enable_tuning: True
44
+ lr: 0.001
45
+ ]
46
+ text_encoder:[
47
+ enable_tuning: True
48
+ lora_cfg:[
49
+ rank: 5
50
+ alpha: 1.0
51
+ where: CLIPAttention
52
+ ]
53
+ lr: 1e-05
54
+ ]
55
+ unet:[
56
+ enable_tuning: True
57
+ lora_cfg:[
58
+ rank: 5
59
+ alpha: 1.0
60
+ where: Attention
61
+ ]
62
+ lr: 0.0001
63
+ ]
64
+ ]
65
+ new_concept_token: <hiro1>+<hiro2>
66
+ initializer_token: <rand-0.013>+boy
67
+ noise_offset: 0.01
68
+ attn_reg_weight: 0.01
69
+ reg_full_identity: False
70
+ use_mask_loss: True
71
+ gradient_checkpoint: False
72
+ enable_xformers: True
73
+ ]
74
+ path:[
75
+ pretrain_network: None
76
+ experiments_root: /home/ujinsong/workspace/ortha/experiments/7657_hiro_ortho
77
+ models: /home/ujinsong/workspace/ortha/experiments/7657_hiro_ortho/models
78
+ log: /home/ujinsong/workspace/ortha/experiments/7657_hiro_ortho
79
+ visualization: /home/ujinsong/workspace/ortha/experiments/7657_hiro_ortho/visualization
80
+ ]
81
+ train:[
82
+ optim_g:[
83
+ type: AdamW
84
+ lr: 0.0
85
+ weight_decay: 0.01
86
+ betas: [0.9, 0.999]
87
+ ]
88
+ unet_kv_drop_rate: 0
89
+ scheduler: linear
90
+ emb_norm_threshold: 0.55
91
+ ]
92
+ val:[
93
+ val_during_save: False
94
+ compose_visualize: False
95
+ alpha_list: [0.7, 1.0]
96
+ sample:[
97
+ num_inference_steps: 50
98
+ guidance_scale: 7.5
99
+ ]
100
+ ]
101
+ logger:[
102
+ print_freq: 10
103
+ save_checkpoint_freq: 10000.0
104
+ ]
105
+ is_train: True
106
+
107
+ 2024-06-15 14:16:09,654 INFO: <hiro1> (49408-49423) is random initialized by: <rand-0.013>
108
+ 2024-06-15 14:16:10,094 INFO: <hiro2> (49424-49439) is random initialized by existing token (boy): 1876
109
+ 2024-06-15 14:16:10,099 INFO: optimizing embedding using lr: 0.001
110
+ 2024-06-15 14:16:10,689 INFO: optimizing text_encoder (48 LoRAs), using lr: 1e-05
111
+ 2024-06-15 14:16:11,747 INFO: optimizing unet (128 LoRAs), using lr: 0.0001
112
+ 2024-06-15 14:16:13,696 INFO: ***** Running training *****
113
+ 2024-06-15 14:16:13,697 INFO: Num examples = 3000
114
+ 2024-06-15 14:16:13,698 INFO: Instantaneous batch size per device = 2
115
+ 2024-06-15 14:16:13,699 INFO: Total train batch size (w. parallel, distributed & accumulation) = 4
116
+ 2024-06-15 14:16:13,699 INFO: Total optimization steps = 750.0
117
+ 2024-06-15 14:16:38,350 INFO: [7657_..][Iter: 10, lr:(9.867e-04,9.867e-06,9.867e-05,)] [eta: 0:27:36] loss: 1.6116e+00 Norm_mean: 3.7861e-01
118
+ 2024-06-15 14:16:49,364 INFO: [7657_..][Iter: 20, lr:(9.733e-04,9.733e-06,9.733e-05,)] [eta: 0:20:38] loss: 2.5793e-01 Norm_mean: 3.9649e-01
119
+ 2024-06-15 14:17:00,266 INFO: [7657_..][Iter: 30, lr:(9.600e-04,9.600e-06,9.600e-05,)] [eta: 0:18:00] loss: 1.3888e+00 Norm_mean: 4.0891e-01
120
+ 2024-06-15 14:17:11,365 INFO: [7657_..][Iter: 40, lr:(9.467e-04,9.467e-06,9.467e-05,)] [eta: 0:16:37] loss: 1.8938e+00 Norm_mean: 4.1814e-01
121
+ 2024-06-15 14:17:22,379 INFO: [7657_..][Iter: 50, lr:(9.333e-04,9.333e-06,9.333e-05,)] [eta: 0:15:41] loss: 9.5401e-02 Norm_mean: 4.2541e-01
122
+ 2024-06-15 14:17:33,370 INFO: [7657_..][Iter: 60, lr:(9.200e-04,9.200e-06,9.200e-05,)] [eta: 0:14:59] loss: 1.9016e-01 Norm_mean: 4.3142e-01
123
+ 2024-06-15 14:17:44,435 INFO: [7657_..][Iter: 70, lr:(9.067e-04,9.067e-06,9.067e-05,)] [eta: 0:14:27] loss: 7.0708e-02 Norm_mean: 4.3790e-01
124
+ 2024-06-15 14:17:55,626 INFO: [7657_..][Iter: 80, lr:(8.933e-04,8.933e-06,8.933e-05,)] [eta: 0:14:01] loss: 1.1433e+00 Norm_mean: 4.4366e-01
125
+ 2024-06-15 14:18:06,820 INFO: [7657_..][Iter: 90, lr:(8.800e-04,8.800e-06,8.800e-05,)] [eta: 0:13:39] loss: 1.5598e-01 Norm_mean: 4.4966e-01
126
+ 2024-06-15 14:18:18,244 INFO: [7657_..][Iter: 100, lr:(8.667e-04,8.667e-06,8.667e-05,)] [eta: 0:13:20] loss: 2.6723e-01 Norm_mean: 4.5584e-01
127
+ 2024-06-15 14:18:29,561 INFO: [7657_..][Iter: 110, lr:(8.533e-04,8.533e-06,8.533e-05,)] [eta: 0:13:02] loss: 1.1469e-01 Norm_mean: 4.6182e-01
128
+ 2024-06-15 14:18:41,047 INFO: [7657_..][Iter: 120, lr:(8.400e-04,8.400e-06,8.400e-05,)] [eta: 0:12:45] loss: 2.2779e-01 Norm_mean: 4.6800e-01
129
+ 2024-06-15 14:18:52,548 INFO: [7657_..][Iter: 130, lr:(8.267e-04,8.267e-06,8.267e-05,)] [eta: 0:12:30] loss: 8.4633e-01 Norm_mean: 4.7347e-01
130
+ 2024-06-15 14:19:04,089 INFO: [7657_..][Iter: 140, lr:(8.133e-04,8.133e-06,8.133e-05,)] [eta: 0:12:15] loss: 6.4646e-01 Norm_mean: 4.7829e-01
131
+ 2024-06-15 14:19:15,488 INFO: [7657_..][Iter: 150, lr:(8.000e-04,8.000e-06,8.000e-05,)] [eta: 0:12:01] loss: 5.4089e-01 Norm_mean: 4.8252e-01
132
+ 2024-06-15 14:19:26,829 INFO: [7657_..][Iter: 160, lr:(7.867e-04,7.867e-06,7.867e-05,)] [eta: 0:11:46] loss: 3.6840e-01 Norm_mean: 4.8734e-01
133
+ 2024-06-15 14:19:38,121 INFO: [7657_..][Iter: 170, lr:(7.733e-04,7.733e-06,7.733e-05,)] [eta: 0:11:32] loss: 1.2167e+00 Norm_mean: 4.9217e-01
134
+ 2024-06-15 14:19:49,588 INFO: [7657_..][Iter: 180, lr:(7.600e-04,7.600e-06,7.600e-05,)] [eta: 0:11:18] loss: 7.4233e-01 Norm_mean: 4.9738e-01
135
+ 2024-06-15 14:20:01,001 INFO: [7657_..][Iter: 190, lr:(7.467e-04,7.467e-06,7.467e-05,)] [eta: 0:11:05] loss: 1.2643e+00 Norm_mean: 5.0184e-01
136
+ 2024-06-15 14:20:12,342 INFO: [7657_..][Iter: 200, lr:(7.333e-04,7.333e-06,7.333e-05,)] [eta: 0:10:51] loss: 1.0204e-01 Norm_mean: 5.0563e-01
137
+ 2024-06-15 14:20:23,717 INFO: [7657_..][Iter: 210, lr:(7.200e-04,7.200e-06,7.200e-05,)] [eta: 0:10:38] loss: 1.4291e+00 Norm_mean: 5.0951e-01
138
+ 2024-06-15 14:20:35,025 INFO: [7657_..][Iter: 220, lr:(7.067e-04,7.067e-06,7.067e-05,)] [eta: 0:10:25] loss: 7.4474e-01 Norm_mean: 5.1317e-01
139
+ 2024-06-15 14:20:46,443 INFO: [7657_..][Iter: 230, lr:(6.933e-04,6.933e-06,6.933e-05,)] [eta: 0:10:12] loss: 4.2296e-01 Norm_mean: 5.1613e-01
140
+ 2024-06-15 14:20:57,885 INFO: [7657_..][Iter: 240, lr:(6.800e-04,6.800e-06,6.800e-05,)] [eta: 0:10:00] loss: 4.2342e-01 Norm_mean: 5.1930e-01
141
+ 2024-06-15 14:21:09,265 INFO: [7657_..][Iter: 250, lr:(6.667e-04,6.667e-06,6.667e-05,)] [eta: 0:09:47] loss: 7.9255e-01 Norm_mean: 5.2241e-01
142
+ 2024-06-15 14:21:20,695 INFO: [7657_..][Iter: 260, lr:(6.533e-04,6.533e-06,6.533e-05,)] [eta: 0:09:35] loss: 6.6829e-01 Norm_mean: 5.2572e-01
143
+ 2024-06-15 14:21:32,058 INFO: [7657_..][Iter: 270, lr:(6.400e-04,6.400e-06,6.400e-05,)] [eta: 0:09:22] loss: 1.0494e-01 Norm_mean: 5.2925e-01
144
+ 2024-06-15 14:21:43,449 INFO: [7657_..][Iter: 280, lr:(6.267e-04,6.267e-06,6.267e-05,)] [eta: 0:09:10] loss: 7.2344e-02 Norm_mean: 5.3281e-01
145
+ 2024-06-15 14:21:54,817 INFO: [7657_..][Iter: 290, lr:(6.133e-04,6.133e-06,6.133e-05,)] [eta: 0:08:58] loss: 6.3012e-01 Norm_mean: 5.3618e-01
146
+ 2024-06-15 14:22:06,240 INFO: [7657_..][Iter: 300, lr:(6.000e-04,6.000e-06,6.000e-05,)] [eta: 0:08:45] loss: 3.3312e-01 Norm_mean: 5.3889e-01
147
+ 2024-06-15 14:22:17,686 INFO: [7657_..][Iter: 310, lr:(5.867e-04,5.867e-06,5.867e-05,)] [eta: 0:08:33] loss: 2.1530e-01 Norm_mean: 5.4141e-01
148
+ 2024-06-15 14:22:29,114 INFO: [7657_..][Iter: 320, lr:(5.733e-04,5.733e-06,5.733e-05,)] [eta: 0:08:21] loss: 2.1486e+00 Norm_mean: 5.4389e-01
149
+ 2024-06-15 14:22:40,459 INFO: [7657_..][Iter: 330, lr:(5.600e-04,5.600e-06,5.600e-05,)] [eta: 0:08:09] loss: 3.7915e-01 Norm_mean: 5.4641e-01
150
+ 2024-06-15 14:22:51,899 INFO: [7657_..][Iter: 340, lr:(5.467e-04,5.467e-06,5.467e-05,)] [eta: 0:07:57] loss: 1.9195e-01 Norm_mean: 5.4890e-01
151
+ 2024-06-15 14:23:03,299 INFO: [7657_..][Iter: 350, lr:(5.333e-04,5.333e-06,5.333e-05,)] [eta: 0:07:45] loss: 6.1196e-01 Norm_mean: 5.5015e-01
152
+ 2024-06-15 14:23:14,702 INFO: [7657_..][Iter: 360, lr:(5.200e-04,5.200e-06,5.200e-05,)] [eta: 0:07:33] loss: 2.8930e-01 Norm_mean: 5.5015e-01
153
+ 2024-06-15 14:23:26,050 INFO: [7657_..][Iter: 370, lr:(5.067e-04,5.067e-06,5.067e-05,)] [eta: 0:07:21] loss: 9.8478e-02 Norm_mean: 5.5015e-01
154
+ 2024-06-15 14:23:37,495 INFO: [7657_..][Iter: 380, lr:(4.933e-04,4.933e-06,4.933e-05,)] [eta: 0:07:09] loss: 1.3770e-01 Norm_mean: 5.5015e-01
155
+ 2024-06-15 14:23:48,735 INFO: [7657_..][Iter: 390, lr:(4.800e-04,4.800e-06,4.800e-05,)] [eta: 0:06:57] loss: 4.6333e-01 Norm_mean: 5.5015e-01
156
+ 2024-06-15 14:24:00,063 INFO: [7657_..][Iter: 400, lr:(4.667e-04,4.667e-06,4.667e-05,)] [eta: 0:06:45] loss: 4.7823e-01 Norm_mean: 5.5015e-01
157
+ 2024-06-15 14:24:11,423 INFO: [7657_..][Iter: 410, lr:(4.533e-04,4.533e-06,4.533e-05,)] [eta: 0:06:34] loss: 3.1513e-01 Norm_mean: 5.5015e-01
158
+ 2024-06-15 14:24:22,760 INFO: [7657_..][Iter: 420, lr:(4.400e-04,4.400e-06,4.400e-05,)] [eta: 0:06:22] loss: 1.1668e-01 Norm_mean: 5.5015e-01
159
+ 2024-06-15 14:24:34,089 INFO: [7657_..][Iter: 430, lr:(4.267e-04,4.267e-06,4.267e-05,)] [eta: 0:06:10] loss: 9.3610e-01 Norm_mean: 5.5015e-01
160
+ 2024-06-15 14:24:45,503 INFO: [7657_..][Iter: 440, lr:(4.133e-04,4.133e-06,4.133e-05,)] [eta: 0:05:58] loss: 1.7819e-01 Norm_mean: 5.5015e-01
161
+ 2024-06-15 14:24:56,754 INFO: [7657_..][Iter: 450, lr:(4.000e-04,4.000e-06,4.000e-05,)] [eta: 0:05:46] loss: 1.4312e+00 Norm_mean: 5.5015e-01
162
+ 2024-06-15 14:25:08,187 INFO: [7657_..][Iter: 460, lr:(3.867e-04,3.867e-06,3.867e-05,)] [eta: 0:05:35] loss: 8.0058e-01 Norm_mean: 5.5015e-01
163
+ 2024-06-15 14:25:19,590 INFO: [7657_..][Iter: 470, lr:(3.733e-04,3.733e-06,3.733e-05,)] [eta: 0:05:23] loss: 9.1174e-01 Norm_mean: 5.5015e-01
164
+ 2024-06-15 14:25:31,082 INFO: [7657_..][Iter: 480, lr:(3.600e-04,3.600e-06,3.600e-05,)] [eta: 0:05:11] loss: 1.8733e-01 Norm_mean: 5.5015e-01
165
+ 2024-06-15 14:25:42,521 INFO: [7657_..][Iter: 490, lr:(3.467e-04,3.467e-06,3.467e-05,)] [eta: 0:05:00] loss: 5.4057e-01 Norm_mean: 5.5015e-01
166
+ 2024-06-15 14:25:53,780 INFO: [7657_..][Iter: 500, lr:(3.333e-04,3.333e-06,3.333e-05,)] [eta: 0:04:48] loss: 1.6629e+00 Norm_mean: 5.5015e-01
167
+ 2024-06-15 14:26:05,275 INFO: [7657_..][Iter: 510, lr:(3.200e-04,3.200e-06,3.200e-05,)] [eta: 0:04:36] loss: 5.0319e-02 Norm_mean: 5.5015e-01
168
+ 2024-06-15 14:26:16,563 INFO: [7657_..][Iter: 520, lr:(3.067e-04,3.067e-06,3.067e-05,)] [eta: 0:04:24] loss: 3.0301e-02 Norm_mean: 5.5015e-01
169
+ 2024-06-15 14:26:27,894 INFO: [7657_..][Iter: 530, lr:(2.933e-04,2.933e-06,2.933e-05,)] [eta: 0:04:13] loss: 1.3825e-01 Norm_mean: 5.5015e-01
170
+ 2024-06-15 14:26:39,210 INFO: [7657_..][Iter: 540, lr:(2.800e-04,2.800e-06,2.800e-05,)] [eta: 0:04:01] loss: 3.7438e-01 Norm_mean: 5.5015e-01
171
+ 2024-06-15 14:26:50,580 INFO: [7657_..][Iter: 550, lr:(2.667e-04,2.667e-06,2.667e-05,)] [eta: 0:03:50] loss: 1.4058e+00 Norm_mean: 5.5015e-01
172
+ 2024-06-15 14:27:01,933 INFO: [7657_..][Iter: 560, lr:(2.533e-04,2.533e-06,2.533e-05,)] [eta: 0:03:38] loss: 1.9322e-02 Norm_mean: 5.5015e-01
173
+ 2024-06-15 14:27:13,321 INFO: [7657_..][Iter: 570, lr:(2.400e-04,2.400e-06,2.400e-05,)] [eta: 0:03:26] loss: 9.2310e-01 Norm_mean: 5.5015e-01
174
+ 2024-06-15 14:27:24,689 INFO: [7657_..][Iter: 580, lr:(2.267e-04,2.267e-06,2.267e-05,)] [eta: 0:03:15] loss: 7.9236e-01 Norm_mean: 5.5015e-01
175
+ 2024-06-15 14:27:36,027 INFO: [7657_..][Iter: 590, lr:(2.133e-04,2.133e-06,2.133e-05,)] [eta: 0:03:03] loss: 7.0721e-01 Norm_mean: 5.5015e-01
176
+ 2024-06-15 14:27:47,574 INFO: [7657_..][Iter: 600, lr:(2.000e-04,2.000e-06,2.000e-05,)] [eta: 0:02:52] loss: 1.1411e+00 Norm_mean: 5.5015e-01
177
+ 2024-06-15 14:27:58,970 INFO: [7657_..][Iter: 610, lr:(1.867e-04,1.867e-06,1.867e-05,)] [eta: 0:02:40] loss: 5.3371e-01 Norm_mean: 5.5015e-01
178
+ 2024-06-15 14:28:10,366 INFO: [7657_..][Iter: 620, lr:(1.733e-04,1.733e-06,1.733e-05,)] [eta: 0:02:28] loss: 3.4431e-01 Norm_mean: 5.5015e-01
179
+ 2024-06-15 14:28:21,642 INFO: [7657_..][Iter: 630, lr:(1.600e-04,1.600e-06,1.600e-05,)] [eta: 0:02:17] loss: 1.0685e+00 Norm_mean: 5.5015e-01
180
+ 2024-06-15 14:28:33,111 INFO: [7657_..][Iter: 640, lr:(1.467e-04,1.467e-06,1.467e-05,)] [eta: 0:02:05] loss: 8.5425e-01 Norm_mean: 5.5015e-01
181
+ 2024-06-15 14:28:44,430 INFO: [7657_..][Iter: 650, lr:(1.333e-04,1.333e-06,1.333e-05,)] [eta: 0:01:54] loss: 1.5162e+00 Norm_mean: 5.5015e-01
182
+ 2024-06-15 14:28:55,957 INFO: [7657_..][Iter: 660, lr:(1.200e-04,1.200e-06,1.200e-05,)] [eta: 0:01:42] loss: 4.4842e-01 Norm_mean: 5.5015e-01
183
+ 2024-06-15 14:29:07,498 INFO: [7657_..][Iter: 670, lr:(1.067e-04,1.067e-06,1.067e-05,)] [eta: 0:01:31] loss: 4.5805e-01 Norm_mean: 5.5015e-01
184
+ 2024-06-15 14:29:19,095 INFO: [7657_..][Iter: 680, lr:(9.333e-05,9.333e-07,9.333e-06,)] [eta: 0:01:19] loss: 9.1257e-01 Norm_mean: 5.5015e-01
185
+ 2024-06-15 14:29:30,589 INFO: [7657_..][Iter: 690, lr:(8.000e-05,8.000e-07,8.000e-06,)] [eta: 0:01:08] loss: 4.7350e-01 Norm_mean: 5.5015e-01
186
+ 2024-06-15 14:29:42,028 INFO: [7657_..][Iter: 700, lr:(6.667e-05,6.667e-07,6.667e-06,)] [eta: 0:00:56] loss: 1.9886e-01 Norm_mean: 5.5015e-01
187
+ 2024-06-15 14:29:53,409 INFO: [7657_..][Iter: 710, lr:(5.333e-05,5.333e-07,5.333e-06,)] [eta: 0:00:44] loss: 3.6839e-02 Norm_mean: 5.5015e-01
188
+ 2024-06-15 14:30:04,787 INFO: [7657_..][Iter: 720, lr:(4.000e-05,4.000e-07,4.000e-06,)] [eta: 0:00:33] loss: 6.6934e-01 Norm_mean: 5.5015e-01
189
+ 2024-06-15 14:30:16,287 INFO: [7657_..][Iter: 730, lr:(2.667e-05,2.667e-07,2.667e-06,)] [eta: 0:00:21] loss: 4.9187e-01 Norm_mean: 5.5015e-01
190
+ 2024-06-15 14:30:27,697 INFO: [7657_..][Iter: 740, lr:(1.333e-05,1.333e-07,1.333e-06,)] [eta: 0:00:10] loss: 9.9657e-01 Norm_mean: 5.5015e-01
191
+ 2024-06-15 14:30:38,903 INFO: [7657_..][Iter: 750, lr:(0.000e+00,0.000e+00,0.000e+00,)] [eta: -1 day, 23:59:59] loss: 9.9407e-01 Norm_mean: 5.5015e-01
192
+ 2024-06-15 14:30:39,027 INFO: Save state to /home/ujinsong/workspace/ortha/experiments/7657_hiro_ortho/models/edlora_model-latest.pth