Hugging Face's logo

llama-duo
/
gemma2b-summarize-claude3sonnet-8k

PEFT
TensorBoard
Safetensors
gemma
alignment-handbook
trl
sft
Generated from Trainer
4-bit precision
bitsandbytes
Model card Files Metrics Community
chansung commited on
Commit
28242eb
•
1 Parent(s): d8fce52

Model save

Browse files
Files changed (6) hide show
  1. README.md +78 -0
  2. adapter_model.safetensors +1 -1
  3. all_results.json +9 -0
  4. runs/Jun05_13-46-32_user-HP-Z8-Fury-G5-Workstation-Desktop-PC/events.out.tfevents.1717562811.user-HP-Z8-Fury-G5-Workstation-Desktop-PC.934553.0 +2 -2
  5. train_results.json +9 -0
  6. trainer_state.json +467 -0
README.md ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: gemma
3
+ library_name: peft
4
+ tags:
5
+ - trl
6
+ - sft
7
+ - generated_from_trainer
8
+ base_model: google/gemma-2b
9
+ datasets:
10
+ - generator
11
+ model-index:
12
+ - name: gemma2b-summarize-claude3sonnet-8k
13
+ results: []
14
+ ---
15
+
16
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
17
+ should probably proofread and complete it, then remove this comment. -->
18
+
19
+ # gemma2b-summarize-claude3sonnet-8k
20
+
21
+ This model is a fine-tuned version of [google/gemma-2b](https://huggingface.co/google/gemma-2b) on the generator dataset.
22
+ It achieves the following results on the evaluation set:
23
+ - Loss: 2.5421
24
+
25
+ ## Model description
26
+
27
+ More information needed
28
+
29
+ ## Intended uses & limitations
30
+
31
+ More information needed
32
+
33
+ ## Training and evaluation data
34
+
35
+ More information needed
36
+
37
+ ## Training procedure
38
+
39
+ ### Training hyperparameters
40
+
41
+ The following hyperparameters were used during training:
42
+ - learning_rate: 0.0002
43
+ - train_batch_size: 8
44
+ - eval_batch_size: 8
45
+ - seed: 42
46
+ - distributed_type: multi-GPU
47
+ - num_devices: 3
48
+ - gradient_accumulation_steps: 2
49
+ - total_train_batch_size: 48
50
+ - total_eval_batch_size: 24
51
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
52
+ - lr_scheduler_type: cosine
53
+ - lr_scheduler_warmup_ratio: 0.1
54
+ - num_epochs: 10
55
+
56
+ ### Training results
57
+
58
+ | Training Loss | Epoch | Step | Validation Loss |
59
+ |:-------------:|:------:|:----:|:---------------:|
60
+ | 1.9711 | 0.9804 | 25 | 2.6222 |
61
+ | 1.3473 | 2.0 | 51 | 2.5571 |
62
+ | 1.1845 | 2.9804 | 76 | 2.5050 |
63
+ | 1.1242 | 4.0 | 102 | 2.5089 |
64
+ | 1.0937 | 4.9804 | 127 | 2.5239 |
65
+ | 1.0632 | 6.0 | 153 | 2.5275 |
66
+ | 1.0377 | 6.9804 | 178 | 2.5316 |
67
+ | 1.0278 | 8.0 | 204 | 2.5399 |
68
+ | 1.0176 | 8.9804 | 229 | 2.5422 |
69
+ | 1.007 | 9.8039 | 250 | 2.5421 |
70
+
71
+
72
+ ### Framework versions
73
+
74
+ - PEFT 0.10.0
75
+ - Transformers 4.40.0
76
+ - Pytorch 2.2.2+cu121
77
+ - Datasets 2.19.1
78
+ - Tokenizers 0.19.1
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f5e8124fc68a5602062a7ece20ba5feb2766fcc60976de0e6e6db446b789c3ea
3
  size 19644912
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:72d03b3922e972c9808fe8730450679c944cc89074324d1c595594d0a4e5a47d
3
  size 19644912
all_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 9.803921568627452,
3
+ "total_flos": 1.4684333472638566e+17,
4
+ "train_loss": 1.2576467142105103,
5
+ "train_runtime": 862.0118,
6
+ "train_samples": 7936,
7
+ "train_samples_per_second": 14.13,
8
+ "train_steps_per_second": 0.29
9
+ }
runs/Jun05_13-46-32_user-HP-Z8-Fury-G5-Workstation-Desktop-PC/events.out.tfevents.1717562811.user-HP-Z8-Fury-G5-Workstation-Desktop-PC.934553.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6b47a8f9360d6724d8e817537047504a6600e4b4698cbd471696bc64bbe7a9ba
3
- size 15891
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a4ec787fd5823d0a557622d2fb9fe67583e164c58d61cef7372e4cd8ffd999b2
3
+ size 19168
train_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 9.803921568627452,
3
+ "total_flos": 1.4684333472638566e+17,
4
+ "train_loss": 1.2576467142105103,
5
+ "train_runtime": 862.0118,
6
+ "train_samples": 7936,
7
+ "train_samples_per_second": 14.13,
8
+ "train_steps_per_second": 0.29
9
+ }
trainer_state.json ADDED
@@ -0,0 +1,467 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 9.803921568627452,
5
+ "eval_steps": 500,
6
+ "global_step": 250,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0392156862745098,
13
+ "grad_norm": 3.171875,
14
+ "learning_rate": 8.000000000000001e-06,
15
+ "loss": 2.8106,
16
+ "step": 1
17
+ },
18
+ {
19
+ "epoch": 0.19607843137254902,
20
+ "grad_norm": 2.046875,
21
+ "learning_rate": 4e-05,
22
+ "loss": 2.7854,
23
+ "step": 5
24
+ },
25
+ {
26
+ "epoch": 0.39215686274509803,
27
+ "grad_norm": 6.34375,
28
+ "learning_rate": 8e-05,
29
+ "loss": 2.7197,
30
+ "step": 10
31
+ },
32
+ {
33
+ "epoch": 0.5882352941176471,
34
+ "grad_norm": 1.7734375,
35
+ "learning_rate": 0.00012,
36
+ "loss": 2.5034,
37
+ "step": 15
38
+ },
39
+ {
40
+ "epoch": 0.7843137254901961,
41
+ "grad_norm": 2.5625,
42
+ "learning_rate": 0.00016,
43
+ "loss": 2.1943,
44
+ "step": 20
45
+ },
46
+ {
47
+ "epoch": 0.9803921568627451,
48
+ "grad_norm": 1.5234375,
49
+ "learning_rate": 0.0002,
50
+ "loss": 1.9711,
51
+ "step": 25
52
+ },
53
+ {
54
+ "epoch": 0.9803921568627451,
55
+ "eval_loss": 2.6221704483032227,
56
+ "eval_runtime": 0.6383,
57
+ "eval_samples_per_second": 15.667,
58
+ "eval_steps_per_second": 1.567,
59
+ "step": 25
60
+ },
61
+ {
62
+ "epoch": 1.1764705882352942,
63
+ "grad_norm": 1.9140625,
64
+ "learning_rate": 0.00019975640502598244,
65
+ "loss": 1.765,
66
+ "step": 30
67
+ },
68
+ {
69
+ "epoch": 1.3725490196078431,
70
+ "grad_norm": 0.7109375,
71
+ "learning_rate": 0.00019902680687415705,
72
+ "loss": 1.5781,
73
+ "step": 35
74
+ },
75
+ {
76
+ "epoch": 1.5686274509803921,
77
+ "grad_norm": 0.474609375,
78
+ "learning_rate": 0.00019781476007338058,
79
+ "loss": 1.4939,
80
+ "step": 40
81
+ },
82
+ {
83
+ "epoch": 1.7647058823529411,
84
+ "grad_norm": 0.7734375,
85
+ "learning_rate": 0.0001961261695938319,
86
+ "loss": 1.4032,
87
+ "step": 45
88
+ },
89
+ {
90
+ "epoch": 1.9607843137254903,
91
+ "grad_norm": 0.51171875,
92
+ "learning_rate": 0.00019396926207859084,
93
+ "loss": 1.3473,
94
+ "step": 50
95
+ },
96
+ {
97
+ "epoch": 2.0,
98
+ "eval_loss": 2.5571093559265137,
99
+ "eval_runtime": 0.5378,
100
+ "eval_samples_per_second": 18.595,
101
+ "eval_steps_per_second": 1.86,
102
+ "step": 51
103
+ },
104
+ {
105
+ "epoch": 2.156862745098039,
106
+ "grad_norm": 0.462890625,
107
+ "learning_rate": 0.0001913545457642601,
108
+ "loss": 1.2973,
109
+ "step": 55
110
+ },
111
+ {
112
+ "epoch": 2.3529411764705883,
113
+ "grad_norm": 0.3671875,
114
+ "learning_rate": 0.00018829475928589271,
115
+ "loss": 1.2645,
116
+ "step": 60
117
+ },
118
+ {
119
+ "epoch": 2.549019607843137,
120
+ "grad_norm": 0.345703125,
121
+ "learning_rate": 0.0001848048096156426,
122
+ "loss": 1.2291,
123
+ "step": 65
124
+ },
125
+ {
126
+ "epoch": 2.7450980392156863,
127
+ "grad_norm": 0.5859375,
128
+ "learning_rate": 0.00018090169943749476,
129
+ "loss": 1.2081,
130
+ "step": 70
131
+ },
132
+ {
133
+ "epoch": 2.9411764705882355,
134
+ "grad_norm": 0.427734375,
135
+ "learning_rate": 0.0001766044443118978,
136
+ "loss": 1.1845,
137
+ "step": 75
138
+ },
139
+ {
140
+ "epoch": 2.980392156862745,
141
+ "eval_loss": 2.5049567222595215,
142
+ "eval_runtime": 0.6655,
143
+ "eval_samples_per_second": 15.026,
144
+ "eval_steps_per_second": 1.503,
145
+ "step": 76
146
+ },
147
+ {
148
+ "epoch": 3.1372549019607843,
149
+ "grad_norm": 0.43359375,
150
+ "learning_rate": 0.0001719339800338651,
151
+ "loss": 1.1781,
152
+ "step": 80
153
+ },
154
+ {
155
+ "epoch": 3.3333333333333335,
156
+ "grad_norm": 0.48828125,
157
+ "learning_rate": 0.00016691306063588583,
158
+ "loss": 1.142,
159
+ "step": 85
160
+ },
161
+ {
162
+ "epoch": 3.5294117647058822,
163
+ "grad_norm": 0.35546875,
164
+ "learning_rate": 0.0001615661475325658,
165
+ "loss": 1.1494,
166
+ "step": 90
167
+ },
168
+ {
169
+ "epoch": 3.7254901960784315,
170
+ "grad_norm": 0.33203125,
171
+ "learning_rate": 0.0001559192903470747,
172
+ "loss": 1.1512,
173
+ "step": 95
174
+ },
175
+ {
176
+ "epoch": 3.9215686274509802,
177
+ "grad_norm": 0.431640625,
178
+ "learning_rate": 0.00015000000000000001,
179
+ "loss": 1.1242,
180
+ "step": 100
181
+ },
182
+ {
183
+ "epoch": 4.0,
184
+ "eval_loss": 2.5088963508605957,
185
+ "eval_runtime": 0.5376,
186
+ "eval_samples_per_second": 18.602,
187
+ "eval_steps_per_second": 1.86,
188
+ "step": 102
189
+ },
190
+ {
191
+ "epoch": 4.117647058823529,
192
+ "grad_norm": 0.390625,
193
+ "learning_rate": 0.00014383711467890774,
194
+ "loss": 1.1068,
195
+ "step": 105
196
+ },
197
+ {
198
+ "epoch": 4.313725490196078,
199
+ "grad_norm": 0.470703125,
200
+ "learning_rate": 0.00013746065934159123,
201
+ "loss": 1.0975,
202
+ "step": 110
203
+ },
204
+ {
205
+ "epoch": 4.509803921568627,
206
+ "grad_norm": 0.52734375,
207
+ "learning_rate": 0.00013090169943749476,
208
+ "loss": 1.0827,
209
+ "step": 115
210
+ },
211
+ {
212
+ "epoch": 4.705882352941177,
213
+ "grad_norm": 0.435546875,
214
+ "learning_rate": 0.00012419218955996676,
215
+ "loss": 1.0938,
216
+ "step": 120
217
+ },
218
+ {
219
+ "epoch": 4.901960784313726,
220
+ "grad_norm": 0.494140625,
221
+ "learning_rate": 0.00011736481776669306,
222
+ "loss": 1.0937,
223
+ "step": 125
224
+ },
225
+ {
226
+ "epoch": 4.980392156862745,
227
+ "eval_loss": 2.523869276046753,
228
+ "eval_runtime": 0.6835,
229
+ "eval_samples_per_second": 14.632,
230
+ "eval_steps_per_second": 1.463,
231
+ "step": 127
232
+ },
233
+ {
234
+ "epoch": 5.098039215686274,
235
+ "grad_norm": 0.376953125,
236
+ "learning_rate": 0.00011045284632676536,
237
+ "loss": 1.0747,
238
+ "step": 130
239
+ },
240
+ {
241
+ "epoch": 5.294117647058823,
242
+ "grad_norm": 0.39453125,
243
+ "learning_rate": 0.00010348994967025012,
244
+ "loss": 1.0627,
245
+ "step": 135
246
+ },
247
+ {
248
+ "epoch": 5.490196078431373,
249
+ "grad_norm": 0.453125,
250
+ "learning_rate": 9.651005032974994e-05,
251
+ "loss": 1.0485,
252
+ "step": 140
253
+ },
254
+ {
255
+ "epoch": 5.686274509803922,
256
+ "grad_norm": 0.470703125,
257
+ "learning_rate": 8.954715367323468e-05,
258
+ "loss": 1.0543,
259
+ "step": 145
260
+ },
261
+ {
262
+ "epoch": 5.882352941176471,
263
+ "grad_norm": 0.431640625,
264
+ "learning_rate": 8.263518223330697e-05,
265
+ "loss": 1.0632,
266
+ "step": 150
267
+ },
268
+ {
269
+ "epoch": 6.0,
270
+ "eval_loss": 2.5274596214294434,
271
+ "eval_runtime": 0.5589,
272
+ "eval_samples_per_second": 17.893,
273
+ "eval_steps_per_second": 1.789,
274
+ "step": 153
275
+ },
276
+ {
277
+ "epoch": 6.078431372549019,
278
+ "grad_norm": 0.55078125,
279
+ "learning_rate": 7.580781044003324e-05,
280
+ "loss": 1.0488,
281
+ "step": 155
282
+ },
283
+ {
284
+ "epoch": 6.2745098039215685,
285
+ "grad_norm": 0.55078125,
286
+ "learning_rate": 6.909830056250527e-05,
287
+ "loss": 1.0318,
288
+ "step": 160
289
+ },
290
+ {
291
+ "epoch": 6.470588235294118,
292
+ "grad_norm": 0.423828125,
293
+ "learning_rate": 6.25393406584088e-05,
294
+ "loss": 1.027,
295
+ "step": 165
296
+ },
297
+ {
298
+ "epoch": 6.666666666666667,
299
+ "grad_norm": 0.5078125,
300
+ "learning_rate": 5.616288532109225e-05,
301
+ "loss": 1.0391,
302
+ "step": 170
303
+ },
304
+ {
305
+ "epoch": 6.862745098039216,
306
+ "grad_norm": 0.412109375,
307
+ "learning_rate": 5.000000000000002e-05,
308
+ "loss": 1.0377,
309
+ "step": 175
310
+ },
311
+ {
312
+ "epoch": 6.980392156862745,
313
+ "eval_loss": 2.5316176414489746,
314
+ "eval_runtime": 0.6547,
315
+ "eval_samples_per_second": 15.274,
316
+ "eval_steps_per_second": 1.527,
317
+ "step": 178
318
+ },
319
+ {
320
+ "epoch": 7.0588235294117645,
321
+ "grad_norm": 0.40625,
322
+ "learning_rate": 4.4080709652925336e-05,
323
+ "loss": 1.0217,
324
+ "step": 180
325
+ },
326
+ {
327
+ "epoch": 7.254901960784314,
328
+ "grad_norm": 0.462890625,
329
+ "learning_rate": 3.843385246743417e-05,
330
+ "loss": 1.0203,
331
+ "step": 185
332
+ },
333
+ {
334
+ "epoch": 7.450980392156863,
335
+ "grad_norm": 0.384765625,
336
+ "learning_rate": 3.308693936411421e-05,
337
+ "loss": 1.0188,
338
+ "step": 190
339
+ },
340
+ {
341
+ "epoch": 7.647058823529412,
342
+ "grad_norm": 0.396484375,
343
+ "learning_rate": 2.8066019966134904e-05,
344
+ "loss": 1.0165,
345
+ "step": 195
346
+ },
347
+ {
348
+ "epoch": 7.8431372549019605,
349
+ "grad_norm": 0.390625,
350
+ "learning_rate": 2.339555568810221e-05,
351
+ "loss": 1.0278,
352
+ "step": 200
353
+ },
354
+ {
355
+ "epoch": 8.0,
356
+ "eval_loss": 2.53991961479187,
357
+ "eval_runtime": 0.5393,
358
+ "eval_samples_per_second": 18.541,
359
+ "eval_steps_per_second": 1.854,
360
+ "step": 204
361
+ },
362
+ {
363
+ "epoch": 8.03921568627451,
364
+ "grad_norm": 0.40234375,
365
+ "learning_rate": 1.9098300562505266e-05,
366
+ "loss": 1.0104,
367
+ "step": 205
368
+ },
369
+ {
370
+ "epoch": 8.235294117647058,
371
+ "grad_norm": 0.41015625,
372
+ "learning_rate": 1.5195190384357404e-05,
373
+ "loss": 1.0056,
374
+ "step": 210
375
+ },
376
+ {
377
+ "epoch": 8.431372549019608,
378
+ "grad_norm": 0.392578125,
379
+ "learning_rate": 1.1705240714107302e-05,
380
+ "loss": 1.0218,
381
+ "step": 215
382
+ },
383
+ {
384
+ "epoch": 8.627450980392156,
385
+ "grad_norm": 0.421875,
386
+ "learning_rate": 8.645454235739903e-06,
387
+ "loss": 1.0136,
388
+ "step": 220
389
+ },
390
+ {
391
+ "epoch": 8.823529411764707,
392
+ "grad_norm": 0.365234375,
393
+ "learning_rate": 6.030737921409169e-06,
394
+ "loss": 1.0176,
395
+ "step": 225
396
+ },
397
+ {
398
+ "epoch": 8.980392156862745,
399
+ "eval_loss": 2.542208671569824,
400
+ "eval_runtime": 0.6862,
401
+ "eval_samples_per_second": 14.574,
402
+ "eval_steps_per_second": 1.457,
403
+ "step": 229
404
+ },
405
+ {
406
+ "epoch": 9.019607843137255,
407
+ "grad_norm": 0.3671875,
408
+ "learning_rate": 3.873830406168111e-06,
409
+ "loss": 1.0092,
410
+ "step": 230
411
+ },
412
+ {
413
+ "epoch": 9.215686274509803,
414
+ "grad_norm": 0.384765625,
415
+ "learning_rate": 2.1852399266194314e-06,
416
+ "loss": 1.0084,
417
+ "step": 235
418
+ },
419
+ {
420
+ "epoch": 9.411764705882353,
421
+ "grad_norm": 0.34765625,
422
+ "learning_rate": 9.731931258429638e-07,
423
+ "loss": 1.0208,
424
+ "step": 240
425
+ },
426
+ {
427
+ "epoch": 9.607843137254902,
428
+ "grad_norm": 0.388671875,
429
+ "learning_rate": 2.4359497401758024e-07,
430
+ "loss": 1.0059,
431
+ "step": 245
432
+ },
433
+ {
434
+ "epoch": 9.803921568627452,
435
+ "grad_norm": 0.435546875,
436
+ "learning_rate": 0.0,
437
+ "loss": 1.007,
438
+ "step": 250
439
+ },
440
+ {
441
+ "epoch": 9.803921568627452,
442
+ "eval_loss": 2.5420966148376465,
443
+ "eval_runtime": 0.5393,
444
+ "eval_samples_per_second": 18.543,
445
+ "eval_steps_per_second": 1.854,
446
+ "step": 250
447
+ },
448
+ {
449
+ "epoch": 9.803921568627452,
450
+ "step": 250,
451
+ "total_flos": 1.4684333472638566e+17,
452
+ "train_loss": 1.2576467142105103,
453
+ "train_runtime": 862.0118,
454
+ "train_samples_per_second": 14.13,
455
+ "train_steps_per_second": 0.29
456
+ }
457
+ ],
458
+ "logging_steps": 5,
459
+ "max_steps": 250,
460
+ "num_input_tokens_seen": 0,
461
+ "num_train_epochs": 10,
462
+ "save_steps": 100,
463
+ "total_flos": 1.4684333472638566e+17,
464
+ "train_batch_size": 8,
465
+ "trial_name": null,
466
+ "trial_params": null
467
+ }