AIFunOver commited on
Commit
2ba6e26
1 Parent(s): 5aba0d2

Upload openvino_tokenizer.xml with huggingface_hub

Browse files
Files changed (1) hide show
  1. openvino_tokenizer.xml +902 -0
openvino_tokenizer.xml ADDED
@@ -0,0 +1,902 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <?xml version="1.0"?>
2
+ <net name="tokenizer" version="11">
3
+ <layers>
4
+ <layer id="0" name="Parameter_306401" type="Parameter" version="opset1">
5
+ <data shape="?" element_type="string" />
6
+ <output>
7
+ <port id="0" precision="STRING" names="Parameter_306401">
8
+ <dim>-1</dim>
9
+ </port>
10
+ </output>
11
+ </layer>
12
+ <layer id="1" name="Constant_306500" type="Const" version="opset1">
13
+ <data element_type="i32" shape="" offset="0" size="4" />
14
+ <output>
15
+ <port id="0" precision="I32" />
16
+ </output>
17
+ </layer>
18
+ <layer id="2" name="Constant_306501" type="Const" version="opset1">
19
+ <data element_type="i32" shape="" offset="4" size="4" />
20
+ <output>
21
+ <port id="0" precision="I32" />
22
+ </output>
23
+ </layer>
24
+ <layer id="3" name="Constant_306502" type="Const" version="opset1">
25
+ <data element_type="i32" shape="1" offset="8" size="4" />
26
+ <output>
27
+ <port id="0" precision="I32">
28
+ <dim>1</dim>
29
+ </port>
30
+ </output>
31
+ </layer>
32
+ <layer id="4" name="Constant_306503" type="Const" version="opset1">
33
+ <data element_type="i32" shape="" offset="0" size="4" />
34
+ <output>
35
+ <port id="0" precision="I32" />
36
+ </output>
37
+ </layer>
38
+ <layer id="5" name="Constant_306504" type="Const" version="opset1">
39
+ <data element_type="i32" shape="" offset="4" size="4" />
40
+ <output>
41
+ <port id="0" precision="I32" />
42
+ </output>
43
+ </layer>
44
+ <layer id="6" name="Constant_306505" type="Const" version="opset1">
45
+ <data element_type="i32" shape="1" offset="12" size="4" />
46
+ <output>
47
+ <port id="0" precision="I32">
48
+ <dim>1</dim>
49
+ </port>
50
+ </output>
51
+ </layer>
52
+ <layer id="7" name="Constant_306407" type="Const" version="opset1">
53
+ <data element_type="i64" shape="" offset="16" size="8" />
54
+ <output>
55
+ <port id="0" precision="I64" />
56
+ </output>
57
+ </layer>
58
+ <layer id="8" name="StringTensorUnpack_306402" type="StringTensorUnpack" version="extension">
59
+ <data mode="begins_ends" />
60
+ <input>
61
+ <port id="0" precision="STRING">
62
+ <dim>-1</dim>
63
+ </port>
64
+ </input>
65
+ <output>
66
+ <port id="1" precision="I32">
67
+ <dim>-1</dim>
68
+ </port>
69
+ <port id="2" precision="I32">
70
+ <dim>-1</dim>
71
+ </port>
72
+ <port id="3" precision="U8">
73
+ <dim>-1</dim>
74
+ </port>
75
+ </output>
76
+ </layer>
77
+ <layer id="9" name="ShapeOf_306403" type="ShapeOf" version="opset3">
78
+ <data output_type="i64" />
79
+ <input>
80
+ <port id="0" precision="I32">
81
+ <dim>-1</dim>
82
+ </port>
83
+ </input>
84
+ <output>
85
+ <port id="1" precision="I64">
86
+ <dim>1</dim>
87
+ </port>
88
+ </output>
89
+ </layer>
90
+ <layer id="10" name="Constant_306404" type="Const" version="opset1">
91
+ <data element_type="i64" shape="" offset="16" size="8" />
92
+ <output>
93
+ <port id="0" precision="I64" />
94
+ </output>
95
+ </layer>
96
+ <layer id="11" name="Constant_306405" type="Const" version="opset1">
97
+ <data element_type="i64" shape="" offset="16" size="8" />
98
+ <output>
99
+ <port id="0" precision="I64" />
100
+ </output>
101
+ </layer>
102
+ <layer id="12" name="Gather_306406" type="Gather" version="opset8">
103
+ <data batch_dims="0" />
104
+ <input>
105
+ <port id="0" precision="I64">
106
+ <dim>1</dim>
107
+ </port>
108
+ <port id="1" precision="I64" />
109
+ <port id="2" precision="I64" />
110
+ </input>
111
+ <output>
112
+ <port id="3" precision="I64" />
113
+ </output>
114
+ </layer>
115
+ <layer id="13" name="Constant_306408" type="Const" version="opset1">
116
+ <data element_type="i64" shape="" offset="24" size="8" />
117
+ <output>
118
+ <port id="0" precision="I64" />
119
+ </output>
120
+ </layer>
121
+ <layer id="14" name="Range_306409" type="Range" version="opset4">
122
+ <data output_type="i32" />
123
+ <input>
124
+ <port id="0" precision="I64" />
125
+ <port id="1" precision="I64" />
126
+ <port id="2" precision="I64" />
127
+ </input>
128
+ <output>
129
+ <port id="3" precision="I32">
130
+ <dim>-1</dim>
131
+ </port>
132
+ </output>
133
+ </layer>
134
+ <layer id="15" name="Constant_306410" type="Const" version="opset1">
135
+ <data element_type="i64" shape="" offset="24" size="8" />
136
+ <output>
137
+ <port id="0" precision="I64" />
138
+ </output>
139
+ </layer>
140
+ <layer id="16" name="Constant_306411" type="Const" version="opset1">
141
+ <data element_type="i64" shape="" offset="24" size="8" />
142
+ <output>
143
+ <port id="0" precision="I64" />
144
+ </output>
145
+ </layer>
146
+ <layer id="17" name="Add_306412" type="Add" version="opset1">
147
+ <data auto_broadcast="numpy" />
148
+ <input>
149
+ <port id="0" precision="I64" />
150
+ <port id="1" precision="I64" />
151
+ </input>
152
+ <output>
153
+ <port id="2" precision="I64" />
154
+ </output>
155
+ </layer>
156
+ <layer id="18" name="Constant_306413" type="Const" version="opset1">
157
+ <data element_type="i64" shape="" offset="24" size="8" />
158
+ <output>
159
+ <port id="0" precision="I64" />
160
+ </output>
161
+ </layer>
162
+ <layer id="19" name="Range_306414" type="Range" version="opset4">
163
+ <data output_type="i32" />
164
+ <input>
165
+ <port id="0" precision="I64" />
166
+ <port id="1" precision="I64" />
167
+ <port id="2" precision="I64" />
168
+ </input>
169
+ <output>
170
+ <port id="3" precision="I32">
171
+ <dim>-1</dim>
172
+ </port>
173
+ </output>
174
+ </layer>
175
+ <layer id="20" name="Constant_306476" type="Const" version="opset1">
176
+ <data element_type="u8" shape="244" offset="32" size="244" />
177
+ <output>
178
+ <port id="0" precision="U8">
179
+ <dim>244</dim>
180
+ </port>
181
+ </output>
182
+ </layer>
183
+ <layer id="21" name="SpecialTokensSplit_306477" type="SpecialTokensSplit" version="extension">
184
+ <input>
185
+ <port id="0" precision="I32">
186
+ <dim>-1</dim>
187
+ </port>
188
+ <port id="1" precision="I32">
189
+ <dim>-1</dim>
190
+ </port>
191
+ <port id="2" precision="I32">
192
+ <dim>-1</dim>
193
+ </port>
194
+ <port id="3" precision="I32">
195
+ <dim>-1</dim>
196
+ </port>
197
+ <port id="4" precision="U8">
198
+ <dim>-1</dim>
199
+ </port>
200
+ <port id="5" precision="U8">
201
+ <dim>244</dim>
202
+ </port>
203
+ </input>
204
+ <output>
205
+ <port id="6" precision="I32">
206
+ <dim>-1</dim>
207
+ </port>
208
+ <port id="7" precision="I32">
209
+ <dim>-1</dim>
210
+ </port>
211
+ <port id="8" precision="I32">
212
+ <dim>-1</dim>
213
+ </port>
214
+ <port id="9" precision="I32">
215
+ <dim>-1</dim>
216
+ </port>
217
+ <port id="10" precision="U8">
218
+ <dim>-1</dim>
219
+ </port>
220
+ <port id="11" precision="BOOL">
221
+ <dim>-1</dim>
222
+ </port>
223
+ </output>
224
+ </layer>
225
+ <layer id="22" name="NormalizeUnicode_306478" type="NormalizeUnicode" version="extension">
226
+ <data normalization_form="NFC" />
227
+ <input>
228
+ <port id="0" precision="I32">
229
+ <dim>-1</dim>
230
+ </port>
231
+ <port id="1" precision="I32">
232
+ <dim>-1</dim>
233
+ </port>
234
+ <port id="2" precision="U8">
235
+ <dim>-1</dim>
236
+ </port>
237
+ <port id="3" precision="BOOL">
238
+ <dim>-1</dim>
239
+ </port>
240
+ </input>
241
+ <output>
242
+ <port id="4" precision="I32">
243
+ <dim>-1</dim>
244
+ </port>
245
+ <port id="5" precision="I32">
246
+ <dim>-1</dim>
247
+ </port>
248
+ <port id="6" precision="U8">
249
+ <dim>-1</dim>
250
+ </port>
251
+ <port id="7" precision="BOOL">
252
+ <dim>-1</dim>
253
+ </port>
254
+ </output>
255
+ </layer>
256
+ <layer id="23" name="Constant_306480" type="Const" version="opset1">
257
+ <data element_type="u8" shape="115" offset="276" size="115" />
258
+ <output>
259
+ <port id="0" precision="U8">
260
+ <dim>115</dim>
261
+ </port>
262
+ </output>
263
+ </layer>
264
+ <layer id="24" name="RegexSplit_306481" type="RegexSplit" version="extension">
265
+ <data behaviour="contiguous" invert="false" max_splits="-1" />
266
+ <input>
267
+ <port id="0" precision="I32">
268
+ <dim>-1</dim>
269
+ </port>
270
+ <port id="1" precision="I32">
271
+ <dim>-1</dim>
272
+ </port>
273
+ <port id="2" precision="I32">
274
+ <dim>-1</dim>
275
+ </port>
276
+ <port id="3" precision="I32">
277
+ <dim>-1</dim>
278
+ </port>
279
+ <port id="4" precision="U8">
280
+ <dim>-1</dim>
281
+ </port>
282
+ <port id="5" precision="BOOL">
283
+ <dim>-1</dim>
284
+ </port>
285
+ <port id="6" precision="U8">
286
+ <dim>115</dim>
287
+ </port>
288
+ </input>
289
+ <output>
290
+ <port id="7" precision="I32">
291
+ <dim>-1</dim>
292
+ </port>
293
+ <port id="8" precision="I32">
294
+ <dim>-1</dim>
295
+ </port>
296
+ <port id="9" precision="I32">
297
+ <dim>-1</dim>
298
+ </port>
299
+ <port id="10" precision="I32">
300
+ <dim>-1</dim>
301
+ </port>
302
+ <port id="11" precision="U8">
303
+ <dim>-1</dim>
304
+ </port>
305
+ <port id="12" precision="BOOL">
306
+ <dim>-1</dim>
307
+ </port>
308
+ </output>
309
+ </layer>
310
+ <layer id="25" name="Constant_306483" type="Const" version="opset1">
311
+ <data element_type="u8" shape="1623592" offset="391" size="1623592" />
312
+ <output>
313
+ <port id="0" precision="U8">
314
+ <dim>1623592</dim>
315
+ </port>
316
+ </output>
317
+ </layer>
318
+ <layer id="26" name="StringTensorUnpack_306484" type="StringTensorUnpack" version="extension">
319
+ <data mode="begins_ends" />
320
+ <input>
321
+ <port id="0" precision="U8">
322
+ <dim>1623592</dim>
323
+ </port>
324
+ </input>
325
+ <output>
326
+ <port id="1" precision="I32">
327
+ <dim>-1</dim>
328
+ </port>
329
+ <port id="2" precision="I32">
330
+ <dim>-1</dim>
331
+ </port>
332
+ <port id="3" precision="U8">
333
+ <dim>-1</dim>
334
+ </port>
335
+ </output>
336
+ </layer>
337
+ <layer id="27" name="Constant_306489" type="Const" version="opset1">
338
+ <data element_type="u8" shape="1100616" offset="1623983" size="1100616" />
339
+ <output>
340
+ <port id="0" precision="U8">
341
+ <dim>1100616</dim>
342
+ </port>
343
+ </output>
344
+ </layer>
345
+ <layer id="28" name="StringTensorUnpack_306490" type="StringTensorUnpack" version="extension">
346
+ <data mode="begins_ends" />
347
+ <input>
348
+ <port id="0" precision="U8">
349
+ <dim>1100616</dim>
350
+ </port>
351
+ </input>
352
+ <output>
353
+ <port id="1" precision="I32">
354
+ <dim>-1</dim>
355
+ </port>
356
+ <port id="2" precision="I32">
357
+ <dim>-1</dim>
358
+ </port>
359
+ <port id="3" precision="U8">
360
+ <dim>-1</dim>
361
+ </port>
362
+ </output>
363
+ </layer>
364
+ <layer id="29" name="Constant_306492" type="Const" version="opset1">
365
+ <data element_type="u8" shape="1106903" offset="2724599" size="1106903" />
366
+ <output>
367
+ <port id="0" precision="U8">
368
+ <dim>1106903</dim>
369
+ </port>
370
+ </output>
371
+ </layer>
372
+ <layer id="30" name="StringTensorUnpack_306493" type="StringTensorUnpack" version="extension">
373
+ <data mode="begins_ends" />
374
+ <input>
375
+ <port id="0" precision="U8">
376
+ <dim>1106903</dim>
377
+ </port>
378
+ </input>
379
+ <output>
380
+ <port id="1" precision="I32">
381
+ <dim>-1</dim>
382
+ </port>
383
+ <port id="2" precision="I32">
384
+ <dim>-1</dim>
385
+ </port>
386
+ <port id="3" precision="U8">
387
+ <dim>-1</dim>
388
+ </port>
389
+ </output>
390
+ </layer>
391
+ <layer id="31" name="Constant_306486" type="Const" version="opset1">
392
+ <data element_type="u8" shape="14600" offset="3831502" size="14600" />
393
+ <output>
394
+ <port id="0" precision="U8">
395
+ <dim>14600</dim>
396
+ </port>
397
+ </output>
398
+ </layer>
399
+ <layer id="32" name="StringTensorUnpack_306487" type="StringTensorUnpack" version="extension">
400
+ <data mode="begins_ends" />
401
+ <input>
402
+ <port id="0" precision="U8">
403
+ <dim>14600</dim>
404
+ </port>
405
+ </input>
406
+ <output>
407
+ <port id="1" precision="I32">
408
+ <dim>-1</dim>
409
+ </port>
410
+ <port id="2" precision="I32">
411
+ <dim>-1</dim>
412
+ </port>
413
+ <port id="3" precision="U8">
414
+ <dim>-1</dim>
415
+ </port>
416
+ </output>
417
+ </layer>
418
+ <layer id="33" name="Constant_306494" type="Const" version="opset1">
419
+ <data element_type="i32" shape="1072" offset="3846102" size="4288" />
420
+ <output>
421
+ <port id="0" precision="I32">
422
+ <dim>1072</dim>
423
+ </port>
424
+ </output>
425
+ </layer>
426
+ <layer id="34" name="BPETokenizer_306495" type="BPETokenizer" version="extension">
427
+ <data unk_token="" fuse_unk="false" suffix_indicator="" end_suffix="" byte_fallback="false" cache_capacity="30268" />
428
+ <input>
429
+ <port id="0" precision="I32">
430
+ <dim>-1</dim>
431
+ </port>
432
+ <port id="1" precision="I32">
433
+ <dim>-1</dim>
434
+ </port>
435
+ <port id="2" precision="I32">
436
+ <dim>-1</dim>
437
+ </port>
438
+ <port id="3" precision="I32">
439
+ <dim>-1</dim>
440
+ </port>
441
+ <port id="4" precision="U8">
442
+ <dim>-1</dim>
443
+ </port>
444
+ <port id="5" precision="I32">
445
+ <dim>-1</dim>
446
+ </port>
447
+ <port id="6" precision="I32">
448
+ <dim>-1</dim>
449
+ </port>
450
+ <port id="7" precision="U8">
451
+ <dim>-1</dim>
452
+ </port>
453
+ <port id="8" precision="I32">
454
+ <dim>-1</dim>
455
+ </port>
456
+ <port id="9" precision="I32">
457
+ <dim>-1</dim>
458
+ </port>
459
+ <port id="10" precision="U8">
460
+ <dim>-1</dim>
461
+ </port>
462
+ <port id="11" precision="I32">
463
+ <dim>-1</dim>
464
+ </port>
465
+ <port id="12" precision="I32">
466
+ <dim>-1</dim>
467
+ </port>
468
+ <port id="13" precision="U8">
469
+ <dim>-1</dim>
470
+ </port>
471
+ <port id="14" precision="I32">
472
+ <dim>-1</dim>
473
+ </port>
474
+ <port id="15" precision="I32">
475
+ <dim>-1</dim>
476
+ </port>
477
+ <port id="16" precision="U8">
478
+ <dim>-1</dim>
479
+ </port>
480
+ <port id="17" precision="I32">
481
+ <dim>1072</dim>
482
+ </port>
483
+ </input>
484
+ <output>
485
+ <port id="18" precision="I32">
486
+ <dim>-1</dim>
487
+ </port>
488
+ <port id="19" precision="I32">
489
+ <dim>-1</dim>
490
+ </port>
491
+ <port id="20" precision="I32">
492
+ <dim>-1</dim>
493
+ </port>
494
+ </output>
495
+ </layer>
496
+ <layer id="35" name="Subtract_306496" type="Subtract" version="opset1">
497
+ <data auto_broadcast="numpy" />
498
+ <input>
499
+ <port id="0" precision="I32">
500
+ <dim>-1</dim>
501
+ </port>
502
+ <port id="1" precision="I32">
503
+ <dim>-1</dim>
504
+ </port>
505
+ </input>
506
+ <output>
507
+ <port id="2" precision="I32">
508
+ <dim>-1</dim>
509
+ </port>
510
+ </output>
511
+ </layer>
512
+ <layer id="36" name="Constant_306497" type="Const" version="opset1">
513
+ <data element_type="i32" shape="" offset="3850390" size="4" />
514
+ <output>
515
+ <port id="0" precision="I32" />
516
+ </output>
517
+ </layer>
518
+ <layer id="37" name="Minimum_306498" type="Minimum" version="opset1">
519
+ <data auto_broadcast="numpy" />
520
+ <input>
521
+ <port id="0" precision="I32">
522
+ <dim>-1</dim>
523
+ </port>
524
+ <port id="1" precision="I32" />
525
+ </input>
526
+ <output>
527
+ <port id="2" precision="I32">
528
+ <dim>-1</dim>
529
+ </port>
530
+ </output>
531
+ </layer>
532
+ <layer id="38" name="Add_306499" type="Add" version="opset1">
533
+ <data auto_broadcast="numpy" />
534
+ <input>
535
+ <port id="0" precision="I32">
536
+ <dim>-1</dim>
537
+ </port>
538
+ <port id="1" precision="I32">
539
+ <dim>-1</dim>
540
+ </port>
541
+ </input>
542
+ <output>
543
+ <port id="2" precision="I32">
544
+ <dim>-1</dim>
545
+ </port>
546
+ </output>
547
+ </layer>
548
+ <layer id="39" name="Constant_306506" type="Const" version="opset1">
549
+ <data element_type="i32" shape="3" offset="3850394" size="12" />
550
+ <output>
551
+ <port id="0" precision="I32">
552
+ <dim>3</dim>
553
+ </port>
554
+ </output>
555
+ </layer>
556
+ <layer id="40" name="CombineSegments_306507" type="CombineSegments" version="extension">
557
+ <input>
558
+ <port id="0" precision="I32" />
559
+ <port id="1" precision="I32" />
560
+ <port id="2" precision="I32">
561
+ <dim>1</dim>
562
+ </port>
563
+ <port id="3" precision="I32" />
564
+ <port id="4" precision="I32" />
565
+ <port id="5" precision="I32">
566
+ <dim>1</dim>
567
+ </port>
568
+ <port id="6" precision="I32">
569
+ <dim>-1</dim>
570
+ </port>
571
+ <port id="7" precision="I32">
572
+ <dim>-1</dim>
573
+ </port>
574
+ <port id="8" precision="I32">
575
+ <dim>-1</dim>
576
+ </port>
577
+ <port id="9" precision="I32">
578
+ <dim>3</dim>
579
+ </port>
580
+ </input>
581
+ <output>
582
+ <port id="10" precision="I32">
583
+ <dim>-1</dim>
584
+ </port>
585
+ <port id="11" precision="I32">
586
+ <dim>-1</dim>
587
+ </port>
588
+ <port id="12" precision="I32">
589
+ <dim>-1</dim>
590
+ </port>
591
+ <port id="13" precision="I32">
592
+ <dim>-1</dim>
593
+ </port>
594
+ <port id="14" precision="I32">
595
+ <dim>-1</dim>
596
+ </port>
597
+ <port id="15" precision="I32">
598
+ <dim>-1</dim>
599
+ </port>
600
+ </output>
601
+ </layer>
602
+ <layer id="41" name="Subtract_306508" type="Subtract" version="opset1">
603
+ <data auto_broadcast="numpy" />
604
+ <input>
605
+ <port id="0" precision="I32">
606
+ <dim>-1</dim>
607
+ </port>
608
+ <port id="1" precision="I32">
609
+ <dim>-1</dim>
610
+ </port>
611
+ </input>
612
+ <output>
613
+ <port id="2" precision="I32">
614
+ <dim>-1</dim>
615
+ </port>
616
+ </output>
617
+ </layer>
618
+ <layer id="42" name="Constant_306509" type="Const" version="opset1">
619
+ <data element_type="i32" shape="" offset="0" size="4" />
620
+ <output>
621
+ <port id="0" precision="I32" />
622
+ </output>
623
+ </layer>
624
+ <layer id="43" name="ReduceMax_306510" type="ReduceMax" version="opset1">
625
+ <data keep_dims="false" />
626
+ <input>
627
+ <port id="0" precision="I32">
628
+ <dim>-1</dim>
629
+ </port>
630
+ <port id="1" precision="I32" />
631
+ </input>
632
+ <output>
633
+ <port id="2" precision="I32" />
634
+ </output>
635
+ </layer>
636
+ <layer id="44" name="Constant_306511" type="Const" version="opset1">
637
+ <data element_type="i32" shape="" offset="3850406" size="4" />
638
+ <output>
639
+ <port id="0" precision="I32" />
640
+ </output>
641
+ </layer>
642
+ <layer id="45" name="RaggedToDense_306512" type="RaggedToDense" version="extension">
643
+ <data pad_right="false" />
644
+ <input>
645
+ <port id="0" precision="I32">
646
+ <dim>-1</dim>
647
+ </port>
648
+ <port id="1" precision="I32">
649
+ <dim>-1</dim>
650
+ </port>
651
+ <port id="2" precision="I32">
652
+ <dim>-1</dim>
653
+ </port>
654
+ <port id="3" precision="I32" />
655
+ <port id="4" precision="I32" />
656
+ </input>
657
+ <output>
658
+ <port id="5" precision="I32">
659
+ <dim>-1</dim>
660
+ <dim>-1</dim>
661
+ </port>
662
+ <port id="6" precision="BOOL">
663
+ <dim>-1</dim>
664
+ <dim>-1</dim>
665
+ </port>
666
+ </output>
667
+ </layer>
668
+ <layer id="46" name="Convert_306513" type="Convert" version="opset1">
669
+ <data destination_type="i32" />
670
+ <input>
671
+ <port id="0" precision="BOOL">
672
+ <dim>-1</dim>
673
+ <dim>-1</dim>
674
+ </port>
675
+ </input>
676
+ <output>
677
+ <port id="1" precision="I32">
678
+ <dim>-1</dim>
679
+ <dim>-1</dim>
680
+ </port>
681
+ </output>
682
+ </layer>
683
+ <layer id="47" name="Convert_306513" type="Convert" version="opset1">
684
+ <data destination_type="i64" />
685
+ <input>
686
+ <port id="0" precision="I32">
687
+ <dim>-1</dim>
688
+ <dim>-1</dim>
689
+ </port>
690
+ </input>
691
+ <output>
692
+ <port id="1" precision="I64" names="attention_mask">
693
+ <dim>-1</dim>
694
+ <dim>-1</dim>
695
+ </port>
696
+ </output>
697
+ </layer>
698
+ <layer id="49" name="Constant_306514" type="Const" version="opset1">
699
+ <data element_type="i32" shape="" offset="3850406" size="4" />
700
+ <output>
701
+ <port id="0" precision="I32" />
702
+ </output>
703
+ </layer>
704
+ <layer id="50" name="RaggedToDense_306515" type="RaggedToDense" version="extension">
705
+ <data pad_right="false" />
706
+ <input>
707
+ <port id="0" precision="I32">
708
+ <dim>-1</dim>
709
+ </port>
710
+ <port id="1" precision="I32">
711
+ <dim>-1</dim>
712
+ </port>
713
+ <port id="2" precision="I32">
714
+ <dim>-1</dim>
715
+ </port>
716
+ <port id="3" precision="I32" />
717
+ <port id="4" precision="I32" />
718
+ </input>
719
+ <output>
720
+ <port id="5" precision="I32">
721
+ <dim>-1</dim>
722
+ <dim>-1</dim>
723
+ </port>
724
+ <port id="6" precision="BOOL">
725
+ <dim>-1</dim>
726
+ <dim>-1</dim>
727
+ </port>
728
+ </output>
729
+ </layer>
730
+ <layer id="51" name="RaggedToDense_306515.0" type="Convert" version="opset1">
731
+ <data destination_type="i64" />
732
+ <input>
733
+ <port id="0" precision="I32">
734
+ <dim>-1</dim>
735
+ <dim>-1</dim>
736
+ </port>
737
+ </input>
738
+ <output>
739
+ <port id="1" precision="I64" names="token_type_ids">
740
+ <dim>-1</dim>
741
+ <dim>-1</dim>
742
+ </port>
743
+ </output>
744
+ </layer>
745
+ <layer id="53" name="RaggedToDense_306512.0" type="Convert" version="opset1">
746
+ <data destination_type="i64" />
747
+ <input>
748
+ <port id="0" precision="I32">
749
+ <dim>-1</dim>
750
+ <dim>-1</dim>
751
+ </port>
752
+ </input>
753
+ <output>
754
+ <port id="1" precision="I64" names="input_ids">
755
+ <dim>-1</dim>
756
+ <dim>-1</dim>
757
+ </port>
758
+ </output>
759
+ </layer>
760
+ <layer id="54" name="Result_306516" type="Result" version="opset1">
761
+ <input>
762
+ <port id="0" precision="I64">
763
+ <dim>-1</dim>
764
+ <dim>-1</dim>
765
+ </port>
766
+ </input>
767
+ </layer>
768
+ <layer id="52" name="Result_306517" type="Result" version="opset1">
769
+ <input>
770
+ <port id="0" precision="I64">
771
+ <dim>-1</dim>
772
+ <dim>-1</dim>
773
+ </port>
774
+ </input>
775
+ </layer>
776
+ <layer id="48" name="Result_306518" type="Result" version="opset1">
777
+ <input>
778
+ <port id="0" precision="I64">
779
+ <dim>-1</dim>
780
+ <dim>-1</dim>
781
+ </port>
782
+ </input>
783
+ </layer>
784
+ </layers>
785
+ <edges>
786
+ <edge from-layer="0" from-port="0" to-layer="8" to-port="0" />
787
+ <edge from-layer="1" from-port="0" to-layer="40" to-port="0" />
788
+ <edge from-layer="2" from-port="0" to-layer="40" to-port="1" />
789
+ <edge from-layer="3" from-port="0" to-layer="40" to-port="2" />
790
+ <edge from-layer="4" from-port="0" to-layer="40" to-port="3" />
791
+ <edge from-layer="5" from-port="0" to-layer="40" to-port="4" />
792
+ <edge from-layer="6" from-port="0" to-layer="40" to-port="5" />
793
+ <edge from-layer="7" from-port="0" to-layer="14" to-port="0" />
794
+ <edge from-layer="8" from-port="1" to-layer="9" to-port="0" />
795
+ <edge from-layer="8" from-port="3" to-layer="21" to-port="4" />
796
+ <edge from-layer="8" from-port="2" to-layer="21" to-port="3" />
797
+ <edge from-layer="8" from-port="1" to-layer="21" to-port="2" />
798
+ <edge from-layer="9" from-port="1" to-layer="12" to-port="0" />
799
+ <edge from-layer="10" from-port="0" to-layer="12" to-port="1" />
800
+ <edge from-layer="11" from-port="0" to-layer="12" to-port="2" />
801
+ <edge from-layer="12" from-port="3" to-layer="14" to-port="1" />
802
+ <edge from-layer="12" from-port="3" to-layer="17" to-port="0" />
803
+ <edge from-layer="13" from-port="0" to-layer="14" to-port="2" />
804
+ <edge from-layer="14" from-port="3" to-layer="21" to-port="0" />
805
+ <edge from-layer="15" from-port="0" to-layer="19" to-port="0" />
806
+ <edge from-layer="16" from-port="0" to-layer="17" to-port="1" />
807
+ <edge from-layer="17" from-port="2" to-layer="19" to-port="1" />
808
+ <edge from-layer="18" from-port="0" to-layer="19" to-port="2" />
809
+ <edge from-layer="19" from-port="3" to-layer="21" to-port="1" />
810
+ <edge from-layer="20" from-port="0" to-layer="21" to-port="5" />
811
+ <edge from-layer="21" from-port="9" to-layer="22" to-port="1" />
812
+ <edge from-layer="21" from-port="10" to-layer="22" to-port="2" />
813
+ <edge from-layer="21" from-port="11" to-layer="22" to-port="3" />
814
+ <edge from-layer="21" from-port="6" to-layer="24" to-port="0" />
815
+ <edge from-layer="21" from-port="7" to-layer="24" to-port="1" />
816
+ <edge from-layer="21" from-port="8" to-layer="22" to-port="0" />
817
+ <edge from-layer="22" from-port="6" to-layer="24" to-port="4" />
818
+ <edge from-layer="22" from-port="7" to-layer="24" to-port="5" />
819
+ <edge from-layer="22" from-port="5" to-layer="24" to-port="3" />
820
+ <edge from-layer="22" from-port="4" to-layer="24" to-port="2" />
821
+ <edge from-layer="23" from-port="0" to-layer="24" to-port="6" />
822
+ <edge from-layer="24" from-port="9" to-layer="34" to-port="2" />
823
+ <edge from-layer="24" from-port="10" to-layer="34" to-port="3" />
824
+ <edge from-layer="24" from-port="11" to-layer="34" to-port="4" />
825
+ <edge from-layer="24" from-port="8" to-layer="34" to-port="1" />
826
+ <edge from-layer="24" from-port="7" to-layer="34" to-port="0" />
827
+ <edge from-layer="25" from-port="0" to-layer="26" to-port="0" />
828
+ <edge from-layer="26" from-port="1" to-layer="34" to-port="5" />
829
+ <edge from-layer="26" from-port="2" to-layer="34" to-port="6" />
830
+ <edge from-layer="26" from-port="3" to-layer="34" to-port="7" />
831
+ <edge from-layer="27" from-port="0" to-layer="28" to-port="0" />
832
+ <edge from-layer="28" from-port="3" to-layer="34" to-port="10" />
833
+ <edge from-layer="28" from-port="1" to-layer="34" to-port="8" />
834
+ <edge from-layer="28" from-port="2" to-layer="34" to-port="9" />
835
+ <edge from-layer="29" from-port="0" to-layer="30" to-port="0" />
836
+ <edge from-layer="30" from-port="3" to-layer="34" to-port="13" />
837
+ <edge from-layer="30" from-port="1" to-layer="34" to-port="11" />
838
+ <edge from-layer="30" from-port="2" to-layer="34" to-port="12" />
839
+ <edge from-layer="31" from-port="0" to-layer="32" to-port="0" />
840
+ <edge from-layer="32" from-port="1" to-layer="34" to-port="14" />
841
+ <edge from-layer="32" from-port="2" to-layer="34" to-port="15" />
842
+ <edge from-layer="32" from-port="3" to-layer="34" to-port="16" />
843
+ <edge from-layer="33" from-port="0" to-layer="34" to-port="17" />
844
+ <edge from-layer="34" from-port="18" to-layer="38" to-port="0" />
845
+ <edge from-layer="34" from-port="20" to-layer="40" to-port="8" />
846
+ <edge from-layer="34" from-port="18" to-layer="40" to-port="6" />
847
+ <edge from-layer="34" from-port="19" to-layer="35" to-port="0" />
848
+ <edge from-layer="34" from-port="18" to-layer="35" to-port="1" />
849
+ <edge from-layer="35" from-port="2" to-layer="37" to-port="0" />
850
+ <edge from-layer="36" from-port="0" to-layer="37" to-port="1" />
851
+ <edge from-layer="37" from-port="2" to-layer="38" to-port="1" />
852
+ <edge from-layer="38" from-port="2" to-layer="40" to-port="7" />
853
+ <edge from-layer="39" from-port="0" to-layer="40" to-port="9" />
854
+ <edge from-layer="40" from-port="11" to-layer="41" to-port="0" />
855
+ <edge from-layer="40" from-port="15" to-layer="50" to-port="2" />
856
+ <edge from-layer="40" from-port="14" to-layer="50" to-port="1" />
857
+ <edge from-layer="40" from-port="13" to-layer="50" to-port="0" />
858
+ <edge from-layer="40" from-port="12" to-layer="45" to-port="2" />
859
+ <edge from-layer="40" from-port="11" to-layer="45" to-port="1" />
860
+ <edge from-layer="40" from-port="10" to-layer="45" to-port="0" />
861
+ <edge from-layer="40" from-port="10" to-layer="41" to-port="1" />
862
+ <edge from-layer="41" from-port="2" to-layer="43" to-port="0" />
863
+ <edge from-layer="42" from-port="0" to-layer="43" to-port="1" />
864
+ <edge from-layer="43" from-port="2" to-layer="45" to-port="3" />
865
+ <edge from-layer="43" from-port="2" to-layer="50" to-port="3" />
866
+ <edge from-layer="44" from-port="0" to-layer="45" to-port="4" />
867
+ <edge from-layer="45" from-port="6" to-layer="46" to-port="0" />
868
+ <edge from-layer="45" from-port="5" to-layer="53" to-port="0" />
869
+ <edge from-layer="46" from-port="1" to-layer="47" to-port="0" />
870
+ <edge from-layer="47" from-port="1" to-layer="48" to-port="0" />
871
+ <edge from-layer="49" from-port="0" to-layer="50" to-port="4" />
872
+ <edge from-layer="50" from-port="5" to-layer="51" to-port="0" />
873
+ <edge from-layer="51" from-port="1" to-layer="52" to-port="0" />
874
+ <edge from-layer="53" from-port="1" to-layer="54" to-port="0" />
875
+ </edges>
876
+ <rt_info>
877
+ <add_attention_mask value="True" />
878
+ <add_prefix_space />
879
+ <add_special_tokens value="True" />
880
+ <chat_template value="[gMASK]&lt;sop>{% for item in messages %}{% if item['tools'] is defined %}&lt;|system|>&#10;你是一个名为 GLM-4 的人工智能助手。你是基于智谱AI训练的语言模型 GLM-4 模型开发的,你的任务是针对用户的问题和要求提供适当的答复和支持。&#10;&#10;# 可用工具{% set tools = item['tools'] %}{% for tool in tools %}{% if tool['type'] == 'function' %}&#10;&#10;## {{ tool['function']['name'] }}&#10;&#10;{{ tool['function'] | tojson(indent=4) }}&#10;在调用上述函数时,请使用 Json 格式表示调用的参数。{% elif tool['type'] == 'python' %}&#10;&#10;## python&#10;&#10;当你向 `python` 发送包含 Python 代码的消息时,该代码将会在一个有状态的 Jupyter notebook 环境中执行。&#10;`python` 返回代码执行的输出,或在执行 60 秒后返回超时。&#10;`/mnt/data` 将会持久化存储你的文件。在此会话中,`python` 无法访问互联网。不要使用 `python` 进行任何网络请求或者在线 API 调用,这些在线内容的访问将不会成功。{% elif tool['type'] == 'simple_browser' %}&#10;&#10;## simple_browser&#10;&#10;你可以使用 `simple_browser` 工具。该工具支持以下函数:&#10;`search(query: str, recency_days: int)`:使用搜索引擎进行查询并显示结果,可以使用 `recency_days` 参数控制搜索内容的时效性。&#10;`mclick(ids: list[int])`:获取一系列指定 id 的页面内容。每次调用时,须选择3-10个页面。选择多个角度的页面,同时尽可能选择可信任的信息来源。考虑到部分页面是无法加载的,你也可以多打开一些可能有用的页面而不用担心内容过多。&#10;`open_url(url: str)`:打开指定的 URL。&#10;&#10;使用 `【{引用 id}†{引用文本}】` 来引用内容。&#10;&#10;操作步骤:1. 使用 `search` 来获得信息列表; 2. 使用 `mclick` 来获取指定 ID 页面的内容; 3. 根据获得的内容进行回复。在回复中应当引用信息来源。&#10; 如果用户提供了 URL,也可以用 `open_url` 直接打开页面。&#10;如果初次搜索结果没有找到合适的信息,也可以再次使用 `search` 进行搜索。{% elif tool['type'] == 'cogview' %}&#10;&#10;## cogview&#10;&#10;如果用户的请求中包含了对图像的描述,你可以使用 `cogview` 来生成图像并展示给用户。你需要向 `cogview` 发送图像描述,规则:&#10;- 发送给 `cogview` 的消息必须使用英语。用户的中文描述必须完全翻译为英语。&#10;- 应当尽可能详细地描述图像生成的需求,需求描述约 100 英文单词。&#10;- 保持用户原始描述的意图。不要虚构内容或者没见过的人物。&#10;- 如无特殊说明,所在地为中国,持有中国立场并遵循中国社会主义价值观。{% endif %}{% endfor %}{% endif %}{% if item['content'] %}&lt;|{{ item['role'] }}|>{{ item['metadata'] }}&#10;{{ item['content'] }}{% endif %}{% endfor %}{% if add_generation_prompt %}&lt;|assistant|>{% endif %}" />
881
+ <clean_up_tokenization_spaces value="False" />
882
+ <detokenizer_input_type value="i64" />
883
+ <eos_token_id value="151329" />
884
+ <handle_special_tokens_with_re />
885
+ <number_of_inputs value="1" />
886
+ <openvino_tokenizers_version value="2024.5.0.0.dev20241030" />
887
+ <openvino_version value="2024.5.0.dev20241030" />
888
+ <original_tokenizer_class value="&lt;class 'transformers_modules.THUDM.glm-4-9b-chat-1m.0aa722c7e0745dd21453427dd44c257dd253304f.tokenization_chatglm.ChatGLM4Tokenizer'>" />
889
+ <pad_token_id value="151329" />
890
+ <sentencepiece_version value="0.2.0" />
891
+ <skip_special_tokens value="True" />
892
+ <streaming_detokenizer value="False" />
893
+ <tiktoken_version value="0.8.0" />
894
+ <tokenizer_output_type value="i64" />
895
+ <tokenizers_version value="0.20.1" />
896
+ <transformers_version value="4.45.2" />
897
+ <use_max_padding value="False" />
898
+ <use_sentencepiece_backend value="False" />
899
+ <utf8_replace_mode />
900
+ <with_detokenizer value="True" />
901
+ </rt_info>
902
+ </net>