Hugging Face's logo

softwareweaver
/
Twilight-XL-2-195B-Mistral-Large-2411-Behemoth

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
softwareweaver commited on
Commit
da758c9
·
verified ·
1 Parent(s): 67a9189

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +40 -0
  2. config.json +27 -0
  3. mergekit_config.yml +9 -0
  4. model-00001-of-00081.safetensors +3 -0
  5. model-00002-of-00081.safetensors +3 -0
  6. model-00003-of-00081.safetensors +3 -0
  7. model-00004-of-00081.safetensors +3 -0
  8. model-00005-of-00081.safetensors +3 -0
  9. model-00006-of-00081.safetensors +3 -0
  10. model-00007-of-00081.safetensors +3 -0
  11. model-00008-of-00081.safetensors +3 -0
  12. model-00009-of-00081.safetensors +3 -0
  13. model-00010-of-00081.safetensors +3 -0
  14. model-00011-of-00081.safetensors +3 -0
  15. model-00012-of-00081.safetensors +3 -0
  16. model-00013-of-00081.safetensors +3 -0
  17. model-00014-of-00081.safetensors +3 -0
  18. model-00015-of-00081.safetensors +3 -0
  19. model-00016-of-00081.safetensors +3 -0
  20. model-00017-of-00081.safetensors +3 -0
  21. model-00018-of-00081.safetensors +3 -0
  22. model-00019-of-00081.safetensors +3 -0
  23. model-00020-of-00081.safetensors +3 -0
  24. model-00021-of-00081.safetensors +3 -0
  25. model-00022-of-00081.safetensors +3 -0
  26. model-00023-of-00081.safetensors +3 -0
  27. model-00024-of-00081.safetensors +3 -0
  28. model-00025-of-00081.safetensors +3 -0
  29. model-00026-of-00081.safetensors +3 -0
  30. model-00027-of-00081.safetensors +3 -0
  31. model-00028-of-00081.safetensors +3 -0
  32. model-00029-of-00081.safetensors +3 -0
  33. model-00030-of-00081.safetensors +3 -0
  34. model-00031-of-00081.safetensors +3 -0
  35. model-00032-of-00081.safetensors +3 -0
  36. model-00033-of-00081.safetensors +3 -0
  37. model-00034-of-00081.safetensors +3 -0
  38. model-00035-of-00081.safetensors +3 -0
  39. model-00036-of-00081.safetensors +3 -0
  40. model-00037-of-00081.safetensors +3 -0
  41. model-00038-of-00081.safetensors +3 -0
  42. model-00039-of-00081.safetensors +3 -0
  43. model-00040-of-00081.safetensors +3 -0
  44. model-00041-of-00081.safetensors +3 -0
  45. model-00042-of-00081.safetensors +3 -0
  46. model-00043-of-00081.safetensors +3 -0
  47. model-00044-of-00081.safetensors +3 -0
  48. model-00045-of-00081.safetensors +3 -0
  49. model-00046-of-00081.safetensors +3 -0
  50. model-00047-of-00081.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - TheDrummer/Behemoth-123B-v2.2
4
+ - mistralai/Mistral-Large-Instruct-2411
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+
10
+ ---
11
+ # Twilight-XL-2
12
+
13
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
+
15
+ ## Merge Details
16
+ ### Merge Method
17
+
18
+ This model was merged using the passthrough merge method.
19
+
20
+ ### Models Merged
21
+
22
+ The following models were included in the merge:
23
+ * [TheDrummer/Behemoth-123B-v2.2](https://huggingface.co/TheDrummer/Behemoth-123B-v2.2)
24
+ * [mistralai/Mistral-Large-Instruct-2411](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411)
25
+
26
+ ### Configuration
27
+
28
+ The following YAML configuration was used to produce this model:
29
+
30
+ ```yaml
31
+ dtype: bfloat16
32
+ merge_method: passthrough
33
+ slices:
34
+ - sources:
35
+ - layer_range: [0, 70]
36
+ model: mistralai/Mistral-Large-Instruct-2411
37
+ - sources:
38
+ - layer_range: [18, 88]
39
+ model: TheDrummer/Behemoth-123B-v2.2
40
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "softwareweaver/Twilight-XL-2-195B-Mistral-Large-2411-Behemoth",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 12288,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 96,
17
+ "num_hidden_layers": 140,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "bfloat16",
24
+ "transformers_version": "4.46.0",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ dtype: bfloat16
2
+ merge_method: passthrough
3
+ slices:
4
+ - sources:
5
+ - layer_range: [0, 70]
6
+ model: mistralai/Mistral-Large-Instruct-2411
7
+ - sources:
8
+ - layer_range: [18, 88]
9
+ model: TheDrummer/Behemoth-123B-v2.2
model-00001-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8ad1a6806cc1c1886cd7b17e90400cb12d940dc30c4293e7055ce49464e59bb
3
+ size 4378928504
model-00002-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ca362e22631e68f500cf9dc52d179ece34d9096943ca018faa91d67411fd2ebf
3
+ size 4907411088
model-00003-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec06f375a34ae6de49cdc9d1a86dbb8a90f8f2d140855d5783f0999700227d43
3
+ size 4806747904
model-00004-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8fbd21ed36f4f869f4549eeb860a63ae0a064580e1505e7f8eb30cf03d03e74d
3
+ size 4831938544
model-00005-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:137ce8c16d8564f7d15b00000478fce0fc1f13c182451f391dfa126d9b3e4048
3
+ size 4831938552
model-00006-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:50a18b7a44d54bf5eac593df97a23a0c405e958e743c21e91a6d8178427db218
3
+ size 4907411096
model-00007-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4984672679ac261cd68365595e7c0ed598850b3c8ec00d2f5e3423ae5d22e89e
3
+ size 4907435776
model-00008-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0443b29196993bdd237674a27ea94f577a7ce228c66c3eb84ef4da14d520b6eb
3
+ size 4781557264
model-00009-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fdad0e3479abbbd44309bc40f6a2c9490734e0676c0b8383e66caf48aeae57ef
3
+ size 4781656640
model-00010-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:57c76c1e54c387d8f5d3e2b97d9754ab160cf63d917bcc9bfa0f15a5e683f708
3
+ size 4882220464
model-00011-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c502c0f70d63de36d488ca7c553968eb2a118c6328178cd7ed6d85eef4b04009
3
+ size 4932601720
model-00012-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c844059bebfc46f70a3d70b8c8c3900505e4bcc99a08f931e662f4e5bd33ce12
3
+ size 4781557264
model-00013-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f2de381c1d0a81221c785447fde73e5a5f6aadd640f893292f39e3ce2ef6cba8
3
+ size 4831938552
model-00014-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33fdfa04aee380b820d276d052c18d36d242504dbd4a26d38afcebe60cab0b17
3
+ size 4831938544
model-00015-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eb014e0865bb847fd367d8ae3eb9a10cb91c331f523c5e30e4082c79ccca15ab
3
+ size 4831938552
model-00016-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:86fb4040e33dcab2cca8184ad676e15cc21435b87f559413d5df3948d4c28ea7
3
+ size 4831938552
model-00017-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5855ff99fa39909b4029f370d0c1d50f56e6d629f5cff83999ecb11be64720e1
3
+ size 4831938552
model-00018-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a784f5e73be14e5b82e260b85936eff3828f5b1ac4d565c060def334d70d2ec6
3
+ size 4882220464
model-00019-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dd89ba3e74579603616170d11238ae2d72f65e99122dfedbe2cf9169afbdb030
3
+ size 4932601720
model-00020-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:011ad1067d3a19c93f55f2e3cce4e00347f696ce7b5cc307b5c911c7314c6c25
3
+ size 4781557264
model-00021-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f460a64f0ec53ba99d819e6df33e0dd3567144cffde18af07e8ddd702c5fc7f2
3
+ size 4781656640
model-00022-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0309241b274428d572e3a4ef7df646611a843dc8b63a0174e9bc54d8a5085d73
3
+ size 4882220464
model-00023-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7cb69c821309ede51b869a64235f11e72bdbd2e16a729f4f05256b7df5b16962
3
+ size 4932601720
model-00024-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0e038308d3f5c0b8fea9c80b323112fd9b0cfa524182c0da712c0b4a5c0b5b55
3
+ size 4781557264
model-00025-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:52b600a4f1fc19f0e293eff835a7542ecc57d5f98739f57b9c7e8c6fe0764132
3
+ size 4831938552
model-00026-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:335b584eb9fb35e4745d06e047ac074becaf08e4af4d8465984411a2e938acc7
3
+ size 4831938544
model-00027-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1d919025b0ec7238fb4dba4b7c0a5ef723ca3760d1c95754d3323942816c23a
3
+ size 4831938552
model-00028-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0425ebf9291f891634931ddfe1ce1dcb5198a861697d2b456b976df8bef936cb
3
+ size 4831938552
model-00029-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e94c6c1780caee2a44706438fd215f0788494c9cb7b7c271489fe981c2effe58
3
+ size 4831938552
model-00030-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:11a54dd8fbdcf8d63d15568aa1d94e4c981ae3456fb170b7fd3664db87162224
3
+ size 4882220464
model-00031-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8145067586cae057fcb8e35a4d3ba0651850138c5197495e86e9d4d82f017d7a
3
+ size 4932601720
model-00032-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c4e923b6e620e38210d21912f93dbdb294e4639b8c12d0b8c06ba7e14932da1e
3
+ size 4781557264
model-00033-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0f1f121060c408f9b4cb23e8fbb898a40c55809b7dc2950c11536308d6aa956
3
+ size 4781656640
model-00034-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:34c2ecfbf2014ee4aad203823bde46f5cb4731b63d632988c257419d987adf52
3
+ size 4882220464
model-00035-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:405dc9254857cf27eabaaf29ab284c18497b64897d316e78ab3525332e74d55d
3
+ size 4932601720
model-00036-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60a300cebbc476b1e1799ddc28457e0843fc6dcb5051a54e1474791f3b3224cf
3
+ size 4781557264
model-00037-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:41e748d2f262fd2b34ca3c4bf6b00c06a462d84466e5a0f708ec43064a312fce
3
+ size 4831938552
model-00038-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b7c3f008495a5e372266b1cd457e2f6365173b4cad4da0c17cf4384583fc3cbd
3
+ size 4831938544
model-00039-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3061b654ae3133fd01076f5d0ec0130d54952744188c83e3b113ea9debb620ef
3
+ size 4831938552
model-00040-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3c8615029b5204b5c94011cf35517dc11081a3d29316aad14b48275907d66e80
3
+ size 4831938552
model-00041-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da30c033ae7052cc07cfe555136a1a157f78af6ca741659dca142054f5fcbe7c
3
+ size 4831938552
model-00042-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0204eb180fcef4eb6c2ccbfd51448d4a2ea93dc3f4ff18dacce278aa5e7aa42d
3
+ size 4882220464
model-00043-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:08e86d941a48afb98894fbd7fb85e2769a7a9bfa5c4a29d97ab5476699eaa40a
3
+ size 4932601720
model-00044-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5d72d9c2e7c051c35b42bb1ce689fe67ecb19ef8b01560bd552d40132e43772
3
+ size 4781557272
model-00045-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6bb7bc0a75db95c62d2eb386d16dca28765efa18ade418dc04d5f99f862487f
3
+ size 4781656632
model-00046-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:69ec7377b4013a9196f5a3d81f7acb7d233ea628d769980f3f4e7123f271bdd0
3
+ size 4882220464
model-00047-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8fb544c1fd842b17b65ab3427b8e30252932b776a08a5f79ef8283690c5cb467
3
+ size 4932601728