| Lumitess | |
| This model was a attempt to create a good llama 3 uncencored model with long context. | |
| At it worked like a charm | |
| Did a merge with breadcrumbs_ties method. | |
| Instruct gradient, Lumimaid and Tess. | |
| Uses llama 3 context | |
| Sampler wise it has a very wide optimal so works with lots of different settings. | |
| base_model: [] | |
| library_name: transformers | |
| tags: | |
| - mergekit | |
| - merge | |
| --- | |
| # model | |
| This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). | |
| ## Merge Details | |
| ### Merge Method | |
| This model was merged using the breadcrumbs_ties merge method using I:\Llama-3-70B-Instruct-Gradient-262k as a base. | |
| ### Models Merged | |
| The following models were included in the merge: | |
| * E:\Llama-3-Lumimaid-70B-v0.1-OAS | |
| * I:\Tess-2.0-Llama-3-70B-v0.2 | |
| ### Configuration | |
| The following YAML configuration was used to produce this model: | |
| ```yaml | |
| models: | |
| - model: I:\Llama-3-70B-Instruct-Gradient-262k | |
| parameters: | |
| weight: 0.20 | |
| density: 0.90 | |
| gamma: 0.01 | |
| - model: I:\Tess-2.0-Llama-3-70B-v0.2 | |
| parameters: | |
| weight: 0.20 | |
| density: 0.90 | |
| gamma: 0.01 | |
| - model: E:\Llama-3-Lumimaid-70B-v0.1-OAS | |
| parameters: | |
| weight: 0.60 | |
| density: 0.90 | |
| gamma: 0.01 | |
| merge_method: breadcrumbs_ties | |
| base_model: I:\Llama-3-70B-Instruct-Gradient-262k | |
| dtype: bfloat16 | |
| ``` | |