|
--- |
|
base_model: |
|
- unsloth/Mistral-Small-Instruct-2409 |
|
library_name: transformers |
|
tags: |
|
- mergekit |
|
- merge |
|
|
|
--- |
|
<!DOCTYPE html> |
|
<html lang="en"> |
|
<head> |
|
<meta charset="UTF-8"> |
|
<meta name="viewport" content="width=device-width, initial-scale=1.0"> |
|
<title>LD-Zephyria-37b Data Card</title> |
|
<link href="https://fonts.googleapis.com/css2?family=Quicksand:wght@400;500;600&display=swap" rel="stylesheet"> |
|
<style> |
|
body, html { |
|
height: 100%; |
|
margin: 0; |
|
padding: 0; |
|
font-family: 'Quicksand', sans-serif; |
|
background: linear-gradient(135deg, #0a1128 0%, #1c2541 100%); |
|
color: #e0e1dd; |
|
font-size: 16px; |
|
} |
|
.container { |
|
width: 100%; |
|
height: 100%; |
|
padding: 20px; |
|
margin: 0; |
|
background-color: rgba(255, 255, 255, 0.05); |
|
border-radius: 12px; |
|
box-shadow: 0 4px 10px rgba(0, 0, 0, 0.3); |
|
backdrop-filter: blur(10px); |
|
border: 1px solid rgba(255, 255, 255, 0.1); |
|
} |
|
.header h1 { |
|
font-size: 28px; |
|
color: #4cc9f0; |
|
margin: 0 0 20px 0; |
|
text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.3); |
|
} |
|
.update-section h2 { |
|
font-size: 24px; |
|
color: #7209b7; |
|
} |
|
.update-section p { |
|
font-size: 16px; |
|
line-height: 1.6; |
|
color: #e0e1dd; |
|
} |
|
.info img { |
|
width: 100%; |
|
border-radius: 10px; |
|
margin-bottom: 15px; |
|
} |
|
a { |
|
color: #4cc9f0; |
|
text-decoration: none; |
|
} |
|
a:hover { |
|
color: #f72585; |
|
} |
|
.button { |
|
display: inline-block; |
|
background-color: #3a0ca3; |
|
color: #e0e1dd; |
|
padding: 10px 20px; |
|
border-radius: 5px; |
|
cursor: pointer; |
|
text-decoration: none; |
|
} |
|
.button:hover { |
|
background-color: #7209b7; |
|
} |
|
pre { |
|
background-color: #1c2541; |
|
padding: 10px; |
|
border-radius: 5px; |
|
overflow-x: auto; |
|
} |
|
code { |
|
font-family: 'Courier New', monospace; |
|
color: #e0e1dd; |
|
} |
|
</style> |
|
</head> |
|
<body> |
|
<div class="container"> |
|
<div class="header"> |
|
<h1>LD-Zephyria-37b [EXPERIMENTAL]</h1> |
|
</div> |
|
<div class="info"> |
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/6W3orrbf8A68l-3p_JxN1.png"> |
|
<h2>Model Information</h2> |
|
<p><strong>Base Model:</strong> unsloth/Mistral-Small-Instruct-2409</p> |
|
<p><strong>Strategy:</strong> Late Duplication</p> |
|
<p><strong>Total Layers:</strong> 55</p> |
|
<p><strong>Duplication Start:</strong> Layer 28 (50.9% of model)</p> |
|
<p><strong>Duplicated Layers:</strong> 21 (38.2% of model)</p> |
|
<p><strong>Unique Final Layers:</strong> 7 (12.7% of model)</p> |
|
<h2>Model Characteristics</h2> |
|
<ul> |
|
<li>Models down_proj and o_proj layers have been nulled and will require healing</li> |
|
<li>Emphasizes complex feature extraction before duplication</li> |
|
<li>Smallest duplicated section among all strategies</li> |
|
<li>Ideal for tasks requiring extensive unique feature processing</li> |
|
<li>May excel in tasks that benefit from a wide range of unique features before refinement</li> |
|
</ul> |
|
<h2>Configuration Visualization</h2> |
|
<pre><code> |
|
[ Unique ][ Duplicated ][ Unique ] |
|
0 ------------------- 27 28 ------------ 48 49 --- 54 |
|
50.9% 38.2% 10.9% |
|
</code></pre> |
|
</div> |
|
</div> |
|
</body> |
|
</html> |
