Commit
•
3d5bea9
1
Parent(s):
ea6fb91
Update README.md
Browse files
README.md
CHANGED
|
@@ -16,6 +16,7 @@ license: apache-2.0
|
|
| 16 |
code { font-family: 'Courier New', monospace; color: #A3BE8C; } </style> </head> <body> <div class="container">
|
| 17 |
<div class="header">
|
| 18 |
<h1>Aura-llama</h1> </div> <div class="info">
|
|
|
|
| 19 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/QYpWMEXTe0_X3A7HyeBm0.webp" alt="Aura-llama image">
|
| 20 |
<p>Now that the cute anime girl has your attention.</p>
|
| 21 |
<p>Aura-llama is using the methodology presented by SOLAR for scaling LLMs called depth up-scaling (DUS), which encompasses architectural modifications with continued pretraining. Using the solar paper as a base, I integrated Llama-3 weights into the upscaled layers, and In the future plan to continue training the model.</p>
|
|
|
|
| 16 |
code { font-family: 'Courier New', monospace; color: #A3BE8C; } </style> </head> <body> <div class="container">
|
| 17 |
<div class="header">
|
| 18 |
<h1>Aura-llama</h1> </div> <div class="info">
|
| 19 |
+
<strong>there was an error in the first model config it has been fixed in the new repo [TheSkullery/Aura-llama-v1](https://huggingface.co/TheSkullery/Aura-llama-v1)
|
| 20 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/QYpWMEXTe0_X3A7HyeBm0.webp" alt="Aura-llama image">
|
| 21 |
<p>Now that the cute anime girl has your attention.</p>
|
| 22 |
<p>Aura-llama is using the methodology presented by SOLAR for scaling LLMs called depth up-scaling (DUS), which encompasses architectural modifications with continued pretraining. Using the solar paper as a base, I integrated Llama-3 weights into the upscaled layers, and In the future plan to continue training the model.</p>
|