winglian
/

llama-3-1m-context-gradient-lora

Model card Files Files and versions Community

llama-3-1m-context-gradient-lora / README.md

winglian's picture

Create README.md

52b97c1 10 months ago

|

history blame contribute delete

179 Bytes

	LoRA extraction from Gradient AI's https://huggingface.co/gradientai/Llama-3-8B-Instruct-Gradient-1048k model.
	LoRA extraction only targeted from the self_attn modules.
	Rank: 1024