Upload LlamaForCausalLM

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

config.json

@@ -0,0 +1,36 @@
+{
+  "_name_or_path": "/root/alpaca/lora-llama3-70b-v2-incontext-shuffle-split2-5e-5-maxepoch20/42/checkpoint-244",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "eos_token_id": 128001,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 28672,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 64,
+  "num_hidden_layers": 80,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.49.0",
+  "use_cache": true,
+  "vocab_size": 128256
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": 128001,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.49.0"
+}

model-00001-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:009bfe2a8ff3cbc194444c0e04c8a11560d24ff1cc919d2a4046b304f32ed787
+size 4806672984

model-00002-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45a79573034c60e9c7be29958455096f664354a122b6523ca1312d4570cf6c92
+size 4362142864

model-00003-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f04d6b4d36892417068127592db1a2030934f8d977e63f3e0bf0bc131a569700
+size 4362142864

model-00004-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ce346bc583dd1d599942e7cf47073e6ec327d8819755631d6c6e4f82b937f84
+size 4966188864

model-00005-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16f04e7a8fab5a8c1b019cd4796c6d85029a77838ac7d1fcc1f7659db2d8a902
+size 4362142864

model-00006-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b85a91d5582718803ec7b44a3cf785cd4783fe9cf2918b1bcc1fc03c6c835f90
+size 4362142864

model-00007-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:282d25d09b344dc38a3f2ca8d7065b674ad9c9f0ea80771ea83933e7bed65571
+size 4966188864

model-00008-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e239e891b1eecdb4d2441d490c178aeb0d956d9933de14a2275b6d9ad5d8ba88
+size 4362142864

model-00009-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2ffcbe171f1aefeefd375e64fd98558331a291fb82bb077f5dc66d50c87754d
+size 4362142880

model-00010-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab17a3cdd02fd13ee43feb68dd69673976452ab55c3d12b71f2f27fae17d0bbe
+size 4966188880

model-00011-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b121b0245483ee8977eccd899ac2ad6d4b213615b4193adc3bfd1d4c93b77771
+size 4362142872

model-00012-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee25283ea10e832d22429b418ee018743be64961f17d5a605567b416c82a4f37
+size 4362142872

model-00013-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa6a1345a99b3d62822216832d96d9f16106901e28c3ea1c56e31635b81aafc5
+size 4966188880

model-00014-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba1a11cabe96a8564777a50e38a43a093dabfb782d465272855e963b0167d07a
+size 4362142872

model-00015-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:13aefacedd12398d6ad6de58721033e39e9c0108417e1f61c243b1df60820e0b
+size 4362142872

model-00016-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8bc1561d4acdab98c64cd1341f9868043f705138409823ad9c2359ddeb115c00
+size 4966188880

model-00017-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c7f17f97d44422579e73d9ffbabc034c6a7195f3789fef10ec6765672de6e9e
+size 4362142872

model-00018-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:defa0c85c8de1bdfa4ba32a1e0523e13766a3b3e62999c6994b7e6f093d5b5b3
+size 4362142872

model-00019-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44edc1f1eb5cfce84316b45675bd06f69cb6092c979fb01fac3faa84e6023f7d
+size 4966188880

model-00020-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc7ca8c5f0749a7e0bed393b34c91b291a744e89a3141970e6b93042c79427cb
+size 4362142872

model-00021-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:774e104b2650e3074bbac54ff609019b5f0f33a4dbd18f8d318c22c0d32935cd
+size 4362142872

model-00022-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd80141743cc6dbd0c37f55db0a82bcbdf6d9403cb4be33f05c373706409c524
+size 4966188880

model-00023-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c80ff7bf6cb7a224f94d85ca4761084e2054c2f1ba8731c699a8a3fa5610281e
+size 4362142872

model-00024-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f519ed6c360621de8ffdf92e6b1a2e98baf663349f5753846eb5aceb8fa0888a
+size 4362142872

model-00025-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c3a7a4275ac81bfbfda26ae274a6b19053fbd1e9f173f0e0e085c5fd80e7aa0
+size 4966188880

model-00026-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88b13f9b34097bc26a5fd4e5c2a630e28d996a4fd5bbb72190c81d81b9db3210
+size 4362142872

model-00027-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19bc7ba99eff6755e3bd9912ef8cad4ed1b5c510080d1d660413e33ccb07439d
+size 4362142872

model-00028-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:515a8490e3436174b2f4e475738b5a57e4ac531ce98bb31bf360bf857932c339
+size 4966188880

model-00029-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:efbb50b121679ca054ffcd35ee5c7d3288824526ce987de09539b654a96f09bb
+size 4362142872

model-00030-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a46f59ee8271e81edf94e8ed19fc1eb16918330d67df2cdb16cb57d7da1256a
+size 4362142872

model-00031-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b53ab711af280d61f5d1a859079f0afb31875add54881f9475a75096936b63d
+size 4966188880

model-00032-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae55cdb2068eb74e9e7c8bf1d197611bcbee166914ecaf0d02e934e33cf35ce4
+size 4362142872

model-00033-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48591d13610e36256cb29bec941a4fa38be0764417cc86e262094d993a1def6c
+size 4362142872

model-00034-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3882610d5e26b2a77adaedab156bea43ccf10df36f7e5793c753012c8c25e082
+size 4966188880

model-00035-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5ac456b7d83c8bbe2150b49210ba282c4abd72442f7357405ff6e86a6f1c92b
+size 4362142872

model-00036-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:566f92c1bcd666190bc10f2996a7a49f0596eb7f25bd91a7bc1c9ec95ea19852
+size 4362142872

model-00037-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf8a58beff5b513e18e24381d96101f67b460d943edb76041cffb0ea7d365965
+size 4966188880

model-00038-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e24cb4469f6eda4cb90410d85c5158c5979c85332c6b92b9e2d15af18777aab8
+size 4362142872

model-00039-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e15da6ecb848d2227febc792d0845f602f43bfe6b6ef76cd8cb7d34b0fe6527
+size 4362142872

model-00040-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aad4f7a8dd3894e897f57a77574e1e3a4efca3b74634b6ce6f4bbea80cceba85
+size 4966188880

model-00041-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57496989e8e92e6f77d044e52aab0a0650cdcb490e14f1ff4412fccad59cf572
+size 4362142872

model-00042-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73e17ced3c66a2a4020152a556533da12a22f39f8efe92066ec9a0903cd6ad40
+size 4362142872

model-00043-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:114972c675fb68728435dd53bd44de504deb0d38eb0070fc209b93d8cb716d0a
+size 4966188880

model-00044-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9972a09296098816c899280a0afd922af8bf1b3ea379c852b6f32c10909ace07
+size 4362142872

model-00045-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4bcd96437b31884671604afee48ecec7d79b9ef53756d7d3b195f8c9b6cc97ed
+size 4362142872

model-00046-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a2d35719cc31bf5300f138e5eb9bfd3c1e521b2734dd854967e18d4a49fb90a
+size 4966188880

model-00047-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87cf9a86812b61b6432aa740826b1f49e6e537c4de768bca73c51b7f3f81830a
+size 4362142872