Text Generation
Transformers
Safetensors
English
llama
climate
conversational
text-generation-inference
Inference Endpoints
ringoreality commited on
Commit
c802feb
·
1 Parent(s): b792fa5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -9
README.md CHANGED
@@ -1,7 +1,5 @@
1
  # Model Card for climategpt/climategpt-70b
2
  - This model is the 70B parameter variant of the ClimateGPT model release.
3
- - Starting from Llama2 70B weights, the model undergoes continued pretraining and instruction finetuning on climate data.
4
- - The model is capable of answering questions and following instructions, especially tailored for the climate domain.
5
 
6
  ## Overview
7
  - **Developed by:** AppTek, Eqtylab, Erasmus AI
@@ -30,13 +28,7 @@ For example, it is possible to evaluate the model with https://github.com/Eleuth
30
  ## Training
31
  - For the Llama2 training data, we refer the user to https://huggingface.co/meta-llama/Llama-2-70b-chat-hf.
32
  - For continued pretraining, 4.2B climate domain tokens (tokenized by the Llama tokenizer) are used.
33
- - For instruction finetuning, about 579K instruction-completion pairs (both in the climate domain but also general domain) are used.
34
-
35
- ## Evaluation
36
- Automatic evaluation is done via https://github.com/EleutherAI/lm-evaluation-harness, into which we also implemented custom evaluation tasks.
37
- TO BE ADDED
38
- We also perform human evaluation with experts in the climate domain.
39
- TO BE ADDED
40
 
41
  ## Environmental Impact
42
  - **Hardware Type:** H100
 
1
  # Model Card for climategpt/climategpt-70b
2
  - This model is the 70B parameter variant of the ClimateGPT model release.
 
 
3
 
4
  ## Overview
5
  - **Developed by:** AppTek, Eqtylab, Erasmus AI
 
28
  ## Training
29
  - For the Llama2 training data, we refer the user to https://huggingface.co/meta-llama/Llama-2-70b-chat-hf.
30
  - For continued pretraining, 4.2B climate domain tokens (tokenized by the Llama tokenizer) are used.
31
+ - For instruction finetuning, about 1.1B instruction-finetuning tokens (both in the climate domain but also general domain) are used.
 
 
 
 
 
 
32
 
33
  ## Environmental Impact
34
  - **Hardware Type:** H100