Commit
·
850fc96
1
Parent(s):
114db8f
edit instructions
Browse files
auto-exl2-upload/INSTRUCTIONS.txt
CHANGED
@@ -21,7 +21,7 @@ After setup is complete then you'll have a file called start-quant. Use this to
|
|
21 |
Make sure that your storage space is 3x the amount of the model's size. To mesure this, take the number of billion parameters and mutliply by two, afterwards mutliply by 3 and that's the recommended storage. There's a chance you may get away with 2.5x the size as well.
|
22 |
Make sure to also have a lot of RAM depending on the model.
|
23 |
|
24 |
-
If you close the terminal or the terminal crashes, check the last BPW it was on and enter the remaining quants you wanted. It should be able to pick up where it left off. Don't type the
|
25 |
|
26 |
To add more options to the quantization process, you can add them to line 170. All options: https://github.com/turboderp/exllamav2/blob/master/doc/convert.md
|
27 |
|
|
|
21 |
Make sure that your storage space is 3x the amount of the model's size. To mesure this, take the number of billion parameters and mutliply by two, afterwards mutliply by 3 and that's the recommended storage. There's a chance you may get away with 2.5x the size as well.
|
22 |
Make sure to also have a lot of RAM depending on the model.
|
23 |
|
24 |
+
If you close the terminal or the terminal crashes, check the last BPW it was on and enter the remaining quants you wanted. It should be able to pick up where it left off. Don't type the BPW of completed quants as it will start from the beginning. You may also use ctrl + c pause at any time during the quant process.
|
25 |
|
26 |
To add more options to the quantization process, you can add them to line 170. All options: https://github.com/turboderp/exllamav2/blob/master/doc/convert.md
|
27 |
|
auto-exl2-upload/auto-exl2-upload.zip
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:a349a20b47f44a0b83615faa7f45efdea6be73e71e50b2f90ff0b84242605b07
|
3 |
+
size 6945
|
exl2-multi-quant-local/INSTRUCTIONS.txt
CHANGED
@@ -21,7 +21,7 @@ After setup is complete then you'll have a file called start-quant. Use this to
|
|
21 |
Make sure that your storage space is 3x the amount of the model's size plus 1 more time per quant. To mesure this, take the number of billion parameters and mutliply by two, afterwards mutliply by 3 and that's the recommended storage. There's a chance you may get away with 2.5x the size as well.
|
22 |
Make sure to also have a lot of RAM depending on the model.
|
23 |
|
24 |
-
If you close the terminal or the terminal crashes, check the last BPW it was on and enter the remaining quants you wanted. It should be able to pick up where it left off. Don't type the
|
25 |
|
26 |
To add more options to the quantization process, you can add them to line 126. All options: https://github.com/turboderp/exllamav2/blob/master/doc/convert.md
|
27 |
|
|
|
21 |
Make sure that your storage space is 3x the amount of the model's size plus 1 more time per quant. To mesure this, take the number of billion parameters and mutliply by two, afterwards mutliply by 3 and that's the recommended storage. There's a chance you may get away with 2.5x the size as well.
|
22 |
Make sure to also have a lot of RAM depending on the model.
|
23 |
|
24 |
+
If you close the terminal or the terminal crashes, check the last BPW it was on and enter the remaining quants you wanted. It should be able to pick up where it left off. Don't type the BPW of completed quants as it will start from the beginning. You may also use ctrl + c pause at any time during the quant process.
|
25 |
|
26 |
To add more options to the quantization process, you can add them to line 126. All options: https://github.com/turboderp/exllamav2/blob/master/doc/convert.md
|
27 |
|
exl2-multi-quant-local/exl2-multi-quant-local.zip
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 5985
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:df071dea5e0f8a33b2828168f9c60b0218b00383a15c3fe16fff9a93bd0f400c
|
3 |
size 5985
|