Running
on
Zero
5
๐
Pixtral 12B EXL2
A demo chatbot using Pixtral 12B with ExllamaV2!
Margaux Ammour
mammour
ยท
AI & ML interests
Instead of looking for something that is potentially harmful, better grasp what we can already make happen
Artificially Augmented Intelligence advocate
Recent Activity
liked
a Space
about 2 months ago
pandora-s/Pixtral-12B-EXL2
new activity
about 2 months ago
Qwen/QwQ-32B-Preview:multi GPU inferencing
new activity
3 months ago
Jacoby746/Casual-Magnum-34B-exl2-4.0bpw:Error during inference
Organizations
mammour's activity
liked
a
Space
about 2 months ago
multi GPU inferencing
2
#18 opened about 2 months ago
by
cjj2003
Error during inference
7
#1 opened 4 months ago
by
Jellon
reacted to
singhsidhukuldeep's
post with ๐
4 months ago
Post
3996
Researchers have developed a novel approach called Logic-of-Thought (LoT) that significantly enhances the logical reasoning capabilities of large language models (LLMs).
Here are the steps on how Logic-of-Thought (LoT) is implemented:
-- 1. Logic Extraction
1. Use Large Language Models (LLMs) to identify sentences containing conditional reasoning relationships from the input context.
2. Generate a collection of sentences with logical relationships.
3. Use LLMs to extract the set of propositional symbols and logical expressions from the collection.
4. Identify propositions with similar meanings and represent them using identical propositional symbols.
5. Analyze the logical relationships between propositions based on their natural language descriptions.
6. Add negation (ยฌ) for propositions that express opposite meanings.
7. Use implication (โ) to connect propositional symbols when a conditional relationship exists.
-- 2. Logic Extension
1. Apply logical reasoning laws to the collection of logical expressions from the Logic Extraction phase.
2. Use a Python program to implement logical deduction and expand the expressions.
3. Apply logical laws such as Double Negation, Contraposition, and Transitivity to derive new logical expressions.
-- 3. Logic Translation
1. Use LLMs to translate the newly generated logical expressions into natural language descriptions.
2. Combine the natural language descriptions of propositional symbols according to the extended logical expressions.
3. Incorporate the translated logical information as a new part of the original input prompt.
-- 4. Integration with Existing Prompting Methods
1. Combine the LoT-generated logical information with the original prompt.
2. Use this enhanced prompt with existing prompting methods like Chain-of-Thought (CoT), Self-Consistency (SC), or Tree-of-Thoughts (ToT).
3. Feed the augmented prompt to the LLM to generate the final answer.
What do you think about LoT?
Here are the steps on how Logic-of-Thought (LoT) is implemented:
-- 1. Logic Extraction
1. Use Large Language Models (LLMs) to identify sentences containing conditional reasoning relationships from the input context.
2. Generate a collection of sentences with logical relationships.
3. Use LLMs to extract the set of propositional symbols and logical expressions from the collection.
4. Identify propositions with similar meanings and represent them using identical propositional symbols.
5. Analyze the logical relationships between propositions based on their natural language descriptions.
6. Add negation (ยฌ) for propositions that express opposite meanings.
7. Use implication (โ) to connect propositional symbols when a conditional relationship exists.
-- 2. Logic Extension
1. Apply logical reasoning laws to the collection of logical expressions from the Logic Extraction phase.
2. Use a Python program to implement logical deduction and expand the expressions.
3. Apply logical laws such as Double Negation, Contraposition, and Transitivity to derive new logical expressions.
-- 3. Logic Translation
1. Use LLMs to translate the newly generated logical expressions into natural language descriptions.
2. Combine the natural language descriptions of propositional symbols according to the extended logical expressions.
3. Incorporate the translated logical information as a new part of the original input prompt.
-- 4. Integration with Existing Prompting Methods
1. Combine the LoT-generated logical information with the original prompt.
2. Use this enhanced prompt with existing prompting methods like Chain-of-Thought (CoT), Self-Consistency (SC), or Tree-of-Thoughts (ToT).
3. Feed the augmented prompt to the LLM to generate the final answer.
What do you think about LoT?
- 1 reply
liked
2
models
4 months ago
updated
a
Space
4 months ago
Please consider changing the instruct format!
19
#10 opened 4 months ago
by
MarinaraSpaghetti
reacted to
bartowski's
post with ๐
5 months ago
Post
4743
@victor
(is this the only way to "DM" on HF?)
Had a funny thought, would it be at all possible to rework what shows up on our personal HF page?
Picture this: I upload a model to an organization, someone who follows me now has no idea that I've uploaded a model or to where, unless they also watch those repos (which also floods them with other notifications)
What if our main Huggingface page was a collection of both models that we've uploaded specifically to our profile, as well as models we've uploaded to organizations? That way it would all be contained in one central followable location, and I wouldn't have concerns about losing followership if I wanted to upload to an organization all of a sudden.
Had a funny thought, would it be at all possible to rework what shows up on our personal HF page?
Picture this: I upload a model to an organization, someone who follows me now has no idea that I've uploaded a model or to where, unless they also watch those repos (which also floods them with other notifications)
What if our main Huggingface page was a collection of both models that we've uploaded specifically to our profile, as well as models we've uploaded to organizations? That way it would all be contained in one central followable location, and I wouldn't have concerns about losing followership if I wanted to upload to an organization all of a sudden.
- 3 replies
reacted to
m-ric's
post with ๐ง
6 months ago
Post
1750
๐ฆ๐๐ ๐ฎ ๐ฟ๐ฒ๐น๐ฒ๐ฎ๐๐ฒ๐ฑ: ๐ก๐ฒ๐ ๐ฆ๐ข๐ง๐ ๐ผ๐ป ๐๐ฒ๐ด๐บ๐ฒ๐ป๐๐ฎ๐๐ถ๐ผ๐ป, ๐ฏ๐ ๐ฐ๐ผ๐บ๐ฏ๐ถ๐ป๐ถ๐ป๐ด ๐๐๐ป๐๐ต๐ฒ๐๐ถ๐ฐ ๐ฑ๐ฎ๐๐ฎ ๐๐ถ๐๐ต ๐ต๐๐บ๐ฎ๐ป ๐ณ๐ฒ๐ฒ๐ฑ๐ฏ๐ฎ๐ฐ๐ธ ๐
It's a model for Object segmentation, for both image and video:
๐ input = a text prompt, or a click on a specific object
๐ output = the model draws a mask around the object. In video segmentation, the mask should follow the object's movements (it is then called a masklet)
๐ช SAM 2 is 6x faster than the previous version, it now also works on a video, and it beats SOTA by far on both image and video segmentation tasks.
How did they pull that?
The main blocker for video segmentation was that data is really hard to collect: to build your training dataset, should you manually draw masks on every frame? That would be way too costly! โก๏ธ As a result, existing video segmentation datasets have a real lack of coverage: few examples, few masklets drawn.
๐ก Key idea: researchers they decided to use a segmentation model to help them collect the dataset.
But then itโs a chicken and egg problem: you need the model to create the dataset and the opposite as well? ๐ค
โ To solve this, they build a data generation system that they scale up progressively in 3 successive manual annotations phases:
๐ฆ๐๐ฒ๐ฝ ๐ญ: Annotators use only SAM + manual editing tools on each frame โ Create 16k masklets across 1.4k videos
๐ฆ๐๐ฒ๐ฝ ๐ฎ: Then train a first SAM 2, add it in the loop to temporally propagate frames, and correct by re-doing a mask manually when an error has occured โ This gets a 5.1x speedup over data collection in phase 1! ๐ Collect 60k masklets
๐ฆ๐๐ฒ๐ฝ ๐ฏ: Now SAM 2 is more powerful, it has the โsingle clickโ prompting option, thus annotators can use it with simple clicks to re-annotate data.
They even add a completely automatic step to generate 350k more masklets!
And in turn, the model perf gradually increases.
I find this a great example of combining synthetic data generation with human annotation ๐
It's a model for Object segmentation, for both image and video:
๐ input = a text prompt, or a click on a specific object
๐ output = the model draws a mask around the object. In video segmentation, the mask should follow the object's movements (it is then called a masklet)
๐ช SAM 2 is 6x faster than the previous version, it now also works on a video, and it beats SOTA by far on both image and video segmentation tasks.
How did they pull that?
The main blocker for video segmentation was that data is really hard to collect: to build your training dataset, should you manually draw masks on every frame? That would be way too costly! โก๏ธ As a result, existing video segmentation datasets have a real lack of coverage: few examples, few masklets drawn.
๐ก Key idea: researchers they decided to use a segmentation model to help them collect the dataset.
But then itโs a chicken and egg problem: you need the model to create the dataset and the opposite as well? ๐ค
โ To solve this, they build a data generation system that they scale up progressively in 3 successive manual annotations phases:
๐ฆ๐๐ฒ๐ฝ ๐ญ: Annotators use only SAM + manual editing tools on each frame โ Create 16k masklets across 1.4k videos
๐ฆ๐๐ฒ๐ฝ ๐ฎ: Then train a first SAM 2, add it in the loop to temporally propagate frames, and correct by re-doing a mask manually when an error has occured โ This gets a 5.1x speedup over data collection in phase 1! ๐ Collect 60k masklets
๐ฆ๐๐ฒ๐ฝ ๐ฏ: Now SAM 2 is more powerful, it has the โsingle clickโ prompting option, thus annotators can use it with simple clicks to re-annotate data.
They even add a completely automatic step to generate 350k more masklets!
And in turn, the model perf gradually increases.
I find this a great example of combining synthetic data generation with human annotation ๐
liked
2
models
7 months ago
liked
a
Space
8 months ago
What is the context length?
3
#2 opened 9 months ago
by
softwareweaver
liked
2
models
9 months ago
reacted to
m-ric's
post with โค๏ธ
10 months ago
Post
2071
[๐๐๐ฐ ๐๐๐ฉ๐๐ซ] ๐๐ฅ๐ฅ ๐ญ๐จ๐ค๐๐ง๐ฌ ๐ฌ๐ก๐จ๐ฎ๐ฅ๐ ๐ง๐จ๐ญ ๐ซ๐๐ช๐ฎ๐ข๐ซ๐ ๐ญ๐ก๐ ๐ฌ๐๐ฆ๐ ๐๐๐๐จ๐ซ๐ญ ๐ญ๐จ ๐๐จ๐ฆ๐ฉ๐ฎ๐ญ๐! โ ๐๐ข๐ฑ๐ญ๐ฎ๐ซ๐ ๐จ๐ ๐๐๐ฉ๐ญ๐ก๐ฌ ๐ซง๐
Google Researchers were unhappy with the way current decoding generally works: all tokens go through the same layers, thus requiring exactly the same effort to compute.
Whereas in reality, completing the answer to a difficult math problem for instance should be more computationally intense than completing the text of the Declaration of Independence: ๐ป๐ผ๐ ๐ฎ๐น๐น ๐๐ผ๐ธ๐ฒ๐ป๐ ๐ฎ๐ฟ๐ฒ ๐ฐ๐ฟ๐ฒ๐ฎ๐๐ฒ๐ฑ ๐ฒ๐พ๐๐ฎ๐น!
โก๏ธ ๐ง๐ต๐ฒ๐ ๐ต๐ฎ๐ฑ ๐๐ต๐ถ๐ ๐ด๐ฒ๐ป๐ถ๐๐ ๐ถ๐ฑ๐ฒ๐ฎ: ๐ก ๐ต๐ฎ๐๐ถ๐ป๐ด ๐ฎ ๐๐ผ๐ธ๐ฒ๐ป ๐ด๐ผ ๐๐ต๐ฟ๐ผ๐๐ด๐ต ๐ฎ ๐ฏ๐น๐ผ๐ฐ๐ธ ๐๐ต๐ผ๐๐น๐ฑ ๐ฏ๐ฒ ๐ผ๐ฝ๐๐ถ๐ผ๐ป๐ฎ๐น. The token can go through the block (thus undergoing expensive self-attention computation) or avoid it through a skip connection.
The routing decision is taken on the block level: each block selects from the total sequence the top-k tokens that will go through it, and the others tokens will skip it. ๐๐ฉ๐ช๐ด ๐ข๐ญ๐ญ๐ฐ๐ธ๐ด ๐ต๐ฐ ๐ค๐ฉ๐ฐ๐ฐ๐ด๐ฆ ๐ต๐ฉ๐ฆ ๐ฆ๐น๐ข๐ค๐ต ๐๐๐ฅ๐๐๐๐ฉ๐ฎ ๐ฐ๐ง ๐ข ๐ฃ๐ญ๐ฐ๐ค๐ฌ, ๐ช.๐ฆ. ๐ต๐ฉ๐ฆ ๐ฑ๐ณ๐ฐ๐ฑ๐ฐ๐ณ๐ต๐ช๐ฐ๐ฏ ๐ฐ๐ง ๐ต๐ฐ๐ฌ๐ฆ๐ฏ๐ด ๐ต๐ฉ๐ข๐ต ๐จ๐ฐ ๐ต๐ฉ๐ณ๐ฐ๐ถ๐จ๐ฉ ๐ช๐ต, ๐ธ๐ฉ๐ช๐ค๐ฉ ๐ฅ๐ช๐ณ๐ฆ๐ค๐ต๐ญ๐บ ๐ช๐ฏ๐ง๐ญ๐ถ๐ฆ๐ฏ๐ค๐ฆ๐ด ๐ต๐ฉ๐ฆ ๐ค๐ฐ๐ฎ๐ฑ๐ถ๐ต๐ข๐ต๐ช๐ฐ๐ฏ๐ข๐ญ ๐ช๐ฏ๐ต๐ฆ๐ฏ๐ด๐ช๐ต๐บ ๐ฐ๐ง ๐ต๐ฉ๐ฆ ๐ง๐ฐ๐ณ๐ธ๐ข๐ณ๐ฅ ๐ฑ๐ข๐ด๐ด.
This yields Mixture-of-Depths (MoD), with spectacular results.
โจ ๐ฅ๐ฒ๐๐๐น๐๐:
๐๏ธ ๐๐ฎ๐ฝ๐ฎ๐ฐ๐ถ๐๐ ๐ฐ๐ฎ๐ป ๐ฏ๐ฒ ๐๐๐ป๐ฒ๐ฑ ๐ฎ๐น๐น ๐๐ต๐ฒ ๐๐ฎ๐ ๐ฑ๐ผ๐๐ป ๐๐ผ ๐ญ๐ฎ.๐ฑ% for every second block: thus 87.5% of tokens just skip the block!
๐ For the same training time and performance, >๐ฒ๐ฌ% ๐ถ๐ป๐ณ๐ฒ๐ฟ๐ฒ๐ป๐ฐ๐ฒ ๐๐ฝ๐ฒ๐ฒ๐ฑ!
๐ค ๐๐ฎ๐ป ๐ฏ๐ฒ ๐ฐ๐ผ๐บ๐ฏ๐ถ๐ป๐ฒ๐ฑ ๐๐ถ๐๐ต ๐ ๐ถ๐ ๐๐๐ฟ๐ฒ-๐ผ๐ณ-๐๐ ๐ฝ๐ฒ๐ฟ๐๐ for further improvements.
๐ ๐ฃ๐ฎ๐ฝ๐ฒ๐ฟ ๐ต๐ฒ๐ฟ๐ฒ ๐ Mixture-of-Depths: Dynamically allocating compute in transformer-based language models (2404.02258)
๐ I added it to my paper collection ๐ m-ric/spinning-up-in-llms-659e698f9dd5a71bd3f579a7
Google Researchers were unhappy with the way current decoding generally works: all tokens go through the same layers, thus requiring exactly the same effort to compute.
Whereas in reality, completing the answer to a difficult math problem for instance should be more computationally intense than completing the text of the Declaration of Independence: ๐ป๐ผ๐ ๐ฎ๐น๐น ๐๐ผ๐ธ๐ฒ๐ป๐ ๐ฎ๐ฟ๐ฒ ๐ฐ๐ฟ๐ฒ๐ฎ๐๐ฒ๐ฑ ๐ฒ๐พ๐๐ฎ๐น!
โก๏ธ ๐ง๐ต๐ฒ๐ ๐ต๐ฎ๐ฑ ๐๐ต๐ถ๐ ๐ด๐ฒ๐ป๐ถ๐๐ ๐ถ๐ฑ๐ฒ๐ฎ: ๐ก ๐ต๐ฎ๐๐ถ๐ป๐ด ๐ฎ ๐๐ผ๐ธ๐ฒ๐ป ๐ด๐ผ ๐๐ต๐ฟ๐ผ๐๐ด๐ต ๐ฎ ๐ฏ๐น๐ผ๐ฐ๐ธ ๐๐ต๐ผ๐๐น๐ฑ ๐ฏ๐ฒ ๐ผ๐ฝ๐๐ถ๐ผ๐ป๐ฎ๐น. The token can go through the block (thus undergoing expensive self-attention computation) or avoid it through a skip connection.
The routing decision is taken on the block level: each block selects from the total sequence the top-k tokens that will go through it, and the others tokens will skip it. ๐๐ฉ๐ช๐ด ๐ข๐ญ๐ญ๐ฐ๐ธ๐ด ๐ต๐ฐ ๐ค๐ฉ๐ฐ๐ฐ๐ด๐ฆ ๐ต๐ฉ๐ฆ ๐ฆ๐น๐ข๐ค๐ต ๐๐๐ฅ๐๐๐๐ฉ๐ฎ ๐ฐ๐ง ๐ข ๐ฃ๐ญ๐ฐ๐ค๐ฌ, ๐ช.๐ฆ. ๐ต๐ฉ๐ฆ ๐ฑ๐ณ๐ฐ๐ฑ๐ฐ๐ณ๐ต๐ช๐ฐ๐ฏ ๐ฐ๐ง ๐ต๐ฐ๐ฌ๐ฆ๐ฏ๐ด ๐ต๐ฉ๐ข๐ต ๐จ๐ฐ ๐ต๐ฉ๐ณ๐ฐ๐ถ๐จ๐ฉ ๐ช๐ต, ๐ธ๐ฉ๐ช๐ค๐ฉ ๐ฅ๐ช๐ณ๐ฆ๐ค๐ต๐ญ๐บ ๐ช๐ฏ๐ง๐ญ๐ถ๐ฆ๐ฏ๐ค๐ฆ๐ด ๐ต๐ฉ๐ฆ ๐ค๐ฐ๐ฎ๐ฑ๐ถ๐ต๐ข๐ต๐ช๐ฐ๐ฏ๐ข๐ญ ๐ช๐ฏ๐ต๐ฆ๐ฏ๐ด๐ช๐ต๐บ ๐ฐ๐ง ๐ต๐ฉ๐ฆ ๐ง๐ฐ๐ณ๐ธ๐ข๐ณ๐ฅ ๐ฑ๐ข๐ด๐ด.
This yields Mixture-of-Depths (MoD), with spectacular results.
โจ ๐ฅ๐ฒ๐๐๐น๐๐:
๐๏ธ ๐๐ฎ๐ฝ๐ฎ๐ฐ๐ถ๐๐ ๐ฐ๐ฎ๐ป ๐ฏ๐ฒ ๐๐๐ป๐ฒ๐ฑ ๐ฎ๐น๐น ๐๐ต๐ฒ ๐๐ฎ๐ ๐ฑ๐ผ๐๐ป ๐๐ผ ๐ญ๐ฎ.๐ฑ% for every second block: thus 87.5% of tokens just skip the block!
๐ For the same training time and performance, >๐ฒ๐ฌ% ๐ถ๐ป๐ณ๐ฒ๐ฟ๐ฒ๐ป๐ฐ๐ฒ ๐๐ฝ๐ฒ๐ฒ๐ฑ!
๐ค ๐๐ฎ๐ป ๐ฏ๐ฒ ๐ฐ๐ผ๐บ๐ฏ๐ถ๐ป๐ฒ๐ฑ ๐๐ถ๐๐ต ๐ ๐ถ๐ ๐๐๐ฟ๐ฒ-๐ผ๐ณ-๐๐ ๐ฝ๐ฒ๐ฟ๐๐ for further improvements.
๐ ๐ฃ๐ฎ๐ฝ๐ฒ๐ฟ ๐ต๐ฒ๐ฟ๐ฒ ๐ Mixture-of-Depths: Dynamically allocating compute in transformer-based language models (2404.02258)
๐ I added it to my paper collection ๐ m-ric/spinning-up-in-llms-659e698f9dd5a71bd3f579a7
- 1 reply
liked
a
model
10 months ago
liked
2
models
11 months ago
