Create README.md

README.md

@@ -0,0 +1,96 @@
+---
+language:
+- en
+- zh
+- fr
+pipeline_tag: text-generation
+---
+This repository is to store all parameters for RWKV6 distillation from Qwen25.
+
+Try to use the model via : https://github.com/yynil/RWKVinLLAMA/blob/main/rwkv_llama/test_custom_cache_rwkv.py
+
+Examples:
+
+1. Layer 2 [0,2] Distilled model:
+Journal planing
+```bash
+python rwkv_llama/test_custom_cache_rwkv.py --config_file configs/step_wise/test_hybrid_2_layer_qwenmlp_local.yaml --ckpt_file /home/yueyulin/model/qwen/layer2.pth --input_text "I will stay in Beijing on Oct 5,6, 2024. Please help me to make a jounal plan for these two days."
+...
+...
+Certainly! Here's a sample journal plan for your stay in Beijing on October 5th and 6th, 2024:
+
+### Day 1: October 5, 2024
+
+**Morning:**
+- **8:00 AM - 9:00 AM:** Wake up and have breakfast at your hotel.
+- **9:30 AM - 12:00 PM:** Visit the Forbidden City (Palace Museum). Explore the grand halls, courtyards, and exhibitions showcasing imperial history.
+- **12:00 PM - 1:00 PM:** Have lunch at a nearby restaurant.
+- **1:30 PM - 4:00 PM:** Walk around Tiananmen Square and take photos of the iconic Tiananmen Gate and the National Museum of China.
+- **4:30 PM - 6:00 PM:** Stroll through the scenic Beihai Park, enjoying the beautiful gardens and lake.
+
+**Evening:**
+- **6:30 PM - 8:00 PM:** Enjoy dinner at a local restaurant, perhaps trying Peking duck or other traditional dishes.
+- **8:30 PM - 10:00 PM:** Watch a performance of Beijing Opera or a traditional Chinese acrobatics show.
+- **10:00 PM:** Return to your hotel and relax.
+
+### Day 2: October 6, 2024
+
+**Morning:**
+- **8:00 AM - 9:00 AM:** Have breakfast at your hotel.
+- **9:30 AM - 12:00 PM:** Visit the Summer Palace, one of the largest and most well-preserved imperial gardens in China. Explore the Hall of Happiness, the Long Corridor, and the Kunming Lake.
+- **12:00 PM - 1:00 PM:** Have lunch at a nearby restaurant.
+- **1:30 PM - 4:00 PM:** Explore the Temple of Heaven, known for its beautiful architecture and historical significance. Take a guided tour to learn about the religious practices and rituals that took place there.
+- **4:30 PM - 6:00 PM:** Visit the National Library of China, which houses an extensive collection of books, manuscripts, and rare documents.
+
+**Evening:**
+- **6:30 PM - 8:00 PM:** Enjoy dinner at a local restaurant, perhaps trying some Sichuan cuisine.
+- **8:30 PM - 10:00 PM:** Take a leisurely walk along Qianmen Street, a historic commercial area with many shops, restaurants, and cultural attractions.
+- **10:00 PM:** Return to your hotel and prepare for departure or further exploration.
+
+### Tips:
+- **Transportation:** Consider using public transportation like the subway or buses to save time and money. The Beijing Subway is efficient and covers most major tourist areas.
+- **Weather:** Check the weather forecast before you go out each day. Beijing can be quite cold in the fall, so dress accordingly.
+- **Safety:** Keep your valuables secure and be aware of your surroundings, especially in crowded areas.
+- **Language:** While many people speak English, it’s always helpful to know some basic Mandarin phrases.
+
+Enjoy your trip to Beijing! If you need more specific recommendations or details, feel free to ask.
+```
+
+Mathmatic solver:
+```bash
+python rwkv_llama/test_custom_cache_rwkv.py --config_file configs/step_wise/test_hybrid_2_layer_qwenmlp_local.yaml --ckpt_file /home/yueyulin/model/qwen/layer2.pth --input_text "## Question: \n\nFarmer Brown has 20 animals on his farm, all either chickens or cows. They have a total of 70 legs, all together. How many of the animals are chickens? ## Instruction \n\nPlease answer this question by first reasoning and then providing your answer."
+...
+...
+To solve this problem, we can use a system of equations based on the information given.
+
+Let's define:
+- \( C \) as the number of chickens.
+- \( K \) as the number of cows.
+
+From the problem, we know two things:
+1. The total number of animals is 20.
+2. The total number of legs is 70.
+
+We can write these as equations:
+1. \( C + K = 20 \)
+2. Chickens have 2 legs each, and cows have 4 legs each, so \( 2C + 4K = 70 \).
+
+First, let's simplify the second equation by dividing everything by 2:
+\[ C + 2K = 35 \]
+
+Now we have the system:
+1. \( C + K = 20 \)
+2. \( C + 2K = 35 \)
+
+Next, we can subtract the first equation from the second to eliminate \( C \):
+\[ (C + 2K) - (C + K) = 35 - 20 \]
+\[ C + 2K - C - K = 15 \]
+\[ K = 15 \]
+
+So, there are 15 cows. Now, we can substitute \( K = 15 \) back into the first equation to find \( C \):
+\[ C + 15 = 20 \]
+\[ C = 20 - 15 \]
+\[ C = 5 \]
+
+Therefore, there are 5 chickens on Farmer Brown's farm.
+```