README.md

---
language:
- en
- zh
- fr
pipeline_tag: text-generation
---
This repository is to store all parameters for RWKV6 distillation from Qwen25.

Try to use the model via : https://github.com/yynil/RWKVinLLAMA/blob/main/rwkv_llama/test_custom_cache_rwkv.py

Examples:

1. Layer 2 [0,2] Distilled model:
Journal planing
```bash
python rwkv_llama/test_custom_cache_rwkv.py --config_file configs/step_wise/test_hybrid_2_layer_qwenmlp_local.yaml --ckpt_file /home/yueyulin/model/qwen/layer2.pth --input_text "I will stay in Beijing on Oct 5,6, 2024. Please help me to make a jounal plan for these two days."
...
...
Certainly! Here's a sample journal plan for your stay in Beijing on October 5th and 6th, 2024:

### Day 1: October 5, 2024

**Morning:**
- **8:00 AM - 9:00 AM:** Wake up and have breakfast at your hotel.
- **9:30 AM - 12:00 PM:** Visit the Forbidden City (Palace Museum). Explore the grand halls, courtyards, and exhibitions showcasing imperial history.
- **12:00 PM - 1:00 PM:** Have lunch at a nearby restaurant.
- **1:30 PM - 4:00 PM:** Walk around Tiananmen Square and take photos of the iconic Tiananmen Gate and the National Museum of China.
- **4:30 PM - 6:00 PM:** Stroll through the scenic Beihai Park, enjoying the beautiful gardens and lake.

**Evening:**
- **6:30 PM - 8:00 PM:** Enjoy dinner at a local restaurant, perhaps trying Peking duck or other traditional dishes.
- **8:30 PM - 10:00 PM:** Watch a performance of Beijing Opera or a traditional Chinese acrobatics show.
- **10:00 PM:** Return to your hotel and relax.

### Day 2: October 6, 2024

**Morning:**
- **8:00 AM - 9:00 AM:** Have breakfast at your hotel.
- **9:30 AM - 12:00 PM:** Visit the Summer Palace, one of the largest and most well-preserved imperial gardens in China. Explore the Hall of Happiness, the Long Corridor, and the Kunming Lake.
- **12:00 PM - 1:00 PM:** Have lunch at a nearby restaurant.
- **1:30 PM - 4:00 PM:** Explore the Temple of Heaven, known for its beautiful architecture and historical significance. Take a guided tour to learn about the religious practices and rituals that took place there.
- **4:30 PM - 6:00 PM:** Visit the National Library of China, which houses an extensive collection of books, manuscripts, and rare documents.

**Evening:**
- **6:30 PM - 8:00 PM:** Enjoy dinner at a local restaurant, perhaps trying some Sichuan cuisine.
- **8:30 PM - 10:00 PM:** Take a leisurely walk along Qianmen Street, a historic commercial area with many shops, restaurants, and cultural attractions.
- **10:00 PM:** Return to your hotel and prepare for departure or further exploration.

### Tips:
- **Transportation:** Consider using public transportation like the subway or buses to save time and money. The Beijing Subway is efficient and covers most major tourist areas.
- **Weather:** Check the weather forecast before you go out each day. Beijing can be quite cold in the fall, so dress accordingly.
- **Safety:** Keep your valuables secure and be aware of your surroundings, especially in crowded areas.
- **Language:** While many people speak English, it’s always helpful to know some basic Mandarin phrases.

Enjoy your trip to Beijing! If you need more specific recommendations or details, feel free to ask.
```

Mathmatic solver:
```bash
python rwkv_llama/test_custom_cache_rwkv.py --config_file configs/step_wise/test_hybrid_2_layer_qwenmlp_local.yaml --ckpt_file /home/yueyulin/model/qwen/layer2.pth --input_text "## Question: \n\nFarmer Brown has 20 animals on his farm, all either chickens or cows. They have a total of 70 legs, all together. How many of the animals are chickens? ## Instruction \n\nPlease answer this question by first reasoning and then providing your answer."
...
...
To solve this problem, we can use a system of equations based on the information given.

Let's define:
- \( C \) as the number of chickens.
- \( K \) as the number of cows.

From the problem, we know two things:
1. The total number of animals is 20.
2. The total number of legs is 70.

We can write these as equations:
1. \( C + K = 20 \)
2. Chickens have 2 legs each, and cows have 4 legs each, so \( 2C + 4K = 70 \).

First, let's simplify the second equation by dividing everything by 2:
\[ C + 2K = 35 \]

Now we have the system:
1. \( C + K = 20 \)
2. \( C + 2K = 35 \)

Next, we can subtract the first equation from the second to eliminate \( C \):
\[ (C + 2K) - (C + K) = 35 - 20 \]
\[ C + 2K - C - K = 15 \]
\[ K = 15 \]

So, there are 15 cows. Now, we can substitute \( K = 15 \) back into the first equation to find \( C \):
\[ C + 15 = 20 \]
\[ C = 20 - 15 \]
\[ C = 5 \]

Therefore, there are 5 chickens on Farmer Brown's farm.
```