AdilZtn HF Staff commited on
Commit
ec04acd
·
verified ·
1 Parent(s): 8fad46f

Push model using huggingface_hub.

Browse files
Files changed (4) hide show
  1. README.md +196 -0
  2. processor.json +28 -0
  3. step_0.safetensors +1 -1
  4. step_1.safetensors +1 -1
README.md ADDED
@@ -0,0 +1,196 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: lerobot
3
+ tags:
4
+ - robotics
5
+ - lerobot
6
+ - safetensors
7
+ pipeline_tag: robotics
8
+ ---
9
+
10
+ # RobotProcessor
11
+
12
+ ## Overview
13
+
14
+ RobotProcessor is a composable, debuggable post-processing pipeline for robot transitions in the LeRobot framework. It orchestrates an ordered collection of small, functional transforms (steps) that are executed left-to-right on each incoming `EnvTransition`.
15
+
16
+ ## Architecture
17
+
18
+ The RobotProcessor provides a modular architecture for processing robot environment transitions through a sequence of composable steps. Each step is a callable that accepts a full `EnvTransition` tuple and returns a potentially modified tuple of the same structure.
19
+
20
+ ### EnvTransition Structure
21
+
22
+ An `EnvTransition` is a 7-tuple containing:
23
+ 1. **observation**: Current state observation
24
+ 2. **action**: Action taken (can be None)
25
+ 3. **reward**: Reward received (float or None)
26
+ 4. **done**: Episode termination flag (bool or None)
27
+ 5. **truncated**: Episode truncation flag (bool or None)
28
+ 6. **info**: Additional information dictionary
29
+ 7. **complementary_data**: Extra data dictionary
30
+
31
+ ## Key Features
32
+
33
+ - **Composable Pipeline**: Chain multiple processing steps in a specific order
34
+ - **State Persistence**: Save and load processor state using SafeTensors format
35
+ - **Hugging Face Hub Integration**: Easy sharing and loading via `save_pretrained()` and `from_pretrained()`
36
+ - **Debugging Support**: Step-through functionality to inspect intermediate transformations
37
+ - **Hook System**: Before/after step hooks for additional processing or monitoring
38
+ - **Device Support**: Move tensor states to different devices (CPU/GPU)
39
+ - **Performance Profiling**: Built-in profiling to identify bottlenecks
40
+
41
+ ## Installation
42
+
43
+ ```bash
44
+ pip install lerobot
45
+ ```
46
+
47
+ ## Usage
48
+
49
+ ### Basic Example
50
+
51
+ ```python
52
+ from lerobot.processor.pipeline import RobotProcessor
53
+ from your_steps import ObservationNormalizer, VelocityCalculator
54
+
55
+ # Create a processor with multiple steps
56
+ processor = RobotProcessor(
57
+ steps=[
58
+ ObservationNormalizer(mean=0, std=1),
59
+ VelocityCalculator(window_size=5),
60
+ ],
61
+ name="my_robot_processor",
62
+ seed=42
63
+ )
64
+
65
+ # Process a transition
66
+ obs, info = env.reset()
67
+ transition = (obs, None, 0.0, False, False, info, {})
68
+ processed_transition = processor(transition)
69
+
70
+ # Extract processed observation
71
+ processed_obs = processed_transition[0]
72
+ ```
73
+
74
+ ### Saving and Loading
75
+
76
+ ```python
77
+ # Save locally
78
+ processor.save_pretrained("./my_processor")
79
+
80
+ # Push to Hugging Face Hub
81
+ processor.push_to_hub("username/my-robot-processor")
82
+
83
+ # Load from Hub
84
+ loaded_processor = RobotProcessor.from_pretrained("username/my-robot-processor")
85
+ ```
86
+
87
+ ### Debugging with Step-Through
88
+
89
+ ```python
90
+ # Inspect intermediate results
91
+ for idx, intermediate_transition in enumerate(processor.step_through(transition)):
92
+ print(f"After step {idx}: {intermediate_transition[0]}") # Print observation
93
+ ```
94
+
95
+ ### Using Hooks
96
+
97
+ ```python
98
+ # Add monitoring hook
99
+ def log_observation(step_idx, transition):
100
+ print(f"Step {step_idx}: obs shape = {transition[0].shape}")
101
+ return None # Don't modify transition
102
+
103
+ processor.register_before_step_hook(log_observation)
104
+ ```
105
+
106
+ ## Creating Custom Steps
107
+
108
+ To create a custom processor step, implement the `ProcessorStep` protocol:
109
+
110
+ ```python
111
+ from lerobot.processor.pipeline import ProcessorStepRegistry, EnvTransition
112
+
113
+ @ProcessorStepRegistry.register("my_custom_step")
114
+ class MyCustomStep:
115
+ def __init__(self, param1=1.0):
116
+ self.param1 = param1
117
+ self.buffer = []
118
+
119
+ def __call__(self, transition: EnvTransition) -> EnvTransition:
120
+ obs, action, reward, done, truncated, info, comp_data = transition
121
+ # Process observation
122
+ processed_obs = obs * self.param1
123
+ return (processed_obs, action, reward, done, truncated, info, comp_data)
124
+
125
+ def get_config(self) -> dict:
126
+ return {"param1": self.param1}
127
+
128
+ def state_dict(self) -> dict:
129
+ # Return only torch.Tensor state
130
+ return {}
131
+
132
+ def load_state_dict(self, state: dict) -> None:
133
+ # Load tensor state
134
+ pass
135
+
136
+ def reset(self) -> None:
137
+ # Clear buffers at episode boundaries
138
+ self.buffer.clear()
139
+ ```
140
+
141
+ ## Advanced Features
142
+
143
+ ### Device Management
144
+
145
+ ```python
146
+ # Move all tensor states to GPU
147
+ processor = processor.to("cuda")
148
+
149
+ # Move to specific device
150
+ processor = processor.to(torch.device("cuda:1"))
151
+ ```
152
+
153
+ ### Performance Profiling
154
+
155
+ ```python
156
+ # Profile step execution times
157
+ profile_results = processor.profile_steps(transition, num_runs=100)
158
+ for step_name, time_ms in profile_results.items():
159
+ print(f"{step_name}: {time_ms:.3f} ms")
160
+ ```
161
+
162
+ ### Processor Slicing
163
+
164
+ ```python
165
+ # Get a single step
166
+ first_step = processor[0]
167
+
168
+ # Create a sub-processor with steps 1-3
169
+ sub_processor = processor[1:4]
170
+ ```
171
+
172
+ ## Model Card Specifications
173
+
174
+ - **Pipeline Tag**: robotics
175
+ - **Library**: lerobot
176
+ - **Format**: safetensors
177
+ - **License**: Apache 2.0
178
+
179
+ ## Limitations
180
+
181
+ - Steps must maintain the 7-tuple structure of EnvTransition
182
+ - All tensor state must be separated from configuration for proper serialization
183
+ - Steps are executed sequentially (no parallel processing within a single transition)
184
+
185
+ ## Citation
186
+
187
+ If you use RobotProcessor in your research, please cite:
188
+
189
+ ```bibtex
190
+ @misc{cadene2024lerobot,
191
+ author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascale, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},
192
+ title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
193
+ howpublished = "\url{https://github.com/huggingface/lerobot}",
194
+ year = {2024}
195
+ }
196
+ ```
processor.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "StatefulProcessor",
3
+ "seed": 42,
4
+ "steps": [
5
+ {
6
+ "registry_name": "adaptive_normalizer",
7
+ "config": {
8
+ "name": "obs_normalizer",
9
+ "learning_rate": 0.005,
10
+ "warmup_steps": 50,
11
+ "outlier_threshold": 2.5,
12
+ "epsilon": 1e-08
13
+ },
14
+ "state_file": "step_0.safetensors"
15
+ },
16
+ {
17
+ "registry_name": "temporal_reward_shaper",
18
+ "config": {
19
+ "name": "reward_shaper",
20
+ "buffer_size": 100,
21
+ "momentum_decay": 0.95,
22
+ "bonus_scale": 0.05,
23
+ "improvement_threshold": 0.1
24
+ },
25
+ "state_file": "step_1.safetensors"
26
+ }
27
+ ]
28
+ }
step_0.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fc697e78331ebbf773f2bf575bef72749e5780f9e9a6f5a8dd3fea04e86f4d9b
3
  size 400
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff0cfbb3f5d62e608e89c16ac6c5150f64a1d2530f1f3c8107fa5fd2405217c0
3
  size 400
step_1.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1a0db5a0de39513175cb411111b1fc48b65f67d7ca0c6bea77921fb6ac9b716b
3
  size 924
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6b06cd9e5594375e8215e5f762605ffe5b7781529dbc55ee7b46743dfd209902
3
  size 924