From Magnetic Tape to RL: How We Transformed a Warehouse into a Dynamic System
Imagine: a 20,000 m² warehouse, a fleet of 30 AGVs, but every morning — a bottleneck. AGVs collide in narrow aisles, idle waiting, and magnetic strips prevent quick route changes when blocked. We removed the fixed layout and replaced it with dynamic routing via multi-agent reinforcement learning (MARL). This same technology allowed cobots on the assembly line to pick parts of any shape without manual programming.
Implementing RL is a practical tool, not an experiment. We have completed 10+ projects for warehouses and assembly lines. Contact us for a free audit of your production.
How RL Solves Production Problems?
Grasping Irregularly Shaped Objects (Bin Picking)
The classical approach requires CAD models and hand-coded trajectories. RL solves this without that: the cobot learns to grasp solely from simulation. State space: point cloud from Intel RealSense D435 + joint angles. Action: 7-DoF continuous gripper movement. The reward is constructed as follows:
def grasp_reward(self): if self._check_grasp_success(): return 1.0 + lift_height_bonus elif self._check_collision(): return -1.0 else: dist = np.linalg.norm(self.gripper_pos - self.target_pos) return -0.01 * dist Sim-to-real transfer is done via Isaac Sim with domain randomization (random mass, friction, lighting). This yields 90+% successful grasps on a real UR10 without fine-tuning.
Multi-Agent Path Finding for AGV Fleet
Note: when there are many AGVs, a centralized planner cannot recalculate routes quickly. Decentralized RL — each agent chooses a direction (N/S/E/W/Stay) based on a local occupancy map. Reward:
rewards = np.zeros(self.n_agv) for i, agv in enumerate(self.agvs): if agv.reached_goal(): rewards[i] += 10.0 if self._collision(i): rewards[i] -= 5.0 rewards[i] -= 0.01 # time penalty rewards[i] -= 0.1 * agv.steps_waiting Result: warehouse throughput 20-30% higher than with fixed routes. The RL approach handles up to 200 vehicles — twice as many as a classical centralized planner.
Job Shop Scheduling with RL
Scheduling N machines and M jobs is NP-hard. An RL agent outperforms the best dispatching rules (SPT, EDD) by 5–15% on makespan. We use a hybrid: RL for fast local decisions, Google OR-Tools for periodic global optimization. This reduced idle time by 18%.
Predictive Maintenance
RL is also applied to predicting equipment failures. By analyzing sensor telemetry (vibration, temperature, current), the agent estimates remaining useful life. This allows maintenance to be planned without production stoppages, reducing unplanned downtime.
Why RL Wins Over Classical Algorithms?
Research in the Journal of Manufacturing Systems shows RL reduces downtime by 30%. Compare: the classical approach requires reprogramming when the object changes, while RL adapts through reward tuning. The table below provides a clear comparison.
| Parameter | Classical Approach (Programming) | RL Approach |
|---|---|---|
| Setup time for new object | 2-5 days (CAD + trajectory) | 0 days (simulation) |
| Adaptation to environment changes | Reprogramming | Reward tuning |
| Scalability to AGV fleet | Centralized planner (100+ AGVs bottleneck) | Decentralized RL (up to 200+ AGVs) |
| Required data | CAD, route maps | Telemetry logs, few hours of simulation |
The RL approach is twice as performant as the classical centralized planner for AGV fleets.
How We Do It: Stack and Case Study
Stack: ROS2 (Humble), MoveIt 2 with OMPL, Isaac Sim + Isaac Lab, PyTorch, Gymnasium-Robotics. For AGVs — custom simulator based on ROS2 Navigation2.
Case study from our practice: automation of an automotive component assembly line. 5 UR10e cobots, 20 AGVs, 12 machines. We implemented:
- RL grasping of parts from bins (6 part types, up to 10 cycles per minute);
- MARL for AGV fleet — delivery time reduced by 35%;
- RL job queue scheduler for machines — idle time reduced by 18%.
The work process is broken into stages:
| Stage | Duration | Actions |
|---|---|---|
| Analytics and data collection | 2-3 weeks | Logistics audit, latency measurement, log collection |
| Architecture design | 1-2 weeks | Algorithm selection, RL problem formulation, environment |
| Simulation training | 4-8 weeks | Iterations with domain randomization, validation |
| Integration and testing | 2-4 weeks | Policy via ROS2 Actions, load testing |
| Production deployment | 2-4 weeks | Monitoring, reward adjustment, training |
What's Included
- RL model for your task (grasping / AGV routing / scheduling) with trained policy
- Integration into ROS2 infrastructure: node-controller, topics, services
- Simulation environment (Isaac Sim / Gazebo) for future fine-tuning
- Documentation: state/action space description, reward design, metric card model
- Training for your team (2-3 sessions of 4 hours each)
- Deployment guarantee: policy adjustment for condition changes within 3 months
How to Set Up RL Grasping for a Cobot
- Collect data with an Intel RealSense D435 camera to create 3D models of objects.
- Set up the simulation environment in Isaac Sim with domain randomization.
- Define state/action space and start training with PPO.
- Validate the policy in simulation over 1000 episodes, then transfer to the real robot.
- Calibrate reward functions based on real test results.
More about reward design
The reward for AGVs includes a bonus for reaching the goal (+10), a collision penalty (−5), and a time penalty (−0.01 per step). Idle time is additionally penalized (−0.1 * steps_waiting). This forces agents to minimize time and avoid conflicts.Approximate Timelines
- AGV fleet routing: 10 to 14 weeks
- Robotic grasping (sim-to-real): 16 to 24 weeks
- Production line RL scheduler: 20 to 32 weeks
Cost is calculated individually based on your KPIs. Get a consultation — our engineers will analyze your production task and propose a solution.
Our Advantages
We have 10+ successful RL deployments on industrial sites. Certified integrators for ROS2 and Isaac Sim. We exclusively use open-source tools — you are not locked into vendor software. We provide full documentation and a policy guarantee. Request a consultation to discuss your project.