45 min read
A First-Timer's Guide to Setting Up a Robot Manipulator
research · knowledge
[WIP -- this page is still being actively updated.]
A lot of my friends (who had never touched a robot before) recently reached out and asked me how to set up a workspace for robot manipulation. I feel reasonably equipped to answer, because not long ago I was in the same position*. And to my surprise, after a bit of searching I didn't find any robotics hardware guide that is (1) vendor-neutral, (2) beginner-friendly, and (3) covers a range of budgets, tasks, and uses. There's plenty of excellent adjacent material, though, which I've linked at the end of this post.
*I started my PhD having never touched a robot, and finished my PhD thesis working on robot manipulation.
Back in 2023, I wrote a note for myself and colleagues on this. A lot has changed since then: some hardware was discontinued, some new hardware appeared, and what people care about most when choosing hardware has shifted (e.g., "which arm has the most pretrained models" has become more important today).
So here's an updated version as of 2026. Hope this helps! Feel free to send in suggestions or point out anything I missed :D
A few disclaimers:
- This guide only covers fixed-base manipulation (e.g., one or two arms fixed to a table). Humanoids, drones, other mobile robots, etc. each deserve their own guide.
- I try to prioritize breadth over depth in this post, so I won't go into as much detail on some topics even though I feel they deserve more discussion. I plan to write follow-up posts on dexterous hands and simulators.
- This guide is intended for anyone who wants the smallest complete setup that can run a meaningful manipulation experiment (e.g., collecting demonstrations, training and evaluating policies). That could be at home, in a startup, or in a research lab -- but not industrial scale, like an Amazon warehouse.
TLDR, or "just tell me what to get"
The rest of the post will explain these choices, but if you want the answer first, here are four archetypes of robot users that you can map yourself to, and what I'd recommend for each:
Desk tinkerer. "I just want something simple to play with."
- Setup: a low-cost open-source leader/follower arm kit clamped to a desk, two webcams, and LeRobot.
- Hardware: ~$300–600.
- Time: a weekend of work takes you from parts to a trained policy.
- Bimanual variant: two kit pairs.
Serious hobbyist. "I want to experiment with building a prototype."
- Setup: a $2k–10k 6-DoF arm (several now offer torque control), a depth camera on the wrist, a matching leader arm or SpaceMouse, and one GPU workstation.
- Hardware: ~$4k–20k.
- Bimanual variants: OpenArm 2.0 ($6,500), or turnkey stations from I2RT or Trossen (~$24k, above this band).
General research. "I want to research general robot learning."
- Setup: a Franka- or UR-class arm, a Robotiq gripper, 2–3 cameras, a proper table, a real-time control PC and a GPU box. This is essentially the DROID reference setup.
- Hardware: ~$45k–75k.
Specialized research. "I want to research high-precision, contact-rich tasks."
- Setup: an arm with joint torque sensing and an open ~1 kHz control interface, an external force-torque sensor, tactile sensing and a rigid mount.
- Hardware: ~$60k–150k+.
| Component | Desk tinkerer | Serious hobbyist | General research | Specialized research |
|---|---|---|---|---|
| Arm | SO-101 leader + follower kit | I2RT YAM, Trossen WidowX, or UFACTORY xArm / Lite 6 | Franka FR3 | Flexiv Rizon 4s or KUKA LBR iiwa 14 |
| Gripper | Stock jaw with TPU fingertips | The arm's own gripper + custom fingertips | Robotiq 2F-85 | Robotiq Hand-E or Weiss WSG 50 + tactile fingertips |
| Cameras | 32 mm USB wrist module + any webcam | RealSense D405 on the wrist + 1–2 static cameras | 2× ZED 2i + ZED Mini (DROID-compatible), or D405 + 2× D435-class | D405 wrist + hardware-synced precision static cameras |
| Force / touch | None | Optional | Built-in joint torques; add a wrist F/T sensor later | External F/T sensor + tactile sensing |
| Teleop | The SO-101 leader arm | A matching leader arm, or a SpaceMouse | Quest 3/3S (DROID used the Quest 2) + a GELLO | Force-feedback leader arm or haptic device |
| Compute | Your laptop + rented GPU | One RTX 5090-class workstation | Real-time control PC + separate GPU box + cloud | Same as general research |
| Software | LeRobot | LeRobot plugin or the vendor's Python SDK | libfranka / franky or franka_ros2, plus openpi | ROS 2 + ros2_control, or Drake |
| Hardware cost | ~$300–600 | ~$4k–20k | ~$45k–75k | ~$60k–150k+ |
| First year, all-in* | ~$0.5k–1.5k | ~$6k–25k | ~$55k–100k | ~$80k–200k+ |
| Time to first learned policy | A weekend of work (1–3 weeks calendar) | 2–6 weeks | 2–3+ months from purchase order | 2–4+ months |
| You should know | Basic Python | Python, Linux, basic networking | Linux admin, C++/Python; ROS helps | All of that + control theory |
*"All-in" adds freight, duties, spares, software and cloud compute. It excludes salaries, which are usually the biggest cost.
For each archetype, it's easiest to copy an existing setup, so you can reuse its data and pretrained models (see Step 1).
Step 1: figure out what you're building
Here are some "first questions" that I would ask my friends when they need help:
What do I want the robot to do?
- Simple pick-and-place? Is force/touch sensing required? Is fine-grained dexterity needed?
- How much error can my task tolerate? Some examples, from easiest to hardest:
- block pick-and-place (cm-level error is fine)
- cloth folding
- peg insertion (with sub-mm clearance)
- Do I want one arm or two? Two arms roughly double the cost and space needed, and more than double the integration work.
Where will my data come from? This assumes you're working on data-driven experiments. Each answer implies a different set of hardware:
- Teleoperation: you need a teleop device (see Teleoperation).
- Handheld devices or human video (e.g., UMI): a robot gripper and wrist camera that match the data-collection device.
- Sim-to-real RL: an arm with a maintained simulation model, and patience for system identification (measuring the real robot's dynamics so the simulator matches it).
- Real-world RL: a compliant controller, a software safety box, a way to reset the scene, and a reward signal.
Do I want to use external data and models? Again assuming data-driven experiments, picking a setup with ample data and pretrained checkpoints could save you a lot of work. (That said, just owning the same setup is not enough -- some work on camera calibration, action conversion, etc. might still be needed before we can zero-shot transfer a downloaded policy.)
How much space, power, and noise tolerance do I have? Do I have just a desk, a whole room, or a shared lab reserved for this setup? How many power outlets and circuits are around? (This matters because a GPU workstation and a robot often shouldn't share a circuit -- see Power.)
What's my budget and timeline? Hardware lead times can range from a few days to a few months. Make sure to start the procurement process early (and compare quotes!).
How experienced am I, and how much hassle do I want?
- Not technical at all, just want to try things: stick to the desk tinkerer setup.
- I can program but have never set up hardware: desk tinkerer or serious hobbyist.
- I'm a pro in AI or software, getting into robotics: any of them.
- I'm a pro in hardware/robotics, getting into AI: (you probably don't need this guide...?) any of them.
Who will maintain it? A solo setup can live with quirks. A shared lab needs written procedures, safety training and spare parts.
Step 2: learn about the components
For each component, this section covers:
- what's out there
- how to choose (space, cost, task, robustness, ease of setup)
- what I'd pick for each archetype
- common pitfalls
1. Arm
How to read a spec sheet:
- Repeatability is not accuracy. Repeatability is how tightly the arm returns to the same taught pose. Accuracy is how close it gets to a commanded position in world coordinates. For learning from demonstrations, repeatability and your camera calibration matter more. Treat sub-0.1 mm claims on sub-$5k arms as marketing until you measure them.
- Payload at reach. The headline payload is usually measured near the base. Kinova's own table gives the Gen3 4 kg of payload at mid-range but only 2 kg at full reach, and that's continuous payload without a gripper. Your gripper, wrist camera and mount all count against it; a Robotiq 2F-85 alone is ~0.9 kg.
- 6 vs. 7 DoF. A 7th joint lets the elbow move without moving the gripper. That helps in clutter and makes teleop feel more human. It also makes inverse kinematics more complex, and your leader arm must match.
- Control interface and rate. Position streaming is enough for most imitation-learning policies, which run at 10–50 Hz. Torque or impedance control at ~1 kHz lets you make the arm compliant in software. That's what force-controlled insertion, wiping and safe RL exploration need.
- "Torque sensing" means three different things:
- real torque sensors in every joint (Franka, Kinova, KUKA iiwa, Flexiv)
- a force-torque sensor at the wrist flange only (UR e-Series)
- an estimate from motor current (most arms under $5k)
- What "open SDK" means in practice:
- Is the protocol documented?
- Can you stream setpoints at a fixed rate from your own code?
- Does it run on plain Ubuntu + Python, or does it need a real-time kernel?
- Is there an official ROS 2 driver and a simulation model?
- Is the GitHub repo active?
What's out there (Oct 2026):
| Tier | Examples and prices | Notes |
|---|---|---|
| Open-source / DIY | SO-101: ~$230 in parts for a leader + follower pair; kits $199–400. OpenArm 2.0: open 7-DoF bimanual, $6,500. Seeed reBot B601: ~$1,800. Koch v1.1: repo frozen since 2024. | SO-101 is position-only with ~1° of servo backlash, which is fine for cm-level tasks. OpenArm and reBot use backdrivable motors with torque estimates. |
| Low-cost commercial ($2k–11k) | AgileX PiPER: $1,999 direct; from $3,780 at a US reseller. I2RT YAM: $2,999. Galaxea A1Z: $2,999. Trossen WidowX Core/AI: $2,995–4,996. UFACTORY: Lite 6 $3,349 delivered; xArm 6 $8,899; xArm 7 $10,499. ARX X5/L5: quote by email. | The tier that changed most since 2024. Many now expose torque control ("MIT mode"), and have a LeRobot plugin and a MuJoCo model. |
| Research cobots (quote-only) | Franka FR3: ~$35k after 2025 US tariffs, per DROID's shopping list. UR e-Series: UR3e/7e/12e/16e; a used 2021 UR5e lists at ~$24k. Kinova Gen3. Flexiv Rizon. KUKA LBR iiwa. Tianji Marvin: Chinese 7-DoF force-controlled arm, quote through a US reseller. | Most published research code targets Franka or UR. |
| Industrial cobots | FANUC CRX, ABB GoFa, Doosan, Kassow | Built for factory programming. Low-level streaming often needs paid add-ons, so check before buying. |
Recent changes (check these before copying an old parts list):
- Trossen's ViperX 300 S and WidowX 250 S, the original ALOHA and Bridge arms, are discontinued.
- The Franka Panda is no longer supported in Franka's robot description repo. The FR3 is the current arm.
- UR's product pages no longer list the UR5e or UR10e. The e-Series page now lists the UR3e/7e/12e/16e. The UR7e (on the CB5 controller) has the UR5e's 850 mm reach, and UR's ROS 2 description uses identical configs for the two. UR also launched a new g-Series in September 2026. Much older code still says "UR5e", so check driver model names.
- SO-100 is superseded by the SO-101.
- The "Tianji" in some lists is Tianji Intelligence (天机智能), a Guangdong company that makes the Marvin force-controlled arm.
My picks:
| Archetype | Pick | Why |
|---|---|---|
| Desk tinkerer | SO-101 | It has native LeRobot support, the biggest beginner community (~20k on the LeRobot Discord), and the largest pool of community data. It's also cheap to break and reprint. |
| Serious hobbyist | I2RT YAM or Trossen WidowX, or xArm / Lite 6 if reliability matters more than torque control | YAM is backdrivable, has a torque mode, and is in stock. Trossen offers US support and ALOHA-compatible kits. UFACTORY arms have industrial-grade position control and a polished SDK; dfdx labs had a Lite 6 moving ~30 minutes after unboxing. |
| General research | Franka FR3 + Robotiq 2F-85 | It matches DROID data and checkpoints, has 1 kHz torque control, and most research code supports it. Alternatives: a UR7e or used UR5e (robust, big used market, and UR now offers direct torque control) or a Kinova Gen3 (light, 1 kHz torque control). |
| Specialized research | Flexiv Rizon 4s or KUKA LBR iiwa 14 | Both have real joint torque sensing plus a ~1 kHz control interface (on the KUKA, that's the FRI option, typically paid). The FR3 also qualifies, and is among the cheapest arms in this class (most rivals are quote-only). |
Pitfalls:
- List price is not landed price. The same PiPER is $1,999 from AgileX and from $3,780 at a US reseller whose price includes shipping and import duties. Always ask for a delivered quote with duties included (see Procurement).
- Low-cost arm quirks (all from GitHub issues):
- Plugging the wrong power supply into Feetech servos burns them.
- PiPER CAN links can fail silently.
- YAM motors drop into damping mode after a 400 ms command timeout.
- ARX's SDK warns that picking the wrong model "may lead to dangerous movements".
- The xArm servo-mode call occasionally blocks for ~190 ms.
- Special note on Franka:
- It needs a real-time kernel.
- It needs a direct cable to the Control box, not the arm.
- Your libfranka version must match the robot's system image.
2. End effector
Parallel-jaw grippers are the default. Nearly every well-known dataset uses two fingers.
| Gripper | Price | Notes |
|---|---|---|
| Bundled gripper on a low-cost arm | Included | Often the weakest part of the setup, but fine for imitation learning. Upgrade the fingertips first. |
| Robotiq 2F-85 | ~$5–6k | 85 mm stroke, 20–235 N. The research standard (DROID, TidyBot++). |
| Robotiq Hand-E | ~$4.7k | True parallel jaws, 50 mm stroke (a 100 mm version exists), 0.025 mm repeatability. Good for insertion. |
| Franka Hand | ~€1.7k (EU price) | 80 mm. Cheap if you have a Franka, but slow (50 mm/s per finger) and controlled by discrete commands. |
| UFACTORY xArm Gripper G2 | $2,199 | 84 mm, for xArm. |
| Weiss WSG 50-110 | Quote | Fast, with force-measuring fingers. UMI's robot-side gripper. |
Fingertips matter more than the gripper brand. UMI's 3D-printed TPU fin-ray fingers, ALOHA 2's griptape upgrade and the SO-101's TPU jaw all make a big difference. Get a 3D printer, keep versioned fingertip designs, and record which fingertip you used in your dataset metadata.
Suction needs real air. A pneumatic vacuum gripper like the Robotiq PowerPick 10 consumes 5.5–10.8 CFM of compressed air. A quiet 60 dB "lab" compressor supplies about 2.2 CFM. Unless you have shop air, buy an electric vacuum gripper:
- the UFACTORY vacuum gripper ($571, for xArm)
- OnRobot VGC10 ($3.8k)
- Robotiq EPick (from $4.7k)
Dexterous hands get their own post. (The open-source LEAP Hand costs under $2k.)
Pitfalls:
- Gripper APIs are often blocking. Run gripper commands in a separate thread, or your 30 Hz control loop stalls.
- USB-to-serial adapters default to a 16 ms latency timer. Set it to 1 ms.
- The gripper's status LED can shine straight into the wrist camera. DROID covers it with Velcro.
- Every time you add a gripper, sensor or camera, update the payload mass, center of mass and tool center point (TCP) in the controller. Wrong values cause bad gravity compensation and false protective stops.
3. Sensors
Cameras
Most popular imitation-learning setups feed their policies low-resolution RGB:
- ALOHA used 640×480 webcams.
- DROID's released data is 320×180.
- LeRobot's SO-101 examples use 640×480 at 30 fps.
So depth is optional for the first two archetypes, and 4K is wasted bandwidth.
The typical layout is one wrist camera plus one or two static cameras. A good wrist camera is:
- small (≤~70 g)
- wide-angle (≥85°)
- able to see things a few centimeters away
- ideally global-shutter, so fast motion doesn't smear
| Camera | Price | Use |
|---|---|---|
| 32×32 mm USB module (e.g., Innomaker 1080P) | ~$18 | SO-101's recommended wrist camera |
| Logitech C920s | ~$60 | Static RGB. Fine, but occasionally drops frames |
| RealSense D405 | $325 | The default wrist depth camera. Sees from ~7 cm, global shutter, 60 g |
| Orbbec Gemini 305 | $229 | Same footprint as the D405; global-shutter color stereo that sees from ~4 cm |
| RealSense D435if / D436 | $425 | Static depth. Can't see closer than ~28 cm, so not for the wrist |
| Orbbec Gemini 335 / 336 | $264–284 | Alternative to D435-class cameras |
| ZED 2i + ZED Mini | $499 + $399 | The DROID-compatible set. Needs an NVIDIA GPU |
| Orbbec Femto Bolt | $418 | Time-of-flight; the Azure Kinect replacement |
| Structured light (Zivid, Photoneo, Mech-Mind) | Quote | Sub-mm precision but slow captures. For "look, then move", not 30 Hz control |
Recent changes:
- RealSense spun out of Intel in July 2025. On September 22, 2026, Cognex announced it will acquire the company, with closing expected in Q4 2026. The old intelrealsense.com links are dead; the code moved to github.com/realsenseai.
- Discontinued:
- the RealSense L515 lidar and T265 tracking camera (end-of-life announced in 2021)
- the Azure Kinect (end of production 2023; Orbbec's Femto Bolt, built with Microsoft, is its successor)
- The ZED 2 that DROID used is no longer sold by Stereolabs. Buy the ZED 2i instead.
Pitfalls:
- USB bandwidth is the #1 integration pain. One USB 3 controller handles about one depth camera streaming 1280×720 depth + color, or 3–4 at 640×480 and below. Front and back ports often share a single controller, which you can check with
lsusb -t. For two or more full-resolution cameras, add a PCIe card with an independent controller per port. - Passive USB 3 cables are unreliable past ~2 m. Use active or optical cables beyond that.
- Camera indices change after a reboot. Always select cameras by serial number.
- Turn off autofocus, auto-exposure and auto white balance. Set anti-flicker to 60 Hz (in the US).
- Time-of-flight cameras interfere with each other. Active-stereo cameras like the RealSense D400s don't.
Force-torque sensing
Built-in force sensing is not the same as an F/T sensor.
- The UR7e's flange sensor is rated ±3.5 N precision and ±4 N accuracy.
- Franka and Kinova estimate the force at the gripper from joint torques. One Franka user reported pose-dependent at-rest offsets of ~1.4–1.7 N.
- External sensors resolve about 0.01–0.15 N at roughly 0.8–8 kHz.
Built-in is enough for:
- collision detection and hand-guiding
- detecting contact above a few newtons
- coarse force-limited pushing and wiping
- HIL-SERL-style RL on a Franka
Buy an external sensor for:
- sub-newton insertion
- policies that take force as an input
- any arm without torque sensing
| Sensor | Price | Notes |
|---|---|---|
| Bota MiniONE | CHF 3,045 | Small (30 g), USB. Good for low-cost arms |
| Bota SensONE | ~CHF 4.9–5.6k | Mounts directly on a UR/FR3 flange; kits for UR, FR3, Kinova |
| ATI Axia80 | ~$5k | Electronics built in |
| ATI Mini45 | ~$9.3k with interface box | The classic research wrist sensor |
| OnRobot HEX | ~$4.4k | |
| UFACTORY F/T sensor | $3.3–3.5k | For xArm |
I couldn't find a 6-axis sensor under $1k with a public price.
Pitfalls:
- Taring only cancels the tool's weight in its current orientation. Rotate the wrist and you need gravity compensation.
- Small sensors die in crashes.
- EtherCAT interfaces need a separate controller program (an "EtherCAT master" stack), so prefer USB, serial or UDP for a first setup.
- ATI sells the sensor and the interface box separately.
Tactile sensing
Tactile sensing is now cheap to try:
- GelSight Mini: $510 ($560 robotics package). Camera-based; shows up as a USB webcam.
- DIGIT: $355. Its official software library was archived in November 2024.
- AnySkin / WowSkin: $128 (sold out when I checked).
- DIY options: FlexiTac pads ($2.5–5 each, with a LeRobot fork that supports tactile input) and 9DTact (~$15).
- Drop-in fingertips for a Robotiq gripper: Robotiq TSF-85 (~€3.1k in the EU) and XELA uSPr (quote).
- Meta's Digit 360 is still not commercially available, so don't plan around it.
Gels are consumables ($42–70, with 4–6 week lead times), so buy spares up front. LeRobot doesn't treat tactile as a first-class data type yet, so expect to write some glue code.
4. Mount, table, workspace
A wobbly table ruins precision, and a robot that isn't bolted down properly can tip. Vendor requirements are stricter than people expect:
- Franka FR3: four M8 bolts at 30 Nm, re-torqued after 100 h. The base must resist 280 Nm of tilting torque, and it must be level to within 0.1°.
- UR: the mounting surface must withstand at least 10× the base joint's full torque and 5× the arm's weight. UR5e-class arms use four M8 bolts at 20 Nm.
Some intuition for those numbers:
- Clamp a 30 cm breadboard to a desk at its edge, and resisting the FR3's 280 Nm takes about 930 N (~95 kg-force) of clamp force.
- A free-standing 60 cm-wide stand would need to weigh ~95 kg before any safety margin.
That's why serious setups bolt down or use heavy frames.
| Option | Price | Good for |
|---|---|---|
| Heavy desk + 2–4 clamps | $5–9 | Desk tinkerer |
| Vendor clamps on a solid desk (the Lite 6 ships with two) | Included | Serious hobbyist |
| The DROID recipe: wheeled standing desk, 30 cm breadboard, 4 clamps, aluminum rails | ~$250–300 in parts | General research, if the setup moves between scenes |
| Welded steel workbench (e.g., Uline 72×36") | ~$780 | General research |
| Vention heavy-duty cobot pedestal / mobile pedestal | $1,161 / $2,245, plus an adapter plate for FR3/UR5-class arms | General or specialized research |
| Floor-anchored pedestal | Varies | Specialized research |
Layout tips:
- Keep the task well inside the arm's reach, not at its edge.
- Mount cameras rigidly and take a reference photo from each. In one study, changing the camera pose hurt policies more than changing the lighting or background.
- Keep a separate table for your own clutter.
- Mark the robot's reach on the floor or table with tape.
5. Teleoperation
The key concept is joint space vs. Cartesian space.
- A leader arm is a small copy of the robot with the same joints. Each leader joint angle goes straight to the matching robot joint. There's no math in between, latency is low, and you can feel the robot's joint limits.
- VR controllers, SpaceMice and phones produce a target pose for the gripper. Software then computes joint angles (inverse kinematics). That can fail near singularities, and it hides self-collisions from the operator.
The evidence favors leader arms. In the one controlled study I found (GELLO: 12 novice operators, 5 bimanual tasks on two UR5s), task success was:
- 92% with a leader arm
- 72% with Quest 2 VR controllers
- 63% with a SpaceMouse
Handheld data-collection grippers are fast too. On a single task, UMI measured 111 demos per hour vs. 35 for SpaceMouse teleop. The cost is an embodiment gap: the handheld device differs from the robot in camera view, gripper and latency.
| Method | Cost | Works with | Notes |
|---|---|---|---|
| DIY leader arm (SO-101 leader, GELLO, U-ARM) | ~$50–300 | One specific arm | Best data quality per dollar. GELLO supports Franka, UR, xArm and YAM; U-ARM covers many more |
| Commercial leader arm | $2,999 (YAM, with force feedback); $4,686 (Trossen WidowX AI) | The vendor's arm | Plug and play |
| VR headset | Quest 3S $349.99; Quest 3 $599.99 (both raised in April 2026) | Any arm with IK | DROID used Quest controllers. The Quest 3 version of oculus_reader is still beta |
| SpaceMouse | ~$150–200 | Any arm with IK | Easy. Great for short corrections during RL, slow for long demos |
| Phone, keyboard or gamepad | Free | Any arm with IK | LeRobot supports all three. Fine for simple tasks |
| Handheld gripper (UMI and successors) | ~$371–450 DIY; Trossen TRumi $950–2,196; I2RT YUMI $599 on sale ($699 list), plus trackers | Robot needs a matching gripper + wrist camera | Fastest data collection; no robot needed while collecting |
| Haptic device (Force Dimension, 3D Systems Touch, Haply) | Quote; high-end ~$40k (2023 estimate) | Any arm with IK | Real force feedback |
| Gloves, exoskeletons | $230–7k+ | Mostly dexterous hands | Mostly research tools today |
My picks:
| Archetype | Teleop |
|---|---|
| Desk tinkerer | The SO-101 leader arm |
| Serious hobbyist | A matching leader arm (GELLO, U-ARM or the vendor's), plus a SpaceMouse for corrections |
| General research | A Quest (for DROID compatibility) plus a GELLO |
| Specialized research | Bilateral force-feedback teleop (Franka↔Franka, or the YAM leader) or a haptic device |
Discontinued: the Quest 2 (DROID's original headset), the Quest Pro and the MANUS Quantum Metagloves. The Leap Motion Controller 2 is getting hard to find.
Pitfalls:
- Control rate matters. In ALOHA's study, dropping teleop from 50 Hz to 5 Hz made tasks take 62% longer.
- Leader arms get heavy. ALOHA added rubber bands for gravity compensation so sessions over 30 minutes were possible.
- Check that the leader and follower poses match before you start streaming. GELLO aborts if any joint differs by more than 0.8 rad.
6. Compute and networking
You need up to three machines:
- Control PC: runs the robot driver loop and safety checks.
- Training GPU: local or cloud.
- Inference GPU: runs the policy near the robot.
Do you need a real-time kernel? A real-time (PREEMPT_RT) Linux kernel makes scheduling latency small and predictable, so a 1 kHz control loop rarely misses its deadline.
- Only the Franka strictly needs one. Its control loop has a 1 ms budget, and 20 dropped packets in a row stop the robot.
- UR recommends one, though a low-latency kernel is often enough.
- The low-cost CAN and serial arms don't need one.
- Ubuntu 26.04 ships the real-time kernel in its main archive. On 22.04/24.04 it comes through Ubuntu Pro, which is free for personal use on up to 5 machines.
- NVIDIA drivers aren't officially supported on real-time kernels. So keep the real-time control PC (no GPU) separate from the GPU box. DROID and TidyBot++ both do this.
How much GPU do you need?
| Policy | GPU memory to fine-tune | Typical time |
|---|---|---|
| ACT | 2–6 GB | 30–60 min on an RTX 4090 (50 demos); 6–14 h on a Mac |
| Diffusion Policy | 8–14 GB | 2–4 h on an RTX 4090 |
| SmolVLA | 10–16 GB | ~4 h on an A100 |
| π0 / π0.5 (openpi) | >22.5 GB with LoRA; >70 GB for a full fine-tune | Hours on an A100 |
- A 32 GB RTX 5090 covers everything up to LoRA fine-tunes of the big models. Its MSRP is $1,999, but street prices are higher during the memory shortage.
- For full fine-tunes, rent an A100 or H100 80 GB at ~$1.2–4 per GPU-hour.
- Training ACT on a rented cloud GPU costs a dollar or two.
- Isaac Sim doesn't run on A100/H100 GPUs, which lack the RT cores it needs.
For inference, π0 takes about 73 ms on a 4090. Over a network, openpi's DROID example says 0.5–1 s per action chunk is normal. Use a wired connection.
Networking:
- One dedicated wired network port per arm, static IPs, no Wi-Fi.
- For a Franka, use a direct cable to the Control box with no switch.
- Put the robots on an isolated subnet with no internet route. It's good security practice, and university IT will probably require it anyway.
7. Software stack
For beginners, this choice may matter more than the arm.
LeRobot (v0.6.1, Aug 2026) is the default for the first two archetypes. It handles teleop, recording, training and rollout, and includes ACT, Diffusion Policy, SmolVLA, π0/π0.5, GR00T and more.
- Native support: SO-100/101, Koch, OMX, OpenArm, reBot.
- Community plugins: xArm, PiPER, ARX5, YAM, Trossen, UR5e.
- Franka only through a fork.
- Requires Python ≥3.12.
Vendor SDKs are the direct route:
- Franka: libfranka + franky (Python)
- UR: ur_rtde
- UFACTORY: xarm-python-sdk
- AgileX: piper_sdk
- I2RT: i2rt
- Trossen: trossen_arm
- ARX: the community arx5-sdk
ROS 2 is worth it if you need any of:
- MoveIt for collision-aware planning
- ros2_control's impedance controllers
- a driver that only exists in ROS
- many heterogeneous sensors
It's also worth it if your team already knows ROS. Otherwise it's overkill for a beginner doing imitation learning. If you use it:
- Pick Jazzy (Ubuntu 24.04, supported until 2029).
- Lyrical (May 2026) is the newer LTS, but most arm drivers don't support it yet.
- Skip Kilted, which reaches end of life in December 2026.
| Arm | LeRobot | ROS 2 | Real-time kernel? |
|---|---|---|---|
| SO-101, Koch | Native | Community | No |
| PiPER, ARX, YAM, Trossen | Plugin | Varies (PiPER: Humble only) | No |
| xArm / Lite 6 | Plugin | Official (Humble, Jazzy) | No |
| UR e-Series | Plugin | Official (Humble → Lyrical) | Recommended |
| Franka FR3 | Fork only | Official (Jazzy) | Required |
| Kinova Gen3 | None found | Official | No for high-level control (the arm runs its own loop); helps for 1 kHz low-level servoing |
Stale projects to avoid for new work:
- Polymetis. DROID's original stack pins libfranka 0.10, which is out of spec for current FR3 firmware (5.9), and one recent FR3 build couldn't get it to compile cleanly.
- deoxys and Octo.
- The original Diffusion Policy repo. Use LeRobot's implementation instead.
Data.
- LeRobot's v3 format stores robot state in Parquet and video as compressed MP4.
- Fifty 30-second episodes from two 640×480 cameras come to roughly 0.3 GB. Raw frames would be 100–300× larger.
- Watch for format mismatches. openpi pins a pre-v3 LeRobot, and GR00T uses its own v2 variant, so you may need to convert.
Licenses. GR00T weights use the NVIDIA Open Model License. AgiBot World data and ManiSkill assets are non-commercial. Check the terms before you build a product on them.
8. Simulation (brief; full post coming)
The main options are MuJoCo (with the MuJoCo Menagerie model collection), Isaac Sim / Isaac Lab (needs an RTX GPU), ManiSkill3, Genesis and Drake.
Before buying an arm, check that a maintained simulation model exists. Menagerie has the FR3, UR5e, Kinova Gen3, xArm, PiPER, ARX L5, YAM, SO-101 and WidowX AI, plus the Robotiq 2F-85 and UMI grippers.
9. Safety (not optional)
Standards in one paragraph. ISO 10218-1/-2 were rewritten in 2025. Most of the cobot-specific content from ISO/TS 15066, such as the body-region force and pressure limits, now lives in ISO 10218-2. The US adopted them as ANSI/A3 R15.06-2025. The key idea for a small lab: "collaborative" describes an application, not a robot. A Franka or UR isn't "safe" until your gripper, task, objects and layout have been risk-assessed.
Know what your e-stop does.
- UR and Franka: the e-stops are Stop Category 1. The robot brakes under control, then cuts power. The Franka drops slightly onto its locking bolts.
- Low-cost servo arms have no safety-rated stop.
- Cutting servo power drops the arm under gravity.
- Unplugging the USB cable does not stop it. Assume the servos stay powered and keep holding or finishing their last command.
- So put a switch within reach that cuts the power supply, through a relay rated for its current. Pad the table under the arm.
Software guards for learned policies. A learned policy can output anything, so wrap it:
| Guard | Example |
|---|---|
| Workspace box | HIL-SERL clips every target to a Cartesian box |
| Per-step limit | LeRobot's max_relative_target is off by default; turn it on |
| Start-up check | GELLO refuses to start if leader and follower differ by >0.8 rad |
| Collision thresholds | libfranka's defaults are "very high (effectively disabling detection)", so set your own |
| Compliant control | SERL limits how far the impedance controller's target can lead the real pose |
| Watchdog | dfdx labs expires teleop input after 100 ms |
| Gripper limits | LeRobot caps gripper torque at 50% so the servo doesn't burn out |
A Franka-specific gotcha: Franka's certified Cartesian safety limits can't be active while your own code controls the robot through FCI (the Franka Control Interface). When you run a policy, your software guards replace those Cartesian limits. The joint-space limits, e-stop and enabling switch still work, so keep them.
Rollout protocol:
- Dry-run in replay or sim.
- Do the first real runs at reduced speed and stiffness, inside a tight box.
- Have one trained person hold the e-stop or enabling switch, with nobody else within reach.
- Log every intervention and stop.
- Widen limits gradually.
Lab rules. University safety offices (UT Austin, Virginia Tech, UIUC, Penn) publish robot-lab rules. Common requirements:
- a task-based risk assessment
- training for everyone who operates the robot
- e-stops at every operator position
- a 250 mm/s speed limit while teaching
- floor tape marking robot zones
LLNL has published a real UR5e risk assessment you can copy. Talk to your safety office before the robot arrives.
Low-cost hazards:
- A wrong-voltage power supply burns servos. SO-101 leader and follower kits can use different voltages, so label them.
- Squeezing a rigid object with a servo gripper for too long burns the motor.
- Cheap uncertified power supplies may not pass your institution's rules.
10. Other important stuff
Power.
- Cobot controllers accept universal 100–240 V. An FR3 averages 140–350 W and peaks near 600 W.
- An RTX 5090 workstation needs ~1,000 W.
- A US 15 A / 120 V circuit is good for 1,440 W continuous. So put the GPU box on a different circuit from the robot, and check which outlets share a breaker.
- Put the robot controller and control PC (not the GPU box) on a UPS so they can shut down cleanly.
Lighting. Use flicker-free LED panels (~$270 a pair), close the blinds, and set camera anti-flicker to 60 Hz. In one real-robot study, new lighting cost only a few points (92% to 83% over 12 trials), while moving the camera roughly halved success (to 46%). Consistent lighting helps, but a rigid camera mount helps more.
Cables.
- Leave slack for every joint configuration and test the extreme poses.
- Zip-tie about every 5 cm.
- Plug cameras directly into the computer.
Tools. A torque wrench (the FR3 base needs 30 Nm), a level, hex keys, a soldering iron, a multimeter, and a 3D printer with at least a 220×220 mm bed ($300–1,200).
Calibration board. A ChArUco board on rigid aluminum costs €116–120 (~$135) from calib.io. If you print your own, glue it to something flat and measure the printed squares.
Test objects. The YCB object set ($1,400 through UMass Lowell) or the NIST assembly task board ($900). Household objects work fine at first.
Receiving.
- Research arms weigh 18–21 kg, so lift with two people.
- Freight surcharges for liftgate or "limited access" delivery (which can include universities) run $75–150 each.
- Keep the crate. Franka requires the original packaging for warranty returns.
Step 3: putting things together
Components that work individually can still fail together. Check each interface before you order:
| Interface | What to check | Example gotcha |
|---|---|---|
| Arm ↔ gripper (mechanical) | Flange pattern and adapter plate; added length and mass | UR uses ISO 9409-1-50-4-M6. Every added part changes the TCP and payload |
| Arm ↔ gripper (electrical) | Tool-port power and comms | UR7e's tool port supplies 1.5 A max, and its RS-485 shares pins with the analog inputs. On a Franka, third-party grippers need an external power supply and cable (DROID does this) |
| Arm ↔ PC | Control rate, real-time needs, network | On a Franka, the network round trip plus your control step must fit in 1 ms, over a direct cable to the Control box |
| Cameras ↔ PC | USB bandwidth, cable length, sync | Two full-resolution depth cameras on one USB controller drop frames. Webcams have no hardware sync |
| Teleop ↔ arm | Joint-by-joint match, signs, offsets | GELLO's README lists two different joint-sign sets for the Franka; check each joint by hand |
| Software ↔ hardware | A driver exists for your stack; versions match | libfranka must match the Franka system image |
| Sim ↔ real | A model exists; the same TCP is used everywhere | Define "end-effector pose" once and use it in the controller, the URDF and the dataset |
| Your data ↔ pretrained models | Camera names, action space, gripper convention, dataset format | DROID uses 0 = open, 1 = closed; OpenVLA's data uses the opposite |
Step 4: procurement
Pricing.
- Research cobots are mostly quote-only. An EU reseller lists the FR3 from €27,650 excluding VAT, and DROID's shopping list (Sept 2025, under the since-struck-down tariffs) put it at ~$35k in the US.
- Low-cost vendors publish prices.
- In every quote, ask: "Is this delivered to my door with duties included (DDP)?" Franka's default terms are FCA Germany, which makes you the importer.
Tariffs (as of October 2026; check again before you order, and this isn't legal advice):
- The Supreme Court struck down the IEEPA tariffs in February 2026.
- New Section 301 tariffs took effect in July 2026. The all-in duty on a finished robot arm is now about 10% from the EU and 12.5% from Japan, Korea and Switzerland.
- Chinese-made arms are also on the 2018 China tariff list (25%). If the layers stack, as trade lawyers say they do, that's roughly 40% all-in. Confirm with a customs broker.
- Canadian goods (Kinova, Robotiq) can be duty-free under USMCA.
- Two investigations that could add robotics tariffs are still pending. My rough rule: budget a 10–25% contingency for orders delivering in 2027.
Restrictions (as of October 2026):
- No enacted US rule bans fixed robot arms. The FCC's July 2026 robot action explicitly excludes fixed, stationary robots.
- The Defense Department added Unitree to its list of Chinese military companies in June 2026, which matters for DoD-funded labs.
- Federal grant rules (2 CFR 200.216) bar Hikvision and Dahua video gear for security uses, and many universities ban them on grants outright, so avoid them.
- Proposed bills target mobile robots and humanoids, not arms.
If you're unsure about a vendor, ask your institution's procurement or research-security office before buying.
Lead times (observed October 2026):
| Product | Lead time |
|---|---|
| I2RT YAM | Ships in ~2 days |
| Robotiq 2F-85 | ~2 weeks (vendor claim; confirm on the quote) |
| Trossen WidowX AI | 2–3 weeks production |
| ZED cameras | 1–2 weeks |
| Franka FR3 / Franka Hand (EU reseller) | ~8 weeks |
| Low-cost kits (SO-101, Lite 6) | Often sold out; stock changes weekly |
Warranty and returns. Twelve months is typical. Restocking fees run from 15% (Trossen) to 20% (I2RT). Repairs can mean shipping the arm back to the vendor's country.
Used arms. UR arms have a real used market; a 2021 UR5e was listed at ~$24k. Before buying, check:
- the controller, pendant, cables and e-stop are included
- current SDKs support the firmware
- the software licenses transfer to you
- the collision history
*During my PhD, most of the hardware I worked with was made by companies outside the US. Warranty service was always a nightmare -- it once took us 6 months to send a robot arm out for inspection and get it back. This seems much less of a problem nowadays, though, as fiercer competition has pushed companies to provide better service in general. Also, many labs are now rich enough to simply keep lots of backup hardware.
Step 5: running your first experiment
1. Bring-up checklist
| # | Step | What to do | Time, my estimates (desk tinkerer / serious hobbyist / general research) |
|---|---|---|---|
| 1 | Assemble (kits only) | Build both arms. Joints must be at mid-range when you attach the servo horns | 2–3 h / — / — |
| 2 | Mount | Rigid surface, bolts torqued to spec, e-stop within reach | 15 min / 30 min / ~1 day |
| 3 | Power and network | Write down an IP plan: one wired port per arm, static IPs | 15–30 min / 30–60 min / 0.5–1 day incl. real-time kernel |
| 4 | Pin firmware and SDK | Record versions in a lab log before the first motion | 15 min / 30 min / 1–2 h |
| 5 | "Hello world" | Read the robot state first. Then move one joint slowly, with a hand on the e-stop | 1–2 h incl. motor setup / <1 h / ~0.5 day |
| 6 | Test the e-stop and limits | Press the e-stop during a slow move. Set and test joint, speed and workspace limits | 30–60 min, then 1 min every session |
| 7 | Gripper | Power, comms and homing; set the payload and TCP | 0.5–2 h |
| 8 | Cameras | Select by serial number, check the USB 3 link speed, lock exposure and focus | 30 min / 1 h / 2–4 h |
| 9 | Teleop | Match leader and follower poses, run slowly, check the gripper direction | 30 min–2 h |
| 10 | Replay | Record one episode and replay it on the robot, which tests the whole data path | 15–30 min |
Rough totals from box to working teleop:
| Archetype | Time |
|---|---|
| Desk tinkerer | A weekend |
| Serious hobbyist | Half a day to two days, plus 1–3 weeks if you write your own software stack |
| General research | 1–2 weeks |
One recent FR3 build had 1 kHz control running within a day, then spent much longer rebuilding a legacy software stack for current firmware.
2. Calibration
Intrinsics describe the lens. RealSense and ZED cameras come factory-calibrated. You calibrate webcams and GoPros yourself:
- capture 10–20 views of a board
- tilt it up to ±45°
- cover the corners of the image
- keep autofocus off
Extrinsics describe where each camera is relative to the robot. Finding them is "hand-eye calibration", and the camera is either on the wrist (eye-in-hand) or fixed in the scene (eye-to-hand).
- Use a ChArUco board with OpenCV's
calibrateHandEye, easy_handeye2 or MoveIt's hand-eye plugin. - Collect many poses, with large rotations between them.
- A careful FR3 setup got ~3 mm residuals, limited by marker detection rather than the robot.
Do you even need extrinsics? Mostly not for the policy itself, but yes for almost everything around it.
- Image-in, action-out policies (ACT, Diffusion Policy, most VLAs) never read extrinsics. But the cameras must not move. In LIBERO-RIG, a 2026 simulation benchmark, perturbing the hardware setup (camera mounting, calibration, gripper pads and so on) cut pooled success across 9 VLA models from 0.93 to 0.77 at middle severity, and to 0.51 at the highest.
- You do need extrinsics for:
- point-cloud policies
- absolute end-effector actions defined in a camera frame
- fusing multiple camera views
- publishing a dataset
- My advice: calibrate anyway at the start of each data-collection campaign, and take a reference photo from each camera. It takes ~15 minutes and gives you a way to detect drift.
Coordinate-frame traps. These cause silent, maddening bugs:
- Quaternion order: SciPy, ROS and Eigen's storage use (x, y, z, w). MuJoCo, PyTorch3D and Eigen's constructor use (w, x, y, z). Isaac Lab switched between major versions.
- Matrix layout: Franka's pose matrices are column-major, so a naive NumPy
reshape(4,4)gives you the transpose. - Frames and units: UR's
baseframe is rotated 180° frombase_link. xArm uses millimeters; ROS uses meters. - Gripper conventions: DROID uses 0 = open, 1 = closed; OpenVLA's data uses the opposite.
A 5-minute sanity test:
- Command +5 cm in base x and +10° about z, then check the direction against tape marks on the table.
- Send gripper = 1 and see whether it opens or closes.
- Unit-test the round trip of every rotation conversion.
- Replay one recorded episode.
3. Your first dataset
Start with pick-and-place into a bin. It tolerates cm-level error, resets easily, and success is unambiguous.
- LeRobot's recipe is at least 50 demonstrations, 10 per start location, with cameras never moved.
- By my estimate, fifty 10–20 second demos take about 30–90 minutes with a leader arm.
While collecting:
- Keep cameras fixed for the whole dataset and the evaluation.
- Apply the camera-only test: you should be able to do the task yourself looking only at the camera images.
- Use marked start positions (tape or a printed grid).
- Use one consistent strategy per task.
- Re-record bad takes immediately.
- Look at your data before training. A LeRobot bug (fixed in July 2026) silently put discarded takes into the videos of re-recorded episodes, affecting a third of one user's dataset.
- Once grasping is reliable, add diversity (new positions, objects, scenes) rather than more of the same.
4. Your first policy
Train ACT in LeRobot. It takes:
- 30–60 minutes on an RTX 3090/4090
- 6–14 hours on an Apple-silicon Mac
- a dollar or two on a rented cloud GPU
Save checkpoints often, and choose them by real rollouts, not validation loss. Validation loss often doesn't track success rate.
The general-research shortcut: if you matched the DROID setup, try π0.5-DROID before collecting anything. Here's every task one FR3 build tried, with no fine-tuning:
| Task | Success |
|---|---|
| Bread → plate | 10/10 |
| All objects → basket | 10/10 |
| Pen → plate, then bread → plate | 8/10 |
| Pen → red cup | 3/10 |
| Bread → outstretched hand | 0/10 |
Not perfect, but a free baseline on day one is a strong argument for matching an established setup.
5. Evaluate properly
This is where most first experiments fall down.
- Decide before you start: the success criterion, the number of trials and the timeout. Write the criterion so a stranger could judge it from video. Adding "just 10 more trials" after peeking at the results is p-hacking.
- Use fixed start positions, the same set for every policy. Overlaying a reference image on the live camera feed helps you reproduce them.
- Run blind, randomized, interleaved A/B tests in the same session. Never test policy A on Monday and B on Tuesday. If you work alone, have a script pick A or B at random and hide which one is running; you just reset and score.
- Report every trial. Tag failure modes (missed grasp, drop, collision, timeout), and record video of every rollout.
- Report held-out conditions separately, such as new positions, new objects or different lighting. Never pool them with in-distribution results.
How much can N trials tell you? (95% Wilson intervals)
| Result | True success rate is probably between |
|---|---|
| 10/10 | 72% and 100% |
| 8/10 | 49% and 94% |
| 18/20 | 70% and 97% |
| 45/50 | 79% and 96% |
8/10 vs. 5/10 is not a significant difference (Fisher's exact test, p ≈ 0.35). With 10 trials per policy, only gaps of roughly 50–60 percentage points reach significance (10/10 vs. 5/10 is the smallest that does), and even a real 40-point difference will usually be missed. Use 20–50 trials per condition for any claim you care about.
6. Realistic timelines (order → first learned policy; my estimates)
| Archetype | Timeline |
|---|---|
| Desk tinkerer | A weekend of hands-on work after parts arrive; 1–3 weeks of calendar time |
| Serious hobbyist | 2–6 weeks |
| General research | 2–3+ months from purchase order (a new FR3 takes ~8 weeks to arrive) |
| Specialized research | 2–4+ months, dominated by integration (F/T, impedance control, fixtures, resets) |
Bonus: maintenance and lessons learned
Pin everything.
- LeRobot shipped 10 releases in the year to August 2026, several with breaking changes.
- Franka's libfranka must match the robot's system image.
- Never update firmware, SDKs or frameworks in the middle of a data-collection campaign. Update on a branch, rerun hello-world, teleop and replay, then merge.
- On a real-time control PC, hold the kernel and driver packages so they don't update automatically.
Back up:
- calibration files (LeRobot keeps them in
~/.cache/huggingface/lerobot/calibration/) - camera calibrations
- datasets and checkpoints
Keep spares:
- 1–2 of each servo (some Dynamixel models have 2-month lead times)
- a servo control board
- official USB 3 camera cables
- fingertips
- a labeled spare power supply of each voltage
A light routine:
- Every session:
- Test the e-stop.
- Clear the workspace.
- Check camera frame rates, and compare views to the reference photos.
- Feel servo temperatures.
- Weekly:
- Re-seat connectors.
- Inspect cables at extreme poses.
- Retighten clamps.
- Spot-check camera calibration.
- Back up.
- After any repair:
- Recalibrate.
- Re-record a reference episode.
- Bump your dataset version.
The most common beginner mistakes, from GitHub issues and forums:
- Using the wrong power supply voltage on servo kits.
- Daisy-chaining an undersized power supply, which causes brown-outs.
- Stalling servos by squeezing objects or pushing into hard stops.
- Mounting servo horns off-center during assembly.
- Using different robot IDs or calibrations for teleop, recording and evaluation.
- Letting serial ports swap names after a reboot.
- Putting cameras on hubs, USB 2 ports or cheap cables.
- Moving a camera or changing the lighting between data collection and evaluation.
- Building on wobbly tables and camera stands.
- Getting the Franka network or real-time kernel setup wrong.
- Leaving software versions unpinned.
- Making coordinate-frame and convention errors.
- Skipping the hardware e-stop or software limits.
- Collecting too few or inconsistent demos, and never looking at the recorded data.
- Choosing checkpoints by validation loss, and reporting "9/10" without error bars.
Further reading
Some adjacent guides that are great learning sources too! Many of these came from fellow researcher friends, and they helped me a lot when I first started working on robotics:
- DROID hardware setup: the most complete public recipe for a research cell, with a shopping list and assembly guide, plus a camera-calibration workflow in the same docs.
- GELLO: low-cost leader arms for Franka, UR and xArm (a YAM version is in the software repo), plus the teleop comparison study cited above.
- UMI: handheld data collection without a robot. Read it for the gripper design and latency matching.
- ALOHA: the low-cost bimanual setup that popularized ACT, with good notes on teleop ergonomics.
- TidyBot++: a mobile manipulator, but its bill of materials and software docs are a model for any build.
- HIL-SERL: real-world RL with a SpaceMouse. The code shows exactly how to build a safety box.
- MIT Robotic Manipulation, Ch. 2: why most arms are position-controlled, and what torque sensing buys you.
- Wenqian Zhang's Franka setup: a recent FR3 bring-up log, including running π0.5-DROID zero-shot.
- dfdx labs' robotics setup: a 2026 desk setup for under €5k, built on an xArm Lite 6 with no ROS and no real-time kernel.
- Kress-Gazit et al., "Robot Learning as an Empirical Science": how to evaluate policies honestly.
- LeRobot docs: the fastest path from zero to a trained policy.