LongCut logo

Hardware Skills for the Age of AI

By Leon Ex Machina

Summary

## Key takeaways - **Hardware's new value: Enabling AI data collection**: The value of hardware in the age of AI is shifting from pure integration to enabling faster, better, or cheaper data collection for AI training. This is exemplified by high-paying roles focused on developing systems for scalable robot training data. [01:57], [02:07] - **Robotics data bottleneck: Beyond human observation**: Simply watching videos of humans perform tasks is insufficient for training robots due to the lack of crucial physical data like forces and torques. Collecting high-quality, large-scale robot training data remains a significant challenge. [02:37], [02:52] - **Scalable data collection: Yumi's simple gripper beats complex hands**: The Yumi Universal Manipulator Interface demonstrates that simpler, single-degree-of-freedom grippers with efficient data collection methods are more valuable for training robots than complex, multi-finger anthropomorphic hands. [04:50], [06:08] - **Sell shovels: Hardware's role in the AI gold rush**: Similar to a gold rush, the most reliable way for hardware engineers to profit is by building the tools and systems that enable AI development, specifically by facilitating large-scale data collection. [06:23], [06:45] - **Dilemma: Specialize in robotics or risk stagnation?**: Hardware engineers face a dilemma: specializing in cutting-edge, but volatile, fields like robotics risks vulnerability if the market shifts, while sticking to traditional work may lead to underpayment and missed opportunities. [07:21], [07:34] - **Side projects: Build skills, not just products**: The primary benefit of hardware side projects is skill development and network growth, rather than immediate financial return. These projects build transferable skills and credibility, positioning individuals for future opportunities. [08:24], [08:43]

Topics Covered

  • Specialization is key for engineering leadership.
  • Hardware's value in AI is enabling faster learning.
  • Robotics data collection faces scale and cost challenges.
  • Simpler hardware enables scalable AI data collection.
  • Build transferable skills through side projects.

Full Transcript

OpenAI just posted a job listing paying

three times the market rate for a

robotics mechanical engineer. What does

this role tell us about value creation

in hardware engineering? And how can we

build skills that are valuable whether

AI humanoids deliver on their promise or

not? After spending a decade across

biotech robotics, Apple autonomous

vehicles, and metawarables, I've learned

the danger of leaving stable roles for

hyped opportunities. I felt it firsthand

leaving a cushy Apple role for

autonomous vehicles only to get laid off

eight months later. It's been a wild

ride since then for myself and many of

my peers, hopping between rocky ships in

this volatile environment. Being a

versatile generalist may seem more

exciting than specializing in the narrow

expertise afforded by most jobs. It may

be the natural result of layoffs or

pivoting out of roles that weren't a

good fit. But every time you switch, you

devalue your private industry knowledge,

reset your company internal influence,

and have to adopt a new corporate

culture and tech stack. The most

successful engineers committed to one

technical domain and became leaders in

their teams and their fields. If I could

do it all over again, I'd specialize.

But in what field or expertise? Humanoid

robotics are hot right now. Top

companies and VCs are betting humanoids

will be the most widely adopted

household robot since the Roomba

launched in 2002. My friends in robotics

are excited for all the unsolved

problems and innovative approaches. But

they're also anxious. The technology is

still years away from commercial

viability. If you're nervous about

specializing in robotics right now,

you're not crazy. I want to share my

observations of technical challenges and

innovations in humanoids. From that,

I'll highlight skills valuable for

roboticists, but also highly

transferable across consumer

electronics, hardware engineering, and

entrepreneurship, so we can hedge

intelligently against any market

volatility coming our way. Part two,

what makes skills valuable? The role is

for a mechanical engineer in robotic

sensing. Six plus years of consumer

industrial electronics experience, two

plus years specialization in developing

robot hardware with tactile and

propricoception sensors like inertial

measurement units or IMUs. Why does it

pay up to 460K plus stock? It's not just

about sensor integration, but developing

hardware systems to enable scalable

robot training data collection. This is

the key insight. In the age of AI, the

value of hardware lies in enabling AI to

learn faster, better, or cheaper. A

quick history recap. OpenAI was actually

one of the first AI robotics pioneers.

Back in 2019, they trained neural

networks to dextrously solve a Rubik's

cube one-handed, but they disbanded the

team a year later in 2020, citing lack

of training data. In late 2022, ChatPT

was released, trained on an internet

scale of data. Now, OpenAI is rebuilding

the team to address the scalable data

collection problem. Robotics currently

lacks this internet scale of robot

training data. Videos of people doing

tasks are lowquality data and

insufficient for training. You can't

become an Olympic level swimmer just by

watching videos of Michael Phelps

channeling a dolphin because videos

don't capture all the invisible forces

applied to the water, the muscle impulse

commands, firing away with every stroke,

nor the torque applied at each joint.

One type of higher quality training data

comes from the demonstrations by human

operators teleoperating or even directly

moving the robot. But data collected

from an operator driving a robot to

generate each demonstration is often

prohibitively expensive to collect at

scale. Think of thousands of robots,

operators, and demonstration stations

running in parallel. Telea operation

also has its limits since humans rely

heavily on compression and sheer force

tactile feedback for dextrous tasks. But

even if you add tactile sensors to

robots, how do you transmit the

information to a human operator? Imagine

trying to tell operator robots to use

chopsticks without the sensation of

touch. Among these challenges, one

stands out as particularly critical.

Dextrous manipulation, the precise

handling of objects in complex scenarios

like piles of clothing or fragile

dishes. These tasks are trivial for us

but surprisingly difficult for robots.

When I first heard about this job, I got

curious. I was at a humanoid robots

conference here in Seattle and stumbled

upon a booth with super highfidelity PZO

resistive tactile sensors. I thought,

could I integrate these into gloves and

robot hands to collect training data?

Would this be a valuable company or at

least a compelling side project? I ran

it by my robotics engineer buddy who

shot that idea right down. Not only are

there several companies like Manis

already making tracking gloves, but

other approaches are better for

collecting training data because they

solve the core problem with tracking

gloves. Human robot hand morphology

differences. Human hands have an

astonishing 27 degrees of freedom.

Companies like Shadow Robot and Clone

Robotics have developed high degree of

freedom anthropomorphic hands, but every

degree of freedom you implement costs

money, space, weight, and controls

complexity. A fullfidelity five-finger

hand with 27 degrees of motion might be

luxurious or wasteful when two or three

fingers are enough for most tasks. So

instead of giving every robot a super

expensive lifelike hand that needs 20

plus motors, you give it two or three

fingers. But now you have a different

problem. How do you teach three-fingered

robot hands using demonstrations from

your human hands? Let me show you how

Top Lab solved this. Stanford and

Columbia University developed Yumi,

Universal Manipulator Interface, a

lowcost open-source data collection

framework. You just mount a GoPro and

two mirrors onto simple 3D printed

handheld grippers. They're just opposing

jaws gear coupled to a spring-loaded

trigger, and operators use them like

hand tools to demonstrate picking things

up, folding clothes, or doing dishes.

The brilliance is in how thoughtfully

simple and efficient Yumi is. A gripper

with just a single degree of freedom,

not 27. Yumi collects data with a single

camera feed recording the environment

and fidial markers of tracking the

gripper's position. The Yumi approach is

more scalable, versatile, and

generalizable to novel environments and

tasks. If you need more dexterity,

there's another promising approach

called Dexop, a passive exoskeleton that

links the human hand to passive robotic

hand movements with mechanical linkages.

Researchers from MIT and Berkeley

developed this approach called Perry

operation that sensorizes and records

human manipulation with maximum

transferability to robots. A human

operator wears passive exoskeleton hands

with vision, propricoception, and

tactile sensors and uses it to

manipulate objects and demonstrate

tasks. Training using the Dexop approach

leads to significantly better task

performance compared to training with

teleoperation data. Notice the pattern

here. The most valuable contribution

isn't building the most complex

hardware. It's building hardware that

enables scale. Yumi's single degree of

freedom beats 27 degrees of freedom

because it's more practical to deploy,

collect data with, and transfer to

actual robot hardware. This is a

transferable insight that goes way

beyond robotics. In the age of AI, one

of the most valuable things you can do

as a designer or hardware engineer is

enable data collection at scale. We're

in this AI gold rush, so many companies

and startups are racing to find

profitable applications of AI. Ilia

Sutzver, co-founder of Open AI, said

data is the fossil fuel for AI, pointing

out that AI progress is hitting a wall

because we've already trained on most of

the useful data available on the

internet. But in any gold rush, the

shest way to make money is to sell

shovels. In this case, it means building

the hardware that collects data useful

for furthering AI. We've seen this

pattern in robotics, but there's a

broader opportunity. There are countless

routines, tasks, and interactions in our

daily lives that could be analyzed and

improved by intelligent hardware. And

there's lots of overlap in the skills

needed to build any sensorized devices.

The pace of work in humanoid robotics

right now is frantic and chaotic. The

technology still seems years away from

being commercially viable, and economic

uncertainty is growing. When the

autonomous vehicles bubble popped in

late 2022, companies far from revenue

shut down or laid off much of their

capital intensive hardware teams, the

job market was flooded with thousands of

hardware engineers looking for jobs.

This creates a dilemma for hardware

engineers. If you're specialized in

cutting edge robotics and the market

shifts, you're vulnerable. But if you're

purely doing traditional mechanical

engineering work, you're likely

underpaid and stagnating, missing out on

the most exciting developments in the

field. Whether robotics and AI booms or

busts, whether we see a recession or

continued growth, you want skills and

credibility that are transferable across

high growth roles and opportunities.

Part three, building transferable

skills. The best way to pick up skills

is to learn by doing and working with a

clear purpose. Hardware is

multiddisiplinary and expensive to

prototype. So people struggle to

identify a product worth building and

then how to build it. The first step is

to connect with smart, creative people

with ideas and passion. Coming up with

ideas gets easier by talking with people

and hearing about their problems and

solutions. Y Cominator Startup School is

a great free platform to link up with

potential collaborators or people to

bounce ideas off of. Next, you'll want

to lower your expectations and just

build for the sake of building. 97% of

hardware startups fail. So, understand

that you're taking long shots where the

expected return isn't millions of

dollars, but better positioning for your

next opportunity. Doing side projects

has two main benefits. One, skill

development. Working on real hardware,

even simple hardware, teaches you

lessons you can't get from courses,

tutorials, or your day job. It pushes

your creativity, resourcefulness,

critical thinking, and grit in ways

unique to being in the hot seat, having

full responsibility. Two, growing your

network and credibility. You bring

something exciting to the table as you

connect with engineers, collaborators,

and users. Builders respect you more for

trying to build something new. Passion

is infectious, and people naturally want

to help those with curiosity and hustle.

Collaborators who've enjoyed working

with you can better vouch for you and

pull you into new opportunities even if

that project doesn't pan out. If you're

searching for an idea with market

potential, I suggest exploring health,

wellness, and productivity. People are

willing to pay to become better versions

of themselves. Look at Eight Sleep,

Whoop, Aura, and Pelaton. These are

billion-dollar hardware startups. Their

hardware analyzes your sleep, health, or

fitness data, then offers insights for a

subscription. These companies succeeded

because they built valuable platforms

around data at scale, the same winning

strategy as open AAI. Evaluate potential

projects through these lenses. Market

validation. Do people actually have this

problem? Talk to 10 potential users.

These conversations fuel your conviction

and inform design decisions. Smart

hardware potential where data or

automation creates clear value. Can you

measure something people care about or

automate something tedious? Can you

provide datadriven insights to change

user behavior? Even if version one is

purely mechanical, is there a compelling

path to adding intelligence learning

value? Will this force you to develop

user research, CAD skills, rapid

prototyping, sensor integration, or

embedded systems expertise? Resource

realistic. Can you prototype this with a

few thousand? Can you build a version

that demonstrates the core value

proposition with limited resources? Let

me tell you what I consider high value,

generalizable skills as a hardware

engineer. User research and problem

validation. Talking to users,

understanding their actual needs versus

what they say they need, determining

what they're willing to pay for a

solution. This precedes detailed design.

It's easy to assume needs, jump to a

solution, and overengineer the wrong

thing. Rapid prototyping and iteration.

The ability to quickly go from concept

to working prototype. This includes

sketching, CAD, 3D printing, and

understanding fabrication techniques.

The ability to put pen to paper and spin

up CAD without analysis paralysis is

critical. Development is inherently

iterative. So prototype to learn, not to

finalize. Sensing and data collection.

This is literally what the OpenAI job is

about. Understanding what sensors to

use, how to integrate them, how to

collect actionable data. Electronics

expertise may seem out of mechanical

engineering scope, but it's invaluable

for excelling in consumer electronics

and systems engineering. These skills

translate across robotics, wearables,

internet of things, and consumer or

industrial hardware. As a bonus,

understanding machine learning and AI

integration points where AI can add

value, what kind of data you need to

collect, and how to design hardware that

enables AI applications. This isn't a

core skill set for mechanical engineers,

but it's strategically valuable if

you're thinking about product leadership

or entrepreneurship. Before getting too

excited about launching a hardware

startup, let me share what legendary

hardware hacker Bunny Hang told me.

>> Harbor isn't like that. It's more like

running a restaurant.

>> Hardware is like running a restaurant.

You have to source components, combine

them, sell them at some margin, and

offer customer service. Building

hardware without VC backing means you're

the business owner, chef, bookkeeper,

server, and dishwasher. Manufacturing

easily costs hundreds of thousands of

dollars to ramp up, and it's hard to

enlist engineers and designers from

their six-f figureure jobs. But you can

still benefit from exploring side

projects without going allin on hardware

entrepreneurship. I'm exploring hardware

ideas and documenting the journey to

bring you along to learn together. If

you found this valuable, subscribe or

join my channel for updates as I share

my findings and struggles. I talk with a

few viewers each week about their

careers and projects. If you want to

chat, there are some discounted

consultation slots on my website. But

honestly, the best thing you can do is

to just start building. Side projects

energize you, make you better at your

job, and empower you to stand on your

own when you need to. Let me know what

you thought of this video or what

excites you in the comments below. I'm

rooting for

Loading...

Loading video analysis...