Hardware Skills for the Age of AI
By Leon Ex Machina
Summary
## Key takeaways - **Hardware's new value: Enabling AI data collection**: The value of hardware in the age of AI is shifting from pure integration to enabling faster, better, or cheaper data collection for AI training. This is exemplified by high-paying roles focused on developing systems for scalable robot training data. [01:57], [02:07] - **Robotics data bottleneck: Beyond human observation**: Simply watching videos of humans perform tasks is insufficient for training robots due to the lack of crucial physical data like forces and torques. Collecting high-quality, large-scale robot training data remains a significant challenge. [02:37], [02:52] - **Scalable data collection: Yumi's simple gripper beats complex hands**: The Yumi Universal Manipulator Interface demonstrates that simpler, single-degree-of-freedom grippers with efficient data collection methods are more valuable for training robots than complex, multi-finger anthropomorphic hands. [04:50], [06:08] - **Sell shovels: Hardware's role in the AI gold rush**: Similar to a gold rush, the most reliable way for hardware engineers to profit is by building the tools and systems that enable AI development, specifically by facilitating large-scale data collection. [06:23], [06:45] - **Dilemma: Specialize in robotics or risk stagnation?**: Hardware engineers face a dilemma: specializing in cutting-edge, but volatile, fields like robotics risks vulnerability if the market shifts, while sticking to traditional work may lead to underpayment and missed opportunities. [07:21], [07:34] - **Side projects: Build skills, not just products**: The primary benefit of hardware side projects is skill development and network growth, rather than immediate financial return. These projects build transferable skills and credibility, positioning individuals for future opportunities. [08:24], [08:43]
Topics Covered
- Specialization is key for engineering leadership.
- Hardware's value in AI is enabling faster learning.
- Robotics data collection faces scale and cost challenges.
- Simpler hardware enables scalable AI data collection.
- Build transferable skills through side projects.
Full Transcript
OpenAI just posted a job listing paying
three times the market rate for a
robotics mechanical engineer. What does
this role tell us about value creation
in hardware engineering? And how can we
build skills that are valuable whether
AI humanoids deliver on their promise or
not? After spending a decade across
biotech robotics, Apple autonomous
vehicles, and metawarables, I've learned
the danger of leaving stable roles for
hyped opportunities. I felt it firsthand
leaving a cushy Apple role for
autonomous vehicles only to get laid off
eight months later. It's been a wild
ride since then for myself and many of
my peers, hopping between rocky ships in
this volatile environment. Being a
versatile generalist may seem more
exciting than specializing in the narrow
expertise afforded by most jobs. It may
be the natural result of layoffs or
pivoting out of roles that weren't a
good fit. But every time you switch, you
devalue your private industry knowledge,
reset your company internal influence,
and have to adopt a new corporate
culture and tech stack. The most
successful engineers committed to one
technical domain and became leaders in
their teams and their fields. If I could
do it all over again, I'd specialize.
But in what field or expertise? Humanoid
robotics are hot right now. Top
companies and VCs are betting humanoids
will be the most widely adopted
household robot since the Roomba
launched in 2002. My friends in robotics
are excited for all the unsolved
problems and innovative approaches. But
they're also anxious. The technology is
still years away from commercial
viability. If you're nervous about
specializing in robotics right now,
you're not crazy. I want to share my
observations of technical challenges and
innovations in humanoids. From that,
I'll highlight skills valuable for
roboticists, but also highly
transferable across consumer
electronics, hardware engineering, and
entrepreneurship, so we can hedge
intelligently against any market
volatility coming our way. Part two,
what makes skills valuable? The role is
for a mechanical engineer in robotic
sensing. Six plus years of consumer
industrial electronics experience, two
plus years specialization in developing
robot hardware with tactile and
propricoception sensors like inertial
measurement units or IMUs. Why does it
pay up to 460K plus stock? It's not just
about sensor integration, but developing
hardware systems to enable scalable
robot training data collection. This is
the key insight. In the age of AI, the
value of hardware lies in enabling AI to
learn faster, better, or cheaper. A
quick history recap. OpenAI was actually
one of the first AI robotics pioneers.
Back in 2019, they trained neural
networks to dextrously solve a Rubik's
cube one-handed, but they disbanded the
team a year later in 2020, citing lack
of training data. In late 2022, ChatPT
was released, trained on an internet
scale of data. Now, OpenAI is rebuilding
the team to address the scalable data
collection problem. Robotics currently
lacks this internet scale of robot
training data. Videos of people doing
tasks are lowquality data and
insufficient for training. You can't
become an Olympic level swimmer just by
watching videos of Michael Phelps
channeling a dolphin because videos
don't capture all the invisible forces
applied to the water, the muscle impulse
commands, firing away with every stroke,
nor the torque applied at each joint.
One type of higher quality training data
comes from the demonstrations by human
operators teleoperating or even directly
moving the robot. But data collected
from an operator driving a robot to
generate each demonstration is often
prohibitively expensive to collect at
scale. Think of thousands of robots,
operators, and demonstration stations
running in parallel. Telea operation
also has its limits since humans rely
heavily on compression and sheer force
tactile feedback for dextrous tasks. But
even if you add tactile sensors to
robots, how do you transmit the
information to a human operator? Imagine
trying to tell operator robots to use
chopsticks without the sensation of
touch. Among these challenges, one
stands out as particularly critical.
Dextrous manipulation, the precise
handling of objects in complex scenarios
like piles of clothing or fragile
dishes. These tasks are trivial for us
but surprisingly difficult for robots.
When I first heard about this job, I got
curious. I was at a humanoid robots
conference here in Seattle and stumbled
upon a booth with super highfidelity PZO
resistive tactile sensors. I thought,
could I integrate these into gloves and
robot hands to collect training data?
Would this be a valuable company or at
least a compelling side project? I ran
it by my robotics engineer buddy who
shot that idea right down. Not only are
there several companies like Manis
already making tracking gloves, but
other approaches are better for
collecting training data because they
solve the core problem with tracking
gloves. Human robot hand morphology
differences. Human hands have an
astonishing 27 degrees of freedom.
Companies like Shadow Robot and Clone
Robotics have developed high degree of
freedom anthropomorphic hands, but every
degree of freedom you implement costs
money, space, weight, and controls
complexity. A fullfidelity five-finger
hand with 27 degrees of motion might be
luxurious or wasteful when two or three
fingers are enough for most tasks. So
instead of giving every robot a super
expensive lifelike hand that needs 20
plus motors, you give it two or three
fingers. But now you have a different
problem. How do you teach three-fingered
robot hands using demonstrations from
your human hands? Let me show you how
Top Lab solved this. Stanford and
Columbia University developed Yumi,
Universal Manipulator Interface, a
lowcost open-source data collection
framework. You just mount a GoPro and
two mirrors onto simple 3D printed
handheld grippers. They're just opposing
jaws gear coupled to a spring-loaded
trigger, and operators use them like
hand tools to demonstrate picking things
up, folding clothes, or doing dishes.
The brilliance is in how thoughtfully
simple and efficient Yumi is. A gripper
with just a single degree of freedom,
not 27. Yumi collects data with a single
camera feed recording the environment
and fidial markers of tracking the
gripper's position. The Yumi approach is
more scalable, versatile, and
generalizable to novel environments and
tasks. If you need more dexterity,
there's another promising approach
called Dexop, a passive exoskeleton that
links the human hand to passive robotic
hand movements with mechanical linkages.
Researchers from MIT and Berkeley
developed this approach called Perry
operation that sensorizes and records
human manipulation with maximum
transferability to robots. A human
operator wears passive exoskeleton hands
with vision, propricoception, and
tactile sensors and uses it to
manipulate objects and demonstrate
tasks. Training using the Dexop approach
leads to significantly better task
performance compared to training with
teleoperation data. Notice the pattern
here. The most valuable contribution
isn't building the most complex
hardware. It's building hardware that
enables scale. Yumi's single degree of
freedom beats 27 degrees of freedom
because it's more practical to deploy,
collect data with, and transfer to
actual robot hardware. This is a
transferable insight that goes way
beyond robotics. In the age of AI, one
of the most valuable things you can do
as a designer or hardware engineer is
enable data collection at scale. We're
in this AI gold rush, so many companies
and startups are racing to find
profitable applications of AI. Ilia
Sutzver, co-founder of Open AI, said
data is the fossil fuel for AI, pointing
out that AI progress is hitting a wall
because we've already trained on most of
the useful data available on the
internet. But in any gold rush, the
shest way to make money is to sell
shovels. In this case, it means building
the hardware that collects data useful
for furthering AI. We've seen this
pattern in robotics, but there's a
broader opportunity. There are countless
routines, tasks, and interactions in our
daily lives that could be analyzed and
improved by intelligent hardware. And
there's lots of overlap in the skills
needed to build any sensorized devices.
The pace of work in humanoid robotics
right now is frantic and chaotic. The
technology still seems years away from
being commercially viable, and economic
uncertainty is growing. When the
autonomous vehicles bubble popped in
late 2022, companies far from revenue
shut down or laid off much of their
capital intensive hardware teams, the
job market was flooded with thousands of
hardware engineers looking for jobs.
This creates a dilemma for hardware
engineers. If you're specialized in
cutting edge robotics and the market
shifts, you're vulnerable. But if you're
purely doing traditional mechanical
engineering work, you're likely
underpaid and stagnating, missing out on
the most exciting developments in the
field. Whether robotics and AI booms or
busts, whether we see a recession or
continued growth, you want skills and
credibility that are transferable across
high growth roles and opportunities.
Part three, building transferable
skills. The best way to pick up skills
is to learn by doing and working with a
clear purpose. Hardware is
multiddisiplinary and expensive to
prototype. So people struggle to
identify a product worth building and
then how to build it. The first step is
to connect with smart, creative people
with ideas and passion. Coming up with
ideas gets easier by talking with people
and hearing about their problems and
solutions. Y Cominator Startup School is
a great free platform to link up with
potential collaborators or people to
bounce ideas off of. Next, you'll want
to lower your expectations and just
build for the sake of building. 97% of
hardware startups fail. So, understand
that you're taking long shots where the
expected return isn't millions of
dollars, but better positioning for your
next opportunity. Doing side projects
has two main benefits. One, skill
development. Working on real hardware,
even simple hardware, teaches you
lessons you can't get from courses,
tutorials, or your day job. It pushes
your creativity, resourcefulness,
critical thinking, and grit in ways
unique to being in the hot seat, having
full responsibility. Two, growing your
network and credibility. You bring
something exciting to the table as you
connect with engineers, collaborators,
and users. Builders respect you more for
trying to build something new. Passion
is infectious, and people naturally want
to help those with curiosity and hustle.
Collaborators who've enjoyed working
with you can better vouch for you and
pull you into new opportunities even if
that project doesn't pan out. If you're
searching for an idea with market
potential, I suggest exploring health,
wellness, and productivity. People are
willing to pay to become better versions
of themselves. Look at Eight Sleep,
Whoop, Aura, and Pelaton. These are
billion-dollar hardware startups. Their
hardware analyzes your sleep, health, or
fitness data, then offers insights for a
subscription. These companies succeeded
because they built valuable platforms
around data at scale, the same winning
strategy as open AAI. Evaluate potential
projects through these lenses. Market
validation. Do people actually have this
problem? Talk to 10 potential users.
These conversations fuel your conviction
and inform design decisions. Smart
hardware potential where data or
automation creates clear value. Can you
measure something people care about or
automate something tedious? Can you
provide datadriven insights to change
user behavior? Even if version one is
purely mechanical, is there a compelling
path to adding intelligence learning
value? Will this force you to develop
user research, CAD skills, rapid
prototyping, sensor integration, or
embedded systems expertise? Resource
realistic. Can you prototype this with a
few thousand? Can you build a version
that demonstrates the core value
proposition with limited resources? Let
me tell you what I consider high value,
generalizable skills as a hardware
engineer. User research and problem
validation. Talking to users,
understanding their actual needs versus
what they say they need, determining
what they're willing to pay for a
solution. This precedes detailed design.
It's easy to assume needs, jump to a
solution, and overengineer the wrong
thing. Rapid prototyping and iteration.
The ability to quickly go from concept
to working prototype. This includes
sketching, CAD, 3D printing, and
understanding fabrication techniques.
The ability to put pen to paper and spin
up CAD without analysis paralysis is
critical. Development is inherently
iterative. So prototype to learn, not to
finalize. Sensing and data collection.
This is literally what the OpenAI job is
about. Understanding what sensors to
use, how to integrate them, how to
collect actionable data. Electronics
expertise may seem out of mechanical
engineering scope, but it's invaluable
for excelling in consumer electronics
and systems engineering. These skills
translate across robotics, wearables,
internet of things, and consumer or
industrial hardware. As a bonus,
understanding machine learning and AI
integration points where AI can add
value, what kind of data you need to
collect, and how to design hardware that
enables AI applications. This isn't a
core skill set for mechanical engineers,
but it's strategically valuable if
you're thinking about product leadership
or entrepreneurship. Before getting too
excited about launching a hardware
startup, let me share what legendary
hardware hacker Bunny Hang told me.
>> Harbor isn't like that. It's more like
running a restaurant.
>> Hardware is like running a restaurant.
You have to source components, combine
them, sell them at some margin, and
offer customer service. Building
hardware without VC backing means you're
the business owner, chef, bookkeeper,
server, and dishwasher. Manufacturing
easily costs hundreds of thousands of
dollars to ramp up, and it's hard to
enlist engineers and designers from
their six-f figureure jobs. But you can
still benefit from exploring side
projects without going allin on hardware
entrepreneurship. I'm exploring hardware
ideas and documenting the journey to
bring you along to learn together. If
you found this valuable, subscribe or
join my channel for updates as I share
my findings and struggles. I talk with a
few viewers each week about their
careers and projects. If you want to
chat, there are some discounted
consultation slots on my website. But
honestly, the best thing you can do is
to just start building. Side projects
energize you, make you better at your
job, and empower you to stand on your
own when you need to. Let me know what
you thought of this video or what
excites you in the comments below. I'm
rooting for
Loading video analysis...