Breaking: Figure Robotics Drops MAJOR Update

Hey y'all, it's Dr. Knowit All. I am

with Dr. Scott Walter once again. Uh,

world traveler Dr. Scott Walter. It's

really exciting to be with you and we have some uh exciting news. So, um,

yeah, I did. You watch it right at 12:00. I was actually out flying with my

12:00. I was actually out flying with my oldest son. He came into town and I took

oldest son. He came into town and I took >> I think it was like 12:06 or I was right at the site at at right at noon Eastern time, you know, and there was nothing there. There's nothing there. There's

there. There's nothing there. There's

nothing there. And [laughter] then the post came out. Evidently, it does take a while for it to trickle through, >> I think. So,

>> yeah. So, I think they were launched at at 9ine, but a lot of times before it makes it to your feed.

>> Oh, no. Actually, it says 12:07, so I think you were probably dead.

>> Okay, maybe it was right. Okay, so maybe it was at 12:07.

>> So, yeah, we'll have to we'll have to like Oh, come on, Brett. You said you said nine o'clock. You missed it by 7 minutes. So,

minutes. So, >> yeah. [laughter]

>> yeah. [laughter] >> Anyway, so this is exciting. The very

first thing I want to um give them kudos for is that they don't have music in this video. Thank you. Thank you as as a

this video. Thank you. Thank you as as a content generator. It's so nice not to

content generator. It's so nice not to have music because I don't have to worry about getting rid of it. Um but anyway, so this is the announcement today.

Figure showing another major milestone uh towards a robot in every home running Helix O2 cleaning a living room fully autonomously. Um before we watch the

autonomously. Um before we watch the video, any statements about the post or anything?

>> Well, no. I I think the the one thing you should know is that um every time Figure puts out a post, it's always 1x and um and it's always autonomous. They

they don't show anything that they kind [laughter] it's it's 1x speed. Yes. And it is always uh it's always autonomous, >> right?

>> And they're so used to it that they actually forget to mention that it's autonomous. And then a lot of questions

autonomous. And then a lot of questions come in this teleopter autonomous. It's

autonomous. And the other thing you'll notice is is that they will do basically a single shot.

>> Um just just so it's very clear. There's

no editing. What what you're seeing is is the whole thing. Yeah.

>> Fixed camera and you know reasonable length. So you know this isn't like a

length. So you know this isn't like a necessarily a long horizon but it's not short. You know it's somewhere in that

short. You know it's somewhere in that middle. It's quite a bit. So you know

middle. It's quite a bit. So you know two and a half minutes is okay. Of

course we've seen them >> post an hourong video before again with one camera so you get an idea of what really can and cannot do. So you get you get a pretty good idea,

>> right? Uh yeah, and I think the fully

>> right? Uh yeah, and I think the fully autonomous part and and of course um just for people who might not know, Helix O2 is not the name of the bot.

That's figure three. Helix 2 is their software that runs the whole thing. It's

a single stack that uh goes photons into controls out basically. So that would be the part. All right, let me uh let me go

the part. All right, let me uh let me go ahead and remove this and add this and Yep. Okay, so I'm going to turn the

Yep. Okay, so I'm going to turn the sound relatively down, but we'll be able to hear it. But, um, all you're hearing is the sound of the room, of course, and the actuators and stuff that the robot is doing. So, uh, we'll go ahead. There

is doing. So, uh, we'll go ahead. There

were a couple of spots in here that I I actually laughed out loud when I saw this. So, I mean, there's just such

this. So, I mean, there's just such human little moments. So, anyway, uh, we don't have to go through the whole thing at once, but just, um, uh, just stop wherever you want. I think the first thing I would love to have known that I

don't see anywhere was what the prompt was, because it's, you know, was it clean up the living room? Was it clean up the coffee cup? put the toys away, turn off the TV. Was it move the

actuator to 27 degrees? You know,

somewhere in between clean up the living room and specifying actuator angles is probably what it actually is. So, uh,

anyway, so hopefully that will be Yeah.

Okay. So, question.

>> Yeah. And I would love to know that. It

would be great. Figure if you guys want to respond, that would be great. Um,

okay. So anyway, so we can see him going in and he's obviously moving the coffee cup so he can >> notice that there's probably a stain or smudge there that needs to be cleaned up.

>> It's a snad for formula 409.

>> Yep. [laughter]

>> Exactly.

>> I was wondering if there's some sort of Easter egg in there with 409.

>> Exactly. And this that was the moment that was the first time I cracked up. I

was like, "Oh gosh, he just like whipped this the towel right over his shoulder."

So, >> and and the question is how much is that a kind of emergent behavior and how much is that been in like the training data?

>> Exactly. Exactly. So, uh so yeah, we can see him doing that and then he's putting the the stuff away and he will the cool part is he returns to that. He doesn't forget about the

to that. He doesn't forget about the fact that that stuff is sitting on the counter. So, and then I thought this was

counter. So, and then I thought this was a trash can at first, but I guess it's a toy hamper.

>> Mhm.

>> So, yeah.

>> Oral hamper. And and your overall impression is this bot right-handed or left-handed?

>> Ooh.

Uh, man, he might actually be fully ambidextrous. I my guess was

ambidextrous. I my guess was right-handed but >> I don't know.

>> You know, the the spray bottle was in his left hand.

>> Mhm.

>> He sprayed with that. Most people would probably do it with their right hand.

>> Um, yeah. You know, the that's sort of a big question because the the training data is going to have a certain amount of right-handedness in there. Yeah,

>> because that's just the the operators, you know, like nine out of 10 of the operators are more than likely going to be right-handed.

>> And this this was the second part that cracked me up. He's just like just just throw it. It's

throw it. It's >> a [laughter] nice nice toss. Yeah, it

was actually >> and and that is kind of a very interesting um you know, I guess you could say a a world model there that that you have is is that it sort of

understands you know from the mass and having maybe toss enough things enough times to have an idea of what you have to do to be able to get that the same way we do when we're playing catch. So

it's not necessarily that's physics are there, it's just that you've had enough experience, you know sort of how much you need to do it and how to aim, >> right? And then notice it also puts the

>> right? And then notice it also puts the towel under its armpit so that it actually has an extra effective extra hand because it had to pick up the coffee cup too.

>> Yeah. And so and uh Yeah. So I that's a very impressive demo and people might be like, "Yeah, okay, whatever." But it's like that's a complete >> cleaning up multi- domain type of thing.

It's like cleaning up the spill, moving the coffee cup, um, cleaning up the toys, resetting the couch, putting the hamper away, and turning off the TV. I

think we talked right over that, but >> Yeah, exactly. Exactly. So, for a second there, I I really thought he was going to sit down and just start watching.

[laughter] It's like, I'm done. Just just go through that. And the question is like,

through that. And the question is like, you know, you can see the TV is actually on, >> right?

>> And um, does it just go in there and automatically turn off or does it notice, oh, the TV's on, it should be off? Right. So, you can see it right

off? Right. So, you can see it right there. It flips off. So, you could just

there. It flips off. So, you could just just see on the TV that it's doing that.

So, yeah. Um,

>> now they do, I think, have other angles of cameras in here because Cory Lynch, who is pretty much an architect of of Helix, >> um, he posted something out there, which was a different angle actually showing

the finger coming down and hitting, okay, >> the off button, which I always have a hard time. And it's one of those like

hard time. And it's one of those like little tiny ones, too. you know, you've got the 64 buttons there and you're trying to find the one that's actually the onoff and it actually um you know, successfully is able to get that. So,

there is a view of that that makes it more interesting to sort of see.

>> Oh, yeah. Yeah. Yeah. Here we go. Okay.

Hold on a sec. Let me uh let me pop this over.

>> I can get that. Uh okay, let me remove this and this. There we go. So, this is one you're talking about, right? Where

we get the the sort of >> Yes. Yes. Yes. He shows that and and

>> Yes. Yes. Yes. He shows that and and then you'll get a kind of a closeup of of the hand and and you can see the dexterity that's there. And first of all, you know, pick picking that up. A

lot of robot hands would have actually a hard time just picking that up like that.

>> Being able to get the fingers to curl underneath you. We we kind of take that

underneath you. We we kind of take that for granted, but you know, it's able to get it readjusted in in the hand and then it's like comes right down there and just hits that, >> right?

>> Uh and it would be nice to know whether it kind of gets a confirmation because just because you hit the button doesn't mean it's off. you then have to look at the TV and again the TV being on prompt it naturally to do that.

>> Yeah. And and that's exactly the um uh the TV remote. Uh the that's exactly where the prompt specificity would be really helpful because I I don't know if you've ever dealt with large language models, but sometimes you have to treat

them like little babies and you have to you have to give them a complete recipe and also say like by the way don't turn the volume way up.

>> So it could be that there's there's a prompt back there. there's some sort of skill that's been sort of set up that when we say tidy up the living room, this is what we mean.

>> Make sure all the pillows are back where there are. Make sure the TV is off. Make

there are. Make sure the TV is off. Make

sure there's nothing on the coffee table. So, there may be like these

table. So, there may be like these checklist of items that it then is going through that's being generated in the background. I'm sure.

background. I'm sure.

>> Right. Or or you could also imagine a a situation where the robot for example, it's like, oh, I just my my boss just left my owner just left for work and this is the routine that you would do,

>> right? So, so and that you would know

>> right? So, so and that you would know that the human has left and therefore like why is the TV still on? That's a

bad, right? Because if I just gone to the bathroom and the thing was cleaning up, I would have been kind of annoyed if it turned off the TV. So, so it's it's just a question of whether it's being aware of the circumstances.

>> No, it's it's saving those watts for tokens, man. [laughter] Looking at that.

tokens, man. [laughter] Looking at that.

It's like, do you realize how much inference we could do with this?

[laughter] >> Yeah, that's probably what it it's like, ah, that was three more tokens just leaving the TV on. [laughter] So, yeah.

>> Exactly.

So yeah. Um, but yeah, so they he's got it's very very cool. And I really do like the close-up things, but also the dexterity. You can see on the right

dexterity. You can see on the right hand, this is the new generation hands with the cameras, right? It's a little hard to tell.

>> It it it might be okay. So there there's like a brand new generation of hands that came out. So I think this is what they're calling figure 3.5.

>> Um, because when 3 came out, it it came out with one set of hands.

>> Yeah. Yeah. And

>> they now have a new set that's able to do abduction.

>> Right. Right. Yeah. And it's just it's a little hard to tell because the the 3.5 is the one with the camera in it. Right.

>> Oh, um I think they had it already.

>> Oh, did three. Oh,

>> three in the camera, but it didn't have the abduction.

>> Yeah. Okay. Uh but you do see as it as it falls. Sorry, the stupid play bar is

it falls. Sorry, the stupid play bar is right. But as it does that, it kind of

right. But as it does that, it kind of like opens its hand and lets it fall in its hand, which is very clever. And of

course bmanual manipulation there because you have to hold it with one hand and push it with the other which again >> not quite as dextrous as a human because of course we would have used our thumbs.

We would have been able to just kind of boop but uh but it's still a pretty amazing capability that it's able to do something like that. So um yeah let's

see what else. And then like I said, and just those couple of human moves were really the things that blew my mind was, let me just turn off the sound because it's slightly annoying, but the fact

that like when it comes in and it's got the towel and then it just like that moment right there is like that is the most human thing to just go like [laughter] right like that. That's

[gasps] just I don't know. I don't know why, but that was that just tickled me because I was like, okay, that that really makes the robot look like it's like a a person inside of a robot suit.

You know, you think C-3PO or something would be doing that rather than an actual robot. So, what did it take us?

actual robot. So, what did it take us?

50 years, right? Star Wars was 77. So,

it's taken us 49 years to get to an actual C3PO.

>> Yeah.

>> Not too bad. Not too bad.

>> Yeah.

And uh it's interesting to see where this these different kind of emergent behaviors come from. So, my favorite one is afterwards after it's cleaned everything up.

>> Yeah. Yeah.

>> Picks up the remote and then when it puts it down, uh, as you probably noticed towards the end there, >> uh, >> Oh, yeah. Yeah. You made a note of that on >> Yeah. made made a little bit of a note.

>> Yeah. made made a little bit of a note.

And you kind of notice what what happens right here is that it it puts it down, then it's like, >> [clears throat] >> uh, no, it's got to be exactly lined up.

It's kind of like monk, you know, it just it's got a little bit of OCD.

>> [laughter] >> Uh so you know again that's that's a question of is that something that was in the training data because that's how the

operators always do it. And this this is what I kind of wonder about sort of emergent behaviors that we're looking at is is that if you're doing a lot of imitation learning at some point you

you're learning it from a variety of different operators who have their own mannerisms and ticks.

>> Yeah.

>> And again their compulsions and everything else. Right.

everything else. Right.

>> So, it might be when they they they straighten something up, they have a certain way of doing it. So, you

shouldn't be surprised if that kind of emerges every now and then, and it's just a question of whether it's it's a much stronger signal for that kind of behavior versus what might be considered more normal or more average behavior of

what everyone is doing.

>> Yeah. Yeah. I'm also curious. So, a lot there are still a lot of hidden things here. Let me take a look at when he goes

here. Let me take a look at when he goes to pick up the uh Formula 40 bottle.

So, uh, I'm thinking because like when he walks in, he's already pre-staged with the stuff he needed. Yeah. Right.

So, it's like, you know, did he have to go pick that up and do that himself or did they put it on his arm, give it to him in the right position to We just don't know some of those details at this point.

>> Yeah. Yeah. Yeah. Exactly. And and

obviously, you all kinds of details here that you don't know about is that like it's it's a little bit of tidying up, but most people say is that they wish their living room was in that state. the

starting state most of the time because it's still already in pretty good shape.

Exactly.

>> So it's it's it's minor things but at the same time we don't want to sit here for you know 20 30 minutes looking at doing the full cleaning. So it's an idea of different kind of tasks you can do you know big questions about okay it

works here but how how diverse different uh environments do you have? How many

kitchens have you built um that are different or let's say living room setups, whatever. If you take it to an

setups, whatever. If you take it to an Airbnb first time site on scene, how it's gonna and and that we don't know and and that would be, you know, a really nice test for for them to be able to go through to see how well it's able to perform in a completely different

environment, >> right? And we can clearly see from the

>> right? And we can clearly see from the the right hand side over here that they're in the figure offices. This is a fake it's a fake living room, which is fine because they said that what's that what did they name that gym that they

made? they made that uh oh I can't

made? they made that uh oh I can't remember >> there is a new I I I I kind of forget it too I don't know if it's the bot gym or something something like that but uh >> uh there's so many different gyms out there now I don't know what to call them

anymore >> yeah so anyway so it's got that and and then of course you know the other piece of this is that we we forget this is total body manipulation which is something Brett said was exceptionally

challenging for them to do so you're seeing it's squatting bending over picking up things by manual manipulation at the same time that it's bending over and not falling on its face, right? That

[laughter] I'm sure they had plenty of times when they were working on it where it was like it just went face.

>> So again, you have like the whole body control, >> right?

>> Uh which is is very impressive that that everyone is working on now. It's, you

know, in the past year that's kind of emerged as as the way to go. And it's

very important to not only get the natural movements, but um you're way more efficient. And as you were able to

more efficient. And as you were able to see with the the kitchen demo um using the you know the hips and the and and the foot to perform different tasks because you have that ability now. So

really it unlocks a lot.

>> You know here's like kind of another question is is what does it mean to tidy up the couch? You know, it's it's you know, se semantically we kind of know what it is, but you sort of wonder how do you train these things that, >> right?

>> Is that right? Is how do you know that the teddy bear is supposed to go in that little hamper and not the pillows, you know? So, clearly it's it kind of knows

know? So, clearly it's it kind of knows what these different objects are and somehow it's got the association, but where did it learn that? That's that's

kind of the things that I I would like to know.

>> Well, and even beyond that, it it was fine with throwing the pillow. That's

close enough where it lands, but it tidied up the remote exactly. So, why is the pillow being there versus exactly in a spot? Okay. Versus the um versus the

a spot? Okay. Versus the um versus the remote. So, you know, it's like it's

remote. So, you know, it's like it's fascinating how it's deciding how it's making these decisions.

>> So, >> yeah. Yeah. Yeah. That that's that's a

>> yeah. Yeah. Yeah. That that's that's a good point. Um you know, in many cases

good point. Um you know, in many cases people throw the pillow because it looks better sometimes. You know, there's just

better sometimes. You know, there's just something about that little kind of goes in. Yeah. Yeah.

in. Yeah. Yeah.

>> Yeah.

>> So, but again, I I h there's so many the problem with all these types of things is the more that they show you, the more you want to know. It's like what was the prompt? What were what was the input

prompt? What were what was the input data? What kind of diversity of data did

data? What kind of diversity of data did they show it? Did everybody throw the pillow? Did some people place the

pillow? Did some people place the pillow?

And even even the pillow placements themselves can be really weird >> cuz I know going to these these Airbnbs and like you know the pillows are in a certain way and then we we move out and we're like [clears throat] exactly how would the pillows and I'm looking [laughter] like man there's so many

different ways of doing it. What's the

correct way? I I don't know.

>> I think the worst one is the beds when they have like six pillows on top and you there's the there's the real pillow, there's like the fufu pillow, there's like the the decorative pillow, there's the decorative pillow on top of the decorative pillow. Yeah. So and that's

decorative pillow. Yeah. So and that's the kind of thing we're talking about here. Clearly there's, you know, green

here. Clearly there's, you know, green there. There's still there's still a lot

there. There's still there's still a lot that has to be done. There's no doubt about it. So, you can easily critique

about it. So, you can easily critique like tons of things. It's like, you know, it's not doesn't do this big questions of like how do I tell it to to clean up my room in particular? I want

the teddy bear there. You know, that that's something for me. You know, how how do you inject those things? And

those things will all be taken care of in time, right?

>> And again, it's like don't worry about the last 10%. [laughter]

>> Yeah. Right now, let's stay focused really on on the 90% and that is the the ability of have a humanoid navigate a a situation like this, have the dexterity and then of course, you know, being able

to do what are kind of the at least these medium-term, you know, and then eventually long horizon uh kind of tasks.

>> Yeah. And and also this is just let's not forget that this hamper is softsided. So, as you could see it as he

softsided. So, as you could see it as he pushed his arm on it, it folded. So

obviously you can't just mash it or else the whole thing will crush. So it's

having to deal with a lot of there there's there's solid objects, there's flexible objects like a towel and like the pillows and there's remote. It's

just it's crazy. And I love the way he kind of shuffles >> down. I guess I call it he Yeah. the way

>> down. I guess I call it he Yeah. the way

it shuffles down as it's doing the pillows >> that that it's just really cool because there's no way and then it's able to turn around and here it turns around which is also incredibly impressive in a

very confined space. It doesn't coffee table, >> right? So,

>> right? So, >> and and that that's sort of the argument of bipedal versus wheeled.

>> Yes. A wheel robot couldn't do this. It

could >> Yes. without the coffee table being more

>> Yes. without the coffee table being more in the middle, right?

>> And you could have a lot of room. So

again, you can come up with a contrived situation that it will work [clears throat] really well for wheeled >> and >> you know there are situations where homes are already set up for that for um

Florida. It's very common. You know

Florida. It's very common. You know

everyone has a very flat home set up for wheelchair access and everything else but you know your real typical home is not that way. It's the spacing is very very different.

>> And I mean I >> I've I've had these discussions with a lot of different roboticists and you you do pretty much have the two factions and they can all make a very very strong

case for one or the other. In some cases it's because they're looking for a particular niche absolutely 100% makes makes sense. Yeah.

makes sense. Yeah.

>> In others that have actually surprised me that you know they're they're able to make a much stronger case for bipedalism than I would have expected when hearing it. So, uh, you know, definitely in the

it. So, uh, you know, definitely in the homes um >> people want wheels because they're stable. They, you know, seem to be

stable. They, you know, seem to be safer, but the reality is you have to reconfigure your home to be accommodating.

>> Yeah. I I think anybody who's ever owned a robot vacuum or not a robot, whatever, an automated vacuum, [laughter] but anybody who has, you have to adjust your home so that it works. There's just

those little disc classic disc vacuum cleaners. um because you have to move

cleaners. um because you have to move the chairs out of the way so that it doesn't run into those, the carpeting, the wires. So that's the kind of thing

the wires. So that's the kind of thing you have to do if you have a standard wheeled robot. And of course the

wheeled robot. And of course the problems get bigger if you have a humansized one as opposed to a a little you know three or four inch tall one. Um

so so there's yeah I mean it's the dropin replacement aspect really at heart. It's like a humanoid robot if it works properly is a drop in replacement for a human. And

that's the advantage.

>> That's the idea. The exact drop. Yeah.

You don't have to change if So that's the difference between, you know, even in factory settings is that there are a lot of factory settings that appear to already be set up for wheeled bots, but there's a lot that really aren't. Even

though you might think, oh, but they got fork trucks, everything they can't.

Yeah, they can drive around 90% of it, but there's like this 10% of the areas that they just can't get into. even the

smaller autonomous vehicles can't get in there. And so if you if you want to go

there. And so if you if you want to go into a brownfield kind of setting, you want to be slotted in. You do not want to have to change anything >> versus a green field where oh yeah, we can reorganize everything and then maybe

we don't need to have it that way. But

there still may be pros and cons to that approach. That's what we'll have to see

approach. That's what we'll have to see is we we're not quite sure what the factory of the future is going to look like.

>> Um right right now factories are kind of designed for people to be in there. So,

um, because there's just so many tasks, we don't know how to automate >> and so people have to be in there. And

the idea of a factory with people in it means that, you know, they talk about lights out, right?

>> It's like, oh, you know, you don't have to have lighting in. Yeah. But there's

all these other things you don't have to have, like you don't necessarily have to have HVAC. [laughter]

have HVAC. [laughter] >> Yeah. Exactly.

>> Yeah. Exactly.

>> All these other things, you don't have to have ceilings of a certain height, you know, just to get air circulation.

all these other little things that you think about that we have to do to make sure it's human friendly. You could

completely reconfigure in such a way that um yeah only a mechanism would be able to go in and and perform these tasks. And if you go in an Amazon

tasks. And if you go in an Amazon warehouse, you're going to see huge volumes of places that people can't get in. I don't think even Bruce Willis

in. I don't think even Bruce Willis could crawl in there. You know, it's [laughter] like places where, you know, they the way the warehousing is set up.

I mean, if you go to like your local Costco, you can see the warehouse there is set up for people to be able to retrieve all that stuff.

>> Right. Right.

>> But the problem is those aisles are very wide and that's space that you cannot store stuff.

>> And you want to figure how do you bring these things in? And so, you know, Amazon is all about, well, how do we do this? And in the end, it's like, well,

this? And in the end, it's like, well, we got to design a space that people can't actually go in there, right? It's

not designed for them. It's designed for a mechanism to the retrieval. So,

they're basically trying to turn their warehouses into like uh RAM. [laughter]

I mean, >> yeah, >> the physical version of RAM is what you're trying to do.

>> It actually reminds me a little bit of the old Have you ever seen like some libraries where they ran out of space and they actually those movable bookshelves where you can open them up and go into Yeah.

>> And there are others where they kind of fold around and come in there, but but in the end, in order to get the stuff, it kind of has to like unfold in a way or unpeel to allow you to go in and access the stuff and come out. Whereas,

you know, you can see these other ways is the retrieval mechanisms are very very different, >> right?

>> And so, so that is is [clears throat] maybe the template of what you would end up seeing in the future because we can take our factories and not make them 2D anymore, but make our factories are kind of two and a halfD.

>> Yeah. Exactly.

>> But to make them fully 3D will be something else, >> right? Yeah. And what you take the So, I

>> right? Yeah. And what you take the So, I mean, you're obviously leaping ahead here. So, the first goal is to create

here. So, the first goal is to create drop in. The second goal is, okay, where

drop in. The second goal is, okay, where do humans fail at this? And obviously

what you're, you know, a human, a forklift, whatever. It takes a certain

forklift, whatever. It takes a certain amount of space to drive a forklift between two aisles, two two shelving units. That's wasted space. What if you

units. That's wasted space. What if you could make it half or a third that size and have this bot that could just go back and forth with an arm that would pull things out? You know, that >> suddenly your warehouse or your factory

is two times as efficient space-wise.

>> Yeah. Yeah. And and things are going to change. I mean, just think about how um

change. I mean, just think about how um you know, Gutenberg's uh printing press and all was very manual.

>> There you just needed a lot of people doing it and then you know, improvements happened over the time to the point that basically [laughter] you take an entire printing press and turn it into this small little box that I can put at the

top of my table and print things out of, you know, much higher quality than Gutenberg. Now, one one slight

Gutenberg. Now, one one slight difference that, you know, people would say is that, you know, people really like the print. It's like, well, there was real pressing going on. I feel like I mean embossment and all all the r there stuff and and that's how and well

was that the whole point or was the point just to put printed word to paper that you used. So you know it it may be that some people will say the only way you can build certain things

>> is this traditional way but in the future we may not care and again it's going to things are going to get more and more compressed. [clears throat]

Well, I there were probably monks back when Gutenberg invented the press that said, "Oh, but you're not doing the actual >> cursive." Exactly. Exactly. The

>> cursive." Exactly. Exactly. The

calligraphy was very, very different.

Oh, yeah. There's definitely a lot of things.

>> Yeah.

>> The good part about it is if people want to do individual printings and stuff, they can do that. You know, that becomes something that is it becomes more artisan, more craft, and not

>> Exactly. Exactly. You're artisan or

>> Exactly. Exactly. You're artisan or craftsman. You're not a laborer. you

craftsman. You're not a laborer. you

you're doing it more out out of love and because of the interest of what's going on as opposed to the drudgery of like h we got to print these things out.

>> Yeah. I I I think you raised a really important point that I want to circle back to though is the factory aspect because he uh figure was very adamant about they were going to go factory

factory factory and of course they went to BMW what gosh maybe two years ago now it's a while um so but but >> it's been interesting how they have

shifted their focus towards the home so do you I have thoughts about that do you have thoughts about why they've shifted at least publicly so explicitly about the home I think in many cases you go

into everything with some sort of priors and assumptions >> and then your your basian gets updated.

[laughter] You know, it's like more information comes in that causes you to kind of change the conclusion of what you're seeing.

>> So, as they're going along, they're probably noticing and seeing a lot of things that say that, hey, you know, um maybe this particular problem that we thought was very very challenging is actually going to be a lot easier uh to

to solve than than we originally thought.

>> Um and and I think there there is some advantage to the diversity of data that you get.

>> The question is if you stay in a niche um are you really as good at that niche as you could be than if you also didn't uh go out of your niche for a little bit and explore some other things that

actually maybe gives you um >> more capabilities. So there's there's a strong argument that could be made for for that that if you really want to be good in manufacturing, it may not hurt to attempt to go and do things in the

home, >> right? Yeah. Cross training. That's what

>> right? Yeah. Cross training. That's what

they that's what all the coaches tell you to do. I I just I'm looking I'm looking at this particular this is from Helix's u sorry a gez it's from figures homepage. Um it it says unlike more

homepage. Um it it says unlike more structured commercial tasks which would be the factory or warehouse the living room changes constantly. objects are

scattered unpredictably blah blah blah and and they actually talk about the narrow navigation path. So they've they I my guess is like you were saying they updated their priors. They were like at

first they're like wow can we make this work in a factory that's really challenging the stand up and do these yeah >> is the factory sometimes is more like your

living room than you think. We we

imagine that it's just this d and yeah there are plenty of tasks where it's just like you're doing the same thing >> but you go into a lot of other and it turns out that people have a variety of different tasks they have to do constantly there's things that are

always changing they have to be very flexible at all times which is why these things could not be automated so that could be part of like I say that that basian update that is you kind of get out there in the factory you realize well wait a minute it isn't always the

same thing you know >> for repetitive for one minute again and again there are these other things and that means there's a certain amount of diversity in understanding it and so it it could be that's where you're seeing

some advantage >> right I mean the other the other argument and I don't know because Brett's never said this but the other argument would be this is just publicity

it's like people the general populace is like oh clean a living room I understand that oh working in a factory I don't understand that I I think that's lower down on the totem pole for them because

I think they're perfectly happy to show you what was like hours and hours of it scanning packages in a factory situation. Uh so I I don't think so, but

situation. Uh so I I don't think so, but one could argue that that is also part of it is just for better public perception, but I don't know. They've

got a lot of money. They don't seem to be in need of like generating cash right now. So, um yeah. So, I've got a comment

now. So, um yeah. So, I've got a comment here. I'm just going to read the first

here. I'm just going to read the first part of this. [laughter] It's from Matias Haymon. No, the second part is

Matias Haymon. No, the second part is like So, it says Tesla better impress with their Optimus demo eventually. I

don't trust their AI development speed to be too impressive. So, what is your response to that? Because, you know, [clears throat] you know, somebody's gonna have to bring up Tesla.

>> Yeah, it's it's true. And uh as you may have noticed, there actually was a comment from Elon about this.

>> Oh, I actually did not. [clears throat]

>> Yes. Yes. Yes. Because

>> see if I can find it.

>> Um not directly underneath it, but under a repost. Um, okay.

repost. Um, okay.

>> I will I'm going to dig up that that comment and find it for you.

>> Okay. Yeah, because I it's not seeing it's not showing directly underneath, but >> No, no, no, no, no, no, no, no. I will.

Um, and if I can find the comment.

>> Here we go. I'm looking. I'm looking.

Was it?

Okay.

Uh, I'm back to three hours ago. So

um >> okay so basically you have to go back four hours >> okay four hours >> uh and the question was autonomously or remotely operated.

>> Um >> and it's it's in your DMs. [snorts] >> Oh thank you. [laughter]

That's probably a good way to look at that. So,

that. So, all right.

So, figure release new video this demonstration. Well, that's interesting.

demonstration. Well, that's interesting.

Um, that that's an interesting question because I think as you said right at the beginning

of this broadcast, like figure has pretty much always done everything autonomous.

>> Yeah. And and the thing is is let's let's be fair, there's a lot of different companies out there, a lot of different technologies, a lot of humanoid companies, right? And you can always say, well, if you follow them really closely, you would know.

>> Yeah.

>> And so for those of us who all I think he's got a day job that keeps him from being able to follow everyone that closely and know what's going on. So it

is a legitimate question that that comes [clears throat] up quite a bit. Um, and

you know, so we all we all know because they're in there and and it was buried, you know, somewhere else. I think in in Corey's post and others that made it very very clear that it was autonomous.

And again, they're so used to it being autonomous, they forget they had to put that in.

>> Yeah, it's one of those things quite honestly that they should probably just mark they should watermark their um videos. Just just have a standard thing

videos. Just just have a standard thing that says fully autonomous 1x speed or something like that, right? and ju just to their social media people like just plop that in there and then there's no never a question. So um but yeah I mean

it's whole body end to end cleanup walkthrough continuously manipulating objects uh a lot of really cool key results. So yeah I'm not blaming him

results. So yeah I'm not blaming him because also that was saw your merit that reposted that. So it wasn't even the original post. So Elon did you know he's like he's like I'm not going to go back and dig through and try to find out

whether it was full. Actually, actually,

actually, the funny thing is is Sawyer actually says right in the top autonomously.

>> Yeah. Uh Oh, yeah.

>> Yeah. [laughter]

That is true. Most people never read everything.

>> Yeah. Well, no, it's more along the lines of saw your merit said that, but I think Elon may have been >> He didn't read it. He didn't read it.

>> He may not have trusted.

>> He may not may not have read it. Yeah.

Or again, you're kind of seeing what's going on and then you just want to know.

So, um, that obviously invited definitely a lot of comments and the the thing is the >> we're getting very close to the end of uh of the first quarter. So, there's

what three weeks left.

>> Yeah.

>> Yeah. And, uh, we haven't really seen any any signs of whether um, you know, what's going on with with V3. Um, and

the thing is is like the the bar keeps going up and and I I think the Elon boxed himself in a corner back in the maybe late summer >> where he talked about that that V3 was

going to everyone's going to be convinced that it's a man in a suit >> and then like three weeks later Jang comes out shows you basically a woman in a suit or what looks like a woman in a

suit and and that and that whole thing.

So it was like ah okay so so it has to be at least as good as that. Yeah. Well,

and I would say this is a man in a suit, too. This this very much like those

too. This this very much like those moments where it throws the towel over.

>> Well, well, that way, but let's say it's like you, you know, it's not because you look at its waist and you go, it's like that doesn't, you know, it's like find someone in the waist that that's that thing. You look at the elbow and you say

thing. You look at the elbow and you say that, well, it doesn't, you know, maybe if it's it was a person wearing a costume, but you know, it's probably a robot. You can't convince me it's a man

robot. You can't convince me it's a man in the suit. No. And I think what they want to do with Optimus is they they want you to look at it and just say, uh, you know, the elbow looks like a human elbow. I don't see an actuator there.

elbow. I don't see an actuator there.

The knees really look like a human knee.

You know, all those things are um that's challenging. That that that's going to

challenging. That that that's going to be very very challenging, you know, was was very very close on that. If you look really closely, you could kind of see where parts of the mechanisms were protruding out a little bit that were

kind of indicators, but that took like a really trained eye. And at the same time, [laughter] he could have it could have been a person in a suit wearing something to make it look like it was a person, you know, that kind of thing.

>> They did eventually show us under the skin. They showed us like all of the uh

skin. They showed us like all of the uh the flexibilities. Yeah, I guess I meant

the flexibilities. Yeah, I guess I meant more in terms of motion here that human.

But yes, obviously that waist that would be a very skinny person to be able [laughter] to fit inside that outfit.

>> Exactly. Exactly. So, so there are parts that let you know that it is a mechanism as opposed to a person. And of course the same thing with uh you know with the Gen two optimist is you know you can tell it's like yeah that's not a person

we can see the mechanism.

>> Yeah. Yep. So uh but I think what exactly what you said is correct. It's

like I'm sure that Tesla is moving very quickly but the problem is that everybody else is too. So they maybe say like hey here's the finish line. And all

of a sudden the finish line went way up here and you're like uh oh we're still not the finish line but the you know the catchup line whatever. So yeah. Yeah.

So, um, yeah, I mean, we've got, um, you know, something that's pretty interesting. Good, a good question. Um,

interesting. Good, a good question. Um,

I think we know what the answer to that is. Um, we'll have to wait and see when,

is. Um, we'll have to wait and see when, you know, what is conf I I I guess maybe technically you could say April 15th is the the end of the first quarter [laughter] because that's when your

taxes are due.

>> Oh, you have to remind me of that. Now I

got to think about my taxes again. Yeah,

exactly. Exactly. [laughter]

>> So, yes, but I guess I guess you could officially stretch it till then, but it's still it's coming down to the wire.

There's not a lot of time left in the quarter. So, yeah,

quarter. So, yeah, >> there's there's not much. Yep.

>> Yep. All right. Well, cool. Um, any

final thoughts? I I am super impressed by this, by the way. I I I don't know if we mentioned that at the beginning, but this is incredibly impressive to see it manipulating all portions of its body, staying in balance, not looking like

it's about to fall over at every second.

Um.

>> Yeah. Yeah.

>> Yeah. And and I guess you got to think about that for a second is like its balance point is different than ours.

>> Yeah.

>> It it's very remember it's it's made out of steel or aluminum.

>> Yeah.

>> Uh- which means its density is very different. Its mass distribution is very

different. Its mass distribution is very different than us.

>> And that our upper torso is basically air. That's where our lungs are, right?

air. That's where our lungs are, right?

>> Yeah.

>> Um so the distribution there is is kind of different. It's filled with batteries

of different. It's filled with batteries which are probably the densest, heaviest part of the entire bot. So, when it's leaning forward, it's its balance point is going to be different. It's right

>> it's going to it's not going to look 100% human. There's going to have to be

100% human. There's going to have to be something different. It's uh to to be

something different. It's uh to to be able to get that, but still it's pretty close to Martin or or maybe it's not as far off as I expect, but you know, I just think that the location of the center and that to know how how

important like the center of gravity is.

You you've probably seen these things where >> um the the the average female distribution for center of gravity is very different than a male. Mhm.

>> And you seen the thing with like lifting up a chair. Like there's one thing leaning over lifting up a chair that women can do.

>> Yeah.

>> That men can't. Men fail at it all the time.

>> Well, I've seen there's another one where you like kneel down or something and women can put their arms behind their backs and men just face plant as soon as >> Yeah. Yeah. Yeah. There's something like

>> Yeah. Yeah. Yeah. There's something like that. And and what that does, it just

that. And and what that does, it just shows you how was like this subtle redistribution of mass around the body >> can have a profound effect >> on something like that. and and robots

are very different as far as their mass distribution, >> right? And and I think it's worth um we

>> right? And and I think it's worth um we didn't really talk about Helix 2 in any detail, but but there was a previous post and I did another video. I'll link

it at the end of this one, but um >> there's three levels of this software.

So, there's the reasoning level, which thinks about on the order of once per second. It's like looking around the

second. It's like looking around the room going like, "Oh, the TV's on, right? Got to turn it off." That kind of

right? Got to turn it off." That kind of thing. Yeah. Then there's the S1, which

thing. Yeah. Then there's the S1, which is the kind of second by- second decisions of oh don't like whack into the table or something like that and then [clears throat] I

think most importantly for what you're talking about here is um the S0 which is very >> which is the which is the very very low level

>> yeah that's that's I think that's something running at 500 or 1000 it runs at like very very high speed >> yeah it's >> and then you have like the next level up which I'm not sure if that's like 30 Hz

or 60 Hz and then like say the higher one which just a couple hertz at most.

>> Yeah. Yeah. But but the important part of that one is that's the part that's actually doing the don't fall over kind of stuff.

>> Yeah.

>> Exactly.

>> And and that that we don't know exactly, but >> I suspect that that was trained in reinforcement learning specifically for this bot. So in other words, if the

this bot. So in other words, if the human center of gravity is let's say four or five centimeters lower than this bot, then that's the S0 is where that

adjustment comes in. That's that's super lowle stuff.

>> But but a lot of that would come from like you say the RL training that's that's being done in the simulation.

>> So the simulation has that information.

It's not learning to walk from humans.

It's grass from humans, but but the whole body control Yeah. that comes from something else. that's not really

something else. that's not really imitating the the human though it may be using some as like suggestions of what you want to do because the whole idea of it is that you want to get the hands to where they need to be to do the task.

>> Yeah.

>> And and so once you kind of know where the hands are, now you have to figure out how do I how do I get the place of my hands? Well, that's what you use the

my hands? Well, that's what you use the arms for to kind of get it. But then

when but then that also has to go back to the constraint. Well, that means like your shoulders have to be somewhere. or

how to get to so well the course torso has of your whole body and stuff like that. So all these things kind of play

that. So all these things kind of play in there and these decisions of like well if I need to get my hands here can I just do it with my uh my arms going back and forth or do I actually have to walk forward that there's so many different strategies that you can use

and that's what they're event they're trying to all figure out and sometimes it has to be done in concert >> because like you really do have to lean forward in order to get down and to get your balance and all that.

>> Right. Right. Uh yeah, and again all of this stuff you can see it's having to manipulate multiple parts of its body, change its center of gravity and where it is when it's picking up the towels,

the sorry the pillows, not the towels.

>> Um >> and but and it also as you look at this, obviously the hands are the major endectors that it's trying to manipulate to get into a position to do the task.

But it also is cool like the hamper, it uses the armpit to grab that.

>> Um it throws the towel over its shoulder. So it's it's utilizing other

shoulder. So it's it's utilizing other parts of its body in some situations as temporary endectors effectively.

>> Exactly. Ex. Exactly. And and um I I think [gasps] one time when we interviewed Bernack, he he kind of put walking and bipalism in like this interesting way I framing I

had never thought of. He he talked about the feet basically manipulating the ground. You know, it's like we always

ground. You know, it's like we always think about the hands, but it's it's always it's kind of the same idea.

They're looking at that. I was like, "Well, yeah, I guess that's that's how you have to kind of look at walking."

And and then with that, you do realize that uh we have points of contact throughout our body that So everything from when you remember how it did like the hip check to to close the drawer and then it also used the foot to lift up

the the door for the the dishwasher.

>> Dishwasher. Yeah.

>> And and the other thing is I think if you go back to that was I think like the end of January there was a post on that.

So if you go to the figure website and just look like under news, >> they have all these white papers there for you. And there is that one from

for you. And there is that one from January 27th I think that um goes through and talks about the S2, S1, and S0 layers for anyone that wants to know a little bit more detail about it.

[snorts] >> Yeah. So I think the full body control,

>> Yeah. So I think the full body control, >> I think uh No, that was 2025. Where the

hell?

>> I think it's that one right there.

Introducing full body control right there.

>> Yeah, this one. Yeah,

>> I think it's in that one. Yeah.

>> Yep. So someplace in here. Like I said, I did a whole Yeah, here it is. System

zero one and two. Yeah.

>> So very very cool stuff. Um, but it's the, you know, in retrospect, probably the obvious way to solve the problem.

It's like system two is where the real human training comes in. That's where

it's like if somebody says tidy up the room, what does that mean? That's the

real highlevel thinking. And then system one and system zero are the kind of reinforcement learning. The bot kind of

reinforcement learning. The bot kind of does the virtual playground, then the real world playground and learns on its own. Now, one other question we So, we

own. Now, one other question we So, we don't know what the prompt was for this.

We don't know how exacting it was. Um,

another thing that we don't know is uh Oh gosh, what was I just gonna say?

[laughter] I just lost what I was gonna say. Um,

system one, system zero. Ah, anyway. Oh

well.

>> So, you're you're doing the pro you you're talking about like what the prompt was, whether as simple as tidy.

You know, I I would I would say it could have been as simple as a prompt like tidy >> up the living room. Um just because you know it was a year ago Corey showed the

example of uh putting the uh the food away in the pantry. Remember so he came in with the groceries >> and it was a very simple prompt. It was

like hey figures you know put this away.

So nothing else.

>> That's right. That's right. Because he

actually spoke to it at that point.

>> Yeah. He actually actually kind of gave it a very simple thing. And again you know it just comes down to to building up a bunch of different skills that you have that you you know what it is. And

at that point, it's like, oh, just take the book off the shelf and start reading it. [laughter]

it. [laughter] >> So, it may be that they really didn't have to do very much. And then there's there's enough understanding of the different situations that um it made

sure to check certain things.

>> Yeah, >> these these these VLMs and all are getting pretty pretty smart nowadays.

It's it's uncanny what they can do, >> right? Yeah. it it and I think again

>> right? Yeah. it it and I think again we've got that major advantage that they actually have access to the world that they they have the ground truth out there. So I think I we'll sign off in

there. So I think I we'll sign off in just a second but I actually wanted to uh just bring this up. This was a post a comment from Bongo Wongo said I'm I'm still not buying it. Nothing to show

reasoning all pre- choreographed. So

that we're just talking about that. Do

you agree with that disagree with that?

Do you think that this shows some level of reasoning about the world?

Not choreographed, but choreographed as Cory Lynch. Um,

Cory Lynch. Um, you knew you had to do that. [laughter]

>> Yeah, we're getting it's it's sort of hard to say. It

they've shown all these things. So, it

it's been trained on to do a lot of these things again and again and again, right?

>> So, pretty much, you know, in its quiver, it knows how to do all these things. So, I'm sure they had like one

things. So, I'm sure they had like one guy just go over and like tidy the pillows again and again and again and just pick up the toys again and again and again and and then eventually somehow they get they they're able to

fuse the whole thing together. So, I

don't think it's it's choreographed in the way that you may think.

>> Yeah.

>> Um like training.

>> Yeah.

>> The Chinese the Chinese uh very impressive Chinese bot.

>> That's very choreographed.

>> Yes. Yeah. That's that's like a tape recorder that's just playing back the motions. Um, it was really impressive.

motions. Um, it was really impressive.

I'm not taking anything away from that, but it's a very different thing. I think

this is hopefully where we're guessing here, but I don't think they're saying move here, then do this, move here, then.

>> No, no, no, no, no. They're not pushing a bunch of buttons because that's what it is when um a lot of these bots that you can buy, they already have a pre-programmed script and like you just kind of push a button and it basically

acts that that whole thing out for you, right? um in this case um you know it's

right? um in this case um you know it's probably chaining these things together on its own based on the fact that the the environment has changed and you know I'm sure um you know maybe we'll see

another one where the coffee table's been moved around everything's you know you just change environment or you go to an Airbnb with kind of a similar configuration setup and see how it behaves >> um and and and that would be more

impressive to let you know that it's not really choreographed but you [laughter] know skepticism is healthy okay so >> Oh Absolutely.

>> There's there's no reason why to not kind of ask that question. And again,

you know what, if we just go back for a second, the the Chinese New Year presentation, which was very very impressive. I don't think people realize

impressive. I don't think people realize what to me what was impressive is that in order to pull that off, they needed to have a lot of robots. And I don't mean like the number of robots were on

the stage.

>> Yeah.

>> I mean, there was a training phase, a real world training phase where doing those catapults and stuff like that.

>> Yeah. I don't know how many catapults they had to do before they finally refined the policy to get it right, >> but I'm pretty sure there are a lot of robots that face planted.

>> Oh, yeah.

>> And and even when they finally got it right, they probably went through and tested it a few times, >> those bots were coming down hard.

>> Yeah.

>> Yeah.

>> And and I don't think that they can do that like a hundred times. They might be able to do it one or two times. So, more

than likely they got it to the point because they were able to produce enough of these bots and and they're close enough to one another that many of the bots that were out there maybe have had never actually performed that stunt.

>> Yeah.

>> They were literally fresh off the line.

The policy was in there. They just sent them out. You know, they probably did a

them out. You know, they probably did a few things just to calibrate, make sure it was right, and then they they able to hit it. So, to to me, that's probably

hit it. So, to to me, that's probably what's more impressive than anything else, like the sheer number that were involved to be able to pull it off, >> right?

[cough and laughter] Because yeah, clearly there was a lot of failure during the process of learning how to >> Oh, there there had to be I'd love to see the outakes. They must have been hilarious [laughter]

because you know it had to end and and there were some impressive like there was what I could only call like the the tenant scene where remember um the robot hurdles over and then it literally runs

backwards >> in time and you're looking at that it's like man and Michael Jackson couldn't do that. you know, that's just [laughter]

that. you know, that's just [laughter] it literally was like the whole thing is backwards.

>> Yeah. So, again, it's it's all of this stuff I feel like is it's not like one's better than the other. They're both

necessary. Like what they're doing and the Chinese bot manufacturers are really showing the limits of actuators, the cap, this is like way beyond what we can do physically. I I I've never been able

do physically. I I I've never been able to do a backflip or, you know, or anything like that. So, you know, that's something that was just not even within John's possibility stack. So, I don't know about you, maybe you could, but there it's very few people in the world

that can do things like back flips and whatever other crazy moves. So,

>> and the thing is that surprising me is that that some some of they're getting a little bit better, but I still don't think they're landing the way you should to absorb the shock.

>> I think, you know, they're they're absorbing a lot of it as as opposed to coming down, making that nice contact, and then trying to absorb it with the actuators. See, like bam, they're coming

actuators. See, like bam, they're coming down hard. And I'm sure they have like

down hard. And I'm sure they have like some padding that helps a little bit, but >> Well, again, speaking of like even the ones that made it could easily have

damage their actuators doing that, so that there's a very limited >> Oh, yeah. Yeah. Exactly. Exactly. So, I

don't think they're necessarily were running the next day. And again, it's it's you may have only seen, I don't know, maybe 12 bots on stage at a time.

I'm not not sure what the number is. It

doesn't mean there was only 12 bots there. Chances are there was like 48 or

there. Chances are there was like 48 or Yeah, there was there was way more that they were just constantly like moving them out and that that some of those they they performed for

>> for 90 seconds and that was it. They

were off stage in the next one because you know some of those stunts that they were doing um are known to overheat the actuators really quickly.

>> Yeah.

>> And and quickly we're talking like a matter of just a like couple of minutes at most. Sometimes within 90 seconds. So

at most. Sometimes within 90 seconds. So

if they're doing a series of those back flips, that's like, okay, get it off the stage. [laughter]

stage. [laughter] >> It comes off the next.

Yeah.

>> But I mean, >> they're also wearing costume, which makes it even harder to stay cool.

>> Crazy. But the thing that they're getting out of this, if you think about it from a practical level, is I bet their actuator design is taking leaps and bounds because it's like if you put it through this, which of these actuator

designs actually survives this insane, you know, kind of over taxing work, which is very different than what we're seeing here, which is a much more mellow, smooth, you know, just do your

job kind of thing. though. So, yeah,

again, all of these pieces are stacking together to create something that's going to be pretty darn magical when it happens [laughter] when it all comes together here. So,

>> yep.

>> All righty. Well, I've kept you on for a long time. Uh, thank you so much for

long time. Uh, thank you so much for talking about this. It's just amazing stuff to see and I just I'm glad you had the opportunity to join me and everyone should definitely thank Scott.

Definitely go to going ballistic 5 onx and check him out and uh yep. Anything

else? Otherwise, we can say goodbye to everybody and thank you so much for the comments and everything.

>> Okay. Yeah. Bye now.

>> Cool. All right. Let me go ahead.