Deep Dive w/Scott: ESP USBIP bridge #adafruit

Hello. We'll get going in just a bit after I confirm everything is going as planned.

Looks like we're live on reream.

And this here, hi DCD, thanks for joining.

Thanks for letting me know it's working.

Hi Mark.

Hi Bata.

All right, I've I've shuffled my windows around. I realized like 30 minutes ago I

around. I realized like 30 minutes ago I was like, "Oh, I meant to finish putting things there." Um,

things there." Um, so yeah, I think we're going. So, hello

everybody. My name is Scott. I work for Adafruit on Circuit Python. Circuit

Python is a version of Python designed for tiny computers called microcontrollers.

Uh let me find a board. Here's a clue.

Um so this is a a microcontroller development board that Adafruit made. Um

here's the example.

And on its back, it's got uh over here under this piece of metal, there's a little tiny chip that is an all-in-one computer. You plug in your uh device and

computer. You plug in your uh device and it shows up as a Circuit Pi drive.

You're able to program Circuit Python right on this little thing. It's got

sensors. It's got a a screen and some buttons and some touch pads and stuff so that you can interface it with other things. So, this is an example of what

things. So, this is an example of what Adafruit makes. Um you can go to

Adafruit makes. Um you can go to Adafruit.com to purchase cool things like that. Um, and I work on the

like that. Um, and I work on the software side. So, I work remotely.

software side. So, I work remotely.

Hello, Hams Labs. I don't know if I said hi to you. Um, I work remote out of Seattle here. And I work on Circuit

Seattle here. And I work on Circuit Python. So, Circuit Python is one option

Python. So, Circuit Python is one option that you have for programming these little tiny computers called microcontrollers.

And so, this is a deep dive uh usually where I just talk about whatever I've been working on uh for Circuit Python.

Um, and this week it's actually tangential or like around Circuit Python but not Circuit Python itself. Um, and

it's just this week is blown by.

Um, so as I've talked about kind of throughout the whole year, like the way that I've been working uh as a software engineer has changed a lot and it's

changed in a way that um I'm relying on uh large language models to generate a lot of the code and and change the code um for me instead of doing a lot of the

typing myself. Um, and what I found is

typing myself. Um, and what I found is that I get to this point where I have all this code, but I have not tested it.

And um, testing it on device is always really revealing about whether like it actually works. And so I've had this

actually works. And so I've had this idea to like actually do hardware in the loop testing as I call it, which is, you know, you generate some circuit Python code, you load it onto a device and

actually verify that it works on the on the device. Um,

the device. Um, and that used to be something that was a nice to have. Um, I did it within the like first few months of being on

Circuit Python. Um,

Circuit Python. Um, we called it Rosie. So, we had like this Rosie CI thing that like I had a Raspberry Pi with like a bunch of boards

all hanging off of it. Um, but it like relied on the UF2 bootloader to be able to function. And then like even then

to function. And then like even then like the Linux kernel at the time this is like in 2017 maybe um like really early on in working

on circuit python for Adafruit like I started in August of 2016 u but I knew that like it was it was going to be critical and and and it's like totally a

best practice and this is an example of where how large language models have really said like no like these these things that were nice to haves for like doing proper proper software engineering

and like they're way more beneficial now than they were before. Um in addition to that, they also are like um more valuable as well because now they have

more more um more things using it, right? So like not only is it testing the code that I'm doing, but it can also could also be

used to test um test LLM generated code. Um, like we could do like pinmxing stuff and like all writing tests is way easier now because like you don't have to type it

all in. You just say like make me a test

all in. You just say like make me a test that does this. Um,

so I part of that part of that experience led me to the belief that I can't really rely on the

the host OS's Linux or the host OS's USB stack to be stable over time. not its

fault, but like they like to make some assumptions about the USB devices being compliant. Um, and

compliant. Um, and that's not always true when you're changing USB code. So, I I've had this idea for a little while. I think it's it

kind of sparked with the native simulator. The Zephron native simulator

simulator. The Zephron native simulator which I've talked about before is definitely a part of that because you can do it off you can do on your host device. Um but it has support for doing

device. Um but it has support for doing USB IP and I so I like started to learn more about what USB IP is. It is the

ability to like connect to a USB device remotely um over a TCP IP connection or

TCP IP server and um it's a Linux thing.

So Linux in Linux you can actually like have two Linuxes where on one you say like hey like you detach this thing and serve it over the network. So like a

keyboard or a mouse and then you can go to the other Linux and say hey like attach to this USB device that's being exported by this other Linux server. Um

so you can use it for like using mouse and keyboard on a remote server in the cloud or something. It's also really commonly used for bridging containers.

So like your host computer has your mouse and keyboard or you know a test device and you got to get it inside a container and containers are good at

doing like networky um sandboxing and things. Um hi isulttech 34. Um

things. Um hi isulttech 34. Um

and so it's been like kind of somewhat popular for that. And I was thinking, you know what would be really cool is if we had a standalone device I've been using. You know what? Let's see if the

using. You know what? Let's see if the overhead works cuz I've got it.

Hey, it works. Or at least it took a picture at one point. Yep, it's live.

So, what I've got here is you It's actually offcreen. [snorts] Um, and I

actually offcreen. [snorts] Um, and I showed this on show and tell, too, but um these are M5 stack boards. They're

ESP32P4s.

And this Ethernet jack is, uh, has PoE, uh, power over Ethernet. So, um, I've got one plug into this device and it's

then plugged into a hub and the hub is plugged into a C6. I think it's a C6.

But you can see that I've got both things here. Um, so what I've been

things here. Um, so what I've been working on this week is both the firmware for the um kind of like the network bridge device and then um making

getting it to work um kind of end to end with the serial stuff. So nowhere in this stack is my um

is my Linux operating system. Um

let's go to the desktop and see if I can make something work. So, I've

added this command called ls USB IP, and it's doing MDNS discovery, and it didn't work. Let's give it another try,

didn't work. Let's give it another try, and if that doesn't work, I'll unplug it and replplug it back in. Um,

it should work.

I haven't actually been using this today, but the idea is that I can use these these P4 boards to like kind of be isolated

um hardware setups for each individual like setup. So, like one for a C6, one

like setup. So, like one for a C6, one for a C3, once for a C2, one for an S2, one for an S3, and like figure out how to do like these very self-contained

like hardware test uh setups so that if I like have this IDF6 changes that I like really just need to test everything, I could just have one of each. I say build for these six things

each. I say build for these six things that are like a variety of things and then run these tests on them in these like test harnesses, uh, remote test

harnesses. um is my goal. Um and like

harnesses. um is my goal. Um and like yeah, I've spent all week on it, but it's like once we get it set up, like it's going to be really nice.

Um let me catch up with the chat here and see what Mark said. So Mark said, "On a non-electronics project, I've really gotten LMS to help me in the last few months. Save saving typing is a great

months. Save saving typing is a great way to put it and saving Googling time."

Yeah, I think I've started to do more research through that, too. Um DCD, that is correct. That's that's what I showed

is correct. That's that's what I showed the unit PoE P4.

Um Hamslab has been doing hardware in the loop stuff using fruit jam and the RP uh 2350 is really suited for that. Um

agreed. Um I really wanted to do Ethernet or Wi-Fi.

Um, and one of the things I ran into that makes me lean even more um, P4 is that, um, the P4 has 16 host pipes. So,

it can talk to 16 endpoints at the same like kind of concurrently. Um, and that may actually be why it's crashing because I did do some re I told Claude

to do some refactoring yesterday and I thought it was working, but maybe not. I

wonder if I plug it in, we can see the debugging. So, I can still plug it into USB, the the host P4 device, plug it into USB, and get debug

output for that firmware as well.

Um, and then I actually have it connected to a second P4 that I'm trying I'm starting to think about. Okay. Like one of the things I had to test and they kind of

triggered this like oh I don't want to have to do this like is proto matter. So

proto matter is the code that um generates se pin sequences to flash images on RGB matrix LED matrices. And

that's the the sort of thing where actually we don't need the matrix. If we

had just a logic capture of that output, we'd know whether it was working or not.

Um or like we could write code that verified that it worked. And basically

like you could have it like draw what the the image actually was showing. Like

you can it's much quicker to do that sort of harness stuff now um with an LLM. Um and so it's just awesome. So,

LLM. Um and so it's just awesome. So,

uh, so yeah, that's kind of like what I'm trying to get to is like, can I can I get some test harnesses up to help me with the IDF6 evaluation?

And once I do that, I can like always use it, right? Like once I have it going, I can I can always use it. So um

and then I mentioned this to Lady Ada whose name is Lam more on Monday and she I think got commits more of the value of this uh USB to IP bridge. Um

particularly because she was like oh but like on Windows there's there's only this like one way of like mounting them into Windows. I was like, "Well, no, no,

into Windows. I was like, "Well, no, no, no. Like, I actually want to do the

no. Like, I actually want to do the Python side, too, so that like in Python, you do all the USB stuff so that you don't have to have your kernel

involved at all. It's just Python that does USB class stuff into a TCP socket.

Um, and actually, let me just show you.

Let's see. Let's see if this works.

Let me unplug the other one so it finds the right one. Let's just take a quick peek and see. I'm curious why it's not working.

I actually have a mailbag as well.

Let's reset it and see what it says.

There's no reset button on it.

Okay, so it's printing out port stuff. I

found a couple bugs.

It looks like the USB is failing.

You know, I changed hubs and I wonder if the hub is bad.

Um, I bought a cheap hub off Amazon and it's possible it's not working.

Um, which is why I only bought one of them. Um, all right. Well, I'm not going

them. Um, all right. Well, I'm not going to worry about it. Um, I've got too much other stuff to talk about because I think people like mailbags, too. So,

I'll do I'll do a mailbag uh here shortly. Um, but yeah, so that's the

shortly. Um, but yeah, so that's the idea. And then let me just show you

idea. And then let me just show you here. So, if you go to Adafruit GitHub,

here. So, if you go to Adafruit GitHub, two of the three repos are up. Now, this

is like me clotting up. I'm not ashamed of that, but I have been testing it. um like

plugging things in and trying to trying to do stuff. So there's this new um thing here and I'm just trying this Chrome like Chrome just released uh vertical tabs

and I don't need that right now. So um

there's this new repo Adafruit US ESP USB IP bridge. So this is based just an IDF project. Um it does actually require

IDF project. Um it does actually require fixes in ESP USB. So, if you look back here, you'll see that we have a forked

version of ESP USB um that has uh it must have a different branch. Oh, um one thing I ran into is

branch. Oh, um one thing I ran into is that the P4 host code doesn't support full speed devices behind a high-speed hub. Um, it is a high-speed uh

hub. Um, it is a high-speed uh controller on the P4, but uh if you put like a circuit another circuit Python device on the other side of a high-speed

hub, um the the thing talking to the high-speed hub has to do like all this split transaction stuff for carrying like full speed transactions over a high-speed link and that's kind of a

pain. So, I finally realized that I

pain. So, I finally realized that I could just put that they call it the high-speed OTG, but you can force it into a full speed mode. And then you're

talking full speed to the hub and then full speed to the devices, too. Um, so

that's one thing I did here. And then

the other thing was like if I unplugged it or no, I was trying I was trying to get the the ESP IDF. Um, so one thing I'm trying to do is get like the IDF.py

Pi and ESP tool working over USB IP and um when you do those tricks to get into the bootloadader like you have to reenumerate the device. So it like very briefly disappears and comes back up and

so there's a fix here to make it so that hubs the hub controller in on the P4 still recognizes when things go away

just briefly. Um so that's uh part of it

just briefly. Um so that's uh part of it and then the other part. So that's

that's the bottom layer. That's on the device. And then on the host side, the

device. And then on the host side, the way that I'm structuring it is that um this here is a library that allows you to use PI USB

um with a device that's available over USB IP. Um and so PI USB is kind of like

USB IP. Um and so PI USB is kind of like the low-level way of talking to USB devices from Python. It's also the API that we mimic in Circuit Python.

[snorts] So this is this is part of it.

So uh building on top of this layer that has like the PI USB boundary means ideally that like our class level drivers so like serial serial drivers

mass storage driver code that's all in Python uh will be ideally portable between uh CPython on a host using this

bridge to USB IP and portable to circuit Python itself. Um so that's one of my

Python itself. Um so that's one of my goals there. Um,

goals there. Um, so yeah. Um, that's what I've been

so yeah. Um, that's what I've been thinking about. I have I just tickled

thinking about. I have I just tickled Claude to try to do another P4 thing where it does logic capture so that like I don't have to have an RGB matrix. I

just logic capture to make sure that all the patterns are still right. Um,

so I started working on that and my idea is that you could probably integrate that into the same device that's doing

the USB IP bridge. Um,

Tia says, "Can you go into more detail or repeat what you said about being patched for the USB IP brief device disconnect events a minute ago?"

Um, sure. So, so when you go from like application mode to bootloadader mode on an ESP, there's this like

quick toggle of things that happen um to cause it to do the reset. And the way that USB hubs work, um, they don't do

enumeration, but they they have to convey back to the root hub, like the ESP in this P4 in this case, that like a device disappeared and came back. Um, so

that like the root hub can do the enumeration again. And the way that it

enumeration again. And the way that it does that is that there's kind of two bit fields I think it is on uh USB hub.

one says like is there a device on each port of the hub? And then also did the did the port status change

um since you last read the status of the port. Um, and so what was happening with

port. Um, and so what was happening with the ESP hub stuff was that to in order to detect a disconnect, it would it would say like, oh, was it there and not there? And if it was there and not

there? And if it was there and not there, then we'll do this process to like tear down what was what we thought was on that port and reinumerate it. Or

um but it wasn't doing the case where like if if the device left and came back in between those two checks of that status, there's this other bit that says

like, "Oh, by the way, it changed." And

so you you can check that too. And so

even if before and after are say, "Yeah, there's a device." There's a third thing that says like, "Oh, by the way, the status changed between the two times that you read me." So, you can know that like even though it's both here and

here, you know, in between that it was gone. Um, and that's important because

gone. Um, and that's important because USB devices when they come onto the network, they have a default address of zero. Um, and that means it needs

zero. Um, and that means it needs enumeration and the root hub has to tell it what address to use. Um, so what was happening for me was that like I was

doing requests to this device and I was managing to tickle it so it disappeared but because it disappeared um without triggering a reinumeration I

was trying to talk to a particular device address that was gone and um the other device didn't have a new address.

So hopefully that's a little clearer. Um,

and it's an example in my mind of like I the only reason I got clouded there is cuz I've had experience doing USB.

Um, so so yeah um the goal the main goal of this P4 thing is to act as a bridge between USB land and um a TCP IP

connection.

However, however, the way that I think I can make it more of an all-in-one testing device is to also expose virtual USB devices. So, in this case where I

USB devices. So, in this case where I said like, oh, I just kicked off this thing to be a logic analyzer.

I think the way that I'll bridge it upwards is like I will pretend there's another USB device plugged in for this logic analyzer code, but it actually

runs in the same CPU or like same uh microcontroller.

Um, but that's my theory. I don't have that working yet. But it's kind of this this idea that like I can use a hub to plug a bunch of stuff in on the on the

hardware test side and then like as I realize what I need exactly, I can integrate it back into the same same thing to cost it down.

Um, Ty says that's cool. Thank you. Does

that mean that two could arrive at address zero both needing reinumeration?

So I think um the way that you get around that is that like the root hub has some control over

the status of hub ports. Um so like the root hub only has one device connected to it, right? So if there's one thing plugged into it, you know that like the

only thing has only one device has device zero. And then there's a process

device zero. And then there's a process for like, okay, now you have hubs and like hubs are actually something that you talk to. Um, and I think because you can talk to it, you can say like, oh,

this port, this port and this port now have devices on them, but like I'm not ready to talk to this one. Let me talk to this one first. I think it's the way that it would happen. Um,

the hub has some control over that. Um,

it's not it's not like they're all on the same connected wires.

I believe I believe that's the way it works.

Um, and yeah, my my brain is kind of dreaming a little bit about making a custom P4 board that does the USB host

hub stuff and then also like has logic connections to common um, developer form factors so that I can take any, you

know, plug into any Arduino shaped thing and you and have this like hardware in the loop testing work on that pin out.

Um, yeah, I think I think the hub's it's the like root hub code's job to like go through enumeration and part of that's

like talking to the hub and and figuring out like where it is in the in the tree of of stuff. Um,

okay.

So, if you want to follow along, I I'm missing the the top piece. So, the third piece that I have that I have not pushed

yet um is that uh PI serial to PI USB adapter.

Um I was tweaking that a little bit, but that's the third repo. And then

right now you can't you can't so the reason to target pi serial is that pi serial is used by the IDF tools and and a lot of other stuff for serial stuff.

Pi pi serial is like the de facto Python interface to serial boards. So um

ideally you'd be able to just say like hey do pi USB at this address at this like um IP address and this like location on that IP address. and then

you'd be able to do whatever you want with PI serial down to that device. Um,

the way that you can extend PI serial now is a little funky. You have to be imported in the same program even though there's like a way to

register with PI serial um it just doesn't happen. Getting it to happen is

doesn't happen. Getting it to happen is a little tricky. So, I had to like modify ESP tool um in order to test with it. But that's not too bad. and and I'm

it. But that's not too bad. and and I'm kind of exploring options of there's this Python library called pluggy where what you do is is you can get metadata

about all the installed packages from something. So you you could say like hey

something. So you you could say like hey I'm pi serial like tell me all the other packages that claim that they're extensions or plugins for me and then

load those two in this like plug-in way.

Um, okay. So, Tia said, "What's the drawback

okay. So, Tia said, "What's the drawback of the OctoPAP probe tentacle versus your Dream P4 version?" Um, here's the

OctoProbe. Um, it was I don't know

OctoProbe. Um, it was I don't know exactly why, but it was like kind of more expensive than I

would like. I I mean I only bought five,

would like. I I mean I only bought five, but um one it's it is designed with like USB in mind. So like I could hang one of these

mind. So like I could hang one of these off one of my uh off of one of my bridges. Um which I definitely kind of was thinking at the

time, but the other is [clears throat] uh and it's got two separate RP2350s,

I think. Um or maybe they're 2040s.

I think. Um or maybe they're 2040s.

I think they're 2040s. Um one's for like being a like twin pin twiddler and then the other ones um being like the SWD

probe. Um

probe. Um but also I'm really lazy. I'm really

lazy and I'm not that interested in like soldering down to this and wiring to it.

Um, so that's part of it is like I'm picturing what I'm picturing now in my brain is like I have this standard P4

board that exports all its GPIO and has three USBs. One for itself, two for host

three USBs. One for itself, two for host and a hub chip in there. So the two because I think you want two host ports, one for your serial link to the device

and one for loading. Um, and maybe more.

That's the nice thing about USB. Um,

and then you want just like a bunch of GPIO pins out and then you want an adapter to a particular form factor cuz like most of what I'm doing is like testing certain dev kits, right? So I

could have an adapter between this board with a lot of pins and a cutie and this board with a lot of pins and a feather and a what, you know, whatever form factor I'm wanting to test. Um, Arduino

Uno is like still quite common on dev boards. So, like being able to just plug

boards. So, like being able to just plug into that and like at least exercise everything in that space

um is really interesting to me. So, um

yeah, lots of dev boards have two so um two USBs.

So, it's 2:30. I do want to get out of here at three because I'm actually I want to watch the moon return stuff. Um,

and yeah, so let's let's just do mailbag.

Let's do mailbag. So, I've got this box here that I just got today that I was excited about. It's a Digi Key box. Um,

excited about. It's a Digi Key box. Um,

if you know me, I they they did not pay me to say this, although I think Adafruit does get paid by Digi Key to do

some other videos. Um, but um I often go on Digi Key and just grab um dev kits, especially inexpensive ones. I just

submitted a feedback to them that I I'd love it if they could sort the new they have a new products designation, but they don't um

they don't sort it by like the newest of new. Like they don't have a date

new. Like they don't have a date associated with it. So like sometimes I'll go back to look at it again and I see a bunch of the things that I've already seen. So I said like, "Hey, it

already seen. So I said like, "Hey, it would be great if you could sort or filter by like newest after this date um sort of stuff.

Um, okay. So, I've got my more dev kits. I've got so many dev kits. It's it it is ridiculous.

kits. It's it it is ridiculous.

Um, let's see what we got here. I got

the plastic or plastic, the not plastic, the cardboard cushioning or the paper cushioning. First thing here is uh RPI

cushioning. First thing here is uh RPI Pico 2040 W5500.

So, the other thing I was working on is Ethernet support, which I still haven't tested.

Um, but this is a board. It's double

wrapped in ESD.

So, this is like a it's like a Pi Pico, but it's got this um giant Ethernet thing hanging off of it.

And it is supported by Zephyr. So, I

want to make a Zephyr board defaf with Ethernet support for this. Um, and

what is the other thing I'm thinking of?

Hey, they have a run button, too. Nice.

Um, one of these is a PoE version, and I don't think this is it.

Um, so that's one. And I've got Oh, yeah. I've got the other one here. This

yeah. I've got the other one here. This

other one comes in a box. So maybe that is I don't know which one's the POE version.

Maybe I didn't get the POE version. Um,

this is similar. I thought one was PoE, one was not. Um, this one comes in a box.

Um, I need to get back to Ethernet. And

Ethernet's one of those things that's like less easy to test in an isolated way. So, I have been thinking about that

way. So, I have been thinking about that a little bit of like, yeah, I'm gonna get broad coverage that I need for the IDF stuff, but this looks pretty similar.

I don't know which one is.

So, sorry, I'm looking at it too.

So, that's the other wizet.

Wasn't that one?

And what else do I have?

So, I was going, this took a little while. It took like 4 days to ship and

while. It took like 4 days to ship and then it took a few days to get here. So,

remembering exactly why I picked all these is a little tricky. Um,

so this says MCX A366.

And I think the reason I was interested in this one is because it has um LCD support if I remember right, which I kind of realized that like maybe that

wasn't the having native LCD support on an ancu is not necessarily that interesting cuz there are I squared C chips that can do it. Um but yeah,

that's what this one does.

Um it's supported by Zephr. So again,

like I justify a lot of these by just saying like, "Oh, they're Zephyr supported."

supported." Sorry, I'm take I'm taking out all the packaging. I don't need all the

packaging. I don't need all the packaging.

Um I think somebody pointed me to the MCXAs because they're like 5 volt tolerant or tolerant or maybe even capable. Um

capable. Um MCX's are all like newest NXPs. You can

see here it's got the segment LCD there.

which I thought would be kind of neat to add an API for that to circuit Python isolated net bit has me made me realize there's real advantages to single board computers for the hardware in the loop

stuff create a Wi-Fi AP has RJ45 or USB or not TCB dump min in the middle all the things yes so I was actually doing that earlier in the week of I was

actually TCP dumping so the cool thing is is like you can use I could use my Linux computer

as the client, the USB IP client with the ESP server, or I could do the reverse of use my use my Python code as

the client to an a USB IP server that I am uh that my own computer is hosting.

Um, and you can TCP dump either of those. I also added the the ability

those. I also added the the ability um which maybe I don't need to TCP dump it but I added the ability to write pcap

and glogs just like um the the herbs. So

like the they the USB IP works the like quite low level of USB. Um and so I also added the ability for like this the bridge the like Python code to the

bridge to just dump it out as well.

Um, okay. So, let me take this one out of

okay. So, let me take this one out of the bag.

MCX 444 34s.

Um, again, this is like in that newer family. Um, it is beefier than the A

family. Um, it is beefier than the A series, I think. And if I remember right, this has Ethernet. So, I've been thinking about like whether the P4 was

the board to go with or the chip to go with. Um, pime partly because of um not

with. Um, pime partly because of um not wanting partly because of um it you can't do the standard or you you can't do fast

assembly on JLC with the P4. You can

only do like standard assembly cuz the like pin pitch is too close. I've gotten

more USB cables with these.

Um, and I'll just show this board.

I think Well, it's pretty small. Maybe it

doesn't have Oh, so this is Oh, this is another LCD.

This another LCD one.

So, uh, just take this as as an example of like something it would be great to harness up. So, like it's got two USBs,

harness up. So, like it's got two USBs, one's to like an onboard debugger, and then although this looks funny, like the pin out here, um, it actually is also an

Arduino. Like there's an an Arduino Uno

Arduino. Like there's an an Arduino Uno footprint inside of this. So, you don't get all the pins, but you at least get that. Um, so that's another example of

that. Um, so that's another example of like, oh, I buy this, I slap like my harness onto it and then load Circuit Python and then be able to test like all

of the functionality of it or a lot of the functionality of it. Um, as a baseline automatically.

Um, so I think that would be really cool, which is part of the reason how I justify all of this. Like these are like my impulse buy amount is like 20 or $30.

However, this one I knew I wanted. Um, and it was more like I think it's like 50 bucks.

So, this is the other reason it was in stock, so I was like, "Oh, I got to grab this." So, this is

this." So, this is very similar, and it better be what I actually want. The box doesn't say it,

actually want. The box doesn't say it, so I will have to check. Um, it is an ESP 32P4 function EV board.

Um, yeah. I don't think the the MCX 444 didn't have Ethernet. It I was looking at the LCD stuff, so I think that's it.

Um, ah, but on this on the back, it does tell me more information. This is why I bought this. It is revision 3.1 of the

bought this. It is revision 3.1 of the ESP32P4.

So, they did a big revision. Um,

they did I didn't know there was a free shipping limit for a digi key. I don't

think I get that. I pay like the $7 shipping. Um, so I have a P4 sitting

shipping. Um, so I have a P4 sitting next to me, but um they did a big chip revision of the P4. They call this is

the the one next to me is 1 something 1.3. And then they like did an actual

1.3. And then they like did an actual like full chip revision. Not just

sometimes you can do like partial revision but revisions by doing just the metal layer, but then if you have to do the masks themselves like all the way

down into the silicon. Um that's that's what this is. So version three of the P4. And there's actually multiple

P4. And there's actually multiple versions of it. This is apparently 3.1.

Um and they do warn you that it's an engineering sample. Um, but because like

engineering sample. Um, but because like the P4 is probably the next chip we're going to get into. It's just like very nice to have different revisions of silicon for them. Um, and then I'm doing

a lot with it right now with all this um, test harness stuff. So, it just gives me another board to test with as well. IDF6

well. IDF6 um, actually defaults to version three of the P4, which is interesting, even though like you can buy other non third

party boards with an ESP32P4 as well.

Um, so yeah, that's my mailbag. That's the

last thing I got. This one is expensive.

I was thinking about this. I was like, maybe I should just base it off this board cuz it's got a lot of what I want.

Um, but the problem is is that it comes with a display, I think, which makes it way more expensive than if I just got the board. So, like, um, here's the

the board. So, like, um, here's the board. It's got a nice label right on

board. It's got a nice label right on the Ethernet jack of what chip revision it is. Um, which is great. I'm sure the

it is. Um, which is great. I'm sure the board itself is this. I think it's almost the same.

Yeah, it says my other one says version 1.5.2 and the new one says 1.6.

So, they're pretty close.

And then it came with comes with a camera and some like adapters for the LCD.

Um, and this is like, yeah, I wish I wish I could get the P4 in a cheaper form factor because I got it also comes with a screen. Now, it it's possible the

screen is actually a different as well.

So, it comes with like a DSI screen. There you can see my light. Um,

screen. There you can see my light. Um,

so that that's why this this is like a $50ish uh dev kit. And this is uh version 1.2 and 1.3,

the HMI subboard.

So I was using that for the DSI testing, but um I don't really need it now.

So that's my mailbag. We've got 20 minutes. Any questions? Any topics you'd

minutes. Any questions? Any topics you'd like me to go towards?

Um, I know the other way I could go if I want to keep talking.

I don't see anything specific.

So, maybe the other thing we should talk about is I spent some time last week and this week doing open claw. So if I just

show you uh so this is my telegram and we can say oh you can't see it

desktop. So, here's Telegram. Here's my

desktop. So, here's Telegram. Here's my

bot that talks to my claw.

And I could say new.

I don't know if it's up. I have a lot of trouble keeping my OpenClaw running. So,

if you don't know, OpenClaw is this project that got really popular, I don't know, like a month ago, where um you give it access to everything and then it's like your personal assistant. And

there's like people that have been buying Mac minis. And I actually happen, this is running on a Mac Mini that I happened to buy like last summer. Um,

oh, say hello to the live stream. I don't know what it's going to do. [laughter]

So, um, this is me talking over Telegram, um to my open claw.

Um, so it said, "Hey, live stream.

Nukebot here. Scott's Circuit Python coding companion. We get into bugs,

coding companion. We get into bugs, fixes, and the occasional rabbit hole.

Should be fun."

Um so this is using uh so open clause is like integrate everything into an LLM loop and there's some

components to it which is like hey like once in a while like wake up and run this thing like I was trying to get it to do um

issue triage. So like, hey, like at 7

issue triage. So like, hey, like at 7 a.m. like try to get all the new GitHub

a.m. like try to get all the new GitHub issues and um so new hasn't really read my bio.

OpenClaw does have a very well I it was pretty brief for me on boarding where you say like who am I and it asks like who am I and who are you and it's so

like and like what do you want to do? So

like that's where it's getting these um this Circuit Python stuff. So it like the Nootbot thing is that I called it Nootbot SC. It knows that I'm Scott and that I

SC. It knows that I'm Scott and that I wanted to get on Circuit Python. So that

that's all stuff that like kind of got baked in as I on boarded it. Um it's

been kind of a pain. It hasn't been working that well and it has certainly hasn't done the issue stuff. But one

thing that it's actually been kind of helpful for is I set it up isolated. So

I I erased my Mac Mini. I made a new Apple account for it. I made an email for it. It has access to that email,

for it. It has access to that email, although I'm not talking to it that way.

It has its own GitHub account. So like

the only credentials of mine that are on there are like my anthropic ones [laughter] so that it could like call the LLMs. Um,

so it hasn't been that useful for um for actual like Circuit Python stuff. Um,

but what it has been more useful is actually just setting other stuff up on this Mac Mini. So the thing that I thought could also be interesting to

look at is that computer's name is Lyft.

Uh, and it's running um it's running Llama CPP. So, this is a way for me to access um

different uh local models that I can run on my 16 gig Mac Mini. Um, and I have a few set up. The ones that I was most interested in, we were talking about

this week, is the two Gemma 4s. Um,

they're small enough to fit on phones.

Does my open claw pay taxes? No, but I do. I just finished my taxes. I was

do. I just finished my taxes. I was

tempted to use Claude to help me with fill out my taxes because I had to like copy over a bunch of like capital gains transactions. Um, but I did not. Um, but

transactions. Um, but I did not. Um, but

it is awfully tempting anytime I hit something tedious.

Does the Mini have much of a GPU? Um,

it's an M4 base one. I don't remember. I bought it

base one. I don't remember. I bought it off Amazon last like when I was on paternity leave.

So like July, June or July of last year.

And the reason I actually bought it was I was I was off off on paternity leave and I wanted to explore this project for mapping which I don't know if I've

talked about but like I have a history of working on maps and I had this idea for maps that I really wanted to try and actually cloud code at that time was

what I used to to generate that code.

It's called Atlas um Atlas app and I like did it for iOS and I needed the Mac Mini to do that because at the time I was using iPhone um and I had just gotten a new car that

had like CarPlay and I wanted to have integration with that but like you need to get permission from Apple to be to even have a development app show up in your car and I never I still haven't

gotten that. So, I actually switched to

gotten that. So, I actually switched to Android and then like it became less useful, but now I have it running um these models. So, um I was I've been

these models. So, um I was I've been really curious about how good Gemma 4B um the E2B is going to be smaller. Um

than the 4B, but I think we could just switch to it and then it will it it will load up the different one. Um and it says model loaded. And as I would I did

this earlier, so I know that it's going to be pretty interesting. Uh write a basic code.py

code.py does an LED blink in circuit Python.

So what I've been thinking is um so Phil has been thinking about this too of like generating cir generating

circuit Python code using an LLM a local LM and I was like look at these Gemma 4 models that just came out. Um like

Google has a demo app that you can get on iOS and Android to to try using these. Um,

these. Um, I'm sorry. I don't know. I don't know

I'm sorry. I don't know. I don't know how to actually make it better or bigger.

Um, so this is a local model. It is not referencing anything besides what's in the model. Um,

the model. Um, Tyth asks, does it have MCP tools shell access on the lift machine? Can it print the free memory after loading the model

and running a prompt? I don't think this llama CPP does.

Um but if we talk to Yeah, we if we loaded that um how much memory so I'm talking to the claw now. So this is like

running claudet I think is being used by llama cpp. Now

we'll ask the claw that's running on the same computer and we'll go back and this window is really tiny so sorry about that. Um it's doing exclamation of the

that. Um it's doing exclamation of the code which looks kind of like garbage.

Um but I think it actually did do it. So um

it says save the following code as your code.py. It imports time board and

code.py. It imports time board and digital IO. So bo both board and digital

digital IO. So bo both board and digital IO are like circuit Python things and it doesn't have any it's not referencing anything as far as I know. There's no

skills. It didn't do any searching. This

is just from the weights of this like model that I'm running on my 16 gig Macch Mini. Um, so it it successfully

Macch Mini. Um, so it it successfully like said here's the pin, do digital in out, and then in the while loop, it does

true for on, sleep, false for off, and that works. So that's pretty awesome.

that works. So that's pretty awesome.

Now, I'm imagining a world where you have like an a mobile app where you say, "Hey, model, write me this thing, and here's a tool for loading it on the device." And it says, "Okay, load this

device." And it says, "Okay, load this on the device." Instead of telling me to do it, it would actually just do it itself. And then um we could get the

itself. And then um we could get the output of it and feed it back, right? Um

the followup that I was trying with earlier is uh now let's actually let's first uh just tell it to blink faster. I

don't know why it doesn't submit.

Mark says, "I haven't gotten to local LMS for coding yet. Curious how they compare to cloud code, though a lot of it is the power of the prompts from what I've read." So yeah, we were just talking about this in our

Slack channel, too.

um of like how good this would be, these local models would be if it was harnessed. So like being able to like

harnessed. So like being able to like reference things like reference what the API docs are and things like that. Um

so I said, "Okay, fast blank now." And

it said like, "Oh, we're pausing for 0.5 seconds. Let's change that to 0.1."

seconds. Let's change that to 0.1."

And now it's it's done that. So it it passed that test. And the other thing I was trying was uh Okay, now do it with a EOPixel.

Um, so this is a like like I said, this is a local model. It's only got 32K context window. I haven't seen it

context window. I haven't seen it compact yet.

Um, so it's thinking, we can see it's thinking trace here. Understand the goal. Identify the

here. Understand the goal. Identify the

requirements. So I think there is like a system prompt that llama CPP is probably injecting into this as well. Um

but now we can see it coming back and now it's saying like oh like now it also has adafruit neopixel picks the pin the number of pixels and the delay. This

looks correct to me although things that look correct aren't necessarily right.

Um but nothing nothing obvious.

And then the while loop where it sets it to red, it shows it it sleeps and then sets it to black. Shows it and sleeps.

It's interesting that it doesn't use auto write to do it. Um,

Mark says, "Setting up a whole system to use the model tools and agents is a lot more work. Was trying to do a rag search

more work. Was trying to do a rag search using just local models was so so success." Yeah, I I didn't try the

success." Yeah, I I didn't try the search stuff. Um, I'm curious to try

search stuff. Um, I'm curious to try this. I was trying this actually within

this. I was trying this actually within um uh within the PI coding agent which I would like to use more of but um it now

costs tokens rather than a subscription um to to Claude. So um but yeah it it uh nailed it. So let's actually just do a

nailed it. So let's actually just do a new chat and let's switch to the smaller model.

And I I already did this earlier, but it is interesting to compare the two. So

the E2B I think actually takes less. And

okay, so it says Llama server is using 34.6 of RAM right now. Um

let's see.

Is did it load up?

Yeah. So

can I go up right basic code.py

basic code.py links and LED.

So we'll see.

Ah yeah, Tim points out that NeoPixel is one that doesn't have the Adafruit prefix. I think you're right.

prefix. I think you're right.

See, that's why that's I think the the value in the those feedback loops, right? Is like if it could autoload it

right? Is like if it could autoload it onto a device, it would just know. Um it

would be able to tell, hey, this one's definitely faster. Oh, it's got a try

definitely faster. Oh, it's got a try finally. Not so good.

finally. Not so good.

Oh, you know what? But I didn't say for circuit Python.

Um okay, that's fine. So this is using RPI GPIO.

It's interesting that it does like the setup cleanup blink phase.

Oh, and it's catching a key keyboard interrupt.

All right. All right. Well, let's see what how it does with do it for circuit Python instead.

Does it mention bash?

Uh, this is just how to run it here. So,

it says so do Python 3 blink yellow and then it explains it.

Do it. And so, but don't explain.

Okay.

So, this is this is like the first time I have really played with these local models and I'm impressed. I'm I'm quite

impressed with these Gemma 4 models and um the Open Claw was actually like really helpful to set that up. Like it's

got its own sandboxed environment. I've

got like on the router like I can talk to this Lyft box, but Lyft can't initiate connections. Like the router

initiate connections. Like the router does it does it that like replies are okay. Um, and look at that. It didn't

okay. Um, and look at that. It didn't

tell me anything. It just printed the code out.

Um, and that's the E2B model. So, let's say now do it with a It's funny that Tim had to point out to me that like

the NeoPixel imports wrong cuz I didn't notice. This is This is Oh, look. It got

notice. This is This is Oh, look. It got

it right. This one got it right. E2B got

it right. It didn't put the Adafruit prefix.

And that looks okay. LED pin

auto write false. It's interesting. It

does auto write false. So that that must be like the way most people do it. Oh,

and it's doing each individual pixel.

It's not calling fill.

Interesting.

Oh, and weird. It put it in a function.

I don't I don't like that for simple examples. Don't put put the blank in a

examples. Don't put put the blank in a function.

I wonder when I'm going to run or I'm going to hit hit a compaction. I'm only

at 5% context now.

Um, so this is really fun. I was already like briefly looked at like, okay, how how much would it cost to try something else? There's a couple people at

else? There's a couple people at Adafruit that got the the framework uh high memory ones, which when they came out, I was like, h, why are they doing this? And I'm like, I kind of want one.

this? And I'm like, I kind of want one.

They're like $2,000, but they are kind of interesting.

Um, but actually these demo models are interesting from the perspective that they're actually like runnable like this. I think we could get this output

this. I think we could get this output in an Android app or an iOS app and that would be amazing. That would be amazing.

Um, but yeah, yeah, I' I'd love to love to try that on on mobile devices because like I've always had this this fantasy of doing Circuit Python code from a

mobile phone and um, typing on a phone and inputting stuff just is terrible. And so like something like this solves that problem and that's just amazing.

Tia says, "Now you're like 2K for that much RAM." Temptation. Yeah. And they

much RAM." Temptation. Yeah. And they

also just announced that they're going to they have some nextgen announcement in the next couple weeks, like 11 days, like the 21st I think it is. I'm a big fan of Frame Framework because I I

really like their laptops. Like I

switched from the 13 to the 16 and I could bring my webcam, like the upgraded webcam with me.

Uh so how did the knowledge get into the model? Um, where was the training

model? Um, where was the training targeted? This is the generalist Gemma 4

targeted? This is the generalist Gemma 4 model and I assume it's it's trained on just you know all of the open source code um

that we have. Um, so I' I've seen people describe it as like it's built into the weights of the model. Um, I haven't done any post-processing or anything on top

of it. I've just taken Gemma 4. I've had

of it. I've just taken Gemma 4. I've had

have had it take gem before and do that.

Mark says, "I wonder what it would cost to take a coding LLM and do a training modification feeding in a ton of circuit Python code." I mean, I think because

Python code." I mean, I think because it's open source, it kind of already had it, right? Like, and I don't want to get

it, right? Like, and I don't want to get in the business of fine-tuning models.

Like, I'm much more interested in in and this is where I'm living with most of my work. It's like I'm interested in the

work. It's like I'm interested in the loop, the feedback loop that you could feedback loops that you can build, right? So like and so it's like skills

right? So like and so it's like skills skills that you're you're giving access to the agent for. Um so give it ways to

look things up and then like also to initiate trying it all directly itself.

Um so yeah, that's where I'm I'm going with it. Um, I think it's really

with it. Um, I think it's really interesting. Um, and I was very

interesting. Um, and I was very impressed with the Gemma 4 models. I was

like, "Wow, it's it's doing that even without any references." So like on my phone, I was like, "Oh, what is Circuit Python?" And it had like some skill

Python?" And it had like some skill that's like Wikipedia. And so it like went and fetched from Wikipedia like what Circuit Python want. I was like, "Oh, that's that's great. It's kind of

cheating." Uh, but this as far as I

cheating." Uh, but this as far as I know, like it didn't do any like it it had some thinking. Um, but as far as I know, this is just all that's in the

weights. Um, and if I'm wrong,

weights. Um, and if I'm wrong, let me know.

But yeah, that's been cool. And I I wouldn't have tried this if I didn't have open claw on this thing. So it's

kind of been useful for that of like I also said like oh what are other similar models that we can try and so it installed those as well.

Um Mark says one failing I've seen to a degree in cloud is when you ask to use a fastchanging library. I don't always

fastchanging library. I don't always look up it doesn't always look up the latest. I am watching the clock. I see

latest. I am watching the clock. I see

it's three o'clock. I'll wrap up right now. Yes, that I I I ran into that claw

now. Yes, that I I I ran into that claw with claude to mark where like I had it generate some circuit Python code and it did the translations in like the old way

that we had done it. So I think it is really important. Um like Gemma 4 was

really important. Um like Gemma 4 was just released so it's like right now it's up to date but um I think having

material like words that um or code that like the it can decide to reference is really powerful.

Um, and that's how you kind of like mitigate the fact that like the weights are a little stale is that you say like, oh, like if you want to look up the latest API, you can

look here or just just search somewhere.

Like I was a lot of this USB IP stuff this week was like and I have the Linux kernel. It's checked out in repos Linux.

kernel. It's checked out in repos Linux.

Like if you need to reference the the way Linux does it, look there. Like I

don't want to have to give you permissions to GitHub and GitHub raw user and all that stuff. Like I've I've cloned it. It's just there. You should

cloned it. It's just there. You should

look at that. All right. Um, thank you all for that adventure. So, this has been another deep dive uh with Scott. I

think I'm around next week, too. So,

I'll see you in a week. Um, hopefully

I'll have like endto-end testing going.

That'll be amazing. And maybe I can make some more progress on the IDF6 stuff.

Um, it was kind of waiting anyway because we decided to like not commit it until after we do 10.2. So, I had I bought I had a little time there. Um, if

you want to support me, you can go to adafrruit.com, purchase hardware there.

Uh, they pay me to do the streams and work on Circuit Python and uh indirectly buy all these dev dead dev boards. Um,

thanks in advance to DCD uh for time codes. Uh, thanks to Tim uh for doing

codes. Uh, thanks to Tim uh for doing deep dives on Tuesdays um and filling in for me when I'm unable to do it on this Friday spot. Um,

Friday spot. Um, with that, thank you all. have a great weekend. I'll pull up the window uh so

weekend. I'll pull up the window uh so we can see a little gray day here in Seattle. Although it's was kind of be

Seattle. Although it's was kind of be going to be warm, so I have shorts on.

Um which worked out.

And I'll see you all on the NASA stream.

Probably I'm going to switch over to watching the four astronauts re-enter the Earth's atmosphere, which happens in

the next couple hours, I think. So, uh,

best wishes to them and I will see you all next week.