ChatGPT Canvas Executes Python in Browser Now! But Who Is It Really For?

hey guys uh so since last week open EA has something like 12 days of shipping and we are on day four they already

announced a new 01 model prom mode for 200 bucks they also announced release of Sora and they also did some stuff around fine-tuning models this is day four and

today is about conas their uh collaborative writing thing inside of chbt that now if we're going to go to chbt it now hides here and maybe we can

say like create poem so when it's on it's going to write it in a separate block you can open it

and you can edit it when it finishes so they announced updates to it before it was only for plus users and yeah here I wrote a blog post about the announcement because I was

like testing it seeing what's new and there are three things that they announced that canvas is now available for everyone before it was only for plus

users and uh they added new support for python so you can now write python there it's highlighted and you can execute python

it can help you debug python or or so they claimed and it also now is possible to integrate them into custom gpts so people making their own custom gpts now

can integrate them with this canvas mode and for me the most interesting part was this python thingy and I started testing it and

honestly I'm bit confused I even made a table it's just the thing is that uh chpt for a while already has what is called chpt code interpreter they

announced it in something like April 23 and it's out for everyone since uh summer of 23 so for longer than a year

and what Cod code interpreter is it's a tool where let me show you maybe like use code

interpreter to draw a chart for me it's interesting they're hiding it kind of it's not here well

uh just imagine some data I want to see how it

works oops typle but here how it works for longer than a year it writes python it then sends code it wrote and files you upload to the chat I will show you a

little bit later with upload of files but it sends files and code to server it starts a server instance sendbox on like

open eii or maybe Microsoft servers sends code files there executes it and sends the results back and chart sees the results uh so if you asked for

analysis it will like you can iterate with it on the analysis it sees the errors and tries to fix them it sees the answers it may uh like see problems with

the answers write more code send more code execute more code it can iterate with you it's assistant so this is what chpt already

has so what's new and different about this new python execution environment let's take a look I wanted to show you how that looks let's go

let's start a new chat uh let's uh enable canas and say something like

create small python Hello World app for me okay so here the way it worked was that

it wrote some python in this kind of block and sent that code to run on the server and got from the server this image

chart now what happens here is that it wrote the code in this convas block it's highlighted we can edit it this is new this is different you cannot edit it in

code interpreter you can tell it what you want to change but otherwise it writes it itself so this is this is what's different about canas you co-

write with it it's a coworker while this just just worker which sometimes you actually would prefer okay let's go back here so it

wrote this and now you can run it up here bom and it run you can see Hello World in console now what is in also

different uh let's ask it this way make it generate a random number as part of hello

world and it rewrites we can see how it rewrites it also highlights the syntax so it's kind of like friendly for people who want to learn to code or want to

added the code let's run it again and we can see that their number is 48 now and it's a random number between one and 100

here random integer number and we can ask now chpt what was the number generated just now and it says that number generated

cannot be known because the code runs dynamically and generates a random number each time it's executor to see the number you will need to run the script let me know if you need to help

so it doesn't see the result uh okay can you throw an error they claimed it helps you to theb things

so let's ask it to throw an error and see what happens then okay let's run it and it's actually did oh if it's let's say that if it's

less than 50 so it didn't throw the error because like it throws the error if number is less than zero while it cannot be less than one let's try again

okay now it did got an error and the way the convers with python works is that when the error happens it does kind of react it says that hey the error was

this and you can fix it so I wonder what it's going to do so yeah in case of Errors it does send them to chat at least something

there's quite a difference in that sense from what uh code interpreter does so let's just compare let's start new one

create script that returns Hello World Plus random number and run it in code

interpreter let's try this so it creates hello world it runs it it sends it to server we are waiting

for it in that sense this is faster right it runs fast like if we run it we can see that we can run it and it's like super

fast okay so here we see that it returned hello world 83 and we see the chat sees that it returned hello world 83 why is this important because if

you're doing data analysis then uh you want chat to see results of the analysis and iterate on them with this you will need to probably copy things

like this or something so I don't know it feels kind of more manual and this is where I kind of get to one of the first

thoughts that I had who is it for because my first thought was that it's for uh data analysts who are playing with small Python scripts doing some kind of

analysis but if it cannot put back results of analysis into chat to work with uh chpt on and you

need to copy paste then this is worse workflow than using Code interpreter especially if we want to use some kind of data and one thing I did was actually

give it a file uh and see if it can actually maybe before that before that let's take a look at one thing though we will get to

uploading files and how they compare I wanted before to discuss what this is and how it actually is different in more way ways than what I shared so

far they also share that it runs in the browser let's uh ask it for a new convas so python for your information

usually cannot run in the browser make a new canvas also H world but this time

also collect all information about environment python is running in

and print it as well thing is that they said that it's fast because it runs in the browser there is no server so it's different it's different than code

interpreter in that regard and one of the first things I did I was curious okay it's run in the browser it runs in web assembly it runs fast this is what

they shared but what does it use and I did this I asked it like output everything everything you can get from it about environment it runs in let's

take a look and we can see that it runs in C Pyon M scripton which is a way how to compile things to run in a browser that were not initially made for the

browser so it's m script in version 31 uh there is no information about processor it's vum 32 as far as machine

goes current directory is home pite and and executable is somewhere here this is what it got out the most interesting

part here is this pite let's take a look what that is I kind of know but let's search so what is

pite pite is a python distribution for browser and no GS based on web assembly so it's a python compiled into a file

that browser can execute basically what they did was that they use this pite loaded into the CH GPT and then send

python stuff to it so it runs there but it has limitations and let me show you what kind of limitations so now I want to show you

about files so here is a folder with files it's uh some numbers about my YouTube channel and here let's give it

to trpd here and let's give it to let's start a new one add convas and add

it here here and say here is CSV with some numbers about my YouTube channel can

you take a look and uh draw a chart for it as well as

calculate median number and and draw at two plus output it so here's a

prompt and yeah use convas and python so here I'm asking this from uh convas and you can see I gave it some

kind of not exactly big CSV file and we can see what kind of code it writes it says like okay load data from mount data total

CSV print the head so print headers do some kind of stuff print the median and it also does something with plot let's run

it and we have an error if this happened with code interpreter it would try to fix it automatically here we need to click on it and see so the error is caused

because the file PAAD does not exist in the environment where the code is running double check the file location andure blah BL blah so it didn't work

because it doesn't have access to a file that I uploaded to chat so it cannot you be used to analyze files this way it doesn't have access to

them um yeah let's copy here the prompt and ask um chpt use code

interpreter and python yeah now let's ask code interpreter same file almost the same prompt and we can see that it's reading the file again from the similar

location it did read an output to chat how the content of file looks like so there is date views so the first thing it's interesting code interpreter is honestly

marvelous it taught and said Okay first I want to Output couple of the like uh examples from the file to understand the structure and then I will write the code

now it wrote the code and it like calculate in the median it returned the median which is 759 and it drew a chart showing the median and the chart of Like

My Views yeah YouTube channel views over time one prompt one iteration just works

this fails so it's not for data analysis um let's let's try one more I am Al an

article comparing it to Claud and before Claud could not do it with artifacts let's again take a look at the file put

it here and give it the same prompt approximately just a little bit adjusted for uh just yeah just remove how to do

it just let's see what uh clot will do so before clot could not use files in code now they do have a feature that is

very similar to code interpreter I'm just not sure I think it does run in in browser not sure so first what they did

is that they run the code and we can see that it's JavaScript and it use Windows FS to read the file it did analysis and now it's doing the other thing which is

artifact so this is inline analysis and this is interactive artifact so it wrote some code and I don't see anything median views is zero no views and yeah

this is my experience also from the article I'm also showing that an article I had the same experience in the morning it's consistent this is how well at this task load Works

consistently can you make it work yes you will need to iterate let's try I see only zero views can you show some

errors of why I do not see a chart so it does something else some kind of debug data but I'm not seeing it's misunderstood it seems it went and

tried to do things inside of its an analysis thingy Pap parse so it's parsing something it's again loading the

file and I have no idea what it's doing remember how fast chpt was at this task I will I am not sure if I will cut out

time or not I know some people like CLA I kind of still prefer chpt depending on the task but

yeah anyways I think I'm going to go further anyways this is what I wanted to show you more or less and there are more

things uh that I wanted to show you that are kind of interesting so I'm not sure when this is going to finish but in the article I think I will link to article

and just tell you the results a little bit there are screenshots to show so what is interesting about this mode the

this one it runs in a browser so I got curious if it's running in the browser like uh code interpreter cannot request files so I will show you it's

interesting uh so image with text so here is an image with text let's go let's take a

link let's go and create a new canvas environment and ask it can you load image

from and show it using plot Library use byon and canvas so we have a code we have a link

to the image we can run it and we can see that it can access the image now we can ask it something like

can you you use some library to extract text from this image so now we asking it to use code to extract text from the

image it runs it like write some code it will use P yeah P TCT it's pretty welln and well working Library let's run it

and we get an error that M scripton does not support processes and library of PCT uses processes so this Library cannot be

used oh oh now let's try to ask the same thing from clot here an upload from chpt with code

interpreter here is link to image can you load it into code interpreter and

run analysis to extract text so preferred uh it cannot load images

directly o can you write python code that uses Network request to get it

no no no no not canvas not canvas do it with code interpreter okay so now it's trying to download image it's right in the code

and it runs it let's see what's going to happen Okay so so this one errored out and what did it

say it seems that the code interpreter environment doesn't have internet access so yes code interpreter is isolated it cannot do Network request it cannot install anything and cannot download

anything including images text or files the only way to get files in is by dropping them into the chat we still can go and take the image copy it let's go

here and paste it into the chat and say o here it is can you extract text from it using Python and code

in so now we uploaded the image we can see that image now it thinks that it's

here it uses by TCT and it got the text make text stand out from backgrounds make text stand out from backgrounds there is some weird syntax in how it

extracted uh let's see what it says it may include some formatting artifacts let me know if you need further refinement or adjustment so this is how you can work with files with code interpreter

while this can't because it cannot load libraries while this loaded the same p

toct and worked so they all have limitations uh chpt has limitations uh with code interpreter that it doesn't have access to Internet but otherwise

it's probably the best tool for most jobs honestly maybe the only other problem is that when you leave the chat uh then the instance on the server dies

so when we return we cannot reuse it and we cannot share it with others they will need to kind of restart their work this is sharable restartable

reusable you can edit but it has a lot of limitations it's also faster but yeah it cannot really work with files yet maybe they will be improving it to

match code interpreter but this early release like does it have any advantages over the code interpreter in my view

no maybe only one there is syntax highlighting and you can edit but otherwise it feels okay it's faster and runs in your browser and it's resumable and

sharable but for actual work code interpreter is better let's also test what uh what clo can do in similar environment

so we can share again try with the link first let's create a new chat here is link to

image create uh artifact that shows the image and also

uses OCR to extract text from it so we're asking something similar and it will say I apologize but I'm not able to

open or access URLs just make HTML with image whose SRC is again I need to iterate I need to

convince it that it can or cannot do something and it still says like I apologize I cannot just do it I know it kind of can do

it okay oh what I need to respectfully Decline and then goes and does something what did it do okay I don't know allow

me to change URL with input let's see if we can get it to work at least in some way again we sometimes for certain types

of things you will need to yeah iterate and convince it and you need to know how to ask depends on the task so so we can give it an image but it

fails to load it because let's open developer tools and it says refuse to load the image because it violates following content security policy

directive so cloud is set up in a way where it cannot request external URLs unless they are WID listed and this is

not wh listed so you could say that it's it has some limited internet access but it's limited this is why I put here

Network requests con was limited interpreter cannot do it and CLA limited just differently so okay but can we give

it the image in chat po I gave you the image can you do can you add it to

artifact and do analysis on it to get the text okay let's see what going to happen if we upload I actually don't think I tried

it exactly in this way it's still a placeholder okay so it this is not what we try to test it used its own image

analysis to T to say the text is there but the image is not here uh okay create new

artifact allow user to paste an image and then use some library to

extract extract text from basted image so it again apologizes it likes to apologize so it doesn't have direct what does it do does it load the library for

me okay can you load some library from CDM again to do

analysis let's try to paste first okay paste it and it says oh it says itself like

yeah let's give it like there is Tess GS which is similar to P Tess that's what uh the code interpreter use but on the python side there is a JavaScript

alternative that should work in the browser what let's see I I remember that back in the day when artifacts just came

out and I was testing them and uh yeah it worked back then now it doesn't uh we will see maybe it does uh when I was doing the block post

and testing it didn't okay I need to drop the image okay let's save the image let's low damage and we get uncut network error fail to

execute import scripts trying to import Tesseract from jet and deliver this is what I got also in the morning

so uh what this tells me is that code like Char GPT with convas and python execution cannot use this kind of libraries because environment doesn't

allow it there are issues in clae and trying to also load this complex kind of libraries and use them while inside of con interpreter it does work as we can

see here and this basically is what what I wanted to show in that sense which brings me to a

question where I'm actually confused what is and I will share the link to a blog post if you want to see more of this kind of examples there is a little bit more there that I did but you

can see that I did similar things I just recreated them in in video for you but conclusions for me are that for real work the code interpreter that's by now

is more than years old one year old is better tool but it has its own limitation it doesn't have network access uh it's harder to reuse and

share but like for data analysis I do use it for data analysis from time to time Cloud artifacts for me they work worse for this kind of thing but if you want some kind of sh able interactive

result they are better but there are I think today there are even better tools like I like websam there are like bolt replate there are tools with which you can create sharable interactive

artifacts that work better than what CLA does now as we saw it couldn't load the library and do Network

requests and but in my view CH con with python execution at the moment is worse than both of them it cannot do

interactive stuff uh it's also it can do Network requests but it doesn't help it because it cannot actually use complex libraries environment in which python

here is limited it's fast it's local but it's limited so it feels to me that only

real use case is learning python with chpt on small scripts limited ones if you are a professional you're using some kind of libraries it will be very hard

to recreate envir working environment here doesn't have access to files doesn't have access to libraries so who is it for this is a question I asked at the start of the

article and after this exploration I still am wondering because there are better tools for this uh one of which is already in chpt

and its code interpreter this is what I wanted to share with you today maybe you have an answer leave it in comments and do you like this kind of content

subscribe and see you next time