AI agents for data engineering and data science | The Agent Factory Podcast

[Music] Hi everyone and welcome to the Asian Factory podcast. The podcast which goes

Factory podcast. The podcast which goes beyond the hype to build production ready AI agents. I'm Smitha Colan and I'm Lucia Sububarian and I'm excited to talk about data today.

Speaking of data, whether you're building dashboards or models, having access to good data is a huge thing.

Data engineers and data scientists struggle with this every day. Data

engineers help make data usable and trustworthy at scale. And data

scientists help turn important business questions into analysis and experiments.

So now imagine if they had AI Asians which could help them by automatically building pipelines, running checks, generating queries and even code for

things like visualization and keeping everything tied in their warehouse or notebooks.

So by the end of this episode, you'll be able to understand how agents and AI tools help you as a data engineer or a data scientist. You will connect agents

data scientist. You will connect agents to BigQuery and Spanner through ADK. You

will use graph and spanner or BigQuery to ground rags and reduce drift. And you

will learn to build data pipelines to have the right data for your agent apps using the BigQuery data engineering agent and the data science agent with

natural language.

Awesome. So before that, let's hop into the agent industry polls. We have some really exciting releases in the world of agents this week.

So Lucia, have you heard of the new computer use model? It's super cool.

Yes, I think this is huge news.

So Gemini API now includes computer use, a model that can literally see and act on your screen. So, think of it as giving an agent a pair of eyes and hands

on a computer. It works by taking screenshots and deciding the next UI action to take. So, things like click, scroll, type, or opening a web page. So,

you as the developer just write a small client that receives those actions and executes them locally. The model then looks at the updated screenshot, reasons

again, and then keeps going until the task is done. So essentially, this loop means agents can automate all sorts of realworld browser tasks such as, you

know, filling out forms, uh, scraping product data, testing user flows, anything that normally needs a human at the keyboard. Under the hood, it's built

the keyboard. Under the hood, it's built with a ton of safety layers. Every

action goes through a safety system that can either approve, block, or ask for human confirmation before something risky happens. So, think like, you know,

risky happens. So, think like, you know, if you're clicking buy now or even accepting cookies or accepting legal terms, you will see that confirmation step pop up. It's also really perfect

for research, testing, or prototyping agents that can interact visually with real apps. So if you want to try it

real apps. So if you want to try it yourself, we will be linking it in the description box below. So yeah, with computer use, we are entering the era of browser control agents. It's a big step

towards truly multimodal AI that can see, reason, and act on the world like we do. In this demo, we're going to be

we do. In this demo, we're going to be using the computer use model to look up its pricing on the documentation page.

Our test.py PI file contains code which uses a computer used model in order to go on to Gemini documentation page and then look up the pricing page. So this

is exactly how it does that. It takes a screenshot of the first web page it opens which is the documentation page and then it finds out how to navigate to the pricing page. And there you have it.

That's the pricing page for the computer use model. To try this out for yourself,

use model. To try this out for yourself, you can head on over to the documentation page, install Playrite and initialize the browser instance as well

as follow these four steps of code in order to build your first agent which is able to interact with your browser.

Wow, that was such a good demo. I feel

like this is next level for automation, accessibility, or even getting better seats at a concert.

Highly recommend using this to get better seats at a concert. Coming up

next we have codemen and codemener is an autonomous AI agent for code security and it operates in two modes. One is

reactive that means that it's instantly patching new vulnerabilities as they come up and the other one is proactive which means rewriting existing code to

secure entire classes of flaws. So why

this is important? As we know, AI is accelerating the discovery of software vulnerabilities, and we're very thankful for that. Uh because it's been helping

for that. Uh because it's been helping us with things like zero days found by tools like big sleep or oss fuss. Uh and

the truth is that human developers, we haven't been able to keep pace with the resulting patch volume. What code vendor helps us with is automating the creation

and validation of high quality security patches at scale.

So it operates using the reasoning power of Gemini deep think and it's equipped with very sophisticated tools for selfcorrection and validation. So this

includes advanced program analysis like static analysis and fuzzing and it pinpoints the root causes and we also have a multi- aent system to ensure that

we still have that functional equivalence. The patches are currently

equivalence. The patches are currently human reviewed, but its track record is very impressive and it has already

streamed upstreamed 72 security fixes to open-source projects. So I feel that

open-source projects. So I feel that this is also a nice breakthrough.

This is a huge gamecher for you know catching software vulnerabilities really early on. I think this is amazing for

early on. I think this is amazing for software developers everywhere. And

that's it for this week's agent industry polls. Now, let's head into the factory

polls. Now, let's head into the factory floor where we'll have Lucia here who is going to show us the data engineering agent, the data science agent, followed

by a really cool demo of creating comics of Spannard concepts using ADK.

All right. Well, welcome to the factory floor and my demo. I will be showing you how different data agents can help you

as a data engineer and a data analyst.

Today I will be showing two agents if that's okay with you.

Yeah, I'm very excited for both of those agents.

Perfect. The first one is going to be the BigQuery data engineering agent.

Have you heard about this?

Not really. This is pretty I I believe this was released pretty recently.

It is. It's actually still in preview.

So that's that's a good call out to make. Uh and we also have the data

make. Uh and we also have the data science agent.

Nice. And so both of these are something that you can deploy on Google Cloud. And

I noticed that for the data science agent, you're using it in Collab. Uh are

you also doing the same thing for the data engineering agent?

Uh actually that's a really really good call out. So yes I am using the data

call out. So yes I am using the data science agent in Vertex in Collab Enterprise and the data engineering

agent is technically a part of BigQuery.

So I I just named them here because I will be going between the two tabs and I hope that this makes it clear for folks watching this video. Uh but both agents

they take natural language through Gemini and they do what we asked them to do. So I will be showing an example of

do. So I will be showing an example of that.

Exciting. Let's get into it.

Absolutely. So as a data engineer, as a data analyst, what sometimes happens is you are faced with massive data sets like in this case I have in BigQuery.

I'm using the Bitquery public data data set, but you get the point. This it's

not unusual to have a lot of complex different data sets coming from different systems. So it's really nice to be able to have some assistance to

get insights out of your data before you start working with it. Of course, the data science agent can do a lot of very powerful data scientist things. Uh but

the reason I brought it to this demo is I want to show how it can help me figure out what the right approach is to make my data good for my agentic

applications. So, as you can see here, I

applications. So, as you can see here, I put this little prompt. It's a very vague prompt, and it's asking the agent

to detect anomalies in the case table.

Um, I should have asked for it to use a very specific case table. So, I'm going to do that now. As

table. So, I'm going to do that now. As

you can see, I have my tables from BigQuery here, and I'm going to select that for it. Oh, it's really cool that you can select tables that you already

have in Bitquery as context to give to Gemini. That's awesome.

Gemini. That's awesome.

Exactly. Exactly. Yes. And and it it's you'll see that it can run um the schema detection and it will figure out things about the data that even sometimes I do

not know, especially when you're new to it. So, let's see what it does. Right

it. So, let's see what it does. Right

now we can see that it has a plan to load and describe data to pre-process the data for anomaly detection and this is a very good use of data science uh to

train isolation forest model to um continue with anomaly detections. It

will provide visualizations which is super good and display it. So let's see what it does with this plan. I'm going

to accept and run it. While this is running, so that we do not waste time, I'm going to go into the data engineering agent. So, you probably saw

engineering agent. So, you probably saw that I came in here and I said new pipeline. And now I have our favorite

pipeline. And now I have our favorite Gemini um option here.

And I'm going to use AI to generate AI because Bitquery has the ability to call

Gemini through the AI functions directly from SQL and I think this is mind-blowing. So I keep talking about

mind-blowing. So I keep talking about this because I really really love this feature. Uh in this case I have

feature. Uh in this case I have I I love this. Uh could you walk us through what your prompt to Gemini was for this agent?

Yes, absolutely. Uh so let me show you first the data set that I have. I have

this data set called catitude and I have accounts and cases and users as well for cats that have complaints.

So, uh, for example, you see the cats are complaining about the the human ignoring their meows and it meowed for 10 minutes straight and my human did not

respond.

Very valid complaints by the way.

It is a very valid complaint. So, all

this of course were Gemini uh generated.

It is actually based on a real Salesforce data set, but this is uh clearly Gemini generated. I also have the accounts table today and note how I

have a billing country here. So the

billing country will show I can just do this because we don't have as many will show Argentina, Canada, Italy and

Japan. So, as you asked, I used a little

Japan. So, as you asked, I used a little prompt here that says, "Using the account table that we just looked at, add a field to the account table that shows the sales region based on the

billing country." And the way I want

billing country." And the way I want this to be is using this AI.

function which calls Gemini 2.5 flash in my case using a connection into Google's clouds internal resources and it's going

to determine the region based on um the country and it's going to pick between North America, Latin America, EMIA and

APAC. So note how I'm giving this agent

APAC. So note how I'm giving this agent a prompt that is basically telling it use this AI.generate generate function

which is absolutely mind-blowing.

This is super cool.

And where is this pipeline going to be stored? So, is this system instructions

stored? So, is this system instructions and yeah, how is this going to be stored? I'm sure a lot of users will

stored? I'm sure a lot of users will wonder that.

And that's that's actually one of the frequently asked questions. Uh you can see that here we have this kind of declarative language. This is data form

declarative language. This is data form in the back end and data form is a declarative language for data engineering

pipelines in the sense of software delivery life cycle pipelines. So if I open this in the create instructions file which act like a system

instructions they are an addition to the prompt and you would use this file here to add your system instructions. But if

I go into the definitions, you can see that we already have the declaration of the catitude accounts table that I asked for and we also have the declaration of

the result of this prompt that we were looking at just now in the pipeline. So

this is nice because a lot of people are already using data form. Some people are using also dbt which is a non Google

cloud product but uh it's of course usable with bigquery and this is great because it helps you

keep a pipeline of your code as related to your data and transport it across your different systems. That's awesome. So it's actually really

That's awesome. So it's actually really accessible for users to access this pipeline even though you know Gemini has helped build it. So that's really cool.

It is. And talking about uh how it is accessible we can also modify this because at the end of the day these are text files in git repository.

So while we were talking um as we saw we have the agent came back it said I have fulfilled your requests here's a summary of the changes it made we saw the the changes that it made and now I asked it

to generate a time dimension and join it with our cases uh table and this is to show that the agent can do a lot of different things. It can do modeling. It

different things. It can do modeling. It

can create data. It can load data from something like a GCS bucket or or create big lake tables. It can do it's it's a really really powerful uh thing that

that what exactly is a time dimension? Is

that something which is useful for natural language to SQL?

It is actually that's a great point. So,

a time dimension, let me show you what what it does here. Um, you can probably get the gist

here. Um, you can probably get the gist of it just by looking at the select statement, but you can see that it's getting the dates from the 2000s to

2050. And it's outputting the actual

2050. And it's outputting the actual date, the year of the quarter, the name of the day, the name of the month, and a bunch of other things. And this is very useful especially when you're doing

natural language to SQL as you say because most users and most business users will ask about how much have I

sold in June. So for it's very useful for those agents to not have to compute that on the fly and to have June as an input as a wear clause readily available and that's exactly what this time

dimension does.

Oh awesome.

So we have the declaration here and I am going to give it another very useful task to keep our data clean for our

agents and that is related to data quality assertions. I'm going to leave

quality assertions. I'm going to leave the data quality assertions here and while this happens I'm going to move into our other agent because it is

waiting for us. I will accept and run this while we talk. So we have the preparation, we have the load and describe data. This is uh showing the

describe data. This is uh showing the collab that has is importing a bunch of libraries in Python. It is declaring the

client and it's doing a beautiful select star from catitude.case.

I went to cloud here because uh sometimes when we see pandas we immediately think we're going to run out of memory for very especially for this

kind of select star. Um we can modify this first if we wanted to. We can

absolutely modify this and make it safer in this case. You saw the cases table right?

Yes.

It's pretty small.

So uh we could modify this code and we could also change this to be uh a big query dataf frames import as opposed to a pandas dataf frame and that would

basically harness the power of big query to still use pandas which is a really really nice feature.

Would that be more efficient especially if you're dealing with larger data sets?

Yes, it will be um more efficient. it

will be faster and this has the risk of running out of memory if this data set is very big and this select star goes unchecked as well. So yes, it's it's a

great question as usual.

Awesome. And I'm just looking at the Gemini chat which is going on in the data science agent on the right side and based on the prompt that you gave it, it's it generated all the steps which

are required and it's running each step one by one. And I think this is so awesome especially things like pre-processing data uh for an anomaly

detection loading and describing the data. These are all things which a lot

data. These are all things which a lot of data scientists this is just like boilerplate code which they always have to write and it's so great that Gemini automates that for them.

Yes. Yes. So I think this is a game changer and even if you're not a data scientist and you're dabbling in data science, this can be very helpful because I mean so far it's been saving a

lot of time.

So I will keep accepting. Uh now it's going to train an isolation forest model. This is where I would rather a

model. This is where I would rather a data scientist look at this because I am not one. But I am trusting that this is

not one. But I am trusting that this is doing what it's supposed to be doing and I'll see what it does in the meantime while this is running

because this may take a little bit. I

will go back into BigQuery. Let's see.

It has generated data quality assertions like I requested for all tables and it's explaining what it has done which is very useful. Some of these make

a lot of sense, right? Like it's making sure that we do not have null ids, right? And I could have been a little

right? And I could have been a little bit more specific. I could have told it exactly what I wanted to do. And

normally I would use the input from the data science agent which is detecting anomalies. And those anomalies may be

anomalies. And those anomalies may be related to wrong data migrations or some pipeline being broken. It may not always be a business anomaly. So that is the

kind of thing that I would incorporate from the output of that data science agent into this kind of a pipeline. So

let's take a look at one of the assertions.

We can see that ID should not be null and it's basically selecting for null ids and account names being null or

accounts being unique.

So this will make sure that when I build a little agent or or something AI related that taps on my data at the very least we do not have null ids which are

going to be important for a lot of things like identifying unique records.

Yeah, this is so cool that like it generated all of the assertions that you needed and also if you feel like you need more, can you prompt it further or

make changes to uh the assertion itself that it generated?

Yes, I would probably prompt it further and ask for whatever this data science agent is detecting as anomalies. Um,

sometimes it's useful to keep running different pipelines at the same time. So

you would add like a second layer of of data assertions further down the pipeline. Sometimes these kind of thing

pipeline. Sometimes these kind of thing like the null detection, you want to run it even in your development environments when you're making copies for developers to do their own select

statements in a lower environment. It's

useful to have this kind of a of a selection and they may come up with their own. So yes, you can always change

their own. So yes, you can always change this. As I said, this is a git repo with

this. As I said, this is a git repo with declarative text. So the the world and

declarative text. So the the world and the repo are your oyster.

Awesome.

And back at our data science agent, it's going into displaying the anomalous data. I have progressed through the

data. I have progressed through the different tasks here. So it has identified some anomalies here. We can

see that this is pretty scattered. So

I'm hoping that it will come up with what anomaly means in in the data and it's listing the data first and then

it will hopefully create another chart that tells us this is what uh anomalous means for your data.

Uh so it analyzes the data and then it identifies the anomalous like uh points data points as well as giving you the reason why there are anomalies or is

that something you have to deduce yourself?

Well, it will give us a little summary at the end when it does here just right on time. I love it. Uh so we have the

on time. I love it. Uh so we have the questions word records in the case table were identified as anomalies and it gives us the the answer and the key

finders. So it train an isolation forest

finders. So it train an isolation forest model.

It identified that 70% of the data set is anomalous. So we can see or we can

is anomalous. So we can see or we can kind of figure that out from this.

Um but um it's it's good that we have that as an actual number because what I would do in this in this scenario is go back to the business and try to

understand why we have this type of anomaly and maybe the business comes back and says this is expected for these or those reasons. In this case this is a

cat's data set and we know that cats can be very whimsical.

So I expected obalish with the data set.

Absolutely.

And we have like visual confirmation of the separation between the three normal points and the seven anomalous points that it found. And this is pretty neat because it's actually tying the visuals

to the analysis that it made. And it's

actually given us insights and next steps on what kind of a feature combination of these seven anomalous records uh do we have and understand the root calls which is basically what I was

saying I would do if I had a business user on the other side that didn't reply just meow I I was just about to say the insights and next steps are actually amazing

especially if you're an enterprise and you're trying to like improve your data set or maybe data collection process.

This kind of gives you steps on like what you can do to improve not just like enterprises but if you're data scientists, data engineers, just dealing with data in general. I think this is awesome.

Thank you. Yes, I think it's awesome too and I hope that folks use it. Now with

all of this data preparation in place, I think we are ready to move on to an application of using databases and clean

data um from an ADK application. What do

you think?

Awesome. Yay.

All right, let me switch to my other demo.

We are full of demos in this episode.

I love it.

So now let me share my VS code here. I

have a little application that I use the ADK to build. It has a bunch of agents

that are going to tap into my Spanner graph database and then it will generate comics using nano banana.

Awesome.

We love Nano Banana. We love that banana.

Uh, but for folks out there who are listening to this, what is Spanner?

Because I've personally not been too familiar with Spanner.

Well, some people call it a wrench. It's

a tool. Sorry, I'm just joking. Uh,

Spanner is uh one of the most basing databases on Earth. Uh so it's um a fully distributed globally distributed

globally available um database that is also immediately consistent. So and that is thanks to some magical atomic clocks

that it has.

But what Spanner helps us do is have our transactions being consistent at scale at a planetary scale. And I'm using

Spanner in this case um because it has graph capabilities. Bitquery will have

graph capabilities. Bitquery will have graph capabilities generally available soon. Uh this was announced at Nex as

soon. Uh this was announced at Nex as well. Um so I could have used BigQuery

well. Um so I could have used BigQuery as well. I am using Bitquery for

as well. I am using Bitquery for something different in this case. But

here I did want to show Spanner because as you say not a lot of people know about Spanner and it's an amazing tool for things like uh finance, banking, um

gaming. So platforms in which you need

gaming. So platforms in which you need um a high uh availability and and

consistency globally at uh super speed.

Awesome.

So showing my my agent in in VS Code. I

have my my code here. We have we're going to start with a question based on the documentation.

And what this um library is going to do is it's going to go into a graph database that I have that I will show

you in a second. in this graph database will be traversed to give back the answer to my question and with that

answer we are going to generate a an engaging comic with nano banana. Oh, so

so you're creating a knowledge uh traversal agent first which is going to have access to the uh graph database that you've created and is the graph

database located locally or is that being connected elsewhere?

So let me flip into Spanner. Um, what I have here are a bunch of tables that I generated using a very specific library

that will pick up all of the documentation from Spanner and turn it into something that can be traversed in a graph. I would say don't

try this at home because the documentation is probably Google's intellectual property. Uh, but I felt

intellectual property. Uh, but I felt entitled to do this with because I'm an employee. Um, but I will probably put out a little um, example of

how to do this with a set of documents that are generally available for the public. But we can see here that I have

public. But we can see here that I have GQL and it's producing in this case a little map of the different documents

that I have in this huge database with the different terms like code samples and and Google Cloud console and training.

So if I go back into VS Code, I can hopefully do ADK web here

and we will see my agents in action.

Awesome. I personally love uh ADK's web UI. It's such a great interactive uh way

UI. It's such a great interactive uh way to see first of all interact with your agents but then also see what's happening behind the scenes.

Yes, absolutely. So as as you're saying I am um using this locally of course we can deploy this in Cloud Run which is

another great uh platform that we have.

And I'm going to ask my agent what are regions just because we were talking about the availability of it. And the interesting

part or when I want to show here and why I was mentioning that MCP in our case cannot fully be used is we see spanner QA chain which is the the library that

we contributed into lang chain. Well, I

mean the the engineering team did. I I

was not a part of this. And it will produce a graph um query like the one we just saw before

to traverse this knowledge graph that I created based on the Spanner documentation.

And you can see that here it basically says um graph docs KB match

concept where ID has region and return the description for that concept.

So I go back into the ADK and we can see that it came up with a result. It says

in spanner regions define wait uh the geographical locations where your data is stored and replicated to ensure high availability and performance. You choose

between regional, dual region or multi-reion instance configurations when you create a spanner instance and it gives a bunch of other stuff. Then we

can see we I have another agent that is generating the prompt for the nano banana agent. Um,

banana agent. Um, and it says, "Create a six panel comic strip in a vibrant tech illustration style. The story explain explains Google

style. The story explain explains Google Cloud Spanner regions and it lays out the characters, a stretch developer named Ada and the friendly helpful robot with a spanner logo. I'm very curious to

see what what it does with the spanner logo. It's not always great." And it

logo. It's not always great." And it describes what the different panels have. So it says panel one addis at her

have. So it says panel one addis at her desk looking worly at the world map.

Then it goes more description with panel two.

So I I love that it's essentially it's generating a prompt for a comic strip to give to Nano Banana based on the question of spanner we asked which is like what are the regional

availabilities?

Exactly. Yes. Um and and you can see that it has the story here and first it shows a regional configuration for local fault tolerance and then it demonstrates

some multi-reion setup where data is replicated across the globe. So I think I couldn't have done a better job of telling the story. It's very creative.

And now it's going to go into a couple of loops of checking the image generation. One thing that I've been

generation. One thing that I've been struggling with is getting the image to have a clear text.

U but if we can see the path here, it starts with four five AE. I'm going to try and fetch that image that it

generated from here and see if I can find it locally because what and here it is. What it will do is

check I have a little loop of um sub aents checking for first the quality of this generation and second that we do not have I think I gave it

more than three typos. I I put a little bit of a level of tolerance because we're still not great at generating flat text and this much text in in image

generation. But you can see this is

generation. But you can see this is actually pretty neat. It says my users are global. If one region goes down, we

are global. If one region goes down, we are in trouble. Uh latency for us users in Asia is so high. And we can see this

developer who's worried. Then we have this little robot here and uh it goes into multi-reion and it has original instances highly available within one

area and then we have but a multi-reion instance replicates automatically routed accessing the nearest healthy one and we can see the garbble the text a little

bit here.

This is awesome. So it's like it's essentially first of all creating such awesome comics and of course you mentioned that there's uh three iterations in case there's any errors

with uh the text or you know the transcripts which are generated. What

happens if you want to do more iterations? Can developers easily first

iterations? Can developers easily first of all get access to this project on GitHub and then try to replicate it?

Yes. So um I haven't published this yet.

I will make it available. We do have a comparison of using graph and not using graph for the same set of documents and I will link into that. It's in the

notebook but I will make this available.

Um that's that's my promise to you. I

will not say exactly when but someday before the end of the year for sure. Uh

and now going to your first question on the iterations. I do have this loop

the iterations. I do have this loop agent going with two sub agents. Ones

that is generating the scoring and the other one who's doing the checker of the images and I capped it at three iterations for this demo. I mean

developers can cap it at whatever they want and they may be able to change the descriptions and the prompting that we're using for the different agents with the instructions. Like here I have

you're an image quality checker. you

will give in a be given a JSON object with a a file path and a score and everything else. So

everything else. So I mean it's up to how much time you have how much uh you want to spend on the models going the the generation of the

images but it's totally doable to get like a very neat image with with the iterations here. I think this is pretty

iterations here. I think this is pretty cool though. I hope that people who are

cool though. I hope that people who are just listening to the podcast uh take a sneak peek around this time in our YouTube channel because I'm very proud of what it did.

Yeah. And I have to say Nana Banana is pretty great at generating text for a lot of compar comparatively. Uh I think it's one of the best out there in terms of generating text.

Absolutely. Yes. The there's there's a little bit of a garble text but but it's still really good and I think this graphics are engaging. I I love the little robot.

So, we have seen in this demo how I've been using the ADK to access Spanner directly. I did a lot of work to make

directly. I did a lot of work to make that uh Spanner um instance and the database and the data it be clean and

accessible so that I could actually get this uh traversed by this graph library.

And we also saw how the data engineering agent and the data science agent can help you with making your data clean and available at the right time for your

agentic applications. And I hope this

agentic applications. And I hope this was an inspiring demo to everybody.

Thank you Lucia. These three demos first of all you killed it. It was awesome to watch and I think a lot of viewers can definitely relate to uh dealing with

data science or data engineering pipelines and an agent such as the ones that you demoed would be so useful in their day-to-day task to automate uh

simple things and also really help to augment uh their data pipelines as well.

And super fun to end off with a comic generated by Gemini uh by Nano Badana.

Yes. Now we're moving on to my favorite section of the podcast which is the developer Q&A. Uh we have a bunch of

developer Q&A. Uh we have a bunch of questions from a few developers here for you Lucia. So the first question is are

you Lucia. So the first question is are both of those agents that you demo the data science agent and the data engineering agent are they available generally?

That's a very frequent question. The

answer is the data science agent as of filming of this podcast is in preview in public preview. So you should be able to

public preview. So you should be able to access it and the data engineering agent is also in preview but you need to follow a link that I will leave in the comments or in the description to get

access to it.

Awesome. So there's definitely ways that users can interact with this and also like try it out themselves at home. Uh

the next question we have is how is the data engineering Asian scalable if you want to perform analysis on multiple tables so not just one or two and also

if you want to use it across multiple data sets and then what would be the deployment strategy to a higher environment that's a great question so the data

engineering agent is based on two very highly scalable platforms one is bigquery and the other one is data form.

So in terms of tapping into other tables in other data sets or projects as long as what is executing the the pipeline

has access to it in terms of permissions you will be okay. You just need to declare it the way that I declared the other tables that I showed today. Now in

terms of the deployment strategy to a higher environment, let's say you're in dev and you want to move to staging or prod or or QA um that is what data form

excels at actually. So these data form artifacts that we are generating you will basically create a release and configure a workflow to deploy take all

of these this assets and move them into whatever combination of project and data sets ID you desire.

Thank you so much Lucia for answering those questions and thank you to everyone for tuning in. We hope this inspired you to try out the data engineering or the data science agents

for yourself. Shout out to my amazing

for yourself. Shout out to my amazing co-host Lucia for joining us. You can

find ways to connect with her in the description box below as well as all the resources to the agents we mentioned in this episode as well. They will be in

the description box as well. With that

said, powering down.

[Music]