Reviewing And Planning Your School's AI Approach: A Strategic Framework [Webinar Replay]

Good afternoon. So, welcome to this session on reviewing and planning your school's AI approach. We're delighted to be joined by Educate Ventures and their

group of educationalists with decades of experience in AI and education set up by Rose Luck. Some of you may know that

Rose Luck. Some of you may know that name. Um they take a very evidence-led

name. Um they take a very evidence-led approach to working and supporting governments, school groups, universities worldwide. and they've worked with the

worldwide. and they've worked with the department of education in the UK, Ministry of Education in Singapore and international school networks. Um, today

they're going to share key insights and learnings from this noble this global network and then I'm here from Third Space Learning and we've been developing an AI tutor that Educate Ventures has

actually evaluated and reviewed. So,

I'll share some practical advice and evidence of how schools have can implement a purpose-driven AI strategy.

Okay. So, as I mentioned, um I'm here from Third Space Learning. So,

I just want to quickly introduce today's speakers. Um I'm Candida Corford and I'm

speakers. Um I'm Candida Corford and I'm chief learning officer at Third Space Learning. I was a teacher on the ground

Learning. I was a teacher on the ground for 10 years and then I've been working at thirdpace learning for 10 years delivering impactful tutoring to thousands of schools already. And then

from educate ventures we've got media um and Ibraim. So Dr. Media Khan is

and Ibraim. So Dr. Media Khan is director of research. She has a PhD in self-regulated learning and AI and brings extensive um expertise in helping

um schools and governments adopt AI safely and effectively. And then we've got Ibraham Bashier and he's the technical projects manager. He's a data scientist and he focuses on designing

and evaluating with practical frameworks.

All right. So

let me give you a quick overview of today's session. So we'll start off by

today's session. So we'll start off by grounding ourselves in the basics. what

AI is and where we are right now with schools and then we'll move on to how schools can begin to develop a strategic approach with AI and um educate ventures

will go through a 4D um framework so looking at government and ethics technology data staff capability and evaluations of AI use cases

and then we'll share practical insights using the development of the AI math tutor as a case study and reflect on what this means for schools exploring AI

today. Uh um okay. So, we'll wrap up

today. Uh um okay. So, we'll wrap up with an AI toolkit where we can um uh share this information. And if you've got any questions, please just uh type in in the chat and we'll uh we've got we

monitoring the chat. So, you um can ask questions and we'll try to answer it.

And then um at the end, we'll send the AI toolkit and the webinar recordering and any answering any of those questions that come unanswered um via email and

I'll send that to you at the end. Um so

let me hand over to um Ibrahim off you go.

Thank you so much Candida. Um so yeah um hi everyone great to be here. Um before

we begin um usually we'd like to go over uh a definition of what AI is. We try

never to do a talk about AI without saying what that is. Um just so we're all on the same page. Um so this is the definition of AI that we like to use.

It's it's technology that analyzes its environment and acts with some autonomy to achieve goals. It's a somewhat broad definition, but the important point to note is that it covers more than just

generative AI uh which is tools like chat GBT uh but also things like machine learning which has been around for a very very long time. Um and in some ways has always been around us. So an example

we'd like to use is is that of texttospech systems. Um, so you talk or speak into your phone or computer, it analyzes your voice, converts into text.

That's an example of a typical machine learning or AI system which isn't generative AI. So just to be clear, you

generative AI. So just to be clear, you know, like AI is more than just your chat GPT. Um, but there is then, you

chat GPT. Um, but there is then, you know, obviously generative AI and that's what all the buzz is about. Um, it's a subset of machine learning. Uh, and the technology on which it was based was

actually developed in 2017. Um, but it wasn't until 2022 with the launch of ChatGpt that it really took off. Uh, and

now you see it basically everywhere. Um,

and with the investments that companies and governments are making into AI, it's safe to say that's here to stay in some shape or form.

Um, so yeah, you know, with that, it brings us a lot of opportunities. Uh,

and particularly generative AI, it brings us a lot of opportunities um, not just for our students but for our educators as well. Um so some of those examples we've listed out here we

have adapt adaptive personalized learning automated marking and feedback AI powered dashboards uh which are particularly powerful for helping us to understand just how students are learning and not just what they're

producing. Um we can get content

producing. Um we can get content generated that we've seen from tools like chatpt and co-pilot. Um and in fact one of these use cases high impact personalized tutoring and the benefits

it might have on learning outcomes is something that we'll discuss more in detail later uh when we look at our evaluation of thirdspace learning's AIT tutor uh Skype.

Um at the same time with these opportunities for educators it presents us with a few challenges not least the messaging that our students receive about AI uh that it's going to make their life easier. it it to some extent

it is but you know like in some ways it's not uh it's because that they have these sophisticated tools they need to be actually more intelligent when using them. Uh we've seen a lot of attention

them. Uh we've seen a lot of attention paid to academic integrity and cheating but there's an awful lot of concern about safeguarding and protecting children's rights uh protecting their

privacy from profiling from fakes from misplaced trust over reliance becoming deskkilled protecting their agency. So

what we really need to understand how to do that you know like protect them from um from all of these safeguarding concerns and amongst educators what we generally find is that people are increasingly aware of the technologies

and are aware that there are risks but unclear how to actually put their understanding into practice um to help protect students but also to enable students and colleagues to use these

tools at the same time.

Um, and so we think the best way to go about this, our motto is learn fast, act more slowly, is that I think it's really important to understand what you want the AI to do for you rather than

thinking about what AI tool am I going to use. Um, it's what do you want it to

to use. Um, it's what do you want it to achieve? Uh, and a great way of doing

achieve? Uh, and a great way of doing that is by taking a strategic approach.

Um, so one of the most common questions we hear is how do we get started? How do

we do this in a way that can demonstrate that we are supporting student learning?

Um, this strategy, the 40 AI strategy framework is based on professor Rose Luckin and her colleague work that her colleagues have done over the last 30 years um that they've been working in AI

and education. Uh in the center we have

and education. Uh in the center we have your existing vision and that vision is fundamental to what you do. uh now just because you have AI it doesn't necessarily mean you need to change that

vision but you need to look at that vision and see how AI might impact uh it mean for example that you can achieve it more quickly or for more people basically what is it that you're trying

to achieve as an organization uh and then on this framework there are four dimensions governance and ethics iterative unit applications technology and data and staff capability uh and we're going to talk about each of them

in turn and they're all highly interconnected Uh and some of the most powerful uses of AI come when these connections are built between these dimensions.

Um so we're going to start with the first one uh which is governance and ethics. Uh and despite our motto of

ethics. Uh and despite our motto of learn fast act more slowly. This is

actually something the one area that we believe people do need to act a bit more quickly. Um yeah, the space of

quickly. Um yeah, the space of governance and ethics uh especially in getting a policy in place uh so that people in your organization, students, staff, colleagues know what it is okay

to do with AI and what they shouldn't be doing with AI. Um so they know what tools they should be using. They know

what's expected and such policies need to be updated regularly. Um one of the best ways to create them is collaboratively with a team of people because more people who know that the policy exists uh the better. Uh, and

we've actually created a free guide that you can get. Um, there's a QR code, but you'll also get it at the end of the session. Uh, we'll send out an email.

session. Uh, we'll send out an email.

It's not a policy. It's a guide to creating a policy because it's really important that you know, you just don't take a policy off the shelf that you create one that works for you. Um, also

part a part of governance and ethics is about evaluating the risks of the AI tools that you use. And this means that once you've decided what kind of AI tools you're you're using, you understand the risks that are associated

with it. Um, one way to do that is

with it. Um, one way to do that is actually by conducting a DPIA or a data protection impact assessment. Uh, which

you actually have to conduct whenever you're using a tool where the data processing is likely to result in a high risk to the rights and freedoms of individuals. Um there is a checklist

individuals. Um there is a checklist available on the ICO website uh the information commissioner office but at the end of the session you'll also receive a guided checklist that we

produced uh that's more specific to AI and education use cases and that can help you conduct uh that impact assessment of you know any AI tool that you might be using.

Um also as an organization we do risk assessments of some of the more popularly popularly used tools out there. So for example, uh this one we

there. So for example, uh this one we built for Sky, you know, as part of the evaluation we were doing. We also did a risk assessment. Um it's something you

risk assessment. Um it's something you can, you know, get from our website. We

post all of these up there just to give um teachers and educators an idea what the risks involved with these tools are and the mitigation strategies they can use. Uh we're not the only organization

use. Uh we're not the only organization do it uh to be doing this. There's

actually some other ones that are doing some really good work on these risk assessments. Common Sense Media, for

assessments. Common Sense Media, for example, they also have some really good. So the idea being that whenever

good. So the idea being that whenever you're using an AI tool, you look at all these things. You look at you do you

these things. You look at you do you have an AI policy for this? Have you

conducted or do you need to conduct a data protection assessment? And whether

there already risk assessments out there for that tool. Um and then finally uh this will also be available to you after the session. Uh we work with an

the session. Uh we work with an organization Avalain to produce this report. It's called from the ground up

report. It's called from the ground up which is a framework to help you go about implementing AI in an ethical thoughtful way. So it's all part of that

thoughtful way. So it's all part of that governance and ethics and you know in this document you basically find out laid out in very practical terms 12 ethical principles what they mean for you and how you can go about thinking

about them the questions you can ask yourself to try and make sure that you're doing AI it's it's ethical.

Um finally we're going to move on sorry next we're going to move on to technology and data. Um so machine learning is basically a perfect storm what we have today is a perfect storm of

three different ingredients data algorithms and processing power. Um and

you need to have the right combination of those three ingredients if you want machine learning to work. Uh and the reason where we are where we are in 2025 with AI is because of a perfect storm of

these three ingredients coming together.

Um and so you know like part of that tech that data part is also you know like one of those ingredients that needs to go in there. Um so you know I'd encourage you all to think about these

four dimensions on this slide but in particular I think the one that's most relevant to educators the rest are mostly related to you know like might be more related to your IT teams is the

data plan which is do you have uh do does your organization have a strategic plan for collecting and using data that's you know aligned to your vision.

So again you know we we we talked about uh evidence-based you know like checking whether uh the interventions that you're making or what you're doing is it actually uh leading is it actually

helping you uh achieve your vision and that's why you need to have a strategic plan about the data that you're collecting to evidence those or signal those learning outcomes. Um, next I'm going to go about staff capability

because again if we want AI to achieve the purposes that we've set out to achieve, excuse me, we need staff to understand who understand how to use the technologies. Uh, to make sure we're

technologies. Uh, to make sure we're supporting our teachers to be equipped with the knowledge of what AI can and can't do and where its limitations might be. Uh, so we need to support teachers

be. Uh, so we need to support teachers through the building of AI literacy to develop skills so that AI can be used effectively and with positive impact in teaching and learning. Um very few

countries right now have clearly defined expectations around AI literacy at the moment. So I'm actually going to use uh

moment. So I'm actually going to use uh for this session on the UNESCO AI competency framework which is available on this QR code. Um this was published in 2024. Uh it's just worth taking a

in 2024. Uh it's just worth taking a moment to look at the six six key principles that they outlined. Um so

essentially as we're approaching the idea around expectations of AI literacy and AI competency, this is all built around these six different principles.

uh ensuring that you're bringing an inclusive digital future to the forefront that we are adapting a human- centered approach to AI that teaches rights and roles are protected, you know, even though they may be redefined,

that we're promoting trustworthy and environmentally sustainable AI within education. um that actually you know in

education. um that actually you know in in which the AI can be applied and is relevant and reflected across all different types of teachers across different levels of teaching experience and that we're ensuring that the use of

AI and AI literacy accompanies that accompanies it is part of that lifelong professional learning that we have set up for teachers within our profession.

So again um as part of again the 4D strategic framework it's having the right policy having the right data and then having your teachers uh having the

right skill set to be able to do that.

Um for the next piece uh oh sorry yeah one more thing um again something that teachers need to be aware of is are these safe AI priorities um so they have that safeguarding piece for their

students. So again they need to be aware

students. So again they need to be aware of again this is also adopted for the UNESCO framework. They need to be aware

UNESCO framework. They need to be aware from you know protecting children from displaced trust from inappropriate tools from bias lack of transparency from fake news. A lot of these things that we

news. A lot of these things that we talked about uh you know so they have their agency they don't become deskilled an example of this you know like how we will discuss in more detail later an

example of how uh we looked at skaya when we evaluated it how that maps to these safe AI priorities so a tool an AI tool that matches these would be for

example uh tell students that it is an AI it's not a human tutor uh it would only connect the collect the necessary data and only for as long as possible.

Um any kind of content that would be on it would be human authored. We would

check for bias. It would only it would be very content gated. So it wouldn't be able to get inappropriate content from elsewhere. Uh the students would have

elsewhere. Uh the students would have some control over pacing and you know they would be able to ask AI for uh explanations or simplifications. So all

of this you know like are things to be aware of when you're evaluating an AI tool. Uh finally for the last part uh

tool. Uh finally for the last part uh the iterative unit applications and you know purpose-driven AI basically how you can apply you know how you can apply

these AI tools to teaching practice. Uh

for that I'll pass on to Dr. Media over to you.

Thanks very much Ibrahim. Hopefully

everybody can see my screen now. Um

thank you. Thank you for that. Um and as Ibrahim was saying um so we've been through three dimensions already for the 4D strategy framework. We looked at governance and ethics, technology and

data and staff capability. The fourth

dimension which is um particularly important um particularly in collaboration with the other three is iterative unit applications. And what

that refers to is evaluating the use of AI on an ongoing iterative basis. The

reason it's important to do it iteratively in an ongoing way is because the technology is changing very very fast firstly and secondly because the impact of the AI will vary depending on the context within which you're applying

it. So being aware of that is really

it. So being aware of that is really important and having those processes set up for that iterative evaluation to occur again really important.

Um so one one framework that we use um to help ensure that any use of AI is purpose- driven um is the ahead framework. Um and the first pillar of

framework. Um and the first pillar of the ahead framework is aligned with educational goals. Um so that goes back

educational goals. Um so that goes back to what Ibrahim said right at the very beginning which is that we need to be making sure the use of AI is schoolled um rather than technologyled. So you

need to be thinking about what are your specific opportunities, what are your specific challenges and based on that how do you want to be using AI? always

put that first. Secondly, we need to be thinking about enhancing human intelligence. So the goal of AI is

intelligence. So the goal of AI is really to augment human intelligence, not to replace that. You should always be thinking about what is the impact of AI on critical skills such as critical thinking metacognition reflection all

of which are becoming increasingly important in this age that we we are now in where AI is prevalent. Um

evidence-based so of course everything should be grounded in learning science research, evidence-based practices. We

do actually know quite a lot about learning design in this area. What works

and what doesn't. And making sure that whatever AI tool you're using does align to that learning science research is incredibly important. Of course, things

incredibly important. Of course, things should be personalized and adaptable to individual learners needs, abilities, and contexts. And finally, it must be

and contexts. And finally, it must be designed and used ethically. Um, so

where an AI tool is not being transparent about its functionalities, limitations, and biases, that can be a cause for concern.

Um there are there's a plethora of AI tools out there and so as we said thinking through exactly really how you want to be using them is important.

Don't be led by the technology rather be led by how you can be using it. I mean

here on this slide we have some examples of how you could be using it. Um so

number one teacher admin. So you might want to use it to generate content um for yourself in lesson planning and enrichment or you might want to use it for emails or report writing as part of admin. Those are both great both great

admin. Those are both great both great use cases. Um, with lesson planning and

use cases. Um, with lesson planning and content generation, we find that you can create lots of content for lots of different types of learners in diff in differentiating ways that wasn't

possible before. Um, so that is a

possible before. Um, so that is a fantastic use case within lesson delivery. You may, for example, decide

delivery. You may, for example, decide that actually you want to get learners to practice specific topics um through the use of AI tools that are out there

um and then really um uh build upon that within class. to

orchestrate your teaching around what's happening with the use of AI. And again,

that's a very powerful use case. Um,

student assessments, there's a lot of formative assessments out there. Um, and

offering personalized rich assessments to every student is a real opportunity for AI. And one use case which is often

for AI. And one use case which is often overlooked, but we think it's quite interesting is around teacher professional development. So you can be

professional development. So you can be using AI to record yourself or your teachers within your classroom and really think about what what are the areas for improvement, get personal

coaching and reflect on practices based on that.

So if I keep moving on um so we talked again about the importance of evaluation. You might be thinking to

evaluation. You might be thinking to yourself, well how do I evaluate what the impact of AI is? And there's not there's no one answer to that. There's a

range of ways that you can be evaluating um the use of AI. Um, one approach we sometimes use is called the logic model.

Um, and an example of the logic model is laid out here. Um, so there are different steps to the logic model. Um,

the first most important is to really think about what impact you're looking to have. If you work backwards, what is

to have. If you work backwards, what is the impact you're looking to have? Like

what is the reason for introducing this AI within the system and then to break that down into midterm outcomes, um, outputs and then the activities,

resources, and input that are required to trial out that use of AI. Um, so for example, let's take the hypothetical example of an AI which is helping you to write essays and is supporting your

students with essay writing. There the

impact might be that you want your students to um become more engaged um and to attain more highly uh reflect more deeply um on essay writing. But

shorter term outcomes might be for example around um behavior in class. um

outputs might be around um the number of times that students are actually engaging with writing essays while the activities and the resources and inputs are more around the materials that are provided both to your students and your

teachers in relation to training on that use of AI.

This is the template that can be quite handy when you're thinking about the purpose of AI. So, who will benefit and how? What are the challenges? What are

how? What are the challenges? What are

the steps needed to implement AI successfully? What are the technical and

successfully? What are the technical and cost implications of implementing AI?

What are the ethical implications of AI?

And what are the staff training implications for implementing this AI effectively? So, this is quite a handy

effectively? So, this is quite a handy template. I think it's going to be

template. I think it's going to be provided to you within your resource pack. It's quite useful when you're

pack. It's quite useful when you're starting up with a new AI use case as a way of really structuring your thinking around it, thinking through the benefits and the challenges as well as some of

the assumptions that you're making.

Um so on that point of evaluations um as Candida mentioned at the beginning we did an evaluation um for Skype um the AI tutor um to see how it aligns with researchbased principles of effective

tutoring and math instructions. It was

an initial study based on the data that's been collected so far but there were some promising initial outcomes. Um

so at this point I'd like to hand over to Candida to say more about this.

All right. Thank you. Um thanks for that media.

Okay so before we dive into the detail and share some practical insights um I just want to give you a feel about what we mean I mean we've been using this

word sky. Sky is our AI onetoone maths

word sky. Sky is our AI onetoone maths tutor and I just want to show you it in action. Um while you're watching it's

action. Um while you're watching it's just going to be a 10-second clip. While

you're watching it, just uh notice how Sky is guiding the learning, asking questions, and getting the student to think and explain. Um, and then we'll go

into the detail after that.

It says that in a library there are 40 books on the trolley and 470 books on the shelves. You need to work out how

the shelves. You need to work out how many books there are altogether. First

of all, tell me the calculation you need to do here.

70 add 40 equals.

Well done. You answered correctly. Why

will more than one column change when you add 40 to 470?

Because it's going to go higher and higher and higher and if it's minus it'll go lower and lower and lower.

Could you please explain a bit more about why more than one column changes when you add 40 to 470?

I'm interested in how the numbers in each column change because it's a higher number.

Think about what happens to the 10 column when you add four tens to 710.

How does this affect the hundreds column?

because look for um there was seven. Yeah.

Okay, that gives you a good sense. Good.

Next page. Um so that gives you a sense of what the AI tutor is about. Um so now let's look at how we've applied purposeful AI design in practice. So the

first step is always to ask what what is the problem that we're trying to solve?

And for us and for schools it is this widening maths attainment gap. Um today

more students are falling behind than ever. I mean it's 27% of 11 year olds

ever. I mean it's 27% of 11 year olds are failing government maths tests, 33% of um 16 year olds. So the problem is

large and it's growing. Um and at the same time budgets are tightening.

they're really shrinking and and schools are being left in a difficult position to to ration their supports really only choosing a subset of students. Um and at

third space we've been working at this challenge for over 10 years now. Um

schools already know and trust us as tutoring experts and then this has positioned us well to scale math support

through AI. So always with education and

through AI. So always with education and pedagogy first. Um it's AI is our

pedagogy first. Um it's AI is our enabler that's helping us widen access and reduce costs for schools but at the core is is good pedagogy.

So how did we actually build an AI tutor that tackles this problem? Um we started off with what matters most. So a strong

pedagogy at the foundation and from that everything else flows. Um each lesson it's grounded in learning science. Um

and the lessons follow a um an I do we do you do approach. We have explicit instruction modeling independent practice extension so that learning

science is really embedded at every lesson. Um our

lesson. Um our content is authored by experts by teachers. Um nothing is AI generated and

teachers. Um nothing is AI generated and our curriculum has been built with 10 years of data. So we focused really tightly on those fundamental high impact

math topics and we've built them sequentially to build understanding.

And then we've also really baked in this best teaching practices. So we we know what good

practices. So we we know what good tutoring looks like and how it should be delivered. Um we know that it should be

delivered. Um we know that it should be personalized. So it should adapt. Um we

personalized. So it should adapt. Um we

know that it needs to address misconceptions. So students are are

misconceptions. So students are are speaking out loud and from from their verbal responses we able to dig a bit deeper to understand what misconception there is. It's not just answering a text

there is. It's not just answering a text text box or a worksheet. And then that gives that enables us to give real time feedback which we know is also important

to make learning gains. But we also underpin it with motivation strategies, growth mindset and builtin um guard rails for safety.

Um so pedagogy is important but so is trust. Um so trust, safety, transparency

trust. Um so trust, safety, transparency and that it's teacher teacher is in control. So let me give you a few

control. So let me give you a few examples of how we did this. All content

um is curriculum appropriate and age appropriate. So because it's created by

appropriate. So because it's created by teachers there there's no bias. A lot of the the maths models are really parallel to what's happening in class. Um we know

that you know keep on saying that teachers retain control. They decide

which lessons to run, which pupils should take part. And then um safeguarding concerns are automatically red flagged. They're escalated and

red flagged. They're escalated and reported to school staff. But we we also flag other issues, disruptive behavior, disengagement, and that's also reported

to to schools. So transparency being really important. Um and then data

really important. Um and then data protection is also super important. No

pupils data is used to train the model and only authorized staff can schedule sessions. So really important

sessions. So really important um things that we embedded right from the beginning and so okay so we've got pedagogy is important. Safety and

transparency being embedded is important. Um but we also built Sky in

important. Um but we also built Sky in close partnerships with schools. So

every element was tested in real classrooms not not in isolation. So we

could see how it performs in authentic contexts. So in the reality, the

contexts. So in the reality, the nitty-gritty of school life, you know, things like background noise, lateness, behavior problems or even at the time of

day that these these interventions are being deployed. Um we ran an extensive

being deployed. Um we ran an extensive pilot program across a diverse set of schools. We did regular visits. We

schools. We did regular visits. We

iteratively developed based on what was working well and where pupils maybe were struggling. Um, and we've got a lot of

struggling. Um, and we've got a lot of data. So, we continue to refine with

data. So, we continue to refine with this. Um, and and also to ensure that

this. Um, and and also to ensure that schools continue to trust the impact.

We've had external research validate external researchers validate this claim.

Um okay.

Uh and then another important aspect is understanding the implications for staff. So flexibility has been central

staff. So flexibility has been central to our design. AI should be supporting teachers not adding to their very busy workload already. So one of Sky's

workload already. So one of Sky's greatest strengths is that it can adapt to each school's needs and schedules.

Okay, as I mentioned, it's teachers teacherless. So the teachers in control.

teacherless. So the teachers in control.

Um but it can work with a single pupil, a group of pupils or whole class. Um we

offer now on deore on on demand support.

So it can be used in lots of different ways. They can be used before SATs. It

ways. They can be used before SATs. It

can be used as an intervention consolidation or or even as pre-teaching. Um and added to this all

pre-teaching. Um and added to this all our lesson all our lessons are tailored through diagnostic questions. So pupils

get what they need most without burdening teachers. um having to to do

burdening teachers. um having to to do that. Um okay. So importantly, schools

that. Um okay. So importantly, schools are noticing the differences um and confirming how the solution is meeting their needs. So pupils they they say

their needs. So pupils they they say pupils are are happier, more confident um they're significantly more uh it's significantly more affordable um and can

reach more students. Um and so with unlimited access um and the flexibility to use Sky whenever they want, it's really they're really seeing the benefits.

Um okay, so we've talked about how Sky was built, what what matters and how it per but what what actually really matters now is how it performs in real classrooms. Um so we've asked to educate

to evaluate it so we can share independent evidence of its impact. Um

and I's going to share that with you. So

I will hand over to him.

Yes. Uh hope you can see my screen and I am not on mute. So um so yeah basically like medi has said uh we were asked to conduct an independent evaluation in sky

and we started doing that by looking at what the research evidence is actually saying about high impact tuning whether it is effective at all. uh and we did find that actually uh the emerging

evidence base points to the impact of uh the the use of high impact tutoring to improve learning outcomes as long as it's aligned with established with the

principles of uh of learning design with established principles of learning design. Um when we evaluated Sky, we

design. Um when we evaluated Sky, we found out that it, you know, their evaluation indicated that these learning design principles have been carefully integrated into Sky. And you know, like

just went over some of these. Uh for

example, one of those is the the scaffolded structure, the I do, we do, you do uh and the emphasis on the step-based learning and the immediate targeted feedback that students receive.

Um when we were doing this evaluation, we looked at a number of things. So

besides uh the research base, we also looked at data that the the tool had been collecting from students um we interviewed several teachers and we also

did a small student survey uh to see how students were using Sky. Uh from the data that we collected from the tool itself uh we found that from the sessions that we uh measured, we found

that students actually demonstrated an increase in confidence when learning math using Sky. Um so for example 77% of the students reported an increase in confidence and only 15% after the lesson

reported that they were less confident about their learning. Uh when for the same students we measured this confidence over time. So whether they were going you know whether confidence was increasing over time over multiple

lessons uh almost 66% showed an increase in confidence only 8.5% showed actually that their confidence decreased.

Um also with the tool what it does is you have at the start uh a check-in questions which check your knowledge about the the topic. Uh and then based on that it kind of personalizes the

learning. Uh and then at the end of it

learning. Uh and then at the end of it it check you know there's a checkout question the diagnostic question that checks how well you learned the topic.

Um so we found that on average students would only get 34% of the check-in questions right. Uh but at the checkout

questions right. Uh but at the checkout questions when they were asked to do that on average they got 92% of those questions uh answered correctly. Uh

additionally and most interestingly for us um we found that there are two learner groups who benefit uh as particular beneficiaries you know like emerg as particular beneficiaries for

this program and those were anxious or quiet pupils or pupils who usually hesitate to ask for help in class. Sky

provided them with a non-judgmental way to ask questions. So, you know, if they had something that they were confused about, they could ask I freely without, you know, feeling that they

would be judged uh for not understanding something. Uh and the second group were

something. Uh and the second group were the students requiring immediate consolidation. So maybe, you know, who

consolidation. So maybe, you know, who were just below the expected standards uh and they really benefited from that targetive adaptive practice the teachers assigned to them.

Uh finally you can see here on this slide some of the results uh from this uh student survey um also from the teachers what we found when we interviewed them uh was for them the

biggest benefit was that sky offers consistent availability so it's available at any time and that it's personalized so depending on how well the student understands the topic um and you know when they used it for uh you

know their students especially those that were slightly below standard it it was very effective in improving their performance Um at the same time there are some things that you know remain

areas of improvement. Uh especially you know there is that tech adoption curve.

Um but you know there were some concerns about Sky being able to you know hear the students properly. So a lot of schools what we found what they did was just creating like you know quiet places

where students could talk to Sky or having high quality headsets. But as

long as that is accounted for uh generally the output has been good. Uh

yeah and overall I would say you know like as part of our evaluation we found good signals early indications that this kind of high impact voice-based

interactive personalized tutoring um can have beneficial outcomes learning outcomes for students. Um so yeah that was basically just an overview of our of

our evaluation. I'll pass it back to

our evaluation. I'll pass it back to Candida to wrap up the session.

Off mute. There we go. Okay, great. So,

thank you. Um,

Sky I I think Sky is a result of really purposeful design and lots of work and I and I hope that um that's given you a practical view of how you could evaluate

and then implement AI in your schools.

Um, here are a bunch of links, an AI toolkit that you can you can scan the QR code. Um, but we will follow this up

code. Um, but we will follow this up with an email. We'll we'll share the the slides and also this recording.

Um, and hopefully you found that helpful. So, a huge thank you to Educate

helpful. So, a huge thank you to Educate Ventures for sharing their expertise and thank you to all of you for joining us.

Um I also see a bunch of questions in the chat and I see we have a a few minutes remaining so maybe we can take some time to answer the questions.

Yeah. Yeah. Yeah.

Um I'm just going to go the ones I can see on my screen right now. So there was that question sorry uh I might not do it in the right order about um so the

question was from Mr. Mant I'm not sure I can't see the whole name. My school

trust allows us to use AI and it has an AI code of conduct and guidelines.

However, we have a very limited amount of tools allowed. As an example, we allowed to use Gemini but not chat GPT.

Is there a reason you can think of that this would be the case? Is it less secure? So, that's a very good question.

secure? So, that's a very good question.

Usually, what we found in our experience is that um schools are already uh already a part of a an ecosystem. So,

they might be using Google classroom, they might be a Microsoft school. uh and

generally that is the number one consideration when they decide to go for one tool over another. Um usually these companies um you know Gemini, Google in

this case might offer you uh a kind of protected uh you know space for you to use that AI tool without your data going to any third party or without it being

used for uh training the model itself.

So in that way it might be a bit more secure but um really you know like the same offerings are there from from other companies from Microsoft from OpenAI as

well ultimately it's mostly about what what ecosystem that uh that school is already integrated in. We I will also say um that despite this about what

tools are allowed or not, we often find that students and teachers um often use tools outside of you know on their in their personal capacity as well. So

there's definitely that's a lot of that happening. Um I don't know if that I

happening. Um I don't know if that I hope that helps answer some of your question.

Um maybe I can grab one of these. um asked

can it be used for multilingual learners and is there a translation element at the moment not which is really exciting we've built um our AI tutor because um

chat like LLM are not that great at math so we make sure that the maths is excellent and the pedigogy is sound but what it's very good at is language and

so we we are going to be experimenting with turning on certain languages and then it will be able to respond respond um in in a different language back which

is quite exciting. Um and then also another question from Juliana which is um what the duration of an AI lesson is.

So it's 30 30 minutes long.

We we can um we we had it longer but we find that 30 minutes is our sweet spot um at the moment. Um, and then there were questions of how do you make sure that you that the AI is an effective way

of teaching? And and I hope that we've

of teaching? And and I hope that we've been able to answer that as we've gone through because we've baked in all of these um effective practices into how

the AI delivers the the session and we also create all the content that needs to be delivered and then it adapts in real time.

Iraham is there I think there was one here about is there training available training?

Yes. Uh maybe media you want to take that one. Uh the question was um are

that one. Uh the question was um are there any training courses for staff to do so that they can learn how to use AI systems or is it just a matter of trial and error for staff who want to use AI?

I'm aware that it depends on what your input is as how well AI does the task or answer the question? If there are training, does it train you in how to set out your question instructions to get the best possible results?

Yeah. Yeah. And I think what we'd say is that there absolutely is training that you can use and but um I think the point we'd make is that training for staff can absolutely firstly include AI literacies, the capabilities,

limitations of AI. There's prompt

engineering which is I think what you're referring to there, you know, so how do you engineer your your cue or your instruction to the AI um appropriately to get the best possible result? And

there is that type of prompt engineering um training that um that is provided um fairly often. We at EVR also recommend

fairly often. We at EVR also recommend going a little bit further and thinking about pedagogical training in relation to AI as well. So what shifts do teachers have to make in terms of their own practice to make the most effective

use of AI? Um and that can be um that can really um result in um going from just using AI as a tool to using in a much more of an augmented intelligence kind of way which is which is really

important. Um the other um question that

important. Um the other um question that I've just seen in the chat and I've lost it now but I think the question was around policies and how do you develop policy have local authority that might

be um already uh providing some level of input there and I think especially if you look at the getting started with policy guide with the QR code that we provided earlier you'll see

that policies when we talk about policy we're almost talking about your approach towards AI and while there will be some things that the local authority will um dictate ate and there's a lot of other things which the school is likely to

have flexibility over such as the parameters around the use of AI specific ethical considerations and there's also a lot of thinking to do around how do you link all of that to the practical

tools that you're giving your teachers in the classroom um so they can actually use the policy in their practice you know both to um use AI safely but also

to get the most out of the AI um so I hope I hope that helps I'm not sure if there are other questions There is another one I'd like to grab.

So, we've got from John McCullen, M Mc Mullen.

Um, he's wanting to know how the AI works with send uh pupils, especially pupils with communication difficulties.

And then he was highlighting how with some of his students. So, looking at the video, his students may not have been able to verbalize uh some some of those answers. Um and and I think this is also

answers. Um and and I think this is also where the kind the kind of magic happens because Sky there's voice voice to text.

So there's a transcription element um and that is amazingly accurate and and really can pick up a pick up a lot. Um

it it's not it doesn't struggle to understand um accents or um different voice tones. Um and it it really can

voice tones. Um and it it really can understand the gist of what the pupil is trying to say. So it's not it's not expecting this like verbatim perfect

answer. And because we are a voice

answer. And because we are a voice onetoone tutor, we really do want to encourage them verbalizing their thinking even if they can't do it

properly and and and we will move through different levels of scaffolding.

And if they don't if it doesn't quite get to verbalizing it properly, then Sky will model that correct verbage using

mathematical terminology and move on. So

um it's it's it doesn't hinder them, it more really helps them along the way.

And then there's with SEN students, there's a a wide variety with amount of data and understanding that we're getting from these sessions. The

personalization is going to just get better and better and better as we look at how we can personalize um with certain um Zen children.

Do we think we should stop here?

Yeah, I think we're on time.

Yeah. Um why don't we then pick up any of these other messages via email? Yes.

Um but hopefully you find that helpful and have a lovely day. Bye everyone.