LongCut logo

Step-by-Step: The State of AI Filmmaking Workflows

By Joey /// VP Land

Summary

## Key takeaways - **Core AI Filmmaking Workflow**: The typical workflow observed from creators in competitions like Cinema Synthetica is text to image to image to video, allowing anyone with a computer to make a convincing film without cameras or resources. [02:19], [02:29] - **Good Story Trumps Visuals**: Amazing visuals won't save you from a bad story; you must start with a good story and good storytelling techniques. [02:51], [02:55] - **Style Consistency via LoRAs**: To ensure consistent visual style across shots, create a LoRA trained on generated images in the desired style like cyberpunk, or use Midjourney's style reference codes. [04:31], [05:19] - **Character Consistency Challenges**: Human brains are highly susceptible to uncanny valley faces, so spend significant effort on character consistency using detailed prompts, LoRAs, or character sheets with multiple poses. [08:28], [09:04] - **Video-to-Video Revolution**: Luma AI's restyle video feature allows shooting low-fi object blocking videos like toys or sheets and transforming them into epic scenes, such as shoes into a race car, shifting workflows dramatically. [23:06], [23:24] - **Top Easy Tools Trio**: For chat-mode ease covering most bases in pre-production and image generation, the top three tools are Runway references, ChatGPT, and Flux Context. [00:07], [11:54]

Topics Covered

  • Story Trumps AI Visuals
  • LoRAs Lock Visual Style
  • Pose Sheets Ensure Character Continuity
  • Video-to-Video Disrupts Workflows

Full Transcript

The top three tools that are just if you're kind of just trying to like in a chat mode, easy to use that kind of hits the bases is going to be runway

references, chat, GPT, and flex context.

>> All right, welcome back everyone to Denois. Uh we're going to do sort of

Denois. Uh we're going to do sort of another deep dive episode uh today looking at AI film making workflows and what sort of the typical workflow that a lot of people figured out is happening

uh tools that are used along each step of the way and also how that's changing a bit with uh more impressive video to video models.

>> Yeah. And if you missed our last episode, we did a deep dive on Comfy UI and some of the fundamentals of image generation and image workflows. Uh give

that one a go before this one if you like.

>> Yeah. Yeah. Check that one out. That

goes into the nitty-gritty of like how AI image generation works. And this one we're going to be a little bit more inflow of >> more creator driven.

>> Yeah. And just sort of the workflow and different tools. And most of these are

different tools. And most of these are going to be off the quote offtheshelf tools. Uh websites that are pretty uh

tools. Uh websites that are pretty uh consumer friendly to use. Um but sort of how you have to kind of think about the process each step of the way. And so a lot of this is adapted. Well, first off, I started building out this flowchart

because I started getting asked questions. you know, people are like, I

questions. you know, people are like, I want to, you know, make an AI movie or I want to use AI to help make a film. What

do I do? Where do I start? Where do I start? Then I was I was thinking about

start? Then I was I was thinking about I'm like, well, you know, I'm like a bunch of different tools and options would pop in my head and it's like there are a lot of questions of like how you want to think about this and also there

are multiple tools along the way that you sort of have to use is tools are getting better at kind of integrating to kind of be a one-stop shop. Yeah. Uh, so

you don't have to hop around between a bunch of tools, but we'll cover all the tools just so you can because some are better at some things and some are better at different things. And it

depends too of like what your need. It's

like do you need a more action based shot? Do you need more static shot? Do

shot? Do you need more static shot? Do

you have characters performing, characters talking, multiple characters?

Uh, so there all kind of things you got to >> Yeah. And regardless what tools you use,

>> Yeah. And regardless what tools you use, uh, we have a very agnostic workflow here that you'll find beneficial just in general. As the tools change and evolve,

general. As the tools change and evolve, the flowchart here that we share won't really change.

>> Yeah. I mean, I've seen uh this also evolved from watching all the creators on Cinema Synthetica, the AI on the lot film making competition and they all basically use some form of this and it's

hard as like text, image, image, video.

That's sort of the gist of it. And that

was sort of in some fashion >> the workflow that all of them use >> today. Today also that is still a good

>> today. Today also that is still a good workflow too. If you just have a

workflow too. If you just have a computer, >> you don't have any resources, you don't have a camera, you don't have like access to film some stuff, you can make something visual, a film.

>> Oh yeah.

>> Uh just using your computer with that workflow.

>> You can make a pretty convincing film if you do it right.

>> Yeah. And have a good story. And this

all goes to the root of, you know, good story and and good and something to to to telling techniques. Amazing visuals

won't save you from a bad story.

>> Exactly. Exactly. All right. So, first

up, starting out with you got to have a good story. Yeah, you got a good story.

good story. Yeah, you got a good story.

You want a shot design yourself and obviously each step of the way you could kind of figure this out yourself or you could lean into some of the AI models. So in here in this case for the tech stuff the general LLMs are

going to come into play. So either

Google Gemini uh claude or chatbt I Gemini 2.5 I've been hearing a lot of really good stuff and I've been using a little bit more and more uh especially with like shot design or kind of

creating prompts for shots. I use cloud a lot. I found cloud's always been sort

a lot. I found cloud's always been sort of the best at creative writing tasks and stuff like this. So you could start off with >> isn't uh Higsfield specifically designed for stuff like this.

>> Higsfield would be more later on of like you want once you're like you want to start generating images or gen or turning images into video. So this is still in the kind of >> ideation >> ideation planning phase like so first off figure out okay I want to make an

action scene of like a guy you know being chased through a mine tunnel on a mine car >> temple run again.

what kind of shots would I need? You can

have help brainstorm the shots and then from there you could also help generate the prompt >> to generate your your image. So go from text to image. Before that though, you

probably want to figure out a consistent style or visual of how you want all the shots to look. Uh so if you don't so you don't generate a new shot each time and it has a completely different visual look. Correct. A couple ways you can do

look. Correct. A couple ways you can do that. One is you just help you describe

that. One is you just help you describe what kind of look you're going for and you create a descriptive text prompt.

Yeah.

>> Uh of the style, >> right?

>> Other way which I found comfy is you can make a Laura.

>> Way better.

>> Way better.

>> And you can guid it with an existing image.

>> Uhhuh.

>> So if you have like a cyberpunk 2077 type look, you know, futuristic Bladeunneres thing, then give the model imagery of that city.

>> Mhm. and it'll sort of start to get a good look and feel of that city from just a few models, from a few images.

>> Would you use existing images of just like reference images from other movies that have the style you're looking for or would you try to generate like image journey images that have the style and

then train it on those images?

>> I don't know. I mean, there's an ethical question there. uh I would just uh

question there. uh I would just uh generate stuff specifically for influencing generation >> if that makes sense >> right create an im like let's say if I create text image in stable diffusion in

that style >> 20 images or so with the cyber punk style and feed that to train the Laura >> okay other option other way for creating style if you're working in midjourney midjourney has srfs which are these codes that trigger and call up a

specific style so cool >> yeah and so and you could also similar to the Laura you can give it a bunch of images and it'll create a style that'll uh call it up by that. This only works if you're working in mid journey, but it

works really well.

>> Yeah. And um the technical term is style transfer. So it'll sort of take uh

transfer. So it'll sort of take uh whatever uh the image is generating and then uh apply like um a filter on top of it, a style transfer filter.

>> Yeah. And then the other way is you can use reference images. And so some of the tools like runway references >> chatgpt uh you can or yeah Google image uh imaging

>> flux context >> flux context you can give it another image and say you want the style of this image and not necessarily any of the details but you just want the aesthetic look of that image and you may have to

run it a few times but you can use that as a consistent to try >> we got to all of the uh spirited away Miyazaki stuff that those were all transfers right but that was Uh it was

it knew that style and those were just people calling up that style. Exactly.

It was they weren't giving it an image of like from Spirited Away and then giving it >> already did that for you.

>> You knew what that style was. But let's

say you're trying to create a unique original style.

>> Yeah.

>> You can give it an image >> and your image of what you're trying to generate and and try to have it uh match that style.

>> Yeah. Like a like a Wes Anderson film has a very specific look. said original

style.

We want you to make original stuff.

>> Yes, I was just pulling an example.

>> But yes, that is a very distinctive style. Okay. So, you want to kind of So,

style. Okay. So, you want to kind of So, this helps because shot to shot, you want to make sure each new first frame feels consistent, like it's in a consistent world with a consistent

cinema look.

>> Yeah. And this is where you as a creator really matter, right? like storyboards

is what's going to drive everything down the pipe. And so if you have an eye for

the pipe. And so if you have an eye for blocking, eye for compos frame composition, for lighting, for cinematography, this is when all of that's going to come into play.

>> Mhm. And then the other thing that's going to be an issue with consistency is characters and making sure that your characters look the same shot to shot.

And so ways to do that, similar to the style, you could either, if it's sort of a genericy character, you can just have a very descriptive text prompt. Um,

>> this is sort of what a lot of people do are currently doing with V3 because V3 is currently just text video. There is

no, you can't give it a frame. So, in

some of the launch videos, it was like the SWAT police action video that Dave Clark did, but that's kind of a easy one to do because yeah, the faces are kind of covered. We're not really seeing the

of covered. We're not really seeing the details of their faces and and it's something that's sort of easy to prompt and get a very consistent looking physical character. But if you're, you

physical character. But if you're, you know, actually building characters where we see their face and they're talking and performing, Laura is another option as well. You can turn it on a character.

as well. You can turn it on a character.

>> Exactly. And, uh, you know, as human beings, I I bash this with a hammer every episode is like we're so susceptible to, uh, looking at people's faces. So, the minute that you have a

faces. So, the minute that you have a slightly uncanny valley type face, a face that doesn't look quite natural, like instantly your project breaks. So,

you want to spend a lot of time and effort here.

>> Yeah. And then we'll talk about actually making them perform and talk later, but that's the other issue. And then also, you know, going back to the tools that we've talked about where you can just sort of give it a single shot.

>> Yeah, >> you um a lot of other tools if you just have a photo of a person that can sometimes work and you like runway, you just give it a photo of the character and you know, say I want them in the scene. Hopefully, sometimes it'll like

scene. Hopefully, sometimes it'll like recreate the person pretty accurately.

Another trick that you could try to do is to create a character sheet. So a

single image that would be much more of like your front face or side profiles, full body. So like a character reference

full body. So like a character reference sheet.

>> Yeah, >> that's a single image that you can give to these models.

>> And that that is done in animation all day long, right? When they're in character development, uh they're they call them pose sheets >> where all that is created in CG

traditionally, but now you're doing it uh in Photoshop >> and then putting that up onto your control net model. Uh we make Muppets does a good job.

>> He does and he has a whole comfy workflow to where you can give it a single image of character and then it'll create an entire pose sheet.

>> Yeah. Ideally, if you have uh 10 characters in your film, every of those 10 characters needs their own Laura >> based and then they'll need their own pose sheet.

>> Mhm. Even with a Laura, you'd want a a post sheet.

>> A pose sheet would be like uh let's say you have a hero male character. So the

post shoot would be male standing like this, standing in the back side, you know, 3/4 view, face close up, face smiling. But I guess I'm saying, would

smiling. But I guess I'm saying, would you train the Laura on that pose sheet or where does the two you have the character Laura and you have the post sheet? So how do the two come into play

sheet? So how do the two come into play if you need both?

>> The Laura build happens before the post sheet.

>> Okay. Right. uh you use the Laura to create the post sheet and then the po sheet influences the generation because you're that character is moving in space.

>> Okay. I think

>> so it'll use both the Laura and the PO sheet together to figure out what that human should look like at whatever angle of whatever framing.

>> Okay. Got what you're saying? Okay. So

now we've got So these are sort of the foundational things you kind of want to figure out before going and to start making your first frames of what is your character your characters going to look like? What's your style going to look

like? What's your style going to look like? And also I didn't get it but like

like? And also I didn't get it but like you're if you're adding a a location >> that's a big deal.

>> Yeah. The scene the the location. Um one

trick that I was did with the AWS the Amazon film. Oh no no I'm talking about

Amazon film. Oh no no I'm talking about that that Culver Cup film making comp thing that I competed in I it was um I had a like a 3D room in Unreal but then

ex you can export a 360 panorama.

>> Oh interesting. And then I style transfer the entire panorama. But then

that way I could just use portions of the background so that if it was like a conversation like this, it was like, oh, if I had the reverse shot, I would at least have a consistent background scene.

>> Yeah. And uh spatial consistency in image generation is still a for like frontier thing. Yeah. Uh, I know Flux

frontier thing. Yeah. Uh, I know Flux Context has a decent way of approaching that and so does Chad GBT, but again, you're you're essentially shortcutting that process by using a 3D model of a

world and giving >> because they have like a world understanding.

>> Yeah, exactly.

>> Yeah. Yeah. I mean, we're going to keep going back to it, but I think the three the top three tools that are like just if you're kind of just trying to like in a chat mode, easy to use, uh, that kind

of hits the bases is going to be runway references, >> chat, GPT, >> and flex context. Those are sort of the top three of just being like, here's an image of a thing in a world. And

>> that's state-of-the-art today like uh recording this.

>> Yeah. And that is only for this uh pre-production portion of the production that we're in. And now we take that and everything we've doing and now we start generating the first frames for each of

the shots that we want. Yep. And so this is either going to be text to image merging it with if we have Lauras or if we have reference images uh in our chat

interface with some of these tools. And

then also this is where also using any of the LLMs comes helps come into play to help you refine and write out >> structure the prompt help you write out the prompt at least give you um it's also good to just have a write the

prompt as a starting point and you can kind of come in and tweak it but they'll write much more detailed prompts than I'll think of off the top of my head.

some um VLMs, visual language models um out there. I haven't done too much

out there. I haven't done too much research into this, but you can give your image to a VLM and then it'll caption it for you correctly and give you a suggested prompt. And

>> oh, like so if you like have a reference frame, you're like, I want this kind of shot, give it that frame, turn it into a text prompt, and then use that >> use that as a input to >> I think Gemini is pretty good at that cuz Gemini is multimodal and you can

give it input >> images and videos and it'll describe or tell you what's happening.

>> Yeah. And remember what we covered in the last video, the clip encoding. It's

it's birectional. So you can go from text to image, image to text back and forth pretty easily. And that's what VLMs do.

>> Okay. Okay. So yeah, that's another good uh trick too if you like have a reference shot where you're like, I want to replicate this Yep.

>> type of angle, feed it, turn it into a text prompt, and then generate your first frame. Okay. So also just to round

first frame. Okay. So also just to round up a sort of the common tools for generating the first frame. This is

going to be all of your text and image uh generator. So midjourney runway

uh generator. So midjourney runway ideoggram flux chatgpt reference. You could use it with the

reference. You could use it with the references feature which we talked about. Also the fourth one is imagin as

about. Also the fourth one is imagin as well Google's imagin which does have pretty good you can give a reference you can give it a character reference a style reference location >> and it does a pretty good job of having

that chat conversational workflow to help modify >> creator that that is the key right otherwise it's just a random thing >> the more control you can have the better that's the thing you're going to be fighting all the time and then we've talked about this before too but you can

also kind of do a hybrid approach with Blender or Unreal and very sort of kit bash a scene together but at least >> roughly frame up a shot and use that as a base image to style transfer.

>> Yeah, I if you're pretty handy with Unreal Engine, I mean, use Metahumans, pose them, right? And then use a camera to kind of block the characters and then bring that still image into one of these

things as reference.

>> And then there might be a final stage too, like sometimes you might get an image, but maybe it's like 90% of the way there of what you want, but it still has some issues, but it's like not worth it to keep regenerating it. So then you might take the image and you have might

have to modify it either within painting to like uh change specific aspects of it and most of the image tools have some form of impainting. So like my journey >> explain to the viewers.

>> Yeah. So in painting is like okay we want to keep this image as is but like I want to change just this computer regional correction.

>> Yeah. I just want to I keep everything the same but I want to change my coffee cup to like a big chalice or something.

And so then you would take the image and you would like paint over the coffee cup and then you would text prompt to make it a chalice >> and it would generate the chalice but blend it theoretically hopefully blend

it in with the image.

>> Oh yeah, it doesn't really I mean most models are really good. Photoshop has

Firefly built in and that is one thing they're just pretty good at of >> taking you know and also I mean sometimes in some cases you might just straight up need traditional Photoshop >> just modifying

>> exactly and don't don't be shy to go back and forth between traditional tools and AI and back >> yeah it doesn't have to be AI all the way. So like a lot of times uh

way. So like a lot of times uh >> this stuff still needs a lot of massaging.

>> Yeah. And a lot of times even just traditional compositing or trying to think of building out your elements separately might be the better option.

And we won't get into that here cuz that gets a little bit more complicated. And

this is kind of more of a vibe filmm we're talking about.

>> I I find that with multiple characters, it's probably easier for you to generate a consistent character and then composite them in Photoshop and then take that reference frame as your image

generation uh video generation frame.

>> So generate the characters as separate elements and build it all as one frame inside Photoshop.

>> Yeah. Yeah. Like it doesn't have to be exactly, you know, uh geometrically correct and stuff like that. the

generation will take care of it, but at least you'll get, you know, your three hero characters in the same shot on that reference frame. If you try to just

reference frame. If you try to just generate that in Comfy UI, you're looking at a pretty crazy workflow, like three different Lauras, three different character sheets.

>> Yeah, I mean, I have seen some hacks now with flux context out of like combining images into a single image and giving that to flux context and that working pretty well in Comfy.

>> You've you've showed that in one of the episodes, >> I think. Yeah. Yeah. Uh, right. There

was like a Yeah, it was a hack or something. Another

something. Another >> You can get five people in there.

>> Yeah. And then also, I mean, yeah, I've seen uh runway demos where you could just kind of I think they built in a sketch tool into runway because people were doing this so much, but you could just kind of roughly sketch out how you want the scene blocked and it sometimes

does a pretty good understanding of replicating that in the image.

>> Yeah.

>> Okay. And so then you've got your first frame >> and then you want to turn that into a video. And now there are some most of

video. And now there are some most of the tools will support image to video, first frame image. And some tools have more control where you could give it both first frame and last frame or first, middle, last frame. But the the

general broad overview of like all the tools that will support image to video got your runway V3. Uh MidJourney just launched video and that's been pretty impressive. So

impressive. So >> people are impressed by it.

>> Yeah. Playing uh Hiluo, the Chinese models that we talked about that also that uh get really good reviews. Uh Luma

AI.

>> Yeah, you can also run this on your own.

So, one one 2.1 which is by Alibaba is is a very popular comfy UI video generator model.

>> Oh, seed dance.

>> Seed dance a new one too. Yes. Yeah.

From uh by dance.

>> Yeah. If you have like a RTX 3090 4090 like last year's GPUs, I would say like a 720p video for 30 seconds, you're looking at like an hour or so.

>> Like it's not that terrible. It's like

old school CG render timelines. And then

I mean this part is always still feels a bit weird to me because it's like you do all this work to get the image the first frame but then it's just like well I just give the first frame to the model and like let it do its thing. So some of these you'll some of the models you'll

be able to give it a text prompt to try to describe what you're looking for and some of them or a lot of them now have sort of pre-built camera moves to which can also help get what you want.

>> Yeah. You want to resolve ambiguity as much as possible. Runway and cling in their older models. They had a kind of a camera control movement where you could sort of map out how you want the camera to move. In Runway, that was only in Gen

to move. In Runway, that was only in Gen 3. It's not in Gen 4 yet. So, Gen 4

3. It's not in Gen 4 yet. So, Gen 4 doesn't have as many features as Gen 3 does, but I'm sure >> eventually that will get added. Uh, we

didn't mention um the one you Higsfield.

>> Yeah, Higsfield is specifically built for cinematic shots.

>> Yeah, I would say it's a bit more for like advertising or like very dynamic highotion shots for like an advertising spot.

>> Sure. But try them all. I mean, a lot of these is just like whatever tool gets you the shot.

>> The the pro here is all these tools are really easy to use >> and they're relatively cheap, right? If

you're looking at like how many cents, how many dollars per video, it's not that much. It's not like going to break

that much. It's not like going to break your bank. The con here is the control

your bank. The con here is the control is still something that we're all trying to figure out.

>> Yeah. Yeah. Getting the control of what you want. Um, which there are some good

you want. Um, which there are some good work uh examples a little bit later. But

other thing that might come into play too if you have characters and you need them to talk then now you got to introduce another tool.

>> Yes.

>> Uh so that could be something like uh Hedra uh which has really good character performance. Um you can give it an image

performance. Um you can give it an image of the first frame and give it direction and the audio.

>> Maybe hen.

>> Hen's another one. Yeah. Runway act one.

>> Yeah.

>> Which also has not been updated for Gen 4, so it's still Gen 3. Okay. And then

there might be elements of compositing these things because it's like you might have elements of like where the character talks and you need the lips to move but then also you need camera movement. So you might this might come

movement. So you might this might come into play where you have to start compositing or treating like the performance separate from the camera move. And

move. And >> I would say V3 probably does the best job of animating faces for speech and matching speech.

>> But V3 is only limited to uh video. So

you're going to have to have a very descriptive prompt. That's why it worked

descriptive prompt. That's why it worked really well in that commercial. The PJ

>> the liquid death one.

>> Not the liquid death. The It was a gambling >> Kelchic.

>> Clashy.

>> Clashy. Something like that. Yeah. But

it was >> that was that was a good one.

>> It was good. But it was like every shot was a single >> scene shot with their own characters in that shot. So you didn't have to worry

that shot. So you didn't have to worry about a character looking, right? There

was no dialogue.

>> No, there's dialogue. It was it was the news interviewer like kind of interviewing people, but every shot was like a different person, different interviewer. So, uh, it didn't

interviewer. So, uh, it didn't >> Yeah, it didn't matter.

>> And that's why you see a lot of sci-fi stuff with like masks on no faces cuz it's still hard.

>> Yeah. Easier to to to deal with the performance, but stuff's getting better.

I mean, Hedra Hedra's new version is really good. We're testing that with a

really good. We're testing that with a project, so we'll report back. Yeah. So,

just another thing to think about, you know.

>> I mean, Joey, what about this? Like, can

the human performances be actual humans with a real camera? And then you cut to your AI generated shot, >> right? And that's the thing, too. You'd

>> right? And that's the thing, too. You'd

have a hybrid workflow of real people in front of a green screen or the totally uh using another workflow like craft jetet >> and then you you new composite that human's face onto your generation if it's

>> oh I see you actually just do the face onto the generation and it's interesting then it could be >> if you're good enough at n say how complicated is that a nuke if you're >> well you got to match move the camera and then take that match move camera

into your generation so the generation's from the same angle >> and uh if it's a static shot it's fairly easy but if it's moving then you're warping that face.

>> Yeah. I was thinking of more just like if you shoot a person in front of a green screen with something like craft with like a with a lowfi 3D background and then you take the background separate and process the background.

>> Exactly.

>> Separately.

>> That totally works too.

>> Could and then run it through uh Bable for relighting so the whole scene matches. That's another workflow that a

matches. That's another workflow that a lot of people are exploring.

>> Yeah. I think AI faces and AI speech is still >> Yeah, that's not not there yet. Yeah, I

mean we've seen AI is very good at these like one shot action shot generations but having that consistency shot >> imagine a 90minute film with there's like hundreds of lines of dialogue right you want a real human

>> it gets tricky there yep >> yeah so that's highle overview gist that's why there are a lot of steps and tools uh there are tools that are sort of combining this all so you can kind of do it in one stop so like free pick has a lot like kind of one-stop shop even

hedra a lot of them are building connecting to the APIs with other tools so you don't have to bounce around a lot. But another thing to think about

lot. But another thing to think about that's sort of changing a lot is uh since Luma AI released the restyle video feature.

>> Yeah, that's great.

>> That's been blowing up and that's been kind of kind of shift >> like thinking about this completely where uh you can just shoot a very lowfi object blocking video. Uh I mean with

John Fingers, his demos are crazy.

>> John Fingers are so like the one where he's just like swimming on a carpet and but it's an astronaut in space.

>> Yeah. Or it's like of someone with the sheet moving and then it's like a stormy ship.

>> He's really imaginative.

>> Yeah. Or he had like two shoes like pulling the strings of the shoes and then turned into like a race car.

>> Race car. Yeah.

>> Yeah. So this and I mean you know we are quote old like I just can't imagine what you know some like kids who take this and run away with it cuz it's like it could just kind of re you can just reim you you don't even have to have the same

rules >> of like matching a shot. Like if I think of a race car shot, I would still probably be like, "Oh, I need some toy cars and I'll turn the toy cars into race cars."

race cars." >> Yeah. And it doesn't have to be video to

>> Yeah. And it doesn't have to be video to video. Think about this way. This will

video. Think about this way. This will

blow your mind.

>> You can go from video to video to video to video. Like you can keep iterating on

to video. Like you can keep iterating on it in Luma to get the quality and the stuff that you need. And I'm sure Luma would like that cuz that's like generation.

>> So yeah. So this type of workflow I think could could just totally throw a lot of the stuff we just talked about out the window. Well, not necessarily because you still need the solid image references, but now you're shifting to a more of a video to video workflow.

>> Yeah. It's like, oh, what if I just take my phone and I could just shoot with a few people or I could just shoot with some toys on a table and now we've got like an epic action film. But also I like, you know, when I'm like, oh, it's

crazy that you just give a first frame >> to the image and then you like let it do its thing. But now, you know, luma is

its thing. But now, you know, luma is mimicking the actual movement that you do, >> cinematography, everything.

>> Luma does take a first frame. You can

either not give it a first frame or you can give it a first frame. So you could combine the elements up to making that first frame. And then that camera

first frame. And then that camera control could be, you know, luma or it could also be a Blender or an Unreal Engine recorded camera movement.

>> You restyle the first frame and then now you're really driving what the image should look like and how the camera should move in that space.

>> Yeah. the the the biggest benefits to physical production and principal photography is character performance, >> direction from the director on the performance and then cinematography.

>> Mhm.

>> Like you want to preserve those aspects of film making as much as possible and I think Luma allows you to do that.

>> And yeah, I mean video to video has been pretty slim on who's offered it. I mean

Runway was the only one that did it for a while and it's still they haven't updated it. It's still only for Gen 3. I

updated it. It's still only for Gen 3. I

mean I'm sure it's only a matter of time before Gen 4 has video to video. Crystal

Ball, >> you want to announce it on the podcast, Crystal Ball, come on.

>> So, also once that comes out too, then it's going to really be because Gen 4 itself right now is pretty crazy. So,

once once the video to video comes out, that's going to be another >> and it's such an exciting time we live in. Like all of these tools we're

in. Like all of these tools we're talking about, they're all great.

>> Yeah.

>> Not one of them is >> how like if you look at the stuff people posting things like from two years ago and it was like, you know, just abstract bizarre.

>> Yeah. Like Vincent Van Go style transfer. That's all we got. Few years

transfer. That's all we got. Few years

like how fast this has come in just 2 years.

>> I know. And it's already pretty usable, but we're going to get to a point where all this stuff is effortlessly usable.

>> Yeah. Yeah. And there won't be a massive um flowchart workflow.

>> Yeah. You won't need Joey and Addie anymore. But hey, we'll we'll move it.

anymore. But hey, we'll we'll move it.

>> You'll have AI Joey and Addie.

>> Yeah. But I think I Good place to wrap it up.

>> Yeah. So, uh, yeah, I mean, this was this is a lot of the tools and workflow that I've been seeing, but if you've got your own workflow or other kind of things that you've been using, uh, that we didn't cover, please let us know in the comments. We'd love to hear it.

the comments. We'd love to hear it.

>> Let us know if this is the type of content you want to see. We'd love to make more of this stuff, but it's up to you, the viewer. So, give us a like and give us a comment.

>> The power is yours. All right, see you next time. They

next time. They

Loading...

Loading video analysis...