Linear Algebra for Machine Learning

this in-depth course provides a comprehensive exploration of all critical linear algebra Concepts necessary for machine learning you'll

learn the mathematical foundations to excel in AI tdiv from lunar Tech developed this course she has created many popular machine learning courses

machine learning is at the Forefront of the Innovation powering the most advanced and transformative systems for the companies like apple Tesla Netflix

Amazon open Ai and many others it enables the creation of the intelligent systems that can predict Trends personalized user experience and

automate complex tasks to develop these practical applications a deep understanding of the underlying mechanics is important this requires a

solid grasp of mathematics behind the machine learning so all these technical details with a particular focus on linear algebra

this all-encompassing course explores the linear algebra in an interactive and machine learning Focus manner welcome to the linear algebra for machine learning

course you will acquire the critical principles needed to build optimize and analyze sophisticated machine learning models from designing customer

algorithms to enhancing curent Technologies this course provides the mathematical foundations with vital interest for the those for pioneering

advancements in machine learning for those dedicated to mastering the mathematical aspect and the technical details behind machine learning our

extensive 26 plus hour course of fundamentals of machine learning within the mathematics boot camp as well as a separate course offers an in-depth

exploration this extensive program includes certification and is tailored for individuals that are serious about advancing their career in the field of

machine learning Andi engineering this crash course in mathematics will serve you as a great starting point by establishing a robust foundation in

linear algebra you will be well prepared to excel as machine learning practitioner equipped with the mathematical knowledge that drives the

Innovation and efficiency in this field so if you're ready I'm really excited and without further Ado let's get started welcome to the course on the

fundamentals of linear algebra presented by Lun Tech Academy my name is D Vasan and today we are going to start with some basic concepts that are important

for understanding linear algebra linear algebra is one of the most applicable areas of mathematics it is used by pure mathematicians that you will see in a universities doing research publishing

research papers but also by the mathematically trained scientists of all disciplines this is really one of those areas in mathematics that you will see

time and time again appearing in your professional life if you want to become a job ready data scientist or you want to do some handson machine learning deep

learning and AI stuff but also linear algebra is used in cryptology it is used in cyber security and in many other areas of computer science and artificial

intelligence so if you want to become this well-rounded professional you want to go beyond using libraries and you want to truly understand the uh

mathematics and the technical side of these different machine learning algorithms from very basic ones like linear regression to most complex ones coming from Deep learning like architectures in neural network how the

optimization algorithms work how the gradient descent works and all these other different methods and models then you are in the right place because you must know linear algebra such that you

will understand these different concepts from very basic ones to most advanced ones in data science machine learning deep learning artificial intelligence

data analytics but also in many other applied science disciplines so before starting this comprehensive course that will give you everything that you need to know about

linear algebra first I'm going to tell you what we assume that you already know because linear algebra it comes from about third PA of Bachelors of different

highly technical studies and here we are assuming that you already know certain Concepts so to ensure that this course stays really on the topic of linear

algebra and that you understand all these Concepts really well for that we need to be able to know different topics

so before we dive into this Concepts let's familiarize ourselves with the basic prerequisites and notation used throughout this course and you will really need to know this in order to

understand these Concepts really well such that instead of memorizing you'll actually just hear me once or maybe twice and then every time you hear later on or you see it in the papers or in

some algorithms you will recognize this is something that we already learned so some key prerequisites overview is here first of all to fully grasp the upcoming material you should

be familiar with some basic concept like real numbers Vector spaces so you don't need to know this idea of vectors though you already most likely are familiar

with this given that you know how to plot different lines you know the idea of exes and wise and how to plot these different graphs but here we are going

to touch base on this every time when we come close to these Concepts I will refresh you your memory and we will go through this numbers the idea of norms and distance measures because when it

comes to the vectors when it comes to the magnitude and all these different topics that we are going to discuss as part of linear algebra knowing the what

Norm is and what is the definition of distance what is the length between two points when we plot it into two-dimensional space or three-dimensional space those are all

very basic concept that usually use as part of a basic pre-algebra or just common algebra courses and lessons in order to truly understand what the new

algebra is about to understand the direction of vectors the angle and then the dimensionality reduction how linear algebra is applied for instance in different algorithms in machine learning

deep learning data science statistics you really need to understand this Cartesian coordinate system so this is not only important for linear algebra but I assume you already know it given

that you have passed those other courses like calcul or usually they are covered as part of pre-algebra or algebra so the cartisian coordinate system I mean here

understanding what is for instance the the common description of them for instance when you when we write like X and then y on the vertical axis and then

we can we have here zero and then we can always plot this different plots you know we have a clear understanding what is this Y is equal to X line we

understand how by knowing certain points we can plot different plots for instance that this is the Y is equal to X line that here it means that if we have here

one then this is just one two this is two so we understand when we have the function of the line and we have a certain value at is our y coordinate or

x coordinate then the corresponding coordinate can be found then you also need to know some basic things that I just didn't mention right now so for

instance that the numbers here can be like 1 2 three up to Infinity so you understand this concepts of infinity and then here the same story then here we

have minus one you know minus two uh and then this is then used later on and we will be pouch basing this one we will be

describing our vectors and how we can visualize our vectors either two dimensional space like we have here because this is two dimensional so we have X and Y but we can also of course

visualize it in three-dimensional Etc so this idea of basic coordinate system is really important usually covered as part of algebra if not

pre-algebra then we have basic triog genetry which means that you need to have a clear understanding what sinus is what cosine is what tangent is and their reciprocals and here I mean that you

know for instance what is cosine function what is s function you know that you have an understanding for instance that what is this line you

know whether it's a sinus line or cosine line you have also an understanding what this Pi is one thing that I didn't mention but it it just goes around all

these topics some basic things that you understand what is X what is y why we use them and this idea of variables and also you need to

understand this idea of square or you know 90 degree angle and then Pythagoras Theorem here we have the same so what is this

relationship between different sides of a triangle that is a very unique triangle and that has one of the angles

as 90° and this idea of you know the sides how this relates to the sinus cosinus tangent cotangent and also how the Pythagorean Pythagorean theorem

applies when we have triangular but it is is no longer with angle that is 90° what is the sum of all the angles of triangle so those are basic stuff that

are com commonly covered as part of trigonometric lessons or part of General geometry then another prerequisite is

this understanding of identities and equations in triog genometric lessons something part of which I already covered and this is goes around of basic

having a basic un uh understanding of algebra and geometry those are super important to understand more Advanced Techniques from linear algebra then we

have finally this idea of orthogonality perpendicularity in vectors so this also comes from geometry and from a trigonometric lessons so you understand

that if we have for instance the two lines that don't have any intersections then we are talking about two orthogonal lines and otherwise for instance if we

have and the two lines like this then we are talking about perpendicular vectors when you have two lines that are actually parallel so they don't have any

intersection and you won't find any point that is common for the two so when it comes to this R so as part of real numbers and Vector spaces R represents

the set of all real numbers so you can be dealing with for instance an integers like 1 2 three this can Al this will also cover all the negative numbers like

-1 - 2 - 3 but also the floting numbers like 1. 223 and all the other numbers

like 1. 223 and all the other numbers that you can think of those are the set of all real numbers so this is in one dimensional

space right so you can see that I'm writing just one number you know two three and other numeric numbers then we have the idea of R2 R3 up to RN where

now all these numbers they represent represent in this case the N it represents the N dimensional aidian space so when it comes to this idea of n

dimensional numbers so for instance R2 here we just mean 2D plane so I'm pretty sure you are familiar with this idea of for instance

xais and Y AIS here we are dealing with two dimensional plane so for every point that we can find here we can describe

them by assigning them a value X so coordinate X and a coordinate y that's exactly what we mean by saying that the

number can be represented in a 2d plane so here we are dealing with this two dimensional space this is our two dimensional Elan space and every number

in here that is part of this R2 can be pictured here can be represented in this visualization so for instance if I have

this number and let's assume that the value on the x-axis is two and we can see here that the corresponding Y is zero I can describe this number which I

will call a I can describe this by writing down first the x coordinate which is two and then the y-coordinate which is zero so I'm then saying that a

which is a point with x coordinate 2 and y coordinate Z it is part of my R2 and it's part of my two dimensional

alian space when it comes to R3 similar thing we can do with that only in that case we need not just x axis and y AIS but we need to add our third

dimension so here for instance when it comes to the r Tre then we need to do y AIS we need to

have xaxis but also we need to have some Z axis so such that every time every point in the

space we can then describe by x y and Zed coordinates so if we write it in terms of the vector something that we will see

very soon as part of our first unit of this course we will then need to represent every number in this three-dimensional Alan Space by writing down first the x coordinate let's say

one and then y coordinate let's say another one and then Z coordinate which is one or even better even easier let's use 0 0 0 which means that we are

dealing with this initial number which is the center of this three-dimensional Alan space when it comes to the N dimensional or the higher dimensional

spaces it's much harder to visualize therefore usually when it comes to visualizations we do usually we usually only visualize the onedimensional two

dimensional and thre dimensional spaces above then it just no longer does make sense to visualize it but we definitely deal with them and they are part of

Applied linear algebra so understanding this spaces is very important for analyzing vectors for their interactions and this holds not

just for this two-dimensional and three-dimensional but really for multidimensional spaces let's now quickly Define this idea of Norm so the norm of a vector

denoted by this V which you can see kind of like similar to the absolute value from pre-algebra you can see here that we have this double straight lines like

from absolute value then we have the name of the vector or the variable name that we are assigning to our vector and then you might notice here on the top of

this this Arrow this basically says that we are deing not with just a variable but really we are dealing with a vector this is really important because you can

see that there makes a huge difference if we have for instance just V or V1 I have to say or just V those are really important and things that you need to

keep in mind when it comes to Leading your algebra and trying to differentiate vectors from a point you will notice that when it comes

to Norm we can represented it either by this notation or this usually it's a common notation in machine learning or

in data science with this two bars and when we do this we automatically also know that we are dealing with aladine distance we call it also L2 norm and

this is something very common and usually used as part of retrogression which is an application of linear algebra and it's used in

regularization so we are regularizing our machine learning algorithms so when you get into machine learning you will see time and time again this notation so next time when you see this then you

know automatically that you are dealing with L2 norm and L2 Norm which is also used a lot in machine learning it is

referring to the usage of L2 Norm to uh in the retrogression and retrogression or L2 regularization is a very popular

regularization techniques as part of machine learning so right now even you can see this intersection or linear

algebra or this idea of norms in machine learning all right so now let's see why we call it actually L2 Norm or often

referred as Eline distance so Eline distance you can see here which is also the in this case this V which describes

the norm of the vector v is equ Al to square roof and then we have all these coordinates assuming that the vector comes from an N dimensional space so you

can see here the RN the V Vector the idian distance or the norm of this vector v is equal to square roof and

then V1 2 plus vs2 S Plus and all this in between numbers plus VN squ so here basically it means take square root of

V1 2 V2 S Plus plus V3 2 blah blah blah plus VN s so basically take all the

units that form this vector and then so are on this vector and use them Square them and then add them and then take the

square root of that that's the distance or I have to say the norm of this Vector so why this is important this idea of norms and equity in distance beside of being used in machine learning and why

is it used so Norms they provide a way to measure the size or the length of a vector in Vector spaces which means that when we

want to measure a distance a similarity relationship between for instance vectors then it becomes much easier to use this idea an Elan distance is not

only used in regularization techniques like L2 regularization or retrogression but it's also used in other machine learning or deep learning Al items as a

way to measure the distance or the relationship or the similarity between two different entities those can be variables those can be two people that

we want to compare in our algorithm or two entities um for instance the Norms or the Al and distance they are also used as part of K me algorithm something

that you might have heard and if you follow later on the machine learning and the clustering section of machine learning you will see that Alan distance is used as part of C's algorithm that

aims to Cluster observations into different groups so this also yet another highly applicable uh topic that you must know in order to understand different linear algebra top topics but

also machine learning topics let's now talk about simple topic that we must know about and refresh our memory very quickly before moving forward to our next topic that is a prerequisite for

this course so the cartisian coordinate system is just a fancy word of describing this idea of X and Y or XY Z

when we just want to visualize them and showcase this numbers related to the space so we just learned the and I just

quickly was talking about this idea of X and and Y and how we can visualize that in plain so the cial coordinate system is a framework for specifying points in

a plane or a space using ordered list of numbers so we know for instance when we plot this then here we need to put X and

Y in our two dimensional space R2 and we know that here in the middle we have zero and here we have 1 2 three four and

the same here one two and then three four which means that everyone that is in the industry whether it's in mathematics in physics in data science

or ml or AI we all universally agree on this system we know this is this ordered list of numbers and we know that if we have for instance a point here then for

this point we know that the xaxis and Y AIS is definitely positive even if we know don't know the corresponding numbers and then once we have more

General lines here so not General but specific lines then we even know the exact coordinates and values here and we definitely know that this number should be so the x coordinate should be between

two and three so first we have the two and then tree and not the other way around so this ordered nature helps us to understand how we can put all these different numbers and organize them in

our two dimensional space and we also know the corresponding y so we know that for instance our Y is not minus three because it's lying in here in this part

of our coordinate system and not somewhere here where the y axis are negative and why do we know that because it's an ordered list of numbers that we

can visualize in this 2D plane and here you also need to keep in mind and we need to remind ourselves about this idea of these four different parts that we

got so we have our here the first part the second part the third part and then the four you know part of our coordinate system and here we we are dealing with a

two dimensional plane but if we were to deal with the three-dimensional plane we no longer have just x-axis and y axis where X AIS were on the horizontal and

y- axis on the vertical but we have our third line which is the Z so we have now three different dimensions so X Y and Z

and we are basically extending our two-dimensional plane to three-dimensional so this system is fundamental for visualizing and working with vectors geometrically so then we

can just use this two dimension uh plane in order to visualize this Vector for instance knowing what are all these points that appear on this Vector

what is its direction where is it headed you know what is the beginning and then we can also find out all the so the relationship of these vectors with all the other vectors for instance if we

have an other Vector here then we can use the coordinates of them and information about vectors to understand that we are dealing with two parallel vectors that don't have anything in

common so no intersection points where to say if we have another Vector like this and we know that here we are dealing with perpendicular you know orthogonal

vectors so this is why those this coordinates Cartesian coordinate system is important and it's not just important for linear algebra but just in general for mathematics and for data science and

for AI and you will see this coordinate system time and time again in different visualizations even when you want to visualize the mean of your data or you want to visualize the probability

distribution function describing your population from statistics or from data science you want to visualize for instance how your optimization is

working or you want to visualize how your model is performing in terms of its evaluation Matrix for all these cases and for any visualizations this idea of

the Cartesian coordinate system is going to become very handy let's now talk about this idea of angles and the idea of circles radian the pi as well as this

degree sign so this comes usually from geometry or tonometry and this is very important when it comes to the vectors because when we have two different

vectors then we want to understand their relationship do they form this less than 90° or so are we dealing with sharp corner sharp angle or with we are

dealing with 90° angle so we are dealing with this type of vectors where we have you know 90° or we are dealing with um

this type of vectors when the angle is 180° which is by the way uh something that we are referring as

Pi and here is one thing that is important here is that it's not just Pi but it's Pi radians why because in mathematics we also have this idea of Pi which is

usually a number that is 3.14 so we should not confuse this Pi with pi radians so the relationship between the two is something that we have also seen as part of our pre-algebra and algebra

courses so if it's something that you want to just refresh your memory on this will be super helpful to check our very initial course on um all these Basics so pre-algebra so this number comes from

per algebra and then this idea of P radians and just in general all this information about what is 180° what is this angle what is 360° and all the

information that comes from triogen metry and geometry can be found in our corresponding course so the next topic is the unit circle unit circle is highly

related to this idea of radians degrees cosine sign but also understanding the Cartesian coordinate system will help you to understand the unit circle so

this also comes from theog gometry and geometry and it's basically a fancy way of saying we have x-axis we have y AIS

we have here zero so our common Cartesian coordinate system only we are trying to focus on this part of the system where we have here one we have

here one so on the x-axis we have one and then here minus one here minus one for y AIS and here y the Y is equal to 1 so we have here all these points and

then we have the circle with the radius of one so here is this you know this is the radius and here we plot this circle and this will help us to understand this

concepts of sinus cosinus you know the Theta is just variable that we use to describe the angle and for instance here we are dealing with

45° this angle is 90° this entire thing is 360° and half of it so this part only is

180° so those are all important part of understanding this idea of unit circle so you might have already guessed that unit circle refers to this idea that we

have here one unit here one unit one unit one unit forming this entire circle so with the radius that is equal to one all right so this is something that

is very easy and this comes from the geometry and pre and triog gometry uh you also need to understand this concept of the sinus and and

cosinus and how sinus and cosinus are related to this what do we refer by the sinus and cosine you know what is this what are these points so 1 Z for

instance we understand that here the x is equal to one and Y is equal to zero so here this point is simply 1 Z so this

point and then we have 2 p radians so what is this idea of P so we know that a p Radian so P radians is simply the 180° which

means that you also need to understand this concept of P2 which is simply the 90° so you can see here one thing that I forgot to mention you need to understand

this concept the relationship between the pi and so Pi radians and radians and this unit circle you need to know that

here the pi ided two is simply this angle and then the entire Pi is this angle and then this entire thing the entire

angle with 360° is equal to 2 pi so 2 pi radians is simply this entire thing so those are

very easy Concepts that come from geometry and trometry and if you want to refresh them then head towards those sores because this will help you to

understand all this concept from scratch let's now continue our refreshment when it comes to so genometric identities and we just spoke about this unit circle we

talked about the sinus cosinus it's really important to relate this back to bit more advanced topics coming from the same do domain and from the same area of

mathematics and here we we need to know this concept before learning linear algebra few other things that um would be really great if you know but it's actually not a must to understand all

these different topics it is the idea of Pythagorean identity so don't confuse this with Pythagoras Theorem this is the Pagan identity so this one that the

square of the S of an angle plus the cosine squared is equal to one and all these different rules that go around the S and cosine and also the what is for

instance the S 2 Theta which is equal to 2 s of theta and cosine of theta you know those are all different rules that would be handed to know and if you are

so far I assume that you also know geometry and fundamentals to triogen ometry which means that you also know these trues but this might be just a great time to go ahead and quickly

refresh your memory on these Concepts because those might become handy in your applied linear algebra and Applied Mathematics Journey but for now I would

say this is not one of the most important things to know to learn this and to go through this course but just something to keep in mind so when it comes to the triog genometric

equations uh this can become very handy later on when we want to prove something in linear algebra so to follow along it's actually a good idea to know for

instance what is how you can solve this different equations and this will go back and refer to the unit circle that we just saw for instance if the sinus

Theta is equal to 1 / 2 then you will need to quickly remember what is that angle for which the sinus is equal to 1

/ 2 then you realize that is actually the angle where you take the p and remember that Pi is equal to

180° and that is the one corresponding to and then Pi / to 6 is simply 180 / to 6 so this is basically the 30 degree so

those are things that you can do when you know for instance all these different sinus and cosinus so you have memorized for these different angles so what is the sinus and cosinus for 30°

for 60° um let me actually remove this to make it easier so this type of problems is very easy to solve when we keep in

mind and we memorize what are these different values for sinus and cosinus when it comes to different angles for instance for the angle equal to zero let

me actually remove this and clean this part for better understanding so if we have for instance 0 degrees then we know that the sinus for this is zero and the

cosine of this is one so we are basically dealing so if I plot a unit circle we are dealing with this number

so remember that sinus and cosinus those refer to the Y and X on our unit circle so keep this one in mind

so if the cosine Theta is then equal to 1 and the sinus so Y is equal to Z we are dealing automatically this number with this number and you can see that

here the angle is also zero so here we are dealing with one and zero coordinate so this is our

cosine of zero angle and this is then our sinus of zero angle so we automatically even from this graph can

see very easy easily that the S of 0° is equal to 0 and the cosine is equal to 1 all right so let's quickly also refresh

our memory on few other degrees so for the 30° which is simply the Pi / to 6 so this is

30° then the sinus or the Y AIS is equal to 1 / 2 and the cosine or the X x value x coordinate is equal to square root of

3 2 so we are dealing with this this corner or angle so 30° so even from here you can see that the coordinates make sense make sense

then we have the pi for another famous value which is corresponding to the 45° it's simply this angle and for this

angle the X AIS which is the coine so this number is equal to 1 / to 2 and then for the sinus the so the y

coordinate is equal to 1 / 2 square root of 2 as you might have guessed because in this number the x-axis and y axis is equal to is the same so you can see that

this distance and this distance is the same because we are dealing with this type of figure so here we have 45° here we have 45° so this values are the same

and this is something that you would know knowing the pag Ian Pagan theorem so then you can go ahead and refresh your memory for the

60° so here I'm referring to the Pi / to three and then the 90° which is the very easy case this one obviously the x-axis

is equal to zero so here you should have zero and the y axis is equal to one so here you should have one and so on all right so we went into quite

detailed here but I think this is a very important topic knowing this idea of a trigonometric equations identities this idea of unit circle are super important

because they are highly applicable to different fields in artificial intelligence data science machine learning and will definitely set you apart all right let's now talk about the

law of signs and cosiness those are things that I won't be going on into too much details I just wanted to quickly showcase to you if you want to get the proof of those definitely check out our

corresponding courses but for here I'm assuming that you already know so you know the law of signs which means that if you have this triangle you know you

have this different sides so you have an angle a the corresponding side is a and then you have angle B corresponding side is B and then here C and the corresponding side is C then you know

that a / to sinus of that angle is equal to B / to the sign of that angle and then is equal to C divided to the sign of that angle so basically take this

value divide it to the sinus of this angle you know right in front of it is equal to taking this value and then dividing into the sinus of this angle so

the proof of this low is outside so out of the scope of this course but knowing this will help you to understand different concepts and then the law of

cosine is simply saying take the side of a Target angle so in our triangular we have here a we have here angle B and the

C and if we go and look into this specific angle so angle C just randomly picking one of the three angles then the side right in front of that angle so the

C c^ squ is equal to if we take this you know the other two sides forming that angle so A and B is equal to a s so this

is just a constant a distance of this side a 2 + b 2 so this side squar minus

2 * a * B times the cosine of that angle this is what we are referring as the law of cosin quite easy we are not going to prove it again if you want to get the

proofs make sure to check our other courses on the geometry and triog genetry we're almost done with the prerequisites just a quick refreshment we saw already the norm here is just a

not EX exle what Norm is and on a specific two dimensional Vector when we have for instance that a vector is equal to three and four which means for the First Dimension let's say on xaxis we

have three and then on Y axis is equal to four then the norm or the Alid distance so this is equal to we take the x value so three and then we Square it

so V you can see here this is the case when n is equal to 2 this is simply equal to square Ro of v1^2 + v2^ 2 and

as V1 is equal to 3 so this is our maybe I can make this just V1 and this is my V2 then the norm or the equan distance

for this Vector so this thing is equal to V1 2 + v2^ 2 which is equal to 3^ 2 +

4 S and this value is square root of 25 and it's equal to 5 so let's now see the difference between aladine distance and

the norm so you could see here the norm here we have just one vector like here and this Norm it has just two corresponding values into two

dimensional space you see here we have just three and then four so this is V1 and V2 when it comes to the Alan distance this is kind of the generalization of this idea of Norm so

the aladine distance between two points a and B in RN so in the N dimensional space is the norm of the vector

connecting a to B so we see that the norm and the elidan distance are highly related to each other only we are talking about the norm when it comes to

one vector but when we have this Vector a and the vector

B this is simply the Alan distance so for the Aline distance we know already this idea of distance how we can measure

it and you can see that this comes very similar to what we see here notation and here we are saying well we have this vector and then it has the two

coordinates in N is equal to 2 in two dimensional space when it comes to the Alan distance Alan distance helps you understand what is this distance between

two points in an N dimensional space so the aladan distance between two points let's say A and B in N dimensional space

is the norm of the vector connecting a to B so for instance if we have a point a and we have a point B we are connecting this and this is the vector

connecting these two points then the aladan distance is simply the norm of this Vector so this is the aladan

distance so we can see that nor and the distance they are highly related to each other in the Alan distance we are using this idea of norm and

specifically the norm two as I mentioned before so here you can see that the definition of aladine distance so the distance between A and B the two point

is equal to square root of A1 - B1 2 + a and then here we have basically A2 - b 2^ 2 and then plus A3 - B 3 squ those

are things that we cover as part of this dot dot dot and then plus up to the last point when we have a n minus BN 2 so here what we mean basically is that if

we have two points here is a and here's B and this s vector and we know all these

different points so A1 B1 A2 B2 A3 B3 blah blah blah and then here a n BN we know all these points lying here in this

distance then we are taking them and using them to calculate the line distance so here for instance if we have

point A and B so in this example let's do quick one specific example when we have a point a which has coordinates 1

and two so this is basically A1 A2 and then point B with points in it like B1 B2 you can notice that the da AB so the

distance or the Eid in distance of these two points which is equal to the norm of this vector or here this is a and this is B

and this is this Vector this is equal to Square < t of 4 - 1 so it takes the B1 so this is B1 and this is

A1 takes the square and then says plus B2 minus H ^ 2 takes the square root of that and says this equal to 5 now you

might be wondering but hey why do we do then instead of 1 - B1 2 we do B1 - A1 2 and the answer to this question lies in the uh properties that we learn as part

of prealgebra because it doesn't matter when we take A1 - B1 squ or B1 - A1 squared because this squared ensures that it doesn't matter which one we take

first and subtract the other now the proof of that is outside of the scope of this of course is this is part of pre-algebra but I just wanted to put this out there to ensure that you are

seeing what we are seeing here because here it says A1 minus B1 but in this example we are taking instead depth B1 and we are subtracting A1 this is a

common thing that we do in prealgebra and just in general in different cting distance or distance related cases so I just wanted to put this here to ensure

that later on this is something that can be clear from the first view right and in here we will quickly refresh our memory on the Pythagorean theorem which

basically says in the right angle triangle so if we have this type of triangle so here we have

90° this is a right angle triangle the square of the length of the the side opposite to the right angle so this side this we over refer C and this

as B uh and then a those two are not very important but this is commonly referred by C so the the side opposite to the right

angle then we know that the square of the C so c^ s is equal to a 2 + b^2 this is super important theorem and a fundamental principle for defining the

Norms the distances in equity and spaces in and in many other applications so the angles play Cru Ro in understanding the direction of the

vectors and you know how they can be measured in degrees or in radians we saw also the pi radian this idea of you know that the P radian is equal to 180° those

are all very important when it comes to linear algebra and just in general application of mathematics in machine learning in Ai and other applications the relationships between this angle

measurements and the triog genometric functions is foundational in solving different problems that are about these vectors and their

orientations for instance this angle of s cosine you know what is this idea of tangent they are very important just to

give you an idea the um uh Tang tangent is specifically used as part of the activation functions we call it tank activation function and knowing this tank will help you to understand the

activation functions that I use as part of deep learning which are more advanced machine learning type of models and they are fundamentals in all these different

new and Cutting Edge techniques large like large large language models Transformers encoder and decoder based algorithms Etc they're also important in

this idea of computing dot products so very important and must know when it comes to linear algebra so this is just an simple example when it comes to this

right angle triangle and Pythagorean theorem and how is applied I will skip this for now it's also important to understand this idea of orthogonality so

the two vectors let's say A and B they are orthogonal to each other if their dotproduct is zero so later on as part

of the vectors when we will talk about dot product we will see what we mean when we say that the dot product is equal to zero and here you can even see

that if the a norm if the a vector so you see here and B Vector if those vectors if we multiply them to each other their dot product is equal to zero

it means they are orthogonal so this angle that they form is equal to 90° orthogonality implies that the vectors from the from a right angle with each

other they are in you know we we are dealing with that in R2 in R Tre so they are super important when it comes also to visualizing them correctly

this concept is visually represented all this you know Vector a and then Vector B and they are perpendicular in the 2D uh coordinate

system all right so when it comes to the applications of orthogonality orthogonality plays a crucial role in various aspect of linear algebra it's fundamental in defining Vector spaces

subspaces in solving a systems of linear equation later on when we pass the vector ideas and we go on to the matrices solving linear systems so

equations with many unknowns and then we use this idea of reductions or gausian reductions we will see how this idea of orthogonality can be important and how

also it relates back to the norm of two vectors so it's fundamental in defining all these different identities and solving system of linear equations and

also orthogonal vectors are used in finding the shortest distance from a point to the plane um something that is important when it comes to the

optimizations and here you can see an example the vector a which is equal to 2 three and then Vector B which is equal to minus 3 and 2 you can see that when

we multiply 2 by minus 3 so we obtain basically the dot product by the way this is something that we are going to cover also as part of this course but for now you can see that if we take this

number we multiply with this so 2 * - 3 we take this number multiply with this so we take three and multiply with two you can see that this equal to minus 6

this is equal to 6 so - 6 + 6 is equal to zero so you can see that the dotproduct of these two vectors is simply equal to zero and this is what we

are referring as orthogonality this means that these two vectors form a right angle where we see here this angle

is equal to 90° why this prerequisites matter and why I meant those understanding this concept is very crucial they underpin this geometric interpretation of linear algebra they

will help you to better understand these Concepts and not just to memorize them but really understand and later on when you go into your machine learning and AI

journey and in your data science Journey seeing these Concepts will help you to better understand those different alori this optimization techniques what we

mean when we say we want our optimization algorithm to move towards local minimum Global minimum but this idea of movement this idea of vectors later on will you will also understand

this different concepts in deep learning how these models work how the neural networks work and those are essential Concepts that you need for solving

different systems of linear equation a core part of this course they also help you in visualizing vectors spaces which are critical to understand this concept

of linear algebra the applications of linear algebra when it comes to the real world applications so those are things that you can definitely must by

following some of our other courses but for this course I assume that you are already familiar with this Concepts right so now we are ready to actually begin and with this prerequisites in

mind you are prepared to start your linear arbra Journey we are going to learn everything in the most efficient way in such a way that you will learn the theory you are going to see many

examples we are going to learn everything in detail but at the same time you're going to learn the must know Concepts and I'm not going to overwhelm you with this most difficult concept

that you will not be seeing in your career I'm going to give you this bare minimum when it comes to really knowing and the must know for linear algebra

such that you will be ready to apply linear algebra in your professional Journey whether you want to get into machine learning deep learning artificial intelligence data science

knowing these different concepts in linear algebra you will be a pro in your field going to give you everything that you need the theory examples

implementations everything in detail but at the same time you will be doing that in the most efficient and time-saving

way so without further Ado let's get started let's Now quickly Define this idea of norm so the normal of a vector

denoted by this uh uh V which you can see kind of like similar to the absolute value from pre-algebra you can see here that we

have this double straight lines like from absolute value then we have the name of the vector or the variable name that we are assigning to our vector and

then you might notice here on the top of this this Arrow this basically says that we are dealing not with just a variable but really we are dealing with a vector

this is really important because you can see that there makes a huge difference if we have for instance just V or V1 I have to say or just V those are really

important and things that you need to keep in mind when it comes to linear algebra and trying to differentiate vectors from a point you will notice that when it comes

to Norm we can uh represented it either by this not ation or this usually it's a common um notation uh in machine

learning or in data science um with this uh two bars and um when we do this we automatically also know L2 norm and this

is something very common and uh usually used as part of um retrogression which is an application of um linear algebra uh and it's used in uh

regularization so we are regularizing our machine learning algorithms so when you get into machine learning you will see time and time again this um notation so uh next time when you see this then

you know automatically that you are dealing with L2 norm and L2 Norm which is also used a lot in machine learning it is referring to the usage of L2 Norm

to uh in the uh regression and regression or L2 regularization is a very popular regularization techniques as part of machine learning so right now

even you can see this uh intersection or linear algebra or um this uh idea of norms in machine learning the norm of this vector v is equal to square roof

and then V1 S Plus V2 squ plus and all this in between numbers plus VN squ so here basically it means take square root

of V1 squ then V2 squar plus V3 squ blah blah BL plus VN 2 so basically take all

the units that form this vector and then so are on this vector and use them Square them and then add them and then take the square root of that that's the

distance or I have to say the norm of this Vector we saw already the norm here is just a not example what Norm is in um on a specific two dimensional Vector

when we have for instance that the vector is equal to three and four which means for the First Dimension let's say on xaxis we have three and then on Y axis is equal to four then the norm or

the AL in distance so this is equal to we take the x value so three and then we Square it so V you can see here this is

the case when n is equal to 2 this is simply equal to square root of V1 2 + v2^ 2 and as V1 is equal to 3

so this is our maybe I can make this just V1 and this is my V2 then the norm or the in distance for this Vector so

this thing is equal to V1 2 + V2 s which is equal to 3^ 2 +

4^ 2 and this value is square root of 25 and it's equal to five so let's now see the difference between aladine distance and the norm so you you could see here

the norm here we have just one vector like here and this Norm it has just two corresponding values into two dimensional space you see here we have

just three and then four so this is V1 and V2 when it comes to the Alan distance this is kind of the generalization of this idea of Norm so

the Alan distance between two points A and B in RN so in the N dimensional space is the norm of the the vector

connecting a to B so we see that the norm and the elidan distance are highly related to each other only we are talking about the norm when it comes to

one vector but when we have this Vector a and the vector

B this is simply the Alan distance so for the Aline distance we know already this idea of distance how we can measure

it and you can see that this comes very similar to what we see here notation and here we are saying well we have this vector and then it has this two

coordinates in N is equal to two in two dimensional space when it comes to the AQ in distance Eline distance helps you understand what is this distance between

two points in an N dimensional space so the aladan distance between two points let's say A and B in n dimens space is

the norm of the vector connecting a to B so for instance if we have a point a and we have a point B we are connecting this and this is the vector connecting these

two points then the aladan distance is simply the norm of this Vector so this is the aladine distance so we can see

that the norm and the distance they are highly related to each other in the Alan distance where using this idea of norm and specifically the norm two as I

mentioned before so here you can see that the definition of alodine distance so the distance between A and B the two point is equal to square root of A1

minus B1 2qu plus a and then here we have basically A2 minus b 2 2ar and then plus A3 minus B3 2 those are things that

we cover as part of this dot dot dot and then plus up to the last point when we have a n minus bn^ 2 so here what we mean basically is that if we

have two points here is a and here is B and this s vector and we know all these

different points so A1 B1 A2 B2 A3 B3 blah blah blah and then here a n BN we know all these points lie in here in

this distance then we are taking them and using them to calculate the Lan distance so here for instance if we have

um point A and B so in this example let's do a quick one specific example when we have a point a which has coordinates 1 and two so this is

basically A1 A2 and then point B with u points in it like B1 B2 you can notice that the da AB so the distance or the

equid distance of these two points which which is equal to the norm of this um vector or here this is a and this is B

and this is this Vector this is equal to square root of 4 - 1 so it takes the B1 so this is B1 and this is

A1 takes the square and then says plus B2 - A2 2 takes the square root of that and says this equal to 5 now you might

be wondering but hey why do we do then instead of 1 - B1 2 we do B1 - A1 2 and the answer to this question lies in the um uh properties that we learn as part

of pre-algebra because it doesn't matter when we take uh A1 - B1 squ or B1 - A1 squ because this squared ensures that it doesn't matter which one we take first

and subtract the other now the proof of that is outside of the scope of this um course is this is part of pre-algebra but I just wanted to put this out there

to ensure that uh you are uh seeing what we are seeing here because here it says A1 minus B1 but in this example we are taking instead depth uh B1 and we are

subtracting A1 this is a common thing that we do in um pre-algebra and just in general uh in different um eling distance or distance related cases so I

just wanted to put this here to ensure that uh later on this is something that can be clear um from the first view why this is important this idea of norms and Al IND distance beside of being used in

machine learning and why is it used so Norms they provide a way to measure the size or the length of a vector in Vector spaces which means that when we want to

measure a distance a similarity a relationship between for instance vectors then it becomes much easier to use this idea an Alan distance is not

only used in regularization techniques like L2 regularization or retrogression but it's also used in other machine learning or deep learning algorithms as

a way to measure the distance or the relationship or the similarity between two different entities those can be variables those can be two people that

we want to compare in our algorithm or two entities um for instance the um Norms or the Al and distance they are also used as part of K algorithm

something that you might have heard and if you follow later on the machine learning and the clustering section of machine learning you will see that Elin distance is used as part of C's algorithm that aims to Cluster

observations into different groups so this is also yet another highly applicable uh topic that you must know in order to understand different linear algebra top topics but also machine

learning topics welcome to the course on the fundamentals of linear arbra my name is D Vasan and today we are going to start with some basic concepts that are important for understanding

linear algebra linear algebra is one of the most applicable areas of mathematics it is used by pure mathematicians that you will see in universities doing

research publishing research papers but also by the mathematically trained scientists of all disciplines this is really one of those areas in mathematics that you will see time and time again

appearing in your professional life if you want to become a job ready uh data scientist or you want to do some handson machine learning deep learning and AI

stuff but also linear algebra is used in cryptology it is used in cyber security and in many other areas of computer science and artificial intelligence so if you want to become

this well-rounded professional you want to go beyond using libraries and you want to truly understand the uh mathematics and the technical side of this different machine learning

algorithms from very basic was like linear regression to most complex ones coming from Deep learning like architectures in neural network how the optimization algorithms work how the

gradient descent works and all these other uh different methods and models then you are in the right place because you must know linear algebra such that you will understand these different

concepts from very basic ones to most advanced ones in the data science machine learning deep learning artificial intelligence data analytics

but also in many other applied science disciplines so before starting this comprehensive course that will give you everything that you need to know about

linear algebra first I'm going to tell you what we assume that you already know because linear algebra it comes from about third uh year of bachelors's um of

different uh highly technical studies and um here um we are assuming that you already know certain Concepts so uh to

ensure that this course Tes really on the topic of linear algebra and that you uh understand all these Concepts really

well for that we need to uh be able to know different topics so before we dive into this Concepts uh let's familiarize ourselves with the basic prerequisites

and notations used throughout this course and you will really need to know this in order to understand these Concepts really well such that instead of memorizing you will actually just

hear me once or maybe twice and then every time you hear later on or you see it in the papers or in some algorithms you will recognize ah this is something that we already

learned so uh some key prerequisites overview is here um first of all to fully grasp the upcoming material you should be familiar with some basic

concept like real numbers Vector spaces so you don't need to know this idea of vectors though you uh already most likely are familiar with this given that

you know how to plot different uh lines you know the idea of x's and y's and how to plot these different graphs but um here we are going to touch base on this

every time when we come close to this Concepts I will refresh you uh your memory and we will go through this numbers the idea of norms and distance

measures because when it comes to the vectors when it comes to the magnitude and all these different uh topics that we are going to discuss as part of linear algebra knowing the what Norm is

and um what is the definition of distance what is the length between uh two points when we plot it in the two-dimensional space or three-dimensional space those are all

very basic concept that usually you see as part of a basic pre-algebra or is uh common algebra ques and um lessons in

order to truly understand what the your algebra is about to understand the direction of vectors the angle and then um the uh dimensionality reduction how

linear algebra is applied for instance in different algorithms in machine learning deep learning data science statistics you really need to understand this Cartesian coordinate system so uh

this is not only important for linear algebra but I assume you already know it given that you have passed those um uh other courses like calculus or usually

they are covered as part of pre-algebra or algebra so the cartisian coordinate system I mean here understanding uh what is for instance the the common um

description of them for instance when you when we write like X and then y on the vertical axis and then we can uh we have here zero and um then uh we can

always PL this different plots you know we we have a clear understanding what is this um Y is equal to X line we understand how by knowing certain points

we can plot different plots for instance that this is the Y is equal to X line that here it means that if we have here one then this is just one two this is

two so we understand when we have the function of the line and we have a certain value where is our y coordinate or x coordinate then the corresponding

uh coordinate can be found then um you also need to know um some basic things that I just didn't mention uh right now so for instance that the numbers here

can be like 1 2 three up to Infinity so you understand this concepts of infinity and then here the same uh story then

here we have minus one you know minus two uh and then this is then used later on and we will be uh touch basing this

is when we will be describing our vectors and how uh we can visualize our vectors either two dimensional space like we have here because this is two dimensional so we have X and Y but we

can also of course visualize it in three-dimensional Etc so this idea of basic coordinate system is really important um usually

covered as part of algebra if not pre-algebra then we have basic triog genetry which means that you need to have a clear understanding what sinus is

what cosine is what tangent is and their reciprocals and here I mean uh that you know for instance um what is cosine function what is s function um you know

that you have an understanding for instance that um uh what is this line you know um whether it's a sinus line or cosine line you

have also an understanding what this Pi is um one thing that I didn't mention but it it just goes um around all these topics some basic things that you

understand what is X what is y why we uh use them and this idea of uh variables uh and also uh you need to

understand this idea of a square uh or you know a 90° uh angle and then uh Pythagoras Theorem here we have the same so what is

this relationship between different sides of the triangle uh that is a very unique triangle and that has one of the

uh angles as 90° um and uh this idea of um you know the sides how this relates to the sinus cosinus tangent cotangent

um and also um how the Pythagorean um Pythagorean theorem applies when we have uh a triangular but it is no longer with

a angle that is 90° what is the sum of all the angles of triangle so those are basic stuff that are com commonly

covered as part of uh trigonometric uh lessons or part of General geometry then another prerequisite um is

this uh understanding of uh identities and equations in triog genometric um lessons something part of which I already covered and this is goes around

of basic having a basic understanding of algebra and geometry those are super important to understand more Advanced Techniques uh from linear algebra then we have finally this idea

of orthogonality perpendicularity in vectors for instance if we have um the two lines like this then we are talking about uh perpendicular vectors when you

have two lines that are actually parallel so they don't have any intersection and you won't find any point that is common for the tube hi there so let's get started with our first module which is foundations of

vectors in this module we are going to talk about fundamentals of linear algebra vectors we are going to make a differentiation with between scalers and vectors we are going to Define them so

first we will learn the theory then we will Implement them into practice by plotting them by looking into different examples then we will look into this representation of vectors by looking

into the magnitude and the direction of it and the representation of them just in general we are going to plot them in our coordinate system then we are going

to see the common notational vectors and indexing of them vectors are super important when it comes to linear algebra and application of it and uh

they matter not only in mathematics but beyond so uh vectors help us in many ways from figuring out how objects move to solving math problems in science and

just in general in technology including in data science machine learning artificial intelligence Etc they are super useful tool so uh

let's start our journey with looking into scalers so scalers they are just plain numbers and by definition a scaler is a single numeric volume often

representing magnitude or quantity for example uh scalers can be describing um the temperature outside

for instance the temperature of um a 22° uh can be represented by a scaler or a height of a person can be represented

it's a scaler so let's assume we have a scaler that we will Define by a letter s it's just a variable this scaler is then

equal to 22 for instance and we are measuring it in degrees so it means that uh if this s measures a room temperature

then the scaler s which is equal to 22° which represents the room temperature it can be for instance 18° or 9° if it's very called uh it just

measures a single volume it represents just a single number or it can be for instance 17 100

2.22 so all these they are just scalers they represent a single numeric volume they often represent a magnitude or a quantity very we will see that scalers

they are a value that represent the magnitude of a vector so uh now when we are clear on this very basic concept of scalers let's

actually move to this idea of vectors so by definition a vector is an ordered array of numbers which can represent both magnitude and direction in space so

uh vectors they are bit more they represent bit more than scalers there are numbers that also show show direction like a car spitting down the

highway or a bow uh being thrown for instance uh when it comes to our previous example we were using this

uh uh room temperature as a way to uh think about the scaler a scaler for instance scaler that we just saw was

this room temperature room temperature which was 22° when it comes to the vector Vector is different for Vector for

instance we can have an example when a bird for instance bird it flies

flies at 10 kilomet per hour and I also add here another information which will make this as a

vector which is that it flies South so here as you can see what I'm doing is that I'm not just oh let me

actually remove this part to make it easier to understand okay so uh in this example let me write it down that the

example bird FES s at 10 kilomet per hour so you can see

that I'm not just adding the scaler which is in this case the magnitude we will see very soon the formal definition of it so I'm writing

down the speed I'm defining the speed but also the direction so I'm saying I know that the bird is flying south that's the direction and I know

also the speed of it which is the magnitude so 10 kilomet per hour so here in the vector I have much more information than in the scaler because

in the scaler I just got temperature room temperature single volue but in case of a vector I not only have um magnitude or speed like 10 kilm per hour

but I have extra information which is the direction of it for instance flying to the South so let's now look into some real examples and plotting them to make

more sense out of this idea of vectors and what is this magnitude what is the direction so let's assume we have a 2d

plane so we have xaxis we have y AIS here like usual we have our z0 Center and we want to plot a simple Vector so

uh usually the way we represent Vector in tutorials or just writing down is by writing the name of the vector this can be just a a random name let's assume

that it's a v letter v and then on the top we are always adding this Arrow so this Arrow it says and it tells the person who is reading that we are

dealing with the vector arrow on the top is that reference so let's assume this uh vector v it starts from the center of

our coordinate system and it goes to this point so let's say in here this is our vector v

so let's assume that this point in here is equal to 4 which means that the x coordinate is four and the y-coordinate

is zero as the um uh Arrow it just as the point in here it has a a y value of zero so you can see that it goes

straight from zero to this one to this point okay so what tells this Vector uh to us is that we have a value that

describes the length of the vector so it goes from 0 to 4 which means that the length is equal to unit four so it's equal to

four um and we have just learned and we were just talking about that the magnitude is the length in this case so the length

describes the magnitude in this case so this means that the magnit ude of this Vector is equal to 4 and then um what else we can see here we can see the

direction of the vector which means that the direction is also something that we can see here this is the direction of the vector so this going straight from

this point to this point in a horizontal way so independent whether I plot this

Vector from 0 to 4 in here or in here here or in here or in here or in here in all cases as long as the length is this

I'm dealing with the same Vector because I am basically in this entire R2 space I have exactly the same Vector

all I care is about the magnitude and the Direction Where will this Vector start and where will it end I am not interested I'm interested

that the uh that the magnitude in this case the length is equal to the direction of the vector so let's now look into another example where we go a bit more difficult on our coordinates

and on our Vector we already saw that we had this Vector where we went let me change the color so this was our vector

v and it went from zero till 4 so this point to be more specific is so this Vector it goes the vector B it goes from

0 0 to 40 so the coordinate X was 4 and the Y was Zero now let's plot another one um

where the direction is no longer horizontal for this Vector let's call it Vector W and for this Vector w we will again start with Z 0 so we will start again in

here but this time we will go bit like this so let's say we go all the way to this

point so this point has a value for an x- axis of three and for y axis it has a value of

four which means it goes from this point to this point and this is the direction

of our vector v so it goes to 34 because this point is 3 0

and this point is 04 so xaxis is 0o x coordinate and y coordinate is 4 so now you can see that the direction of this

Vector is like this while the direction of the vector v was like this and like in case of vector v i

again no longer care about where exactly my Vector W stars and ends but all I care is about its magnitude so the length and the direction so for

instance I can have the same Vector in here the same Vector in here as long as the length the magnitude is the same and the direction I am

dealing with the same Vector that's all I care so the magnitude and the direction is all that you care about all right so now about the length um that's

uh something that you can see very easily from this specific example because by using the Pythagoras Theorem or P Pythagorean theorem we can see very

quickly that as the length of this side of our uh right angle 30° so right triangle we can see that

this side is three this side is four which means that this side is 5 because 4 2 + 3^ 2 then we take the

square root of that square root of 25 and it's equal to 5 so the length or the magnitude of this vector v is simply equal to

5 all right this was about this uh specific vectors let's now look into the uh common representation of the vectors so we always use the magnitude as well

as the direction you know to represent the vectors and they commonly are represented by two different uh ways let's now look into the first way that the vectors can be represented and then

we will move on to the next one so when it comes to the vector v so we saw that vector v was moving from

0 till uh to the point of 40 so we can represent the vector B by 4 and

zero when it comes to the vector w we can represent that uh Vector so Vector w we can again do the parenthesis and we

can say that it's equal to 3 four so by using the coordinates from the coordinate system we can then represent our uh vectors so this is just one way

of representing a vector another way of representing these vectors is by using this Square braces given that we are in a two dimensional space first we will

mention here the four then we will mention the zero in here twoo so we can say three and four this is yet another

way of represented the vectors in a two dimensional space so if we were to have a threedimensional space so let me

actually show it on a new page so if we were to um if we were um to have vectors in three dimensional space so we are dealing with

R3 so we have points that can be described by X Y and Z so coordinate space like this so X and

and the Y and then the Z then every point so let's say we have this Vector then we had to represent it by a value

let's say x uh X1 y1 and Z1 or um better let me actually use a different

letters a b and c and this would be my vector v and I could also represent this Vector

B is the same so vector v can be represented as a b and c so one thing that you can notice is that unlike the

R2 now I have three different entries what we are also referring as rows and we just got one column so um we can uh

often represent and usually that's a common way of representing vectors by using this um columns so columns help us

to represent our vectors and you can see very clearly then when it comes to the two dimensional space so when we have

R2 so then our vectors have just two rows so three and four four zero like in here when it comes to three dimensional space we have three entries and so on so

the same holds of course also for for instance R5 then for R5 um our vectors so coordinate space can be for instance

x y Zed and then GMA and then let's say Delta and then the coordinates uh of a vector in that space can be V and then

arrow is equals sh and then we would have uh let's say A B C D E you get the idea so depending on the space the coordinate space and the dimension of

that space then the corresponding vectors can be represented accordingly so the vectors are quantities that have both magnitude and direction as we just

so distinguishing them from scalers which only have magnitude so we saw that the scalers got only magnitude while in case of vectors we saw both for the

vector v and for the vector w we didn't we didn't only have the magnitude so the length of the vector but also the corresponding

Direction so uh when it comes to the um vectors so the this is exactly what we just saw in our example a vector in a

two dimensional space so in R2 uh can be represented by using this Square braces and the corresponding entries for X and Y where X is basically

the x coordinate in our coordinate system so in our X and Y system whenever you have this x and y

coordinate then uh this x coordinate Will then describe your magnitude and the y coordinate Will then describe your second entry that you need to put when

representing your vectors so here the X and Y indicate the movement in the horizontal and in the vertical Dimensions respectively so for

X's it's always the x coordinate so how far you move towards the horizontal Direction in here in here or independent in here so always take the x coordinate

that is the value that you need to put first and then the Y need to be put it in here so indexing in vectors when it comes to the um indexing the standard

mathematical notation uh indices in the N vectors goes from I is equal to 1 to I is equal to n so the um notation here

can be bit ambiguous so AI uh could mean the E element of AI uh the a vector or the each Vector in a collection so let's start with a simple one and then move

move on to this next part so what this means and what this means we will look into now so uh usually uh when we have a

um n dimensional space we are having hard time visualizing it therefore we use this two dimensional space or maximum three-dimensional space in order to get an understanding of what these

vectors are so we just s examples of them uh when uh creating our vectors in

um V and V uh and W in uh R2 and also in R3 but we can have similar vectors also in R4 in

R5 or all the way down to RN where n can be 100 200 500 any number as large as you want the thing is is that

visualizing R 4 R5 RN is very hard but we can still benefit from this great properties of the vectors metrices and in general linear algebra in order to

describe different things that have more than three dimensions therefore we have this a bit more ambiguous notation where

we use r n and this n can be any real number and it can be all the way to Infinity so very large number and uh let's say we have a vector

in this RN then this Vector is usually described by using similar Square uh brackets like before only with uh more

entries so like before we got just one column so that's something that we didn't uh change but here we have instead of just two entries or three entries like in the two dimensional or

three dimensional spaces now we have A1 A2 A3 all the way down to a n minus one

and a n so we got in total n elements in our column and this describes our uh single Vector so this Vector in an N

dimensional space this we can call also a so one thing that we just saw is that it was saying in our definition and notation that uh we might also be

dealing with the E Vector in a collection which means that sometimes you will see this while here the A1 A2 they are

vector themselves so here we saw that these are just entries so A1 is a number A2 is a number A3 is a number a n is just a number but it's

also possible uh when you have a much more difficult and complicated case that you got an A let's write it down with a capital

letter A which is equal to A1 or let's actually remove this so we got let's say

A1 A2 A3 all the way down to a n minus one and a

n where you can already see what is going on so instead of having just a number as an entries instead we have vectors in here so our first element is

actually Vector our second element is actually Vector so A2 Arrow A3 Arrow all the way down to a n arrow so while here this can be for instance some numbers

let's say one one one all the way down to one one here we have a vector vector another vector and all the way down here yet

another Vector where for instance let me remove this part where for

instance A1 arrow is actually equal to

A1 1 A1 2 A1 3 all the way down to A1 n one thing that you will notice here is that unlike in

here here I got double indices so I got here a11 and then A1 2 and then a13 all

the way to A1 n so the first index it doesn't change as I have here a one so I'm writing down the index corresponding

to this Vector but the second index it changes per entry indicating which element specifically in the vector I'm talking about so from the first index

you can identify the vector that I'm referring to which is A1 and from the second index you can see the

corresponding um entry or the volume you that that um element is positioned in this Vector so you can see that this

values for instance in the um Vector one so A1 to be more specific but then it is in the first position this is in the second position in the third position

all the way down to the end position so this is something that is really important to understand well because this notation is going to appear time

and time again across various applications of matrices and vectors so is really important to understand well therefore I want to go one more time

through this to make sure that we are clear on what this indexes represent so whenever we have an index uh an a vector

that we want to uh represent and it's um it has just um it is just a vector which means that it's not a nested vector

vector in a vector then um we can Define it by let's say a and then on top an array and it's equal to and here we can

have A1 A2 all the way down to a n so you can see what we are also referring

as dimension of this Vector is equal to n by 1 so I got n entries and just one column so n by

one which means that this already gives me an indication that most likely this A1 is a number this A2 is a number this a and is a number so let's say this

equal to 1 2 uh three blah blah blah and then here I have let's say 100 but if I'm dealing with the nested

Vector later we will see that this can be represented by a matrix then um I can also Define

this by capital letter A which is a common way to refer to either matrices or nested vectors and then this

is equal to A1 Arrow A2 Arrow A3 Arrow this already sends a message to the reader that we are dealing with no

longer U constants within a vector but rather vectors in a vector and uh what can we see here is that the dimension of

this nested vector or which we can also refer to as a matrix here the number of rows so the number of entries this

elements we can see it's equal to n but then this time the number of values that form these vectors is no longer one because we are not dealing with just a

constant this is not some constant but rather this is yet another Vector so let's assume this Vector has a length of

M so let's say this has a length of M then the dimension of this Matrix a is equal to M so something that we will see

also when talking about matrices so let me actually clarify this bit more for better understanding let's

say we look into one of those um one uh one other example of an entry so let's say we look into this specific Vector

which is in the uh the third uh vector within this Vector capital A so this A3

Vector so one thing to see here already is that I assumed that these vectors they got M elements and keep in mind that all these vectors they should be of

the same size so it means that I already know that this specific Vector A3 has M

elements so m elements so I'm representing this uh A3 Vector from here I'm taking this out from this entire uh

nested a vector and I just want to represent this and now unlike this elements that got an arrow on the top

this time I will have uh constants forming the A3 Vector so I no longer have vectors but I have elements in it

so in here I will have a a let me actually write down all the A's but to refer and to make sure that I recognize that I'm dealing with the

third a vector so a Tre Arrow here I will put three Tre all the way here Tre so they all come from the same third A3 Vector but then their positions is

different because this is let's say uh one two and then all the way down to Ed position

so this indices help us to keep track what are the um position that these

values are taking part in the vector A3 errow this might seem bit complicated at the moment but once we move on onto bit more complex material like uh matrices

it will make much more sense this is bit of an extra I just wanted to Showcase this but this is what uh is at its core and what you need to uh understand at

the moment to understand this concept of vectors so you need to know that vectors can be represented by this arrow on the top so let's say Vector a and it has let's say n elements then you can write

the square brackets and then you will need to mention A1 A2 all the way to a n which means that you have n different entries describing your vector so you have A1 which is the first element in

your vector A2 the second element all the way to a n which is the end element where here you can see for instance so if I had here A3 that uh A1 is simply

equal to one A2 is equal to 2 A3 is equal to 3 all the way to a n is equal to 100 so this numbers I'm basically

taking and I'm representing them I'm putting them in here within Square braces in order to get a representation of my Vector so my Vector a has all these different entries and different

entries and it starts with one and it ends with 100 this is a vector and then when it comes to the vectors within vectors here we need to be a bit more

careful CU here we not just have uh constant values forming a vector but we have vectors that form yet not vectors

so our Vector a our nested Vector a which we uh later will refer as Matrix a has actually entries that also are

vectors so we have a 1 Vector A2 Vector A3 Vector they are not just constants but only own they are vectors so here for instance we have defined also an

example of it we have said let's look into this third specific Vector that is part of a which is A3 uh vector and uh

that one has M different elements here we have then the index referring to the which Vector from the nested Vector a it is which is the third

one because we have taken it from here but then on its own this Vector has different members and different members to be more specific therefore we have also an index to keep track of the

position of this value one to up to M and this can be yet another uh this time it can contain some elements an example

of which is for instance Z 1 2 all the way to let's say 500 and this can be different numbers it doesn't need to be ordered it doesn't need to have a

specific pattern they can be just random numbers describing this A3 Vector so hopefully this makes sense if it doesn't don't worry because we are going to see

this time and time again I just wanted to give you a brief of an intro such that you can uh remember this when we

come uh back to bit more uh complex topics like uh indexing in matrices so now let's talk about special vectors and operation

here we are going to talk about zero vectors unit vectors the concept of sparcity in vectors as well as vectors in higher Dimensions like we just saw about this n dimensional space we will

also talk about different operations we can apply when it comes to vectors like uh addition subtraction and then later on in the next module we will also talk

about multiplication we will also be looking into the properties of vector addition after we have looked into some detailed examples when it comes to operations on

vectors all right so let's start with the zero vectors and unit vectors when it comes to zero vectors you can see here already that um the zero and arrow

on the top it basically refers to the vector like we saw before only with the difference that all its members are zero

so you can see here that we have zero and then an arrow and then underneath here we have some number tree and then this is described by this common

representation with the square braces and then three different members z0 0 so all zero and then it says in R

Tre okay so why are we doing this well uh when it comes to uh different linear Lal operation sometimes we just need to add zero vectors or we just want to

create zero vectors it's just easier to work with you we want to uh just create an empty uh Vector we want we know the length but we want to keep it empty such

Laton we can add something on the top or knowing that when we add a zero on a number the number stays the same we can make use of this property to uh do

different um uh tricks when it comes to programming in Python in SCAR or in C++ Etc so therefore this idea of zero vectors can become very handy now one

thing that you need to notice here is that we are not just writing down this zero to emphasize we are dealing with the vector but like uh before we have

this error on the top emphasizing that we are dealing with a vector then what we are doing is that we are also adding

the dimension of this Vector so in what dimension in what space are we um uh creating this zero Vector that this

Vector is located is it in r R 2 in RN in R3 in this specific case you can see that in this example the uh index that we got here is three which basically

indicates we are dealing with a zero Vector in threedimensional space so in the R3 uh in general we would just note this

by n keeping the uh notation general which means that we are dealing with 0 0 all the way down to zero so it has n one

dimension in r n all right so this is about zero vectors it is just a way to uh make our programming life easier also to use it

in different uh algorithms when it comes to bit more advanced algebra uh the next type of special vectors that we will look into is this

unit vectors so vectors with a single element equal to one and all the others zero denoted as EI for the E unit Vector

in N dimensions are referred by unit vectors so uh what we mean here when it comes to the unit

vectors um if we have for instance E1 it means that we have a vector where the e in this case the first element is equal

to one so you can see that E1 is equal to 1 0 0 so in the first element we got one and the remaining is zero and this is really important that we are dealing

with vectors that contain only elements of zeros and ones and the only member that is equal to the only element in that Vector that is equal to one is the

E element in the entire Vector all the remaining ones are zero and you can see here that the dimension is no longer

specified but just the um index of the entry where the um uh the uh one is located so let's look at another example

in here for instance when it comes to the um uh unit Vector yet another unit Vector is E2 which basically means that

in the second element so in the second place uh the uh Vector contains one and all the other members are zero so here you can see Zero here it can see Zero

only in the second element we have one and then in the E3 what we have here is that the third element is one and all the other ones are zero so let's

actually look into uh one um bigger Vector uh in higher Dimension to make it even more sense so first I will Define and assume that we are dealing with a

vector in RN so in an N dimensional space this gives me an idea that we are dealing with um so we are not dealing with nested Vector we are dealing with a

simple and dimensional Vector so it has n rows and one column so using the square braces I'm going to represent my

Vector so I have all these different members n members C so e let's say it is

E5 so what does this mean it means that I is equal to 5 and this I element so the fifth element is equal to one and all the other entries the elements in

this Vector are zeros so let's look into this is z0 0 Z I'm approaching the fifth element

in my Vector so it's this one this is one and the remaining all zeros so this is a unit Vector in an N dimensional

space and I'm defining it by E5 because my fifth element is equal to one now those are very handy when it

comes to some other uh techniques in linear algebra and just in general think about techniques like um uh row etum form solving linear equation something

that we will see as part of the next unit so many things um we can do by using unit vectors unit vectors are super important so you need to

understand this concept uh very well such that later on you will understand uh more advanced concepts in linear algebra let now look into the topic of

sparsity in vectors so by definition a sparse Vector is characterized by having many of its entries as zero so its parity pattern indicates the position of

a nonzero entries so uh what we are basically saying is that if we are dealing with a vector that contains too many zeros we

are dealing with the sparse Vector so uh this sparsity pattern indicates uh all Al the positions of a nonzero elements

so um if we have um unit Vector it means that we are already dealing with a sparse uh Vector this is a concept that is super important when it comes to

linear algebra but also in general data science machine learning and AI because having a spity in your vector it means that you don't have much of an

information usually a value zero it means you don't know much about that specific volum and if you got just too many of zeros and too few numbers which

do um provide information it means that you are dealing with a vector that doesn't provide you much information and there's always a problem when it comes to data science machine learning and AI

so sparcity is something that you need to be aware of you need to know how to recognize it and you also need to know whe there's a problem in your specific case or not so let's look into an

example let's say we are dealing with this Vector X that has five different elements so X is a vector coming from um

five dimensional space so we have for instance an element of three the first entry then we have z0 in the second and third uh entries then we have an entry

um four which coincident also contains value four and then the last element in our five dimensional Vector X is equal to zero now what do we see here we see that the

majority of elements of a vector X is equal to zero because we got in total five elements and then we got three of it

actually uh being equal to zero and only two of them containing information like equal to three and four so only two elements that are not zero so non-zero

elements it means that 3 / to 4 which is basically 60% 60% of all the entries in

the vector X are equal to zero so the 60% it means that is above half so above 50% 60% of all the information in this

Vector um the majority is simply equal to zero this type of vectors we are uh calling sparse vectors and sparcity is really important concept uh that we need

to keep in mind later on so uh while we can visualize vectors in two and three dimensions in linear algebra like we just saw in case of this

n dimensional vectors visualizing uh the this type of higher dimensional vectors becomes very difficult so uh this mathematical

flexibility uh to work with uh this type of uh information so when we can represent uh information many with many entries we can represent it by vector s

which we can actually not visualize becomes very handy for complex data structures for different simulations in physics and much more so uh we just saw

in couple of examples uh how we can represent vectors in a high dimensional space using this Square braces and this common Vector notation representation we

saw that in an N dimensional space we could uh very easily represent this uh very large Matrix or vectors uh by just um using this Vector

representation for instance if we got a vector that had any different entries where n is for instance thousand so let's say we have thousand then uh we can

represent uh this uh vector or this information by using common Vector notation so A1 A2 all the way to a th000

so of course we cannot visualize this it just doesn't make sense we can visualize two dimensional vectors we can visualize three dimensional vectors but we cannot

uh visualize thousand dimensional vectors so Vector that comes from uh r, but what we can do is still make use

of this very useful information in order to uh do different operations when and later on we will see that uh this property and specifically

this part of linear algebra it helps us to work with vectors in any number of Dimensions whether thousands million billions this mathematical flexibility

is super important for more complex data structures uh for metrix multiplications when for instance we are doing different uh algorithms including how we can

represent uh very large matrices very large feature spaces all this different information we can represent just by making use of

vectors coming from this specific uh part of linear algebra let's now finish of this module by looking into some applications of

vectors so one common application of making use of vectors is uh when we are performing different operations while having words and we want to count those

words so this is a super common application of vectors and we can account this words and you can even plot a histogram over how often each of these

words appear in a document so a vector of a length n for instance can represent the number of times each of these words in a dictionary of n words appears in a

document so uh just for the sake of Simplicity let's assume that we got um dictionary that contains only three words of course in the reality uh the um

dictionary what we also refer often as Corpus it contains much many much more many words but for the Simplicity we will assume that we just got three different words in our dictionary so

that's a total now let's assume that we got a document uh with these different words and we want to count how many times each of those words that we got in

dictionary actually appear in our document so uh if our document is described by this Vector so it contains

an entry of 25 2 and zero it means that in our our document we got

25 word one in now from our dictionary so in the position one two * word two

and zero * word three so basically we have a predetermined set of words in our

dictionary in this case three words word one word two and word three and they have a specific IND specific position in our vector and when we are putting these

values in here then the machine or the uh computer the program will understand that if we have 25 in the first position

then the word one in the dictionary appeared 25 times in our document whereas the second word appeared only two times and the last word for three

didn't appear at all so zero times in the entire document so let's look into a practical example actually to make even more

sense so um this is by the way a common practice to count different variations of a word there are common application in engrams large language models Transformers they are just the

Cornerstone of many language models when we want to count the words in the document to understand how often the word appears because this gives us a

idea what this document is about knowing how many times the same word appears in that uh document it gives us an indication of the topic of uh the do

document also we can make use of a to do sentiment analysis to understand what this document is about not only in terms of the topic but also is it a positive

is it a natural or a negative uh document so to say so uh for instance if we got uh the following words uh that

correspond to our dictionary and in our dictionary we got just um let's say 6 different words then what we can do is

that we can say 3 2 1 let's say Zer 4 two and the corresponding words are word row

[Music] number horse is and then document what this means is that we have

a text what we refer as a document that contains three times the word word that contains two time the word row contains one time the word number zero

times the word horse and four times the word eel and two times the word document so uh this is basically a

common way represent presenting the uh frequency of the words in the document let me actually give you uh another example

and in here I want to emphasize another thing the concept of stop words so uh let's say I make

this 10 and then here I say there is a three times the word I

two times the word uh reading two * the word library four times the word

book 0er * the word shower and 10 times the word

uh so uh you can see a that in here we are dealing with the document that contains 10 times the word uh which is what something that we refer as a stop

word so those are things that actually don't give us too much information about what the document is about because uh it's just used everywhere but it is

appearing too often so you can see 10 times the most frequently appearing word this is what we refer as a stop word and then another thing that we can observe

the second thing we can observe is that we are dealing most like ly with a document that describes library reading uh because you see the words like reading you see the word like book

library but another word shower that is totally unrelated to reading book or library is appearing zero times so even by looking at discounts we can already

get an idea what a topic of this document is about so uh you can see already know from this very basic example where I made too many

assumptions regarding how small the the uh dictionary should be uh you can even see now how we can use discounts in our

dictionary from our text in order to get idea about the topic of the document or topic of the conversation it can be topic of the uh tweets if you have a

tweet data it can be topic uh regarding book if you have many book um uh book text it can be for instance the topic of

the review if you got a reviews from uh Amazon for instance using this count can help you to get a topic regarding topic

from that text then you can also use it to remove the stop wordss because usually the stop words are the most frequently P words it can also give you an idea about the sentiment for instance

here we are dealing with natural sentiment it's not positive it's not negative it's just reading a book in library that kind of topic so all this can be super helpful when it comes to

natural language processing that's a field where this uh text processing text cing and then using that for modeling purposes is what uh what plays a central

role it also plays a super important role in the large language models in the Transformer models and uh in simple

matters like uh back of words or uh in the uh TF IDF all these they are based on this idea of counting words and how

we can use it information and you can see how vectors come into play in the different applications of linear algebra in data science natural language

processing in artificial intelligence in machine learning so they are super important another application of vectors

can be representing customer purchases for example an N Vector P so let's say p can record a customer purchases over

time with pi being the quantity or dollar value of an item I now what does this mean so let's say we have Vector P that represents the customer purchases

and we are dealing with a single customer and we are just saving over time that information how many time this customer has made purchases over

time so the quantity is in the um dollars so the dollar value of item I purchase so we are basically keeping

track of uh what is the value of the item I that the customer has purchased so what we can

do is we can assume that in here actually it already Mak that assumption it says n Vector which means that the

number of rows or number of um items that the customer purchases is n now what the um the problem says that

it represents is that in each entry and here we have in total n entries we got a dollar value of item I

which means that here if I have P1 P2 all the way to PN and here somewhere in the middle I got Pi in the East position

it means p Pi represents the value of item I so for example if I'm dealing

with a customer that buys um let's say uh courses and uh the first item that the

customer is buying is a mathematics course so I'm writing mathematics course and this is the first

course that it buys e is by the way just a um way to refer to the E purchase so let's say um here somewhere in the

middle the um customer decides to buy a deep learning course deep learning

learning course and then it continues buying uh the customer continues buying courses and the last course that a customer buys

is let's say um career coaching course now let's say the mathematics course

costs uh around $1,000 let's say the uh deep learning course costs $33,000 and then let's say the career

coaching service which is usually one of the most apply and personalized one can cost all the way to $5,000 now we see that in the each

position this is the East position let me change the color by the way so let's say this is the East position this is the first position and this is the last position so those are

just indices we can see that in the E position we got the 3,000 which means that the p e is equal

equal to $3,000 so this indicates that in the East purchase the customer purchased deep learning course and the value of that

item was equal to $3,000 all right so now we are ready to go on to next major topic which is about vector addition and subtraction so we

are going to do some operations and apply this operations two vectors so let's first formally Define this ideal of vector addition so uh two vectors of

the same size are added by adding their corresponding elements the result is a vector of the same size so uh let's

unpack this it says two vectors of the same size are added by their corresponding elements so here it refers

to two different vectors let's say vector v and Vector W and it says let's add them what we refer as vector

addition and says for that what we need to do is to take all the elements of v and then all the elements of w and using

their corresponding elements so indices that helps us to understand where those elements are located we are using in order to add each

element in the vector v to the element of the vector W in the same position and do note that in the second part it says

the result is a vector of the same size because we are adding two different vectors of the same size it's mentioning here it means if we add two different

vectors to the same uh that have the same size we are going to end up with a vector that has the same size now once I go into the examples it

will make much more per let's quickly also look into this concept of substraction so on its own uh substraction is very similar to this idea of addition so if we have a

substraction let's say we have vector v We substract Vector W then we are doing basically uh what we just did to the

addition only instead of uh doing add we are doing subtract so again we are just uh we are just subtracting from vector v

Vector W they have the same size so we end up having the result which is a vector of the same size only one thing

that you can see is that this can be also written as V Vector plus and then minus W so we basically can represent

subtraction um on its own as a way of adding only we take the negative so the um opposite directed Vector so this will

make even much more uh once we go on to the examples so let's look into our first operation example where we are adding two different vectors this a basic example

we got just two dimensional two vectors we got Vector a that has entries two three and Vector B that has entries one form and what we are doing is that we

are adding Vector a to Vector B we just learned that a we need to have the same size of vectors so you can see that Vector a has a dimension 2 by one vector

B has a dimension of 2 by one so their sizes is the same both they got two entries only two elements and at the same time we just

learned that what we need to do is to take their corresponding elements and add them to each other now what does

this mean it means that we take from a the first element two and then we take the first element

of the second Vector which is the B so we take the two from here and one from here the first element of a and the first element of B and then we are

adding them to each other 2 + 1 is equal to three and then the same holds for the second and Tre so three which is the second element of vector a and then four

which is the second element of vector B we are saying 3 + 4 is = to 7 so let me write it down even in a simpler manner such that it will make much more sense

so Vector a has elements 2 three in the first element we got two in the second element we got three so a then we want

to add B which has in the first element element equal to 1 and the second element is equal to 4 this means that if we want to add these vectors 2

3+ 1 4 this is equal to we need to take two we need to add one so this element and this element and then we need to

take three we need to add to four so this one and this one which is equal to 2 + 1 is equal to 3 3 + 4 is equal to 7

so we got uh Vector 37 do you note that this Vector the result Vector contains again two

elements and just one column so 2 by 1 so you notice that the sign that the size is the same of this result Vector now let's actually generalize

this concept before moving on to the next example so if we got let's say Vector

a that contains n elements A1 A2 all the way down to a n and it is

from n dimensional space and we got Vector B that also has n elements so remember

that they both need to have the same size so B1 B2 all the way to b

n so they come also so B comes also from n dimensional space so then when we add a to B this is

equal to A1 A2 all the way to a n

plus B1 B2 all the way to BN so n by one n by one the sizes this

is equal to let me actually use this color to make it even more visible so I for the

first entry for my result factor I will get A1 + B1 then A2 + B2 so all the way

down onto the end element which is a n plus then me use a different color A1 B1 B2 b

n so you can notice is now in general terms what we are doing here so we are taking the A1 coming from the vector a

we are adding in the same uh position the value that comes from Vector B which is B1 we are saying take

the A1 Plus B1 this is the uh first element so the position stays the same and then in the result Vector so we take all the corresponding values that are

have the same position in the corresponding Vector first from Vector a and then Vector B we are adding them and this forms our new vector and this new

Vector will again have a size n by one so you can see that a the sizes of the two vectors are the same both have n elements and then we are using their

corresponding elements to add them to each other element wise and then we are getting the result that has the same size so n by 1 so this is a more General

description of how you can add two vectors let's now look into this specific example so we have a vector

with the entry 073 so this comes from R three you can see so three dimensional vectors the second Vector is 1 2 0 and

then the final result is 1 193 so how we got this we took zero we added 1 7 we added two and then three we added zero so you can see all these

elements element Y and then this is equal to 0 + 1 is 1 7 + 2 is 9 and then 3 + 0 is 3 exactly what we got here so

again the same sizes and the result is from the same size so quite straightforward now when it comes to the vector substruction what are we doing

that um so what are we doing here so we are doing kind of very similar thing we are taking this element one we are subtracting the other one in this first

element then we are taking the nine in the second position and subtracting this again from the second position of the second vector and we are putting in here

1 and then 1 - 1 is = to 0 9 y - 1 is = to 8 so we get result Factor 08 like in here and you can see that the sizes stay

the same so also in this case let's write more General um this idea of subtraction if we got a vector a

from RN so n dimensional space and it can be represented by A1 A2 all the way

down to a n so it has n elements n by one and then we got B also from RN so coming from the n

dimensional space which means that it got n elements so B1 B2 all the way down

to BN again with the same size n by one then a minus B is simply equal

to a A1 let me actually use the same colors to make it easier to

follow so let me first draw my Square races and then here I will use blue for

a and then red for the uh color for second Vector which is B here I will use black

minus then given that the same size should be for the result Vector I I already know that I expect n different elements for this and then

here I'm taking this first element that comes from Vector a I substracting from this the first

element that comes from Vector B so element wise subtraction B1 and I'm already getting the result for the first

element in my result Vector so you can see A1 minus B1 I'm taking this element and this element and subtracting them

from each other to get A1 minus B1 and then the same holds for all the other values only coming from different

elements from Vector a subtracting from this the corresponding values element Wise from

the vector B so B2 B3 all the way to a n so you can see that in my result Vector a vector minus B Vector in the

first element I get A1 minus B1 then A2 - B2 then A3 - B3 in the third element all the way down to the end element which is equal to oh this should be b a

n minus BN so um this already should makes uh much more sense so every time we take the element in the same position from one vector than the other we subtract

from each other in order to get the corresponding element in the final Vector all right so let's now uh before moving on to the

properties um I want to to show you um this only in a coordinate space so what this means in terms of visualization in

a coordinate space so uh let's say we have a coordinate space this is my Y

axis this is my x axis so this is X and the Y and this is my Center so 0

0 and what I'm doing here is simply I want to have Vector a let's say

this is just um Vector a simple one with the coordinates um let's say four and minus

2 and I got Vector B let me use a different color Vector

B that has coordinates let's say

minus 4 and four so let's actually visualize them let's first start with D Vector a uh

which has a x value of four three four one 2 three and

four and the Y value minus 2 so this is my Vector a and let's now visualize the vector B

so minus 4 and 4 which means that let me actually extend

this this is minus 4 so the x coordinate is min - 4 so it should be here and then

the y coordinate is four so 1 2 3 and 4 it's this one which means that my Vector B is this

one all right so you can see now that the vector a is in here and the vector B is in

here now what I want to do is to add these two vectors to each other so what I want to do is to take this Vector

a and add to this the vector B which

is is equal to 4 - 4 = 0 and then - 2 + 4 is = 2 so zero and then two it

is z and two two so this is my result Vector so now when we are clear on how we can in vector s how we can perform these

different operations and what it means in practice when it comes to looking at the vectors in a cordan space and adding them or subtracting them we are ready to

look into the properties of vector additions so this is something that will definitely seem familiar to you uh from pre-algebra where we are basically using

all these properties that we already know that holds for uh numeric values for the scalers that being transferred to this Vector space so we are going to

talk about this four different properties that a vectors have the first one is the cumulative property which says that if we add a vector a to Vector

B then this is the same as adding a vector B to Vector a so basically the order of the vectors doesn't really matter when it comes down to adding them

so formally A + B is equal to B+ a for any vectors A and B of the same size then we have associative property which

says A + B + C is equal to a + b + C we can write both As A + B + C now what does this mean we know from pre-algebra

that this parenthesis means first do this addition and then do the the rest of operations in here it basically says

if you add a to the B first and then you add the C is the same as first you add B to the C and then on the top of that you

add D Vector a so then the third property is addition of zero vectors which says if we add a zero Vector to Vector a then this is equal to adding a

vector zero to a and this is equal to Vector a so adding the zero Vector has basically no impact on the vector whatsoever

then the final property is subtracting a vector from itself which means if we take the vector we substract the same Vector from itself so a minus a and we

get a zero Vector so a minus a is equal to zero vector and this heals the zero Vector now let's look into each of those properties one by one and let's uh look

into specific examples uh in some cases we will prove this on the example that we have to make this Concepts much more clear so let's start with this

cumulative property of vector additions so we want to see whether A+ B is equal to B + a so let's say we have a vector

a that has coordinates or magnitude and direction that is equal to one and two then we have a vector uh let's say

B that has a magnitude and direction of - 2 and 3 so the first thing that we want to

check is indeed whether the A + B is equal to B + a so therefore let's first calculate this part and then we will calculate

this part that I will Define by one and two and we will see whether we are indeed having the same value the same vector or not so let's see so we have

here a so A + B which is the first value that

we want to calculate a plus b is = to 1 2+ - 23 and we learned before that this is

simply equal 2 take this value and then add this one so 1 + - 2 and then 2 +

3 so this gives us a vector 1 - 2 is = to - 1 and 2 + 3 is = 5 so we get that A

+ B is = to -1 5 this Vector now let's look at the second quantity so B Vector

B plus Vector a this is equal to - 2 3 + 1 2 and this is equal to - 2 + 1 and

then 3 + 2 this gives us - 2 + 1 is = to- 1 and 3 + 2 is equal to 5 so we can

already see from here that the quantity one is indeed equal to quantity 2 which proves that indeed the A + B is equal to

B+ a what this basically means is that adding two different vectors the direction or the order is not important whether you add a on the top of the b or

B to a it doesn't matter at the end is the same and actually you can also see it if you uh combine this or if you do this in more general terms so let's say

if we have a vector a which is equal to in an

N dimensional space A1 A2 up to a n so it has n by one dimension and you have a vector B with the same size from the

same RN Dimension and it has element B1 B2 up to BN and the dimension is equal to M by1

then if we calculate first A+

B and this is equal to Simply A1 + B1 A2 + B2 up to a n +

BN and if you calculate the second uh amount which is B+ a

this is equal to B1 + A1 B2 + A2 up to

BN + a n you can see that A1 + B1 is equal to B1 + A1 simply from prealgebra

you know that if those are all constants for instance 2 + 3 is equal 3 + 2 in the

same way A2 + B2 is = to B2 + A2 and then here up to a n + BN is equal to BN

+ a n what this means is that all these elements they are basically the same which means that we already have a proof

so we get this proof and we can see that even for the general term independent what this Vector a is what this Vector B

is that a + b is equal to B + a this is exactly what we saw before in the first property which is called

commutative property of the vectors that a plus b is equal to B+ a now let's move on to the other property which is called associative property of the vectors now

what this property does and says is that a plus b so first we do this plus C is equal to a + b + C and this is then

equal to a + b + C now let's then see um this specific property on an actual example so what this basically says is

that if we have this example where a is equal to actually I had this before let me simply just remove this part let's

then add our third Vector which is C and let's call it let's say it has a representation of four and

five then the IDE behind this property is that what we need to prove here that A + B within the parenthesis plus C is

equal to a plus B+ C and then this is equal to a plus b

plus C so let's see actually whether this is indeed true for this specific case now this should come very intuitive so I'm going to do it very

quickly so first we have this quantity this one then we have this one and the third one let's do it very quickly so A +

B plus C is equal to one Tu plus and then we had C so it is

simply 4 five and then this is equal to we saw before when doing this that we were getting

1 - 2 2 + 3 and then we add this four five this is simply equal to 1 - 2 is

-1 and 2 + 3 is 5 + 4 5 now given that it doesn't really

matter no longer that we have uh here parenthesis or not this basically means

that this volue is simply equal to -1 + 4 so here -1 + 4 here 5 +

5 so this is then equal to three and then 10 all right let's then now quickly do the

second amount which says first add the vector B to Vector C and only then add the vector a on the top what this means is that we need to

take one two this is Vector a and we will only add this once we have added D minus 23 the vector B plus to the Vector

45 okay so we can see that we are just leaving this in here let's first add this two minus 2 + 4 3 +

5 so this gives us 1 2+ - 2 + 4 is uh 2 and then 3 + 5 is 8

so this gives us let me remove this calculations so this gives us 1 + 2 is =

to 3 and then 2 + 8 is equal to 10 okay great so now we got already the quantity 1 being equal to quantity 2 let's check

whether this is all equal to this one it should already be um something that you see now given that um we know just from mathematics that parentheses doesn't really matter when it comes to the

scalers and adding two vectors is basically very close to this idea of addited property um of the edited property of the scalers but just let's

quickly do it to be 100% sure so when we take this Vector a to the B and to the C we had all this this is equal to one

want to added to minus 2 three and then added this to four and five now what this is equal

to let me actually write this in bit shorter way such that it can be all fit

in in the small place so 1 2 + - 2 3 + 4

5 this is equal to basically 1 - 2 + + 4 and then 2 + 3 + 5 now what is this

number 1 - 2 + 4 is simply equal to 1 - 2 is = to -1 and then + 4 is equal to 3 so first element is three 2 + 3 + 5 is

equal to 5 + 5 which is equal to 10 perfect so now we get the confirmation that indeed a + b plus c C is = to A + B

+ C is = to A + B + C so let's quickly also look into this addition of zero vector and the subtracting a vector from itself

properties and uh the detailed explanation of this or example of this I will leave it to you so when it comes to this A+ um 0 is equal to 0 + a is equal

to a so this property let's say if a is equal to this 23 and then we are adding on this a plus some zero Vector which

basically means take two three and then added the same size of zero Vector you can see that this is the same as adding

this zeros on these values now what do we get we get that this is equal to 2 + 0 is 2 and then 3 + 0 is

three there we go so we already see very quickly that it doesn't really matter whether we add a zero Vector to this original a vector or not we in all cases

it just adding a zero Vector has no effect and seeing from the commutative property that a plus b is equal to B+ a

we already know that if um a + 0 is equal to uh a and is equal to this then also 0 + a will be the same and we can

see indeed that we just saw that a a + 0 is simply equal to a so we basically have quickly proven all this now when it comes to the subtracting Vector from itself I think

this is a very nice one just to see how we um uh take the same vector and subtract from that value and we get zero and this is very similar to working with

just real numbers in the same way as 3 minus 3 is equal to Z also when we have a vector consisting of the scalers like

a is equal 2 23 in the same manner if we take this a and we subtract it from itself so A Min

- A then what we will get is 23 - 23 and this will give us 2 - 2 is 0 and then 3 - 3 is zero so we get a vector zero so

zero Vector so now when we are clear on how we can perform different operations on our vectors and also we know uh what are

the prop properties of uh adding and subtracting uh different vectors we are ready to move on to a bit more advanced topics so uh in this module we are going

to discuss this idea of scalar multiplication we're going to look into the example how uh what happens and how we can do the uh Vector multiplication

with the scalar then we are going to uh look into the span of vectors what it means to have a sp of vectors uh what is this IDE of linear combination and the

relationship between the span and linear combination and the unit vectors then we are going to look into the application of scalar Vector multiplication in audio

scaling uh example and then finally we are going to finish off this module by looking into the length of a vector and a DOT product and we are going to uh go

back to this idea of distance understanding vector magnitude and understanding Vector l so let's get started now before we

look into this idea of span and linear combination I quickly wanted to look into this idea of scalar multiplication and the um specific definition of it so

formally the scalar multiplication involves multiplying each component of a vector by scalar value effectively scaling the vector's magnitude so what

do I mean here let's say we have a vector and I will write it in the general terms to keep everything General so let's see we have a vector a let me

pick up my pen a and this Vector a is from n dimensional space so it is from

RN and it can be represented by A1 A2 up to a n and I have this magnitude um of a

vector and now I want to scale this uh Vector for which I know the magnitude and the direction I want to scale it with a scaler and we learned before that

the scaler is just a number so um scaler in this case I will be uh referring it to uh by C so c will be my scaler and uh

this comes from R which means that it's a real number let me actually use a different color to make it easier to

follow okay so my scaler will be with the color uh red so C and C comes from R

so what do I mean by scalar multiplication I mean that I want to find what is this c

times a this is what we mean by scalar multiplying with Vector now what does

this definition say it says when we are multiplying scalar we Vector so the scalar multiplication meaning multiplying Vector with the scalar it

involves multiplying each component of a vector by a scalar volume so if we translate it to this specific example it means

that this amount so this amount is equal to taking C and multiply find it with

each element of this Vector so each component of vector and what are the components of my Vector the A1 A2 a up to the point of

a n so all these components so that means that the first element of this new Vector the scalar

multiplication result will be C * A1 C * A2

dot dot dot so all this middle elements and at the end again c times and then a n and then in both cases of course the

number of elements doesn't change so the so the number of rows of my Vector doesn't change it's n so here also n and then number of columns is the same so

it's just a column Vector so one column so what we see here is that we go

from a 1 to C * A1 we go from A2 to C C * A2 up to the a n transforms into C * a

n so we see very easily that I keep all the elements from this Vector I take them in here and instead what I'm doing is that I'm multiplying every element

from this vector by the scaler C so this is exactly what this definition says and let's actually go

ahead and do a Hands-On example with some real numbers to have this um method and to have this uh definition very clear in our mind because we are going

to make use of this fundamental operation scalar multiplication on and on in the upcoming lectures and just in general in your journey in any applied

sciences so this is an example of scalar multiplication uh here what we are doing is that we want to multiply this Vector C so in this case the vector is defined

by a letter C and then on the top we can see the arrow indicating that this is the vector now and here we refer the scalar by a letter K we are saying we

want to perform scalar multiplication which means that we want to multiply the uh a vector C by the scaler

K so how we can do that so what we want is to multiply K by C and we just learned that for that what we need to do

let me write this over so this equal to minus 2 multiply it by 4 - 3 this is my Vector so this is

the K and this is the C this is equal to so I take my scaler and I multiply it with the each

of the ele element of the C so - 2 * 4 and then - 2 *

-3 so - 2 * 4 is = to - 8 and then - 2 * - 3 so- minus it goes away it becomes a

plus and 2 * 3 is 6 so my end result the K * C is equal to - 8 6 this is my final

result so let's quickly also do yet another example and this one is a unique one because it's relating to this idea

of U multiplying something with a zero uh which is something that we also uh know from a high school that when we multiply number let's say seven by zero

we are getting zero and here in this example the uh problem is describe the effect of a scalar multiplication by

zero on any Vector which means what we are doing is that in this example is we want to know what is this result

of any Vector let's say Vector uh C so we will use the same example C only this time instead of multiplying it with scalar k equal to minus 2 our scalar

will be zero which means that c is equal to 4 - 3 and then K is now equal to zero and we want to find out what is this K *

C let me actually write down the K with a different color k is equal to zero so what we want to find out is K * and then

C and this is that equal to0 so I'm taking the k0 times then I'm taking each of the

elements of C which is four and then minus 3 and I know that when multiplying the number with is 0 it gives me 0o

which means that I end up with 0 here 0 * 4 is 0 0 * - 3 is also 0 so I end up

with a zero Vector now this gives me an idea already that I can make a general conclusion that independent of the type

of vector that I have independ and what are this values in my C uh if I have any

Vector C and I'm multiplying it with zero then this will always give me a vector of zero because all the members

of this final Vector will be just zeros so if for instance the C comes from uh let's say r n so it has n different

elements it comes from n dimensional space then my final result of 0 * C so

this zero Vector this one so zero that this one will come also from RN so you

will be having a vector so 0 * c will then be equal to z0 blah blah

blah blah zero so n time zeros so this is then the idea of multiplying so scaling a vector with zero and this

is our example two all right so let's now move on on to our application of scalar vectal multiplication and then after this we will go back to this idea

of linear combinations and dispense so in this specific application we have a scalar Vector multiplication and we are looking into application of audio scaling

so the scalar Vector multiplication audio processing uh this can change the volume for instance of an audio signal without altering its content so um you

might have noticed that um when uh when you are listening to video you can simply increase the volume of that video or decrease it but you will notice that the content doesn't change you are just

increasing the volume or decreasing it even on the TV when you are watching a show you are increasing The Voice or decreasing now what you're basically doing behind and this is super interesting is that

behind the scenes what is happening is that there is simply um audio that um contains that show and the audio of that

show is being multiplied with a scaler and that scale is simply the volume scale if you scale it in such way that

you want to decrease the volume so the audio will then have a lower volume then you are simply multiplying it uh your

vector containing the audio information in such way that those newer volume indications they will be they will be containing lower

numbers hope this makes sense let's look into the example this make uh this will definitely clear this out so um let's assume we have an a vector a that

represents the audio signal and we want to multiply Vector a a by scalar B to adjust the volume so B is some sort of

number it can be so B comes from R so is a real number while a is simply a vector given that it

doesn't mentioning here I'm assuming that a comes from RN so it comes from r n dimensional space so imagine of a as

this Vector A1 A2 blah blah blah blah to a n and each of these values it basically describes uh an uh the audio

signal so it represents um uh an amount so it contains an amount that represents the audio signal of your uh video or uh your uh

show and then the b in this case for instance in this example you can see that the B is then uh equal to for

instance 1.2 1 / 2 or B is equal to Min - 1 / 2 so you can see that b is equal

to 1 / 2 which basically is a fensive of saying that b is equal to 0.5 or B can

be equal to minus1 / 2 which is minus 0.5 now then it says then the B * a

which basically means multiplying our um scalar beta by the Vector containing the audio

signal a so this B * a is perceived as the same audio signal but at the lower volume now why lower because you can see

that b is equal to 0.5 or minus 0.5 it means that once you take all these elements of your a and you multiply it with a number that is smaller than one

in this case 0.5 then all these numbers will decrease which means that also your audio volume will decrease so let me actually uh show you

an example so let's say our talk show is very short and you know the audio

variation is very low you have a vector a that is quite small it comes from a three dimensional space so R Tre and it

has numbers like three uh six and then five so 3x1 vector and then we have our audio adjustment scalar beta which is

equal to 0.5 now when we take the beta we're multiply it by our audio signal then what we

do times is clear so times what we are doing is that we are simply taking all

the elements of our a so Three 6 and five and what we are doing is that we are multiplying it by

0.5 0.5 and 0.5 or you can also say 1 / 2 so what this is equal is that 3 * 0.5 is

1.5 6 * 0.5 is 3 and then 5 * 0.5 is 2.5 and you can see that all this numbers 1.5 3 and 2.5 they are smaller

and specifically two times times less than all the original values in the um original audio so original audio is a

which was 3 6 and 5 and the new audio the the scaled one is so audio scaled so

B * a is equal to 1.5 3 and 2.5 so you can clearly see this transformation where this element three

is larger than 1.5 6 is larger than three and then the last element five is larger than 2.5 which means that this audio

audio is much at a higher volume so the volume two times higher than this

audio so this is basically the idea of uh applying scalar multiplication to our audio pre-processing I will leave the other example to you that will show that

when your scaler is equal to minus 0.5 you again will end up with the lower volume only that time the volume will be much much lower than the original one so

now that we know how we can perform scale multiplication in theory as well as we have looked into an example how we can do it in terms of the numbers and multiply apping them and we have also

seen uh applying SK multiplication in practice uh so we have seen in this audio processing stage the uh multiplication process we are ready to

look into the visualization of it this will help us to get a better understanding on uh what exactly happens when we are scaling different vectors

let's look actually in the following example so let's assume we have a vector oh let me remove

that so let's usum we have a vector and the vector is let me get a color this

one for instance a vector a and this Vector a consists of elements one and

two so where does this Vector Li the vector is with um one so here in our coord system this is our xaxis this is

our y AIS and here we got uh let me actually pick another color let's say black

one and then we got one and then two right this is two this is one so it is this one so the line that we get here it

is this one so this is our Vector a now let's assume I want to multiply my Vector a so I want to scale my Vector

a by a constant Tree by scalar tree so I have a scaler let's say I call K and this k a different number let's say k is

equal to three so what I wanted to do is to perform a scale of multiplication so I

want to obtain K multiplied by a and we learned that this is simply equal to three times and

then one two and then this is equal to 3 * 1 3 * 2 which is equal to 3 and

then six so let's also visualize this scaled uh Vector so let me pick this yellow color

this will be our scaled Vector so we have done scale multiplication and we are going to visualize that so we have three and six so this is three 1 2 three

and this is six so we have this point so you should already see what is going on here

so we got 3A here so you can see that this part is

our Vector a and this longer one is 3 a and even visually you can see that this longer Vector is simply the three times

of the shorter Vector so we got this and then if you add on the top of this the same three times you will then end up with

the original so scaled version of that so basically this is a this is a this is

a we combine three different so we scale a three times and we simply get a three times longer version with the same direction so you can see that when we

are scaling even visually it makes sense so we are scaling our Vector a three times and we are just getting that Vector so we are transforming

oh let me remove this so basically we are taking this vector and

we are scaling it up to this point if I would do it only two times then it would be something like

this or one and a half times it would be something like this so only half of

it so now this should make much more sense let us actually do yet another example to uh make sure that we are clear on this visualizations because we are going to make use of it when uh

looking into this idea of linear combination in a span so let's say we have a vector B and this Vector B has

elements zero and three so let's visualize and uh plot this Vector so it contains elements Z zero

and three so zero and three so this is the X element and the Y element on the Y AIS we can see this is three which means

that our Vector B is this Vector all right perfect so this is our B let's now multiply so scale our Vector

B by scaler t Q so let's say we want to get 2 * B so what is this amount this is equal

to 2 * 2 times and I'm simply taking each of those elements zero and then three so this is then equal to 2 * 0 is 0 and

then 2 * 3 is equal to 6 so this is my new scaled Vector 2 * time B Vector this one so let's

visualize this the xais value is zero so we are still here and then the y- axis value is six so what is sixth this thing

all right so you already should see that this is very similar what we had before so this is 2

B all right so this all uh should make sense uh also we learned as part of the um High School when visualizing different plots so this is quite similar

to this idea of having Y is equal to X and then scaling it getting like Y is equal to 2x so in this case only we know

exactly where the vector starts and ends uh so we have a much more specific definition instead of having all this infinite number of points on the

line but the idea stays the same so we are taking this vector and we are then scaling it two times so we get 2 B

vector and I could do the same only instead what I could also do is I could do like uh 0.5 or 1 / 2 * B so I take the half of it which means I would get

this vector or I could multiply it with minus one so minus - 1 * B so I was scale with minus1

and and then I will simply get the negative version of my original Vector so this thing this

would be minus b or min-1 * B so this is basically the idea of uh scaling multiplication when visualizing it in our coordinate system cartisian

coordinate system and now when we know all this we are ready to move on on this idea of linear combin and now when we know all this we are

ready to move on on this idea of linear combination so let's now formally Define this ideal linear combinations a linear combination of vectors A1 up to a m

using scalers B1 up to BM or what we also refer as beta 1 to Beta m is the vector beta 1 * A1 Plus up to Beta M * a

m and the scalers are called the coefficient of linear combination and any Vector B in N Dimensions can be expressed as a linear

combination of the standard unit vectors E1 up to n the coefficients in this combination are then the entries of B itself well this is whole bunch of

information uh let's unpack them one by one firstly um I want to mention about this m so far we have seen this idea of n so I just wanted P to experiment with

a different one just to ensure that we are clear that you can use any source of identifier to describe the size of your

um uh number of vectors that you got and uh in this case we got M different vectors because so far we were using this n in order to describe the size of

a vector and now we are no longer talking about the size of a vector but the number of vectors therefore I specifically didn't use use the letter N

so here m is simply the number of vectors so don't confuse this with this thing where we were plotting this and we

were saying this A1 A2 up to a n because in here we basically mean that we are dealing with some Vector a and this has

n different elements whereas in here we are already moving from this idea of one vector and now we are talking about mve multiple vectors so we have M different

vectors they all look like kind of this only with bit more complex indexing that we also saw before all right but we will learn this

um that's not an issue I just wanted to mention this to ensure we are at the same page so then let's move on to this

idea of using scalers beta 1 till beta M so it's a common uh practice in linear algebra in just in general in mathematics but also definitely in data

science statistics and in artificial intelligence to use beta 1 as a way to describe the coefficient so what do you mean by coefficient it is just a scalar

so it's just a constant or a number so in this case for instance this beta 1 can be 0.5 beta 1 can be uh two bet one

can be let's say 100 it just describes how much we are multiplying scaling this Vector A1 so so far we have done a lot

of scal and multiplication already lot of details there and we have seen different times different scalers that we use we can use um zero as a scaler we

can use any other number as long as it's a real number so this beta 1 should belong uh in the a real number space so

it's a real number and of course the same holds for uh all the betas so we have M different vectors which means we are going to have M different scalers because each of those vectors we are

going to multiply with their corresponding or respective scalers so beta one is basically the scaler uh or

the um uh coefficient that we are using to scale a one maybe I can actually write this down on a new page such that we can

save this as a SL Light page for you let's write it down so what do we have as this idea of linear combination so a linear combination simply involves

taking several vectors uh to go from this uh formal definition to more practical uh terms so we got this

A1 A2 up to a and what we want to do is to take the linear combination of this m different

vectors so we got m is the number of vectors and to get a linear combination we need to uh scale each of those

vectors which means that we need to have this different scalers let's say beta 1 for

A1 and then plus beta 2 for A2 so each time we are scaling each of those vectors where beta 1 is the uh scaler or

the coefficient of the vector A1 and we are multiplying we are performing scalar multiplication of our scalar beta 1 with

the vector A1 and then we are adding to this our beta 2 which is the coefficient corresponding to the vector A2 and then

adding beta 3 * A3 and then dot dot dot so all these different uh vectors up to the point of beta

M time a m and all this so A1 A2 up to a those

are all vectors belonging to the m space so those are all vectors coming from the um M dimensional

space so um in here this is the linear combination of our

M different vectors and the uh beta

1 beta 2 up to Beta M those are all constants so those are scalers or real

numbers that belong to R so those are real numbers all right so now when we are clear on that let's also unpack this idea of coefficients so the scalers are

called the coefficients of linear combination so basically all this

members so beta 1 beta 2 of two beta M that belong to real number space they are called

coefficients this is what we are referring as coefficients and this coefficients this IDE and name is super important because you will see this time and time again appearing in your uh very

basic machine learning models or some other applications of linear algebra because the end goal is always to find these coefficients so these coefficients

those are numbers that we are using to scale these different vectors and uh the idea of coefficients is very Central

because those are numbers that Define how exactly we are combin ining all these different vectors because this beta 1 beta 2 Beta 3 they can be

different numbers real numbers and every time when we are choosing these coefficients or these betas we will then end up with a different combination of

these vectors so we are basically mixing all these different vectors and the way we mix it and how we will mix it it will depend on the values of this beta 1 beta

2 Beta 3 up to Beta m so these coefficients so therefore coefficients are super important and they Define the end results from our linear

combination so any Vector B in N Dimensions can be expressed as a linear combination of standard unit vectors E1

up to n so when looking into this um idea of unit vectors uh we saw already

what this E1 is what is E2 is up to e n and we saw that E1 is for instance if it's from an N dimensional

space and it says from n dimensions then E1 simply means 1 0 0

dot dot dot dot Z then E2 means 0 1 Z dot dot dot dot zero so we already saw this this is not something new that we are seeing so 0 Z blah blah blah and

then one at the end and what this definition basically says is that any Vector b as long as the B comes from n

dimensional space we can represent this by using this uh unit vectors and by linearly combining

them so this is yet another part of this definition and we are going to by the way um go through each of the parts of this definition one by one going to each

of the examp examples as well as visualizing them so now I just want to quickly unpack all the parts in this definition before moving on to step by step examples and

explanation so this is about this linear combination of any n dimensional uh Vector B that we can uh create by using a linear combination of these unit

vectors I will come to this in a bit so then the final part of this definition is that the coefficient in this

combination are the entries of B itself so it says that the coefficients so beta 1 up to Beta m in this linear

combination that we can create are the entries of B itself so we will come to this section once we are done with the first part so

first let's have a good understanding of what this linear combination is and also touch base and we will also formally Define the idea of span and after that

we will move on on uh representing and expressing any Vector B in N Dimension Space by using standard unit vectors E1

up to e n and this idea of coefficients and then entries of B so let's start with the first one so let's assume we have two different

vectors we have Vector a and this Vector a is equal to one 2 so let's plot

this one and two in our coordinate space that is this one which means that our Vector a is this

one and let's assume that we have a vector B and this Vector

B is equal to Z 03 so 0 is here and then three is here which means that our Vector B is this

one this is Vector B now I want to create a linear combination of this Vector a and Vector

B so we just learned from the formal definition that in order to do so I need a beta 1 to multiply the vector

a and then I need beta 2 which is the coefficient corresponding to to my second Vector in order to multiply the second Vector which is

B Vector B okay so I'm getting the linear combination of A and B by taking any beta 1 and beta 2 which are real

numbers so beta 1 and beta 2 belong to R so they are real numbers and then I'm getting a linear combination of the two

so let's look into a few examples of a linear combination of vector A and B depending on the different choice of the

coefficients like beta 1 and beta 2 so example one is that beta 1 is equal to zero and then beta beta 2 is equal to Z

Now what is the linear combination of A and B when my coefficients beta 1 and beta 2 both are zero it just means that I'm

getting 0 time A Plus 0 times B which is of

course 0 * 1 0 * 2 plus and then multiplying Vector B with

a scaler zero which is 0 * 0 0 * 3 so let's quickly do this what this value is this is equal to 0 * 1 is 0 0 * 2 is

0 0 * 0 is equal to 0 0 * 3 is equal to 0 and this is then equal to 0 + 0 0 0 +

0 is0 so I'm basically getting a vector zero all right so this is then equal to

zero so this equal to vector is zero so I can also say that this vector or it's actually a point so this

point is simply a linear combination of these two vectors now this is a super basic case let's look at another case when our in

our second example the beta 1 and beta 2 so our coefficients they are actually not zero there are some other nonzero real

numbers so in this example I will then take beta 1 = to 3 and then beta 2 is = to

2 and then what I will do is that I will take actually I will take the um minus two then I can also get rid of one of the elements and I can get actually a zero

for one of the elements I will show you in a bit so then the linear combination of A and B using these coefficients beta

1 and beta 2 where beta 1 isal to 3 and beta 2 is equal to minus 2 is then equal to so this

amount this amount is equal = 2 3 * 1

2 and then plus we got - 2

* 0 and three now what does this give us 3 * 1 is = 3 3 * 2 is = 6 plus and

then - 2 * 0 is = to 0 and then - 2 * 3 is = - 6 so you might have already noticed why I picked the

beta 2 equal to minus 2 I wanted these two numbers to actually cancel each other so you see because 6 + - 6 is equal to Z so what do I get in my final

result as a linear combination of these two vectors I get 3 + 0 so 3 + 0

so 3 + 0 and then 6 + - 6 and this gives me 3 + 0 is 3 6 + - 6

is zero there we go so this is my linear combination of the vector A and B when using the coefficients equal to 3 and

minus 2 respectively so this value is actually = to three and zero in this case all right so let me actually clean

this up because I also want to visualize this idea and then we will go uh back to this uh linear combination let just summarize uh what we got before moving

on to the plotting part so if we simply take a and we add to this B so this is the first case so this is as you might have already guessed this is also l your

combination here we are saying take 1 * a and take a 1 * B and this is yet in our linear combination here the beta 1

is equal to 1 and then beta 2 is equal to 1 so this linear combination gives us a vector that is 1 + 0 is = to 1 and

then 2 + 3 is = 5 this is our first linear combination when the beta 1 and beta 2 is equal to one this is a basic case so doesn't require too much

explanation here we have seen already this let's now look into the other example that we saw when we use uh the

zeros as our coefficient so that is 0 * A+ 0 * B then this gave

us 0 0 this was our second linear combination when beta 1 and beta 2 were both equal to zero and then the third linear

combination that we saw was that 3 * A+ - 2 * B this gave

us 3 and zero this was our third linear combination where beta 1 was three and then beta 2 was minus

2 so so then the linear combination of these two vectors is basically all the possible combinations of these

two vectors that I can get when scaling or when multiplying these two different vectors by different sorts of uh vector

by different sorts of scalars so in all these different cases what I'm simply doing is I'm taking different sorts of coefficients beta 1 and beta 2 and then

I'm getting the linear combination of these two vectors we saw that in the simple case when we take a and we had to B so basically the coefficients are

equal to 1 so 1 * a + 1 * B then the corresponding linear combination is equal to one and five it means that we are getting this vectors so one and five

is in here which means that we are getting this one this Vector if we get if we take the zero as a scal so beta 1 and beta 2 are both equal to zero then

the linear combination of these two vectors is simply the vector zero which means that it is this point then if if we take the linear combination using

three and minus 2 as coefficients then we are getting this so 0o and three so one two and three this is three then this is our linear

combination I can also take any other uh like scaled version of my B and of my a and then I will get entirely different

sort of vector so let me actually show you a few more times um a couple of other examples so let's say I keep my a

so I just take the beta 1 equal to 1 but instead I scale my Vector B two times so this was at three I'm taking two times

of my Beta which means that I'm here then I can take this I can add this to my a so this is 2

B this will give me another linear combination of these two different vectors I can also you might recall that

we said that the starting point and the end point doesn't really matter for for us what matters is that we uh have the same magnitude and the same direction

for our vectors so this means that for me the vector being here and the vector being here doesn't matter when I scale it with two I can be here with three I

can be here so this is the same as my B only 3 * B this is basically basically scaling B with three and this in here

means that my Beta 2 is simply equal to tree and then this means that I can combine this with my a which was in here

you remember so this here this means that I get yet another linear combination of these vectors which means that I'm

taking 3B and I'm on this my a so 1 * a my beta 1 is equal to 1 my Beta 2 is equal to 3

which means that the linear combination of this is equal to one two plus and then 3 * B is equal to 0 and then 3 * 3 is 9 this is then the new linear

combination which is 1 and 11 so the new linear combination is equal to 1 and 11 so this thing

which is the same as this thing and then you can go on and on you can also calculate the same with a

negative B so you can take B and then you can scale it with minus one so this is minus b or you can go in here in here

the same holds for a so you can scale it all the way to here or in the negative side so so this already uh give us the

idea that we will go into to the next point which is the span so when it comes to the linear combination and in this specific case when we have these two

vectors we can combine these two vectors in anyway and uh we can mix them up by using different sorts of coefficients of

beta 1 and beta 2 and we will can we can get any Vector in our R2 so this means that any vector in our R2 we can

represent by using only these two vectors and this is not always the case for this specific case we are dealing with two vectors that we can use to

represent any Vector in our r two so what I mean here is that let me clean this

up so independent what kind of vector you will give me in the R2 so it has two

different elements it is 2x one I can use a linear combination of A and B so a linear combination of A and B is beta 1

* a plus beta 2 * B in order to represent this

Vector X1 and X2 therefore we are saying and we will come to this um in the next slide too that the

spend of the vectors A and B so this is the set of all possible combinations of these two vectors is equal to R2 because

any Vector in R2 can be represented as a linear combination of these two vectors so we have a linear combination of A and

B and I'm saying that I can represent any Vector so here vector X and this Vector X I'm representing by X1 and X2

and X1 and X2 can be any real numbers so X1 and X2 they belong to R and X is

simply a two-dimensional Vector so X1 and X2 those can be any numbers 0 1 2us 100 anything and I'm saying any number

in this two dimensional space so whether it is this one any Vector this one this one or this vector or this one any Vector that you give me in two

dimensional space I can find a linear combination of this A and B that is equal to that Vector so I can represent that Vector as a linear combination of

vector A and B that we saw before so let's actually prove that so I'm going to represent this uh X1 and X2 by a linear combination of this Vector A and

B and how we can do that so we have beta 1 * a plus beta 2 * B it is equal to X1 and X2 where beta 1 and beta 2 so beta 1

and beta 2 they are constants so they are also real numbers so let's unpack this which is

beta 1 * 1 2 + beta 2 * 03 and this should be equal to X1 and X2

now this is

equivalent of so beta 1 * 1 beta 1 * 2 plus beta 2 * 0 and then beta 2 * 3 and

this should be equal to X1 X2 so this is my beta 1 a this is my Beta 2 B and this is my

X all right so now what we get is that and this is equivalent beta 1 * 1 is equal to beta 1 beta 1 * 2 is 2

beta 1 plus then here beta 2 * 0 is 0 and then beta 2 * 3 is 3 beta

2 so we have learned U from the uh operations on the vectors that beta one uh so in this case when we are adding two vectors so beta 1 +0 is the uh

amount that we need to put as our first element so when we are adding two vectors we just need to take their corresponding elements we need to add them up so equal to beta 1 + 0 and then

2 beta 1 + 3 beta 2 this is the result and this should be equal to X1 and X2 at least this is what I'm claiming so this zero doesn't matter so

we what we are getting from here is that beta 1 is = to X1 and then 2 beta 1 + 3

b. 2 is equal to X2 this is the two

b. 2 is equal to X2 this is the two expressions that we are getting based on all these different calculations so let me actually remove all

this so we have beta 1 is equal to X1 and 2 beta 1 + 3 beta 2 is equal to

X2 so here given that we have already that beta 1 is equal to X1 and here we have two unknowns I'm going to fill in the value for beta 1 in here so I'm

going to take this and I'm going to fill in it in here so for this volue so remember that beta 1 and beta 2

are two unknowns and each one and next to are just uh numbers that we will get when we uh know exactly the vector and we just want to represent the vector as

a linear combination of two vectors so when I take this uh value for beta 1 which is equal to X1 and I'm going to

fill that in in here it means that I'm going to get from here that beta 1 is equal to X1 and 2 *

X1 because beta 1 is equal to X1 and here I got beta 1 I'm just filling in that value for beta 1 which is equal to X1 so 2 *

X1 and then the rest I'm just taking over 3 beta 2 is equal to X2 let me remove this and from here what

we are getting is that beta 1 is equal to X1 and I will solve this equation for the unknown which is equal beta 2 so I will just take the three beta 2 from

left hand side I will leave it there and I will take this and I will take it over to the right so I'm taking X2 over and this two X1 so this part I'm just taking

to the right two of the equation so Min - 2 X1 which then on its turn is equal to so

it goes to B 1 is = to X1 and then beta 2 is = to X2 - 2 X1 / 2

three perfect so what do we get here what is our end result and why is it significant so what we are getting here

is that based on all this information without knowing beta 1 and beta 2 we got that beta 1 should be equal to X1 and

beta 2 should be equal to X2 - 2x1 / to three this means that

independent what kind of x's you will give me so what kind of vector we have in our R2 so this X1 and X2 they are

just real numbers we can always find beta 1 and beta 2 that we can use to represent that X1 X2 so our X Vector as

a linear combination of these two vectors let me actually give you an example so let's remove

this so let's assume we have a vector X and this x is is equal to four and let's say

three so if we got this Vector X and we are saying we can use this Vector A and B to represent X as a linear combination

of vector A and B which means that I can find I can find real number beta 1 and beta 2 that I can use to multiply the vector A and B respectively combine them

together so their linear combination that will be equal to this Vector X so this is my X1 this is my

X2 so this is equal to 4 and three Now using this let's actually see whether that is

true so based on this example my beta 1 should be equal to X1 which is four my Beta 2 should be equal to X2 which is 3

so beta 2 should be equal to X2 which is 3 - 2 * X1 which is 4 / to three and

what's this number this means that my beta 1 should be equal to 4 and my Beta 2 should be equal to 3 - 8 so 3 - 8 / to

3 and this is equal to Minus 5 / to 3 so this means that I use a coefficients beta 1 is equal to 4 and

beta 2 = to - 5 / to 3 to represent my Vector X as a linear combination of vector a and Vector B so let's actually

prove that too as a final step so let's see where the four

times Vector a which is 1 2 + - 5 / to 3 whether this is indeed

equal to Vector X so my Vector B is 03 so this is the first part and I want to prove that this is indeed

equal to X and we already know what x is so this is equal to 4 * 1 4 * 2 plus and

then here we got - 5 / 3 * 0 and then - 5 / to 3 * 3 this is equal

to 4 * 1 is = to 4 4 * 2 is = to 8 and then here we need to subtract minus

53 5 5 / 3 * 0 is equal to 0 so this one is zero and then minus 5 / to 3 so 5/3 *

3 this ones are canceling out and we got 8 + - 5 so here the plus and here minus just to make sure we got everything

right and this is equal to 4 and then 8 + - 5 is equal to 3 so you can see already that this amount

that we got here is equal to X which was equal to 4 / to3 and this helps us to uh

verify and to know for sure that indeed while given any Vector in a two dimensional space in R2 X independent

what this X1 is or X2 is we can always find a pair of beta 1 and beta 2 that

will ensure that the beta 1 a plus beta 2 B is actually equal to this

x where X A and B they are part of R2 and a is equal to 1 2 and then B is

equal to 03 so we can represent any Vector in our two-dimensional space as a linear comp combination of this Vector a with

elements 1 2 and um Vector B with elements 03 and that's exactly what we saw here because we could find any vector and we can represent this Vector as a linear combination of this Vector A

and B this Vector as a linear combination of this A and B this Vector as a linear combination in any vector or a point in this plane we can represent

as a linear combination of this Vector a and Vector B and in this specific case with this Vector a and Vector B we are saying that Vector a and Vector B they

spin R2 so Vector A and B

span R 2 now we will come to these definitions of the span and uh just in general for different sorts of vectors we will see what this IDE of span is but

for now given that we just proved that we can represent any Vector in R2 as a linear combination of these two vectors A and B therefore we can say and we

usually say it in linear algebra that the vector a and Vector B they spend R2 before moving on onto this concept of Spence that we just touched upon in our

example I wanted to quickly go back to this example that I promised to discuss uh which was part of the definition of the linear combinations and unit vectors

because we saw in our definition and let me just show you that uh the uh definition was providing these two highlights these two bullet points and

was saying any Vector B in N Dimensions can be expressed as a linear combination of the standard unit vectors E1 to up to e n and the coefficients in this

combination are the entries of B itself so let's look into the example and see what we mean by that in this specific example we have this Vector B it is

coming from the three dimensional space which we can see given that we have three different uh three entries so three uh elements in our Vector so it's

3 by one and this means that b belongs to R3 and in here we can see that B can be

written as a linear combination of these three vectors so you can see that b e is

equal to -1 * this Vector 1 0 0 so this one then we have + 3 * 0 1 0 Vector so

this one and plus 5 * this third Vector which is 0 01 now we already know from the unit

vectors that E1 is equal to 1 0 0 assuming that we are in three

dimensional space E2 is equal to 0 1

0 and E3 is equal to 0 01 you can notice that that's exactly what we got here this Vector is E1 this Vector is E2 and

this Vector is E3 where E1 E2 and E3 belong to three-dimensional space okay so another thing that we can

see is that here we got coefficients minus one here three and here five so this is basically how beta 1 beta 2 and

beta 3 using the common conventions that we saw before when describing the linear combination so let's actually check that and then we will comment on these values

so let's check whether -1 * E1 + 3 * E2 + 5 * E3 is indeed equal to this B so this is equal to -1 * this Vector gives

us -1 0 0 three times this E2 gives us 0 3 and

zero and then 5 * E3 gives us 0 0 5 and this is equal to

-1 + 0 + 0 is = -1 0 + 3 + 0 is = 3 and then 0 + 0 + 5 is equal to 5 now what do we get here we see that

this which is equal to this it is equal to this Vector B indeed okay so now when we have indeed checked that B can be

represented as a linear combination of this three vectors this unit vectors E1 E2 E3 another thing that we can notice

and I'm sure you already did is that those coefficients they are not just randomly picked coefficients

those are the entries of this Vector B so this is exactly what that definition was about it was saying that any Vector B including this example in in this case

threedimensional space can be Express as a linear combination of the standard unit vectors E1 A2 E3 Etc so this

coefficients in this combination so you can see that the beta 1 beta 2 and beta 3 which are our coefficients in our linear combination they are the entries so this

values of the B itself so the same will hold for four-dimensional case five dimensional

case n dimensional case so this means that if we write this down for General case just to ensure that we are clear on

this part of the definition so if we got B Vector in N dimensional space so it

got B1 B2 up to BN as the elements of it comes from RN then we can represent this B

as a linear combination of unit vectors coming from the N dimensional space so we got E1 E2 up to e n that belong to n

dimensional space and we can represent this B as a linear combination of these unit vectors so by using beta 1

time so this this is a common Convention of the coefficient as you Rec called time C1 then B beta 2 * E2 blah blah

blah plus beta n * e n and what is important here is that this beta 1 beta 2 and beta n those are not just some coefficients

but we already know what these coefficients are because we can then represent this

beta by taking the values so those are the entries the elements of the vector B itself so it is B1

* B1 plus b2 time E2 dot dot dot

plus BN times e

n where B1 B2 up Q BN they are all real numbers so basically knowing what these

Vector is these elements of this Vector we can always describe and express it as a linear combination of the standard unit vectors and if you're

wondering why is this important in some cases when performing different operations or working on different algorithms it just becomes handy to represent your vector as a linear

combination of multiple vectors and in those cases exactly you can make use of this property of linear combinations to express your n dimensional Vector b as a linear

combination of the standard unit vectors because everything is then down to you by having this Vector B you will already know what are the entries that you can use as your coefficients in this case

beta 1 beta 2 so those are all these values coming from your vector itself and then the remaining is also none because you know exactly what these unit vectors are and how they are represented

so here for instance the E1 is basically one 0 0 blah blah blah blah 0 and then this is n by1 Vector here the E2 is

equal to 0 1 Z blah blah blah and then zero here so n * 1 again up to the point where you have the N where you have all

the zeros only the last element is one again n by one vector so this is the idea behind this second part of this definition which says that any Vector

being in N dimensional space can be expressed as a linear combination of the standard unit vectors E1 up to e n let's now talk about other concept which is

also super important which is the span of vectors so by definition the span of a set of vectors is a set of all possible linear combinations of these

vectors so if V is equal to V1 V2 up to V K and is a set of vectors then the span of V is written as a span V and it

includes any vectors that can be expressed as C1 V1 up to C2 V2 up to CK

VK so basically it is a common uh notation uh to say that if we got for

instance vectors V1 V2 up to VN so we have n different vectors then we say that the span

of V1 V2 up to VN that this is simply the notation that

we use in order to describe the span of these vectors and we briefly spoke about this concept of span when we were looking into our example that we saw

before so you might recall vectors A and B that we had and we saw and we said that the span of a and b is the entire

space in the two dimensional uh real number space so we said that span of a and b is equal to R2 where our Vector a

was simply equal to one 2 and B was equal to 03 so we proved that the span

of one two and 03 was the entire R2 and how we knew that because

we proved that any Vector in R2 could be represented as a linear combination of these uh two vectors so you might recall that we solved this equations we saw

that in depend what kind of X1 and X2 uh one will give us we can always use the uh um we we found this amount let me see

where I can find it back I no longer have this so we saw that for uh specific values of um beta 1 and beta 2 we can

always get a linear combination of this A and B in order to get our desired factor x so beta 1 * a plus beta 2 * e will

always then be equal to X1 and X2 if our Vector a and Vector B are those but of course this doesn't hold for all the vectors so not for all two-dimensional A

and B uh we can say that the span of these vectors is the entire R2 therefore to better understand this concept of span and this concept of span of vectors

I wanted to distinguish five different cases one of which we already spoke about and that is the case when we had this Vector a and Vector B and we said that the span of a and b is the entire

R2 but we will also look into the case when we for instance have a span of the zero Vector the span of a single vector and the span of perpendicular vectors we

might also look if there is time left we will also look into the span of parallel vectors so let's now look into this cases one by

one so let's say we have a vector of zero so we have a zero vector so this is a very simple case we will start with the simplest case and we will move on B

to two more advanced cases if we have a vector a that is a zero Vector 0 0 then

independent what kind of scaler we will use to scale this so let's say um we Define it by C so C * Z independent what

kind of scale we will use this will always end up being equal to 0 0 so if C is equal

to0 C * 0 will be equal to 0 if C is equal to 1 C * 0 will be equal

to Z or C is equal to 100 C * 0 will still be 0 0 so independent what kind of scal

we will be using what kind of lead linear combination we will create from our Vector a this will always stay in here so the

point the vector will always stay in here in our two Dimension space so this is completely different from what we saw before when we could create and we could take any Vector in our R2 and we could

represent it as a linear combination of these two vectors that we saw in the previous example so in this specific case

um scaling the zero with independent of any scalers we use this will not change the magnitude nor it will change the direction of our Vector so no matter how

we scale it we still get zero this means that the span of zero

Vector is just the zero Vector itself so you can see that independent what I scale the zero Vector I always end up with the same zero Vector so therefore

this span of the zero Vector is equal to zero because by definition this Spen of set of vectors is the collection of all possible vectors that I can reach by

performing linear combination and in this case I will always Reach This Z 0 Vector so all possible collections of

these vectors are the vector 0 0 which is single vector and the same as the input so this is the basic case now

let's move on onto bit more uh Advanced case so bit more complicated than this one but itself also very easy which is

when we got a single Vector a so let's say a is equal to one and two now I want to know what

is the span of a in order to know what is the span of a we simply need to understand what are all these possible collections of

vectors that I can get when I'm uh combining um a I'm multiplying a with different coefficients so what are the

all possible linear combinations of this Vector because I got just single Vector a so a is one one two which means one in

here and then two here my a is this vector and let's look into uh different uh scalar multiplications of this Vector

so let's say I want to calculate C * a so the scalar multiplication of this where C is equal

to C is equal to 2 C is equal to 3 C is equal to uh Min -1 C is = to - 3 and of course C is = to

1 so in all these cases when C is equal to 1 then the linear combination in this case just the scale multiplication of

this single Vector a so 1 * a is simply equal to one and two so the same Vector

a c is equal to 2 this will give me 2 and four C is equal three this will give me 3 and

6 C is = to min-1 will give me -1 - 2 for my a and then C is equal to -3 will give me -3 and then - 6

so let's plot each of those so if we got for instance C is equal to one case you can see that we already got that Vector in here so it is this

Vector when C is equal to two then we got this one so two and four so where is that it is in

here let me use another color it is in here when when we got C is equal to three then we got so this one three and

six so this is three and this is six so it gives me this Vector in the next example so in the next linear

combination we have C is equal to minus one so we got Min -1 and Min -2 so where is min -1 it is in here where is min-2 it is in here so I'm

getting this vector and then finally when I have let me change the color when I have C is equal to minus 3

so this case then I got minus 3 and 6 which means that here is my minus 3 here is my minus 6 so we got this thing

so you already should see what is going on here when we got just the single Vector for which we need to know what is a linear combination and that Vector is not a zero Vector it has nonzero

elements but um it's still it is just a single Vector then all its linear combinations given that it is simply a scaled multiplication of

it we are all getting them on the same line so you can see all the linear combinations of this single Vector is just a scaled version of it and it lies

on the same line so what this tells us is that essentially you can move along the line defined by this Vector a but you cannot

leave it so you cannot get a vector that is in here that is in here that is in here in here so you cannot leave this uh

line you will always stay on this line so this line essentially spend of

a so when we got a single vector and that Vector is not equal to Z Vector then the span of a is equal to and this

can be expressed as C * a given that the C is a real number so we already saw this independent of what kind of scalar we

will take any linear combination of it will end up simply the C * a so therefore we are generally izing this and we are seeing that the span of a so

to set of all possible linear combinations of this a is simply equal to C * a given that the C is a real

number this is basically the spend of a real uh uh Vector in a two dimensional space let's now look into the next case

the next example when we will calculate or we will Define the span of a perpendicular vectors so let's look in into another example when we are looking

for a case when the um when we want to find out the span of perpendicular vectors so imagine we have these two vectors Vector a and Vector B where a is

equal to 1 0 and then B is equal to 0 1 so we are still in our lovely uh two-dimensional space so let's first visualize the vector a it's quite basic

it is this one and then Vector B it is simply this one so we can already see why they are perpendicular so you can see that they are forming this um 90°

angle so right angle here and then we know that the span so that's exactly what we want to find out

so the span of a and b and this is what we want to find out and we know that the span of two

vectors is the set of all possible linear combination of these vectors so we want to see what are these all

possible inar combinations of C1 so all the possible outcomes that we will get when we get a linear combinations of these two vectors so

basically C1 * a plus C2 * B

because those are all the linear combinations of these two vectors C1 * a + C2 *

B nothing thing that we can see here quickly is that C1 * a so this part those are all the scaled versions of a

so scaling multiplications of a and this second term in the linear combination those are all the scal variations so scalar multiplications of

vector B which means that and we already have seen this time and time uh again that when it comes to Vector a all its

linear combinations they will lie on the same line so let me take this color so if I do 2 a so C1 is equal C2 then I

will be in here if C1 is equal to three then I will be here C1 is equal to 4 I will be here C1 is equal to 10 I will be

in here and then the opposite holds as well if C1 is equal to for instance minus uh 2 then I will be in here if it's equal to Minus 5 I will be uh my

Vector will look like this and so on so this means that all the scaled multiplications of vector a will lie on this

line so I can also say that the span of see uh the span of a so span of a is simply equal

to C1 a so you can see in here so on this line basically so this

is C1 a so this basically means independent what kind of C1 I will take with is 1 2

3 0 - 5 - 100 I will always end up on this line so this line so this about the uh scaled

multiplication of a but of course to create this linear combination of A and B we also have the second element which is the all possible scaled

multiplications with a vector B so C2 B2 so let's see what that looks like so if I for instance take C2 is equal to

Z I will be in here if I take C2 is equal to uh 2 I will be in here C2 is equal to 5 I will be in here C2 is equal

to Minus 5 I will be here so you are already seeing what is happening here so all the possible scaled multiplications with Vector B will be on

this line so now we are then getting that the span of B will then be equal

to C2 and then B and here I'm not uh using formal

notation I'm just trying to um I'm just trying to uh draft the idea of the spin of vector a and uh span of vector B because we are not uh done yet we still

need to combine the two in order to find the span of vectors A and B when they are perpendicular so when this angle is simply

90° okay so let's also add this on our plot so this is C2 and then

B so this already gives us an idea that all the possible combination of the two so when we add these two elements to each

other the outcome will always lie on these two lines but there is no way that we can

find any other coefficient for C1 or C2 that can help us to get a value that will be so a vector that will be in here or in here or in here or in here that's

just not possible so just you can try to go ahead and solve that equations like we did before before and you will see that there there is no way that you can

pick here a line and you can represent it as a linear combination of these two vectors it just not possible and later

on we will see why but just keep in mind for now that once we have this this type of vectors when two vectors are

perpendicular then um we cannot find a line a vector that is outside of the two lines so here you can see the xaxis

and the Y AIS but it can also be like this it can also be like this but then you cannot find any other line that lies outside of this area that you can uh

create a linear combination of these two different vectors and then you say then you cannot say that you can create a linear combination of these two vectors

A and B so therefore when it comes to defining the span of the two

perpendicular line we say that the span of a and b given

that A and B are perpendicular but also given that these values in this case you know a is equal to 1 is z b is equal to 0 and one then their spend you might

have already guessed is equal to C1 a plus C2 B given that C1 and C2 are

of course real numbers so in this case C1 and and C2 as expected are just scalar so they are just some real numbers coming from R and

uh this A and B those are vectors that are being spent and in this case specifically the vector a is equal to this one zero and Vector B is equal to 0

1 and this expression that we see here this pen this simply describes the set of all possible vectors that can be formed by adding the scaled versions of

this A and B so C1 a plus C2 B in order to form this linear combination so this set this set of C1 a plus c2b which is a linear combination all possible linear

combinations of the two vectors so um it effectively covers the entire plane illustrating that any point

in 2B space can be reached by some combination of A and B let's now move towards our final example that we saw also as part of our definition for the

span of vectors in order to check and to learn how we can usually check uh whether the two vectors they really spend the entire space so in this case

we got two vectors we got Vector uh V1 which is equal to one two and Vector V2 which is it's equal to three4 so in here

and also um we uh have in our example that it says the span of V1 and V2 is all over the R2 because any Vector in R2

can be expressed as a linear combination of V1 and V2 so the example basically is saying that if we know that um we can

express any Vector in R2 as a linear combination of V1 and V2 2 then we say that the span of V1 and V2 is the entire

R2 so let's actually go ahead and prove that from our example so we have X which we can represent as X1 and X2 and X1 and

X2 are just real numbers and we got V1 which is 1 2 V2 which is 3 4 and we got in our example that uh we need to prove

that the span of V1 and V2 is the entire

R2 so for that what we need to do is we need to prove that we can express our coefficients C1 and C2 in such way using

X1 and X2 that independent of what these X1 and X2 are so what kind of X Vector

we have whether this is like one two or this is 04 or this is th000 and uh 5,000

independent what kind of vector we get uh we give here so X1 and X2 values as long as those are real uh numbers we can

always find a set of C1 and C2 that we can use as coefficients in order to create a linear combination from vectors

V1 and V2 and in that case we say then the span of V1 and V2 is the in par R2 okay so let's go ahead and actually

prove that using our previous knowledge that we already gained so keeping in mind that C1 and C2 are unknown numbers for us whereas X1 and X2 are just a way

to describe those elements in our Vector X that will be provided to us so X1 and X2 will be basically know and C1 and C2 are the unknowns that we are chasing so

for that the first thing that I'm going to do is to describe this linear combination that we got here C1 V1 plus C2 V2 with actual equations unknown

equations and the way I'm going to do it is by simply filling in this Vector V1 and Vector V2 um

values so we have C1 and then C2 here and then here I got one two

plus and then three and four here and what is this amount so this is equal

to let me actually go on to the next row so we can create um set of equations

for this so this is equal to X and we get C1 *

1+ C2 * 3 is = 2 and remember that this is X and we said that the x is equal to X1 and X2 so

this basically equal to we can B right here is equal to X1 and X2 so this is then equal 2

let's not skip all the steps X1 and X2 so here then the second elements need to be added so C1 *

2 and C2 * 4 which is then the same as

C1 + 3 C2 and then 2 C1 + 4 C2 and then this we are saying this Vector is equal to X1 and

X2 so this is what we have here and let's move from the vectors to equations so given that we have this we are allowed to say that this gives us

actually two equations this means that this element this element from this part should be equal to this and this element should be equal to this now let's write

it down we see that c1+ 3 C2 should be equal to X1 2 c1+ 4

C2 is equal to X2 this is all that we see in here let's remove this to keep the space

clean now what this means is that we have two equations with two unknowns C1 and C2 and X1 and X2 are the

numbers that will be provided to us as part of our Vector so what we want to prove is that we can describe and we can

express C1 and C2 which are our unknowns using X1 and X2 so you see here this is C1 C2 those are our nouns and we

want to describe them by using X1 x 1 and X2 and very soon we will also see why so for now let's try to express those two

unknowns using our nouns like X1 and X2 so here I already see that C1 is alone so there is no scaler so I will make use

of that opportunity to keep the C1 on the left hand side and I will take this this amount to the right so I will say

C1 is equal to X1 minus 3 C2 two okay slightly better so I have C1 at the left I do have X1 in the right but I

also have three C2 in here but another thing that you will notice is that in my second expression here I got 2 C1 + 4 C2

plus X2 I want to have the C2 only because then I will have an expression

of my C2 only using X1 and X2 um so numbers that are that will be provided to me that are n so for that what I'm going to do is basically trying to solve

uh two equations with two unknowns exactly the same um process so I'm going to take this C1 from the first equation and I'm going to fill in in the second

equation so I am going to say two times and here I'm going to fill in that C1

expression from here so X1 - 3 C2 2 so this is my C1 plus just taking over

this part so 4 C2 is equal to X2 okay perfect so now what I end up with is C1 is = to X1 - 3 C2 just taking it over

and then here I'm opening parenthesis which is 2 X1 - 6 c2+ 4 C2 is equal to here I forgot an X2

is equal to X2 okay one step closer why because I in my second equation I no longer have a C1 I only have a C2 which is great which means

that this gives me an indication that I can rewrite the C2 which is unknown with nouns with X1 and

X2 so let's make use of that opportunity the first equation I would just take over so C1 is equal to and then

X1 - 3 C2 and then here I will do 2 X1 and then here

we got 2 * c2s which means we can combine them so Min - c - 6 * C2 + 4 C2

it gives me - 2 * C2 and this is equal to X2 all right let's now solve that

part so what I want to have is just the C2 in the left hand side which means I need to bring all this to the right and I need to get rid of them such that I

can leave the C2 in the left entirely alone so this is what I'm basically chasing for that I'm going to once again

rewrite C1 is equal to X1 - 3 C2 and this time I'm going to take them minus 2 C2 here I'm going to leave that

in the left but then this one I'm going to bring to the right so X2 - 2 X1 here I need to be very careful to not

make a mistake CU that will mess up my entire calculation all right so now we are one step closer just taking over the first

equation again so C1 is equal to X1 - 3 C2 and here what I need to do to get rid of this minus two is to divide the two

sides so both 2 minus 2 CU that will help me to keep the C2 only in the left

alone without any scaler so the C2 is then equal to X2 minus 2

X1 / 2 - 2 so this is what I end up with perfect so we are very close stay with

me so uh here what we are getting is that C2 is equal to this amount we see that now we no longer have any other C

in here which is great and remember that X1 and X2 will be numbers that it will be provided to us I just wanted to give everything General and then uh another

thing that I want to fix is this C2 because this C2 is an unknown and I want to fill in uh this value of C2 in here

such that for the C1 I will have a similar picture so in the left hand side I will have C1 in the right hand side I will Express the C1 with no number so X1

and X2 but not the C2 or others all right so let's then go ahead and do that first I will write C2 in a simpler way

so C2 is equal to here I got a min I will just write here minus so I will take the minus over here and then I will write X2 - 2 X1 to U be super careful

with this minus therefore I'm using parenthesis so now I'm going to use this C2 and I'm going to fill that in in here so C1 is equal to

X1 minus 3 * I can also make it plus because minus of here so minus of here and minus of here

will cancel out therefore I will do plus three times and then X2 - 3

X1 / 22 and this then gives me

C1 is equal to X1 + 3 / 2 * X2 - 3 sorry 2 almost made a

mistake 2 X1 and then C2 is = to X2 - 2 X1 /

2 2 and here we got minus

Perfect all right awesome so now we have expressed X1 and

X2 well careful with this X1 and X2 we only noun numbers now what I'm going to do is that I'm going to prove that independent what kind of X we will be

taking here we will end up getting the C1 and C2 using this what we just found here that will give us a linear combination of these two vectors that

will be equal to that eight so for that so to prove that this pen of V1 and vs2 is the entire R2 I need to prove that independent what kind of X I will take

so X1 and X2 I can always find the C1 and C2 that I um just calculate in here using that X1 and X2 that I can then use

to combine with my V1 and V2 to find the linear combination of these two vectors with that C1 and C2 which will be equal to this

x so for that what I need to do first is to take such a uh random X so let's say

my X is equal to 0 and 4 this means that my X1 is equal to 0 and X2 is equal to 4 what this means is that this gives me C1

which is equal to and here X1 so I'm basically filling these two values for here to obtain my C1 so C1 corresponding

to this specific Vector X so X1 is equal to 0 which means I end up C1 is = 0 + 3 / 2 * X2 is = 4

so 4 minus and then 2 * X1 is equal to - 2 * 0 which is

0 and then C2 is = to minus and then X2 is equal to 4 so 4 and then minus 2 * XY

is = to 0 0 and then this divided to two now what are those numbers so C1 is equal

to 3 / 2 * 4 which is 3 * 2 so 6 and then C2 is equal to- 4 and then minus so this is zero

this cancels out which means 4 / 2 is 2 and then C2 is equal to minus

2 so basically I have calculated the coefficients C1 and C2 by just knowing

what is this Vector so knowing X the provide X1 and X2 I have calculated my C1 and C2 using my

derivations in here so let's now get rid of this this calculations to clear some space and to do the final part which is

compute the linear combination of vector V1 and V2 for this specific coefficients well knowing what this given Vector now

is example random Vector so the C1 is equal to 6 which means 6 * and then Vector V1 is one 2 so this is first part

of my linear combination plus and then C2 is equal to - 2

times then here 3 4 what is this this is equal to 6 and then 6 * 2 is 12

plus now let's calculate the second part - 2 * 3 is - 6 and - 2 * 4 is

-8 so what does this give us 6 - 6 and 12 - 8 this gives

us zero and four nice so this confirms that we have done everything also correctly which is

great because we have seen that using this C1 and C2 that we have just calculated we have successfully uh

computed the 6 V1 so linear combination of this uh two vectors V1

plus - 2 minus 2 and then V2 and we have seen that this linear combination is equal to 04 which is

exactly our 8 so in this way we have proven that independent what kind of vector we will pick what kind of X we will pick here we can always find and

calculate the corresponding coefficients C1 and C2 in the same way as I just did and then by using those when we calculate the linear combination of

these two vectors with this specific coefficient this will be exactly equal to X and this proves that independent what kind of vector we have in our R2 we

can always express that as a linear combination of the vector V1 and V2 and this proves and this concludes our proof

that span of V1 and V2 is the entire R2 all right so we are very close to

finishing up this unit so the next topic we are going to talk about is a linear Independence and all this important stuff that we learned as part of the previous modules are going to become

super handy as part of this specific concept so we just spoke about the idea of span we have plotted a lot of vectors we have seen the linear combination of

that and how we can find out whether the span of multiple vectors is the entire space uh for instance the R2 or it is just the line or it's maybe the zero

Vector we have seen many examples and many operations we have we have also seen this idea of unit vectors and we are finally ready to come to this very important concept which is a concept of

linear Independence so by definition linear Independence says that the set of vectors is linearly independent if no

Vector in a set can be written as a linear combination of the others otherwise they are linearly

dependent so vectors V1 V2 up to VN are linearly independent if and only

if the only solution to the equation C1 V1 + C2 V2 plus CN VN is equal to zero

is C1 is equal to C2 up to CN is equal to Zer in other words in a l linearly independent set the equation C1 V1 + C2

V2 plus CN VN is put zero has only the trial solution where all CIS are zeros so now what do we mean here there

is a ton of information in this definition so let's unpack them firstly it's really important to uh keep in mind this idea of

Independence and dependence Independence and dependence there are things that we commonly use in data science in artificial intelligence in statistics so

those are really important so we basically have linear independent condition so there is a certain condition that our vectors should

satisfy vectors in our set in our Vector space for them to be named as linearly independent and otherwise we are calling them linearly dependent and you can see

that here there are a couple of Parts as part of this definition first it talks about um being unable to create a vector

in the vector set while using the remaining vectors in our set so it says if you can use the remaining vectors in

your vector space and linear create a linear combination of them so linearly combine them and we have already seen the definition of linear combination so

if we cannot create such linear combination from the remaining vectors to get our Target vector then we are saying that we have a

linearly independent vectors so if we want to say that all our vectors in our Vector set they are linearly independent it means that each of those vectors we

should not be able to recreate out of the remaining vectors so we should not be able to find coefficients to create linear combination using the remaining

vectors in order to get our Target vector now what do I mean by this target Vector what do I mean by this linear combination uh I will come to this in a

bit for now let's just try to unpack this definition cuz uh with examples uh we will definitely go through this step by step in detail such that this ideal

linear Independence and dependence is super clear so in the second part of the definition it says vectors V1 V2 up to

VN are linearly independent if and only if the only solution to the equation and we have here in the left hand side you might recognize the linear combination

of our vectors V1 up to VN so in in the right hand side you have zero so you are saying our linear combination of vectors is equal to zero if and only if C1 C2 up

to CN is equal to zero so linear Independence basically claims that we will have linearly independent vectors

only if and only in the condition when um the only way we can create linear combination of these vectors equal to

zero only if those coefficients are zero there is no other way that we can get a linear combination that is equal to zero

while those coefficients are not zero so the only way that we can get a linear combination out of all our vectors equal zero is only when all of the

coefficients C1 1 C2 up to CN is equal to zero that's something that we will come later to this again this something also that we are going to come back in

our next module and the next one so um this one will be also super clear once we go through those modules but for now keep in mind that the uh linear

combination of all these vectors can only be zero in case when all these coefficients are equal to zero so and then we have the third part in our

definition which says that in other words in a linearly independent set the

equation C1 V1 plus C2 V2 up to CN VN is equal to zero has only the trivial solution where all CIS are zero so this explanation is basically what we just

spoke about as part of this second part where we said that only in case the coefficients are all zero we can have a linear combination of our vector V1 V2

up to VN which is equal to Zer and why we would like this linear combination to be equal to zero because it's a common way to find solution to

our linear system so this is something that we will also see as part of the next module when we'll be discussing the idea of solving linear systems we will

go into more uh Advanced topics but for now in order to understand this idea of linear Independence we should just keep

in mind that we cannot find any CIS so C1 C2 so any coefficients that is not equal to zero and then expect that the linear

combination of these uh linearly independent vectors is equal to zero so that's the uh if and only uh if and only uh if part which means that this holds

from both sides on one hand we have V1 V2 up to VN which are linearly independent only if the linear equation so the linear combination of all these

vectors is equal to zero if all these coefficients are zero but also the other way around holds as well so if we have a linear combination that is equal to zero only if those coefficients are zero that

means that we are dealing with a linearly independent vectors this is the if and only if part which means that we have this uh conditions from both sides

if one holds the other one holds but also the other way around all right so let's now look into specific examples that will make our journey in understanding linear

dependence much more convenient so let's say we have our coordinate system and we have these two different vectors so we have Vector let's

say 2 and three which is our Vector a and we have a vector B that is equal

2 6 and N so those two are our vectors and what we want to understand is where those two

vectors are linearly independent or linearly dependent so one thing that you can quickly notice is that b looks quite

similar to a in terms of its scal so there is a way that we can recreate

Vector B by using Vector a now you can see that if I take Vector

a which is equal to 23 if I take Vector a and I multiply it by three so three * Vector a this is a

scale multiplication then what I can get is three times and then I have here two

three and this is then equal to 3 * 2 is 6 3 * 3 is 9 this gives me 6

and 9 which is our uh Vector now another thing that you can notice that that is exactly my B so you can see that those

two are similar which means that three * a is equal to B now what this means is that I can

recreate Vector B by using Vector a so in our definition we saw that a set of vectors is linearly independent if no

Vector in the set can be written as a linear combination of the others so here I can take this 3A as a

way to write down a linear combination so 3A + 0 *

B is then equal to B which is basically saying 3 a is equal to B so by using these two vectors in a

set I can then create a linear combination of the two and actually even basic way of writing this is saying I

can use the vector a to write a linear combination from this so 3A is a linear combination so just a scaled multiplication in this case of course

but if we have just two vectors our Target Vector is B and I want to write this uh I want to see whether I can rewrite the vector b as a linear combination of the remaining vectors

which is Vector a so I can then write Vector B as a linear combination of vector a because I can say that 3 * a is equal to Vector

B so this means that Vector a and Vector B they are linearly dependent this means that I can use

Vector a to recreate Vector B and of course I can also do the other way around right what I can do is that I can just

take Vector B so I can take Vector B I can multiply it by one ided to

three 1 / 3 is real number so I'm just performing a linear combination using B and this will give me 6 / to 3 is 2 9 /

to 3 is Tre and I'm getting exactly what I have under a so I can then also rewrite Vector a by using Vector B so I

created a linear combination using Vector B in order to get a vector a and that's exactly the opposite what we have learned here

because we should not be able to write this a vectors using the other ones in our set cuz otherwise we have a linearly dependent set

therefore we are saying that Vector a and Vector B they are not a set that is linearly independent but they are linearly dependent before moving on to

another example I also wanted to visualize these vectors just to see what is going on with this pan and uh how the two linearly dependent vectors look like

in R2 so this is our r t we have a vector a which has two tree elements so we know already the magnitude and the direction this is two this is three

which means here let me actually use another color so 2 three which means this is my

Vector a and then my Vector B is simply six and N so it is this

one so you can already see what is going on so this is Vector a and this entire thing is Vector B

and you can see that those two vectors no matter how I combine them I can I will always get the combination so

linear combination of the two on this line if I want to get um Vector that is for instance in

here I can never find a scalers of

C1 and C2 in such way that these vectors so A and B they can form a linear combination that will give me

this Vector there is no way that I can do that and that's why uh we say that this span of this two

vectors so span of A and B with this A and B is this line and we cannot express any of these

other vectors like this one or this one using a linear combination of these vectors A and B the only linear combinations that we can recreate using

these vectors A and B are on this line so you can see that even if I have two different vectors I actually just got um

single Vector because I have two Tre and both of these vectors they are actually um uh scaled multiplication of the other

one so B is equal to I'm missing here something 1 / 3 so B is simply equal to 3 * a and then a is

equal to 1 / to 3 * B so in both cases they are simply a version of scaled multiplication of this Vector Q3

so a is simply equal to B * 13 and then B is equal to 3 *

a and both of them they are actually based on this Vector 2 Tre on this Vector

a so therefore they both actually form and they span or round this single line and they are

both linear we also call it collinear and they are linearly dependent okay so let's now move on to the next uh example where we will have

bit more interesting case and we will look into this example when we have linear Independence look into another example bit more interesting one as we want to see

whether those two are linearly independent or not so the first Vector that we got is the vector a the vector a

is equal to 6 and Z so it is this Vector this is Vector a the vector B it is this

one and it contains element of Z 0 and 7 so it is

this Vector this is the vector B now in our definition of linearly independent vectors we saw that the idea of linear

Independence is that the two vectors can only be linear independent if we cannot rewrite one of them by using the other

so this means that we cannot rewrite a in terms of B and we cannot rewrite B in terms of a so there is no way that we can scale the vector a to get Vector B

and there is no way that we can scale Vector B with vect with some uh scaler in order to get the vector a so there is no way that we can create a linear

combination of this one vector to get the other one and the other way around so let's see whether this is the case just from uh trial and error we have a

vector a which contains elements 6 and zero for us to go from A to

B that has elements from so we need to go from 6 to zero in this case and we need to go from 0 to 7 now we can automatically already see from the

second element that there is no way that we can go from 0 to 7 you cannot find any scaler C that you can multiply with zero in

order to get seven there is no way that you can do that because any number any real

number that is a real number if you multiply it with zero it will never become seven and of course another thing that

you can notice here also very quickly is the other way around right so here if you go from this zero to six there is no way you can go from this Z to six

because there is no such C that you can take this zero and multiplying it with that so here our

scaler and you get this equal to six this is just not possible so what we are seeing here is that there is no way that we can somehow

change this vector so there is no way that we can scale them in such way so this is minus

B so all the scales scaled version of this or all the um scaled multiplications of vector B they will always be on this

line and then the same holds for a as well so all the scaled multiplications of a will be on this line so then one thing we can quickly see

here is that given that those two are perpendicular this

pen of A and B is the entire R2 so we can see that by using those two lines we can recreate any other line in

this R2 and this is highly related to this idea of linear Independence and given that we cannot come up with a linear combination using

the a vectors to recreate the other one in this case given that we cannot recreate a using B and we cannot recreate B using a so no linear

combination that exist that we can use to recreate Bay using a and the other way around we are saying that Vector a

and Vector B are are linearly independent let's now look into another example that will uh clarify this linear

Independence concept so we have three different vectors and the first Vector is Vector a 1 0 0 Vector B uh 0 1 0 our

second vector and the third Vector 0 0 1 you can notice that we are in R Tree and then the example goes on and it says that those three vectors are linearly

independent and as an explanation we have that there is no way to add these vectors together with any scalar multiples to equal the zero Vector

unless all scalers are zero now before even going on to next part it's actually very quickly um uh provable that those three vectors are linearly independent

and you cannot create a linear combination of one using the remaining of the two let's look into this example in more detail so we have three vectors

A1 A2 sorry B so we got a B and

C which are 1 0 0 0 1

0 and 0 01 now you can quickly see that if we are in R3 and this is actually our unit Vector

E1 this is our unit Vector E2 and this is our unit Vector E3 because in that positions we got our ones and the remaining they are all zero and this is

actually very similar to the previous example because we can quickly see how we are we will not be able to recreate one vector using the other ones by even

looking at the positions of the zeros so for us to recreate Vector a which is

equal to 1 0 0 it means that we should be able to

find a linear combination C1 C2 and then using these vectors this is the vector

B 0 1 0 plus C2 * Vector C which is 0 0

1 so in here basically we are already seeing a

problem because we have here here an element one and we somehow need to be able to find

C1 and C2 in such way that 1 is equal to C1 * 0

+ C2 time Z but we know that there is no C1 and C2 that we can find such this uh expression actually is true because C1

and C2 they should be real numbers and there are no real numbers that we can find to multiply with zero such that this will end up to one because this is

always equal to zero and we basically get 1 is equal to Z which is not true and of course the same holds the other way around you can prove that B

can never be um recreated by using the linear combination of a and c and also the C can never be recreated by using a

linear combination of A and B therefore we are saying given that

a can't be written as linear combination

of B and C B can be

written so the same only this time A and C and then C

hunt B written as linear combination of A and B those

vectors A B and C they are linearly independent and if stronger you can

actually go ahead and prove that this Spen of these three vectors is the r Tre but that's outside of the scope of this example so we will just pass but I will

leave that um to you to prove all right so now when we are done with that let's actually move on to the last module which is the dot product and

its applications so uh the length of a vector and Dot product is a con cep that um we um are familiar from the high

school so the length of a vector is deeply related to this do product idea the dotproduct of a vector v WID itself

gives this a square of the length of V what basically um it means is that this dotproduct of vector v so this thing

which means take the vector v and multiplying it with the with the other vector v is simply equal

to the square of a length of B so this is way to express the length of

the uh of the vector B and once we square that that is the dot product so that's basically this definition what is about so we know what this definition of

the distance is and we Define it by this and then we take the square of that distance and there is our DOT

product and we are going to see this IDE of dot product a lot especially when it comes to uh matrix multiplication Vector multiplications also in many

applications of linear algebra you will see this idea of that product coming again uh and coming back to us so uh this is a concept that we really need to

understand so in the two dimensional space let's say we have a vector B which is um consisting of the two elements X and Y then the dot product and the link

are related by V by V this is the way we denote the dot product so we just simply use the dot and the name also makes sense because we are saying we are using

the dot to perform dot product so we are multiplying to two we are creating the product of this Vector with itself and

this is equal to x² + Y 2 which is equal to the uh um squared of the distance of this Vector now you might recall from

the high school that we have learned this idea of distance so if we have x-axis Y axis then we basically use

this uh x² + y sare to uh get the uh you know the formula for from our Circle and then uh we have the x square + y Square we take the square root of it and then

this is our distance so once we take the square root of that square of that from this uh square root of x square + y Square then we are simply getting this

two cancel out which is equal to x² + y^2 So This is highly related to this idea because we are again talking about distances and we are simply taking the

distance we are squaring them up and then we are getting the dot product so this the double uh straight line

this is just a notation that we use and we spoke about this also before this comes um from the pre-algebra and this um this is highly

important related to this idea of pythagore theorem and how we compute the distances so for instance when we have

this uh Square triangular so we have this um uh rectangle here and we have here the 90 uh

uh great so here we have the right uh right um angle and here we have our C which is uh the side right in front of

this uh 90° angle and here we have the A and the B and we say that the c² is equal to a sare + b

sare and if I were to actually write this in terms of X and Y so if this side is X and this side is

y and this is my Z let's say then z s would be equal to x² + y² and this is something that we can see

here too and the two terms are highly related so the Z squ is equal to x² + Y 2 and this is

simply equal to Z * Z right and this is something that we know from High School welcome to the module one of this new unit when we are going to talk about

about matrices as well as linear systems so those are all fundamental concepts that you will see time and time again when applying linear algebra not only in mathematics techs but also in applied

sciences like data science artificial intelligence when training different machine learning models and trying to see what is this mathematics behind machine learning models different

optimization techniques when you want to solve different problems using linear algebra so in this first module as part of foundations of linear systems and

matrices we're going to introduce this concept of linear systems and then we are going to talk about the general linear systems we are going to uh see this common labeling of the coefficients

this idea of indices that refer to the rows and the columns we are going to see what is this differentiation between homogeneous and nonhomogeneous systems

so without further Ado let's get started so uh the linear systems form the uh bedr of linear algebra modeling this

array of problems thanks to this advancements in these linear systems and Sol in it in Computing we can now solve a large amount of problems in a very

efficient and a fast way so uh the general linear systems can be represented by this uh set of M equations with n

unknown in the previous unit when we were looking into this uh linear combination of vectors we saw this

notation which was A1 and then we we had C1 multiplied or rather let me keep me uh let me keep the same notation so

we had this linear combination of vectors so we had beta 1 and then we had

A1 Plus beta 2 and then A2 and those are all vectors plus A3 so beta 3 * A3 dot

dot dot and then beta m times a m this is the notation that we saw before and we said we want to come up we wanted to come up with the linear

combination of these different vectors A1 A2 A3 up to a and then we use that in order to get a sense of whether we are dealing with linearly independent

variables vectors or linearly dependent vectors and then we also commented on the span that these vectors take now when it comes to um the uh

vectors and just in general linear systems we can represent what we had before now in terms of with a bigger

system so in terms of M equations and with n unknowns so here what you can see here is that we have M different

equations so we have beta B1 B2 up to BM so you can see it in here and then each of these equations it contains n

unknowns so you can see that the unknowns stays the same so the unknowns are those X1 X2 up

to xn so X1 X2 up to xn are the set of all n unknowns and then M equations that you

can see in here are all these equations so a11 X1 + a12 X2 dot dot dot and then a1n and then xn is equal to

B1 and here one thing that is really important to keep in mind is that the indexing is what we need to focus on so

we need to keep this one in mind this a i j and this XI so this is something that we also

spoke about when uh discussing the linear combination of vectors we slightly uh touched upon on this topic so let's now dive into this this

indexing and how do we indexes a i j what are this A's what are this JS and here you can see that we have a11 and

then a12 and then up to the a1n and this is in our equation one and then we have in our equation two A1

two let may actually write this with different color so in our equation two we got a 21 a 23 up to

a2n and this A's that you see here those are just real numbers so a11 can be 1 A1 2

can be three A1 n can be 100 and then the same also holds for this B1 for this B2 and for this BM and all these values

A's and B's they are just real numbers the only unknowns that we got here are those so the X1 X2 up to

xn all right so what about the indexing now so we got a i j and as you can see in this

case the first thing that we can see here it stays everywhere the same which is the one so we got here one we got here one and up to the point we got here

one whereas the second Index this one it does change it grows gradually with

one and it becomes it goes from 1 to two and up to n so you can see here that the first

index first index or index I it

goes from one it doesn't change it's just one so it is one one and one so here in all cases for this equation I is

equal to 1 but another thing that you can notice here is that the index 2 unlike index I so the second index which

is the J so you see here that the second index is referred as J this is a general way of defining the indexes so here J is

equal to 1 2 dot dot dot and then n so basically the I doesn't change in the

same row but the G changes and then of course we have slightly different in terms of I but then the same for J for our second

equation so here I is equal to 2 and then J is again equal to one and then two dot dot dot and then

n and then here up to for the last equation our I is equal

to M and then our J is again equal to one till two dot dot dot so you might notice that I was

looking at this from the row perspective so I was saying pair equation or pair Row the I doesn't change but then the J

stays the same and then it is either one two up to n but the set is the same so it is it contains all these different elements here so one one two and then n

but it contains all these different real numbers going from one till n because we are combining and we are creating this combination the sum of all these values

a11 and then X1 a12 X2 A1 n xn and another thing that you can also notice here is

that here with the second index so with this J J is equal to one then here the X's corresponding index is

also one when the J is equal to two then the ex's corresponding index is also two and then here the same story and you

will notice that while the coefficient contains two indices 1 one one 2 or 1 n which are the two indices for the

coefficients for the unknowns we got but just single index which goes from one till n so basically 4 a for the

coefficients so I let me write with the right

color so I can be one 2 all the way to M whereas in case of J it can be one to all the way to

n and the indices are basically used to help

us to keep track of in which row we are and what is the um

variable that the coefficient belongs to because knowing this second Index this helps us to understand that we are dealing with a coefficient that

corresponds to this first unknown the first variable X1 and then the same holds in here as you can

see in here and in here we are dealing with the same variable X1 therefore the second index the index J is then the

same both in the first equation and in the second one in both cases it's equal to one okay so now when we are clear on that let's

also understand this high level concept because you will see this system of linear systems this m equations and N unknowns appearing a lot not only in

terms of calculating and finding the solution to this linear system but this actually has a very common application when it comes to um running regression

linear regression specifically and one thing that you can notice here is that here we got also this B1 B2 up to

BM and you will notice that here the index also uh goes from one but then this time to M so when it comes to the

rows we have M rows or M equations therefore we also expect when it comes to Counting from the top that at the bottom we will see an M whereas

if we count from this side so kind of like imagine it like a column then we see that it goes from one till

n so those are common observations and reference to um number of observations and number of uh features that you will

see in your data when dealing with data analysis or modeling data so just this uh just keep those things in mind this uh abbrevation of M and then n m

equations and unknowns because this will become very handy and the same also holds for this indexing just to keep in mind that this I and this J what those

indices are and how for instance the first you know the I the first index changes when we go from up to the bottom

and how the second index J goes and changes when we go from left to the right when we go through the columns but we are going to it is also in the uh

upcoming slides so uh we can we will have time to practice it so um this is what we are calling a coefficient labeling the coefficient uh

a i j so this thing in a linear system they are labeled where the first index represents the row and the second index

denotes the column so when we see a i j we know that this refers to the row and the J refers to

the column so this is something that we use in order to understand where exactly in

our metric something that we can we will see very soon where exactly our unit or our uh member that is part of our Matrix

where exactly is that located in which row and in which column the systematic labeling is super important because this helps us to keep the structure and this helps us to

understand uh what does this uh coefficient represent what what is this row that it belongs and what is the column it belongs so for which equation

and for which unknown we have already solved the problem such that we can know what this uh coefficient represents so before moving on onto the actual linear systems and the definition

of metrices let's quickly understand this distinction between homogeneous and non-homogeneous because this will help us to also get an understanding how we

can solve a system of linear systems so a system is homogeneous if all the constant terms b i are zero otherwise

it's non homogeneous so identifying this helps us to really understand the nature of the solution set that we need to get and to understand what kind of strategy

we need to use in order to solve this problem now what do I mean by bi we is so that we had this system of M

equations with n unknowns and we saw that that we have in the right hand side this B1 B2 up to BM which means that we

had this m different equations with n different unknowns and to find a solution to the system it means finding this value values corresponding

to X1 X1 here X2 X2 xn so basically finding the set of X1 X2 up to xn that solves this problem and for us to know

how to solve this problem we need to know whether this B1 is equal to zero or not this B2 is equal to zero or not and

then this BM is equal to zero or not this is very similar to this idea of solving any sorts of um problems that

contain unknowns for instance if we have three x is equal to let's say five solving this is

entirely different than if we know that the tree exal to Z so this is a simplified version of

course but the IDE is the same knowing that this B1 B2 up to BM this R zero this gives us an idea how we can solve this problem and later on we will see

this distinction between non-homogeneous and homogeneous system and whenever these BS so whenever this B1 B2 up to BM whenever these BS are zero then we are

saying that the system is homogeneous and we need to solve a homogeneous system otherwi wise we are dealing with nonhomogeneous system so this means that

the bis are not all zero let's now move on to the second module which is about the matrices so we are going to define the Matrix we are going to see the definition of it as well as the notation

this idea of rows columns Dimensions uh some of which we have already touched upon but we are going to uh go into the depth of it we are going to learn properly as well as we are

going to see many examples then we are going to talk about Matrix types so here we will talk about identity Matrix diagonal matrices and also special type of matrices like matrices containing

only zeros and only ones so by definition add a matrix is a rectangular array of real numbers that

are arranged in rows and in columns for example an M byn Matrix a can be

represented as follows so let's look into this definition and this reference to Matrix we call this Matrix or

Matrix a and every Matrix it can be described

by this rows and columns where we always have this uh way of describing this Matrix always should be

defined by the number of rows and number of columns so this is super

important and let's look into this specific Matrix so we have a matrix a and all these values they are members of

this Matrix they form the Matrix and we already saw this labeling of a i j where we said that I is

referred to the row so you might recall that those were all these equations that we got so this horizontal lines where I

was equal to 1 I I was equal to two I was equal to three up to the point of I was equal to M and then we had this J so

this thing and then J was referred to the columns and we had J

was here one and then two and then three up to the point of n so one 2 3 and

N this is exactly what you can see here so in this Matrix we got all these elements a11 is a number a12 is a number up to the a1n is a number those are all

real numbers and one thing that you can notice here is that here we got a11 so this is our first row and First Column

here we got A1 two this is our first row and second column and then we got up to the point of a1n actually let me just

write this down even at a bigger scale such that I can make more noes so let's assume we have this Matrix a and this Matrix a

a if I'm bigger and we got all these different elements so we start with our first row

and here we have A1 1 so here the row that I will write with

let's say with blue the r is equal to 1 and then

the column is one so this is Row one this is Row one row one and this is column

one let me write it with red this is column one this is column two this is column three dot dot dot and

this is column n and this is row two this is Row three

dot dot dot and this is row M so in total I got M rows and N columns I will

come to this notation that I'm putting here later for now let's keep track of the rows and the columns to get a good understanding what this indices were about that we just

learned so every time I will also mention this reference to a i j to keep track of this and also let me write it

with the right colors so a i this is the row and J which is the

column so all the elements I'm just defining by this a because it just a way to reference a part that comes from a matrix it's a just common way to write

the higher matrix by capital letter A whereas its members we will write with the um with the lower case a so this is

Matrix Matrix a all right so here in the second row

But First Column we got a two and then one because it is still in the First Column and then when it

comes to this element we have here a the row is the first one because we are in the first

row but then we are in the second column so this one should be two then we go on to the next element in

our first row so a one and then three and then dot dot

dot the last element is an a as we are still in the first row it will be one the I but then given we are in the last column the column index or the J will be

equal to n because we got in total n columns so we are now ready to go into

the second row so here given that we already have our first element a21 this is in our second row and the

First Column so the I is equal to here two and G is equal to 1 let's now write down the element in the second draw second column as you might have already

guessed I is equal to here 1 I is equal to here two s and then uh the J is equal to 2 and then we go on to the next element

which is in the second row and the third column so it's a the row index is 2 so I is equal to 2 and

then the column index is three dot dot dot and then we got a as we are in the second row it is

the I is equal to two and as we are in the last column the J is equal to n now you might have already guessed when I was writing this down that whenever you

are in the row and you move on to all the elements in the same Row the I so the row index it stays the same only you need to uh update the

column index so here for instance you got one one one here also one so all the way down in the same row or one which logically makes sense because we are in

the same row so the row index should not change but instead you should change the column index like here column one column two column three all the way to column n

so those are our columns dot dot dot so let me make this distinction and those are our

rows as you can see so this kind of mentally helps us to understand why we are writing all these indices

over time once you practice more with this this will become more natural very quickly remove this so now our ride rest very quickly

so as you might have already guessed we are in the third row so we have a tree so everywhere I will just write down

the ace so first write down the A's and then the rows the row index will stay the same as I in the same row but then I will increase the columns gradually so

we are in the column one and the column two column three up to the column n so now the remaining stuff you can actually

write down yourself to just practice let's now move on on to the last row and last column so in the last

row we got a a a up to here and in the last show the uh row index is M which means that here I need

to have M M M everywhere I need to have M and then the column index is 1 2 3 all the way to

n so this last column is very interesting too you can see here that we have the opposite of what we have here

because in the last column we see that the uh column index is the same so it is everywhere n Only the first index the

index of the row it changes it goes from 1 2 3 up to M which is of course logical because we said that in the last column if we are looking it from the perspective of column so all these

values this A's so the all the ends they are logical because they we are in the last column we are in the same column but then the row changes here we are in the row one here we are in the row two

Row three of two row M therefore we have also at the end a m n now let's talk about this idea of

MN we said that our Matrix a has

M as a number of rows and n as a number of columns which you can see by the way

also here so we always refer the dimension of a

matrix so the dimension dimension of Matrix a by these two

numbers so first we always write down the number of rows in this case M then as the second element we are

writing the number of columns in this case n we are always putting this small X in between two kind of emphasize M by

n Matrix and we most of the time use the square braces to Showcase that we are dealing with Dimension and in this case we are saying the dimension of Matrix a

is equal to M byn so we are dealing with M byn Matrix this is a common convention used in linear algebra in mathematics General but also used in data science uh

in machine learning artificial intelligence so whenever you are dealing with matrices a it is a common convention to talk about this idea of dimensions and the idea of Dimensions is

super important when it comes to the idea of multiplication multiplying Vector with Matrix Matrix with Matrix so this dot product Dimensions play a

central role in here so keep this one in mind once we uh get to the point of that products this one will become very handy so let's now look into a specific

example where we see simple Matrix a so in this case you can see that we are dealing with a matrix that has a 2x3

Dimensions so like we just learned 2x3 means that we got two

rows and three columns that's something that you can also see here very quickly so you have a small Matrix on the small matrix it's

really easy to actually count so you can see that we got Row one and row two and we got column 1 column two and column three so this basically confirms this

Dimensions therefore we are also saying that we have a 2 by three Matrix and like usual we first

write down the number of rows and then the number of columns you can see here that here we have this elements for our Matrix so a is equal to 1 2 3 for the

first row and then uh 4 5 6 for the second row so from this actually I think it's a good exercise to just uh very our understanding of indices and from this

um we can write down that for instance all these different elements uh like a 1 1 is equal to 1 A1 2 which means that we

are in the first row and in the second column so we have this element is equal

to two and then we got a and then one Tre so we are in the third column so

this one 1 is equal

to 3 and then a 21 is equal to 4 a 22 is equal to 5 and then a 23 is equal to 6

so this is actually a good way to practice our understanding of indices our understanding of this Matrix structure and the understanding of dimension of the Matrix which in this

case is 2x3 so this is yet another different definition of a matrix structure when it comes to the rows coms and dimensions so this is exactly what

we just spoke about on our example and let's just quickly look at the formal definition so the rows of a matrix are the horizontal lines of the of the

entries while the comms are the vertical lines so basically it's saying those are let me remove

this so the rows are are the horizontal line and the columns are those vertical lines those are the columns this helps us to form these columns so column one

column two and column three whereas this horizontal lines it helps us to create the rows so Row one and row two so then we have the dimensions of

Matrix are given by the number of rows and columns it has so an M by n Matrix has M rows and N columns that's

something that that we already saw so let's look into some special type of matrices one Matrix type is the

identity Matrix so we saw before we had this Identity or unit Vector now we have identity Matrix so the two are quite

similar so like before when we had our unit vectors we had this for instance E1 in three dimension we had 1 0 0 then we

had our E2 which had 0 1 0 and then we had our E3 which was 0 01 so you might recall this about our identity vectors

or we were calling it unit factors you might notice very quickly that we have formed an identity Matrix i n which is a

square Matrix with one on the diagonal and zeros elsewhere is basically a matrix that is built using those unit vectors so here we have E1 here we have

E2 and here we have E3 so you can also see that this 3x3 Matrix because we got three

rows and three columns so you can see that here we have on the diagonal so we call this diagonal on this diagonal we have all

ones and in here outside of the diagonal they are all zeros and this is the definition of

identity Matrix it is this i n Matrix where n is the dimension of a matrix and given that it's a square Matrix it means

that the dimension of it is n by n so all the rows so the number of rows is equal to the number of columns on the diagonal we have all these ones and

every where else we got zeros and do note that we are forming this identity Matrix simply by combining

these different uh unit vectors so like here E1 E2 and E3 so let me actually uh give you yet

another example but of much higher Dimension so of this identity Matrix so

let's say we have I and then this I uh let us actually use this notation i

n so let's say we got i n what this means is that we got actually this large

matrix it's a square Matrix which means that it is n by n so it has n as the number of rows

and n as number of columns so the dimension is n by n you got n as number of columns too because

it's a square and let us actually write down that how that Matrix looks like it's a large Matrix the N is the size of

that Matrix so here we got on the diagonal we got one here we got one here we got one dot dot dot up to the last

point one and the index of this one here so this is the first row this the First Column basically and everything else is simply

zero so here we got z0 0 dot dot dot zero here we got 0 0 all the way down to zero here also zero all the way down to Z

and then here also zero so everywhere here and here we all got zeros only on this

diagonal we actually got once so basically by using our common

notation we can say that in the D in the identity Matrix we got a 1 1 = to a 22 =

to a 33 equal to all the way to a NN equal to 1 and then when it comes down

to the rest of the cases so all the other observations let's

say a 21

a 31 or a 41 anything so anything that is not um a11 or a22 anything that is not on the

diagonal it is simply equal to zero we also say in those cases that a i j is equal to

1 if I is equal to J because then it means that we are talking about item that is on diagonal because both the row

index is equal to the column index otherwise the a i

j is equal to Zer if I is not equal to J so this is in the nutshell how a large identity Matrix in general can be

defined so let's now move on to another type of Matrix which is the diagonal matrix so by definition a diagonal

matrix is a matrix where all of diagonal elements are zero so what does this mean we is saw um example of a diagonal

matrix which was our identity Matrix because identity Matrix is an example of a diagonal matrix and what do I mean by

that in our just seen example we saw that only on the diagonal we had all these nonzero elements but the rest were all zeros so all the off diagonal

elements were zeros like in here and in here exactly the same holdes for the diagonal matrices only unlike in the identity

Matrix we no longer need to have this diagonal elements equal to one those can be any other numbers so as long as we

have this um elements D1 D2 D3 that are not zeros but then of the diagonal numbers so all these elements they are zero then we are dealing with the

diagonal matrix so in this case we got a 3X3 diagonal matrix because we have uh three rows and three columns and here we

can see that the um the first so the a11 the first element from the first draw and First Column is equal to D1 so a 22

is equal to D2 and then a33 is equal to D3 so D1 D2 and D3 those are all so D1

D2 and D3 those are all real numbers now when it comes to the uh this

numbers for example it can be that D is let's say 2 five 6 on diagonal then we

have those zeros this is a diagonal matrix it can also be

that D is equal to minus 3 and then 0 0 and then 5 8 and then here we have zeros

so again we have on the diagonal all these elements and the off diagonal elements so if all the off diagonal elements are zero then we are dealing

with diagonal matrix and if you wondering well what happens if on the diagonal we got zero do we still have a diagonal matrix it's actually a great

question but yes indeed we are dealing with the diagonal matrix as long as all the off diagonal elements are zero so

for instance if we got D is equal here we have zero here we have 0 0

0 and then 7 and then 0o and then 8 and then 0 0 so we got this of diagonal elements so here are the diagonal

elements and all the of diagonal elements are those given that all the of diagonal elements are zero which is the

definition of the diagonal matrix then we can say that our D Matrix in here is indeed a diagonal matrix let's now look into yet another

type of Matrix which is a special type of Matrix and it's called one's Matrix so by definition one's Matrix is denoted

by 1 M1 so you can see here and here it mens the dimension of it so the number of rows and number of columns and it's a

matrix in which all the elements are one so this is a very unique Matrix we often use it during the programming so in data science data analytics but also

in um uh when creating like data structures when designing algorithms this becomes very very handy and this idea of one's Matrix is that all the

elements are just one it means that if we want to create a placeholder in such way that we can then multiply any number in here with some other number and get

that number then it can be done very easily because we know that when we multiply a number with one then we get

that number so a * 1 is = to a x * 1 is = to X now this is a exactly this property exactly is what motivates us to

create and to have this type of ones matrices it means that we are defining matrix by its Dimension so it is M by n

and here the m is equal to two and then n is equal to three because we got two rows and three columns but you can see that all the elements are the same and

they are equal to one so a11 is equal to A1 2 is equal to a13 is equal to a uh 21

and is equal to a 22 and a 23 and they are all equal to one and this is the definition of one's Matrix you can have

um on Matrix of the size 4 by 10 On's Matrix of the size

th let say 10,000 by 100 Etc so any number any real number

so M and then n are real numbers you can use in order to create this large M by n1's

matrix let's now look into our final special type of Matrix before moving on onto the next module which is about zero

matrices so similar to this one Matrix a zero Matrix denoted by 0 m by N is a matrix in which all the elements are the

same with the one difference that this time all the elements are equal to zero so in the on Matrix all the elements were ones but in the zero Matrix all the

elements are zero this type of matrices become very handy also during the programming creating um different algorithms during design encoding um but

for slightly different purposes usually we create the zero matrices as a placeholder such that in the beginning we can have this uh tups or we can have

this um uh arrays or nested Loops um that we want to perform and then gradually add these values to the

existing Mt array so if we create this Zer Matrix and um this is a placeholder then next time we can always add on this

this new data that we get and then we know that zero plus a number is always equal to number which means that once we have this updated information of a we

can add this to the zero and we will then have this new updated information in our system therefore the zero Matrix is often used as a way to uh have this

placeholder with the provided Dimension where we can always add new information and the information can be updated so in this specific case we got

um a zero Matrix that has two rows and three columns so you can see two rows and three

columns so m is equal to 2 and then n is equal to three three perfect so we are done with module 2 and now we are ready

to go on to our next module which is the core Matrix operations so when it comes to matrices we often perform Matrix

additions Matrix subtraction but also Matrix um scalar multiplication of this Matrix so multiplying Matrix with a scaler and then Matrix um multiplication

just in general so taking two matrices and multiplying them we are going to look into this concept in detail we are going to see many examples like before we are going to dive deeper into this

such that we lay the ground on uh to the next module which is solving a system of M equations with an unknown so solving

this General um linear system so for the beginning uh we will be looking into this Matrix operations where we are adding or subtracting

matrices so by definition the sum of two matrices A and B of the same dimensions is obtained by adding their corresponding elements so by taking the

element i j from both matrices and adding them to each other so in this case you can see that Matrix A and B are

here and uh the uh definition says we just simply need to take the corresponding elements corresponding elements from the row I and the column J

take them add them and this will become an element in our final um Matrix because when we are adding two matrices

of the same size the result is yet another Matrix so we will use the Matrix a to add to Matrix B and this will give

us a matrix A + B and this i j simply refers to the indices corresponding to the row and the column we will look into an example in a

bit and this will make much more sense and the same holds also for the difference so by definition the difference of the two matrices A and B of the same dimensions is obtained by

subtracting their corresponding Elements which means that in order to obtain this Matrix a minus B this is a new Matrix we

simply need to look for each element so we are going to index them for a row I and J we are going to do this pairwise element wise subtractions we are going

to see what is that element corresponding to the row I and column G in The Matrix a which we say is a i j we are going to subtract from this the

element in the row I and column G that comes from Matrix B and this will give us our new Matrix which is a minus

B so let's now look into an example in this Matrix Matrix um uh a and Matrix B are used and Matrix a is of the size 3x3

Matrix 3 Matrix B is of the size 3x 3 in order to obtain a plus b what we are doing is that we are

performing element wise additions now let's verify this so what we are doing here is that

we are saying a plus b let me actually get a larger area here so let's say we have the two

matrices I want to add the two in such way that we do everything one by one such that this idea of a plus b and addition of the matrices will make

sense so we want to find out a plus b for that what we are going to do is that we are going to make use of this

definition that A + B and then I J is equal to a i j+ b i j which is a fancy

way or mathematical way or describing that for each element we need to go and look for the row I and column J and take that element from the um column from

that uh Matrix a and from the Matrix B so this means that for a + b this is going to be a

matrix that will have the same number of rows and the same number of columns as two matrices because both A and B are 3x3 which means also their sum is going

to be 3x3 so this going to be 3x3 and here we are going to do so we are going to take for the first row in the First

Column so for A+ b 1 1 so first row and First Column we need to go to the first row and First

Column of Matrix a and the first row and First Column of Matrix B and we need to add these two elements so we need to do

1 + 1 and then we need to go on to the second column so the first row and the second column which means that we need to be here

in both matrices so here we have 0 + 2 and then we got 2 + 3 and then we got 0 + 0 so

you can see it in here and then we have 1 + 0 and then we

have 3 + 1 0 + 1 and then 0 + 2 and then 1 + three which gives us

so 1 + 1 is = to 2 0 + 2 is = 2 and then 2 + 3 is = to 5 0 + 0 is equal to 0 0 +

1 is = to 1 1 + 0 is = to 1 0 + 2 is = to 2 and then 3 + 1 is = 4 1 + 3 is = to

4 which means that our A + B is equal to this Matrix that we got in here so you can see that we are getting exactly what

we uh what we have here only we have done it manually one by one so the same idea holds exactly when we have a minus B only instead of adding you will have

to do here minuses so minus minus so everywhere minus so 1 - 1 0 - 2 2 - 3 Etc so let's look into another addition

so in this case by definition it is defined as this element wise uh of the adding of these two matrices here the only difference in this definition is

that it's saying it's calling this a plus b as C so this new Matrix that we are getting as a result of adding a to B

it's calling C so basically it's the same as calling this Matrix as C you will see also this type of definitions so in this case The Matrix C is equal to

a plus b which basically means that for each row with index I and with each column with index J go and look for row

I and index J take the corresponding elements from Matrix a and Matrix B add them in order to get that corresponding element in our new Matrix C and you can

see that in this example that's exactly what we are doing we have a we have B we are taking this element and this one so 1 + 1 we are getting here two and then 0

+ 2 we are getting two here 2 + 3 is 5 and then 0 + 0 is = 0 1 + 0 is = to 1 and then 3 + 1 is equal to

4 so now we already go to the next topic which is about scalar multiplication of a matrix so by definition scalar multiplication of a matrix a by scalar

Alpha results in new Matrix where each entry of a is multiplied by Alpha the idea of scalar multiplication matrices is actually quite similar to this idea

of scaled multiplication in vectors so uh we have already seen in the lecture of the vector multiplication that when

we were having this scaler C and we had this Vector a then uh when we are multiplying C

which is a real number with Vector a then we simply need to take all the elements of vector a so A1 A2 all the

way down to a n and we need to multiply them by this same scaler so see this is what we were doing with vectors and that's exactly the idea

behind matrices and when uh doing the scalar multiplication of matrices only instead of multiplying only just one

vector with this scaler C now we need to apply this to all the rows and all the columns so here we got this one column and Matrix is simply a combination of

multiple vectors which means that we need to multiply all these elements of all the vectors of all the columns in this Matrix so let's actually look into

a specific example so in this case we have a matrix a and this Matrix a is this thing and we

have a scaler which is three so in here our Alpha is equal to three or you can call it C or anything so you can see

that when we are scaling The Matrix with a scaler in this case Tre with this Matrix what we are doing is that we are simply taking each of these elements and

multiplying it with this scum so 1 by 3 is 3 2x 3 is 6 3x 3 is 9 and 4x 3 is

12 this is the idea behind this entire scal multiplication ofation Matrix in more general terms if we for instance

have a matrix a so let's actually look into a high level General example where we have a DA Matrix M by n so we got M

rows and N columns and we want to get a scal multiplication of this Matrix and um scaler that we have here as in our

definition it is defined by this alpha alpha is just a number you can qu C you can qu B anything so in this case our scaler

alpha alpha is coming from R so it's a real number so Alpha time a is then

simply equal to to this new Matrix where all of these elements are simply multiplied by this scal so I will

just take over all these values H1 up to a M1 and then A1 2 a22 all the way down

to a M2 and then let me also add the last column just for fun here a 2 N and then here a m

n so here this new scaled M multiplies so so scaled uh Matrix a so Alpha * a is simply equal to Alpha time all these

elements are simply multiplied by the scale it is as simple as that so that's the simple idea behind um

Matrix as scaling so when you are doing scalar multiplication of this Matrix you simp take all the values and you multiply them element by element per row

and per column by that single scalar Alpha do note that you are multiplying them all without exclusion with exactly the same number which is that

Alpha so let's now look into the definition of matrix multiplication so here we are no longer multiplying a matrix with a scalar but

we are multiplying Matrix with Matrix so the product of an M by n Matrix a and an N by P Matrix B results in an M by P

Matrix C where each entry cig is computed as the dotproduct of the e Road of a and the J column of B now what does

this mean firstly let's look and unpack this part of the definition so we got Matrix

a that is M by n and then we got Matrix B which is

n by P what this means is that in this case Matrix a has M rows and N

columns and Matrix B has n rows and P cups so this is then simply the

dimension dimension of the two matrices so then it's saying that by definition the product of these two

matrices so the product of A and B the product of the two B is equal to to this Matrix

C and each entry cig J so c i j is computed as the dotproduct

of the each row of a and the Jade column of B now this part might seem bit difficult but once we look into the

actual example and we illustrate this on our common high level General expressions of Matrix am and their multiplication this will make much more

sense for now before coming to this one I just wanted to refresh our memory on one thing I said before when discussing also this idea of improving uh this uh

different properties of vectors that when we want to multiply a vector with a matrix or Matrix with Matrix or vector with a

vector we need to ensure that from the first element the number number of comms is equal to the number of rows of the second element this is also very important for this specific case and

just in general for matrix multiplication so you can notice here that the number of coms here is equal to

the number of rows in here and the order is very important so in case of matrix multiplication the order is really important which means that if you have a

matrix a and you want to multiply with the Matrix B then the number of

columns of a should be equal to the number of rows of

B otherwise you cannot multiply those two matrices with each other so in case you got a matrix a that doesn't have the same number of columns as the rows of

number of the Matrix B then there are some alternative things that you can do including this idea of the transpose that we saw also doing when Computing the dot product between this Vector a

and Vector B that's something that we also do in programming when we are dealing with this Matrix and we want to compute this relationship between two matrices but the number of columns of

one of the first one is not equal to the number of rows of the second one we are simply uh manipul ating this matrices or removing some data if that's not hurting

our problem maybe uh flipping so transposing our Matrix or applying any other source of operation to it to ensure that the two matrices that we are

multiplying with each other the first one's number of columns is equal to the second one's number of rows that's just the low and that's something that you should follow if you want to multiply

these two matrices all right so now let's move on onto this idea of multiplying and Dot product let's look into a specific

example and this will uh help us to understand this process better so before doing that I just want

to quickly show you this general idea so if we have a matrix a that is M by n which means that it looks something like

this like A1 1 a 2 one up to the point of a

M1 and then here we got let's say a 1 2 a 22 up to the point of a M2 and then at

the end we got a MN and here we got A1 n so let me also add this one 2 N and we

got a matrix B this Matrix B is n by P so it has n rows and P columns so we are fine in terms of Dimension

here and we got here b11 B21 up to the point of b m sorry b n

in this case let's not confuse the letters so b n 1 B1 2 B 22 up to the

point of b n 2 because n now is the number of rows for Matrix B unlike for the Matrix a up to

B1 p and here b 2 p and here after the point of B and then n p this is the last element in order to perform um

multiplication between these two matrices so to obtain a matrix C which is a equal to a *

B what we need to do is we simply need to take pair case or pair Row for the row I for

instance we need to take this element so this row and we need to multiply it with this so we need to find the dot product

between this row and this column then we need to move on on to the next one and then for the second element we will then

take this row and we will multiply it with this one so this is then something that we need to do in order to obtain these

elements and you might have already noticed that we got this m by n and n by P so you might have already guessed what

will be the dimension of the C if we got that the dimension of a is equal to M by n and the dimension of B is equal

2 N by P then the results Matrix after M multiplying the two so Matrix c will be will be having a

number of rows equal to this and the number of columns equal to this so This middle part basically disappears and the number of rows of the first Matrix will

be then the number of rows of this result Matrix C and the number of columns or the second Matrix so Matrix B will then be our final number of columns so we will then have a matrix C that

will have a dimension so Dimension so dimension of C will then be equal

to M by P so we will have M rows and P

columns so how we are going to compute this so for c i j which means row

I and column J let's look into the definition of it it's saying c i j is computed as a dotproduct of the each row

and the Jade column so each row from a and Jade column of B what where is the each Road of a the each Road of a is

somewhere here so each Road of a it is uh the A and then

I one then a and then I 2 and then a and then I Tre

dot dot dot and then a i and then then we got in total n columns n and we always do the transpose right

when Computing this um dot product so we then take the transpose so we take this row row I and we multiply it so we do

the dot product between this one this is the a i and the B J

this is column J it is somewhere here so it is B and then we got the first element which is one and then J

and then b 2 J B 3j dot dot dot up to B and then in

total we got n rows in B so n and then the J is the column so it stays the

same so this is then the dot product between e row that comes from Matrix a and the J

column that comes from Matrix B so it's always like that actually so we always take row by row so we take this different so every time we

take just a row and we multiply with the corresponding column and then we get the dot product between this row that comes from the first Matrix and then the column that

comes from the second Matrix in that specific order in order to get our DOT product and that specific valum and what is this amount actually

so when we calculate this do product you can quickly see that we have a

i1 multiplied by B 1 J plus a I2 multiplied by b 2 J and then dot dot dot

a i n multiplied by b n g and this new Matrix c will then have all these elements so

C11 C 21 and then c31 dot dot dot and then C the last

row as the number of rows of C is m c m see here M so C and then here it will be

one 2 C 22 C3 and then 2 up to the point of cm and then two and then here the last

col will be C1 and then p is the number of coms in C so C1 p and then C2 p and then c3p dot dot dot and then

c m and then P okay so this is what we get this is our

final Matrix C when multiplying Matrix a and Matrix B so let me clean this up C is to now if you want to find out

what is C11 you can easily fill in this general formula that uh that we just calculated the I is equal to 1 and then J is equal to 1 and this will give you

C11 by using this formula if you want to get the C and P then just fill in the I is equal to M and then J is equal to P in order to get this value C and P so

you can already see the amount of calculations you need to do in order to get all these elements from this l large matrices A and B let's actually look into a simple example to clarify this so

we have a matrix a here and Matrix B here and we want to do a multiplication of the two and we have just learned how to do it let's actually do it one by one

so we got a matrix a which is equal to 1 2 3 4 with Dimensions 2 by 2 then

we got a matrix B which has values two Z and then one two so it is 2 by two and I

want to find what is c that is equal to a * B and I know already by looking at these Dimensions that c is going to be

equal to 2 by 2 so you might recall that I said that when looking at this final result the number of rows or the final

um Matrix will be this so the number of rows of the initial Matrix a and then the number of columns of this final Vector c will be the number of vectors

number of columns of this second Matrix B so two therefore I know already before even doing calculations that the uh

product Matrix c equal to a * B is going to have a dimension 2x two let's actually do a calculation to check this so C is then equal to a * B and it's

equal to 1 2 3 4 4 multiplied by 2 0 1 2 okay

so I expect to have four different elements here here here and here so to

obtain the C11 so it is C11 in here what I need to do is that I need to look at the first row and in the first column in

here so first row from a and the First Column of B and I'm doing the dot product which means 1 * 2 + 2 * 1 1 * 2

is 2 2 * 1 is 1 so here I'm getting 1 * 2 + 2 * 1 which basically gives me 2 + 2

and that's equal to 4 so here I'm just writing [Music] down 1 * 2 + 2 * 1 now when I want to

get this value which is C12 this means that I want to get the first row and the second column and that's exactly what I'm doing so I'm

going back and I'm saying let's look at the first row but this time will look at the second column coming from the uh from The Matrix B so 1 *

0 0 + 2 * 2 and then I do the same only this time for the second row which means I'm picking this row and then this column

so it is three * 2 + 4 * 1 and for the final element c22

I'm taking the second row and the second

column which gives me three * 0 plus 4 * 2 now what does this gives

me this gives me this 4x4 Matrix where 1 * 2 + 2 * 1 is 4 1 * 0 + 2 * 2 is 4 3 *

2 + 4 is = to 6 + 4 which is 10 and then 3 * 0 + 4 * 2 is = to

8 so let's check 4 4108 that's exactly what we have here so as you could see here the idea is that every time to follow what element I'm

looking for for the CI J and then I just go to the E rows from the first Matrix and the J column from the second Matrix

and I do the dot product of the A and then I and then K let's say so I'm going to

the E Row from the first Matrix and I'm taking all the elements which means I don't even need to mention this index it just means the entire each row coming

from the Matrix a and then I'm doing the dot product between this row and the column that comes from the Matrix B

which means B and then J which then will give me the cig so I'm looking at this and taking this multiplying this dot product and this

gives me the first element then the first row and then the second column which gives me the uh second element in the first row in my Matrix so this one

and so on so hope this makes sense uh if it doesn't make sure to reach out because it's a very important concept and uh let's also look into another example to make sure that we got this

right so in this case as you can see we have another matrices so set of A and B matrices again 2 by two a simple

one and we want to know what is a so let's say we call this C we already know C should be 2 by 2 and what we are doing is basically for

C11 we are saying let's look at the first row so first row and the First Column coming from the second Matrix B

and let's do the dot product so 2 * 1 2 * 1 2 * 1 4 * 5 4 * 5 we get this and

then when we want to find what is C oh what is C and then one two so in the first row but in the second element in

our final Matrix so I is equal to one and J is equal to 2 it means we need to look at the first row from The Matrix a

but this time the second column from The Matrix B so

it is 2 by 3 2 by 3 4 * 7 4 * 7 and this gives us a number 13 four even if you calculate you can see that 2 * 1 is

equal to 2 4 * 5 is 5 so 2 4 * 5 is 20 so 2 + 20 is 22 in here and then you can do the rest of calculations and this

will be a good practice to see how we can do a basic matrix multiplication the idea is actually quite straightforward when it comes to multiplying it it just

it comes with a practice when we see all these uh much bigger matrices so um this is another example I will

leave this one to you to complete it just uh to keep in mind we always do uh so we always look at the dimension first in here 2 * 2 and 2 * 2 which gives me

an impression already what I can expect the result will be 2 by two and when it comes to the uh cross elements just

ensure to always look to the E row and the J column this comes from Matrix a and this comes from Matrix B take them compute the dot

product and then you will find your C uh your final result let's call it um kig because in this case we have a

matrix C already welcome to the module 4 of this course when we are talking about matrices and linear systems so in this module we are going to dive deeper into

this uh idea of linear systems with matrices and solve linear systems using different techniques and specifically we are going to learn the uh concept behind

solving linear systems using matrices named gausian elimination and gaussian reduction welcome to the module one in this unit so in this uh case we are going to talk about algebraic lows for

matrices we are going to discuss four different properties for matrices uh and the first one is the communative laow for Matrix addition the associative law

for matrices the distributive laow for matrices both the left and the right one and then finally we're going to talk about the scalar multiplication laow for matrices so the algebraic lows or

matrices they are like in case of real numbers like in case of vectors they help us to do different operations on these entities they are very similar to

the real numbers and the vector cases where we for instance um so that if for

instance A + B uh is equal to B+ C C or a * um b + C is equal to a b + a c those

are all sorts of lows that we uh learn as part of high school prealgebra and we have applied it to real numbers we know how helpful those can be and similar

type of lows we have also for the matrices and we got in this case four different laws that we will be discussing the first one is what we are

referring as associative law the second one is the distributive low the SEC the third one is the scalar multiplication low and the fourth one is the communative low for addition so these

laws help us to do different metrix operations they help us to manipulate algebraically this metrices and then uh this can help us to solve different

sorts of problems including solving a system of linear equations so let's start with the commutative law for Matrix addition so

the Matrix addition is cumulative um which means that A+ B is equal to B plus a so unlike the matrix multiplication that we have seen in the previous

lessons uh where the order did matter and we said that we um had to uh ensure that the number of columns of the first Matrix is equal to the number of rows of

the second Matrix in case of addition that's this is not the case so we should not care about the order whenever we want to add two matrices the other thing that we need to keep in mind though is

that the two matrices needs to have the same size so I mean that both Matrix a and Matrix B need to have a dimension so

dimension of Matrix a should be equal to dimension of Matrix B and let's say should be equal to M

by n but for the rest we don't really need to care uh which one we will put first will we put first a and then add the b

or we will do the other way around so we will then First Take B and then we will add a so this is the idea behind communative low for Matrix

addition so first let's look into all this uh lows and then we will also look into the corresponding examples so for this specific case it

might actually also be helpful to write down the general formula which will um make sense out of this um low for the uh

uh which is a communative low for the Matrix additions so let's say we got a matrix a which is M by

n and this Matrix can be represented as a11 dot dot dot a M1 so this is something that we saw time and time again so I'll just quickly write it down

the common notation for this and then here we have the last column which is a

MN and this is the Matrix a then we got Matrix B which is again M by n and can be represented as

b11 and then dot dot dot and then B M1 dot dot dot b1n dot dot dot b MN

so the communative lows says that A + B should be equal to B + a let's check that whether this

is the case let's first compute this part and then we will do this one well the first

one means that we get so A+ B is and we remember remember how we add matrices right so we know that we

just need to pick their corresponding elements and add them to each other so we get a11 plus

b11 then dot dot dot and then a M1 plus b M1 this is why also D is really important that they got um the same Dimension which means that they got

exactly the same amount of elements the same uh amount of columns and the same amount of rows um in terms of the uh

Matrix size so then here we have a 1 n and then plus B1

n then dot dot dot and then a M1 and then plus b m here I need to put

n we are in the last element of the Matrix so BMN and that is it this is our Matrix A

+ B let's now look into the Matrix b + a so what that amount is so the

Matrix b + a will then be equal to b11 plus

a11 dot dot dot and then B M1 plus a

M1 then dot dot dot and then b 1 n plus a 1 n then dot dot dot and then

the last element will be B MN and then Plus a m n

so in here if we remember from the real numbers we know that a

+ b is equal to B + a for instance if a is equal to 2 and then B is equal to 1

then a + b is = to 2 + 1 which is equal to 3 and then b + 1 is = to 1 + 2 and it's equal to 3 so we know that indeed

for the real numbers a plus b is equal to B + a and making use of that property

we can already state that b11 + a11 is equal to A1 1 + B1 1 and then the

general case is that a i j plus b i j is equal

to b i j plus a i j where I is the index of the

rows and then J is the index of the columns from the coefficient labeling so using this property from the real numbers given that all these values

in the m Matrix are real numbers we can quickly see that the Matrix B+ a that we just got in

here is equal to this Matrix a plus b which means that 1 is equal to B and this proves

that A + B is equal to B + a this is the communative property of the Matrix additions so the next law is the

associative law for matrices which says that the in case of Matrix addition a + B+ C isal to A + B +

C so basically this time we go from here to adding one more

element which is the third Matrix Matrix C so we are saying A + B + C is equal to a +

B+ C so it doesn't matter whether we will First Take The Matrix a and then B and then add them up and then we add

Matrix C or if we first take the Matrix B and C add them up and then we add a to this sum it doesn't matter we will see an example of this in

a bit and then the uh second part of this associative law for matrices says that

for matrix multiplication a * B * C is equal to a * B * C so again in terms of the

um order when it comes to this specific multiplication so it doesn't matter whether we will first multiply a by B and then by C or we will first multiply

B by C and then we add the a at the end we will end up with the same amount so a * B and then * C is equal to B * C C and

then in the left hand side we add the a so a * B * C so these properties help us to add or multiply matrices without really worrying about this idea of

grouping of the terms so we can always group them and perform all sorts of operations so this is this first property that we see in

here let's say we have this uh Matrix a matrix B and Matrix C so let's prove that in the order doesn't matter and this

associative property holdes so let's prove that so for that the first thing we need to do is to

compute a plus b so this part so A + B + C what is that first I need to compute this

part and then I will add C which is the second part so A + B is equal to my a is equal

to 1 2 3 4 plus and my B is equal to 5

6 7 8 this is then equal to so 1 + 5 is = to

6 2 + 6 is = 8 3 + 7 is = to 10 and then 4 + 8 is equal to 12 this is my A + B

this is the first part now the second part is then to add to this A +

B this C this I can by the way also call some Matrix D so I can say that this is equal

to D+ C so let's find out what is this amount so a plus b or what we're referring as D

is 6 8 10 12 we just calculated it in here I'm also adding now my Matrix C

which is 91 10 12 so 9 10 11 12 what is this amount it is 6 + 9 is 15 8 + 10 is

18 10 + 11 is 21 12 + 12 is 24 this is my final Matrix

so I have checked

that D A + B + C is equal to 15 18 21 and 24 this is the first part let's now go

ahead and check whether this is equal to the second part which is this part so

this is one this is two so this is then A + B + C as you can see it in here let's now calculate that amount

and like previously we will do it in an order so first we need to calculate this part and then the entire thing so B+ C is then equal

to the B was 5 6 7 8 5 6 7 8 plus and the C

was 9 10 11 12 9 10 11

12 what is the much 5 + 9 is 14 6 + 10 is 16 7 + 11 is 18 and 8 + 12 is

20 this is the first amount let's refer refer this as a d or we can even call it by some other

letter let's say k this is Matrix K so B plus C is k then the second part is to take this

B+ C so B+ C which we have referred as K say

K and then we are adding to this the A and specifically just to ensure that we stay with the same order I'm saying I will add from the left side the

a to this Matrix K and this obviously means this is equal to so A+ b + C this

is what I'm referring by just uh in a more simpler note a I'm just using K in here so this is my B plus C or what I'm

referring also as a k and this amount is equal

to what is my a my a is 1 2 3 4 1 2 3 4 plus and what is B plus C we

just calculated that that's the K so 14 16 18 20

T So 1 + 14 is 15 2 + 16 is 18 3 + 18 is 21 4 + 20 is

24 so we have learned that the A + B + C is this

Vector now is this Vector equal to the a plus b and then plus C well here we got

this 15 18 21 24 15 18 21 24 so we have just proved

that the first part is equal to second part which means that we have proved that indeed the order doesn't matter and A +

B + C is equal to a + B+ C so this calculation confirms that the both sides of this equations they are in indeed equal and this confirms the

associative low for the Matrix addition so let's now look into the distributive law for matrices which says that Matrix addition and multiplication

they satisfy the distributive property which means that if we have a left distribution a * b + C is equal to a + a c

and then in the right distribution we basically have the Matrix multiplying from the right from hence the name right

distribution A + B * C is equal to a C + BC you might very quickly see and recognize from here that we have very

similar actually exactly uh the same rule only for real numbers we know that a * b + C is equal to and then we open the parentheses with say this is equal

to this times this so AB plus this times this a c you can see that we have exactly the same here only in the capital letters so in the real numbers

we have exactly the same low so the same we have also for our left distribution when it comes to Matrix additional multiplication and the same

we have only with a different order here you can see the C so this one is basically uh with the different order instead of having the Matrix multiplied

in the left here we have from the right and this is similar to the property that

A + B * C is equal to C * a which is a c plus c * B which is BC an example uh where we will prove that the

distributive law for matrices um is indeed true and I have skipped deliberately the uh example for this one

because uh this a b * C is equal to a * b c so the associative law for matrix multiplication it's something that you can calculate for yourself using the

same a b and c matrices only this includes multiplication of these two matrices and it's something that we are going to do as part of this example so

instead of doing and redoing this multiplication I thought that it's great to leave that for you as a practice and instead focus on bit more complex

problem like this one that one way or the other includes the same matrix multiplication so I need to calculate the a * B in this case which means that

by providing you this example I'm also including what is needed to do the previous example only it would be a great way to practice the material for yourself so let's now move into proving

the distributive low for matrices so we got this mat matrices a b and c and here I'm going to apply matrix multiplication the same as that is needed for the

previous uh case and here what we need to prove is that a * b + C is equal to a * AC so this is the first part this is

the second part so let's go and calculate them separately so for the first one we need to calculate

a * b + C which is then something that we can calculate by

first doing the addition so we will first do the addition of matrices B and C and then once we are done with that we will then do a * b +

C so that's the second part so let's go ahead and do that calculation so first we

got b + C what is B plus C B is 5 67 8 5 6 7 8 plus and the C is minus one 0 0 minus

one so on the diagonal we got min-1 and minus one and then of diagonal lower and upper part we got zero and what is this

Matrix this is equal to 5 - 1 so 5 + -1 is equal to 4 6 + 0 is = 6 7 + 0 is = 7

and then 8 - 1 is = 7 this is our B plus C which we can refer also as Matrix D so let's call this D which means that now

we are interested in a * B so for this second

part we need to take this Matrix a so a

* D is then equals to we need to take the Matrix a which is 1 2 3 4 and we need to multiply it with

this Matrix that we just got because this is the B plus C or the D that we were referring 46

77 okay so let me remove this part cuz we are going to need some space for this and let's do this calculation this

is 2x two and this is 2x two I will do the calculations in here so we need to end up with the Matrix

that is also 2 by 2 because we know 2x 2 Matrix times 2x two we will pick this part so the number of rows and the number of columns of the second one this

will be our resulting Matrix which is 2 by two all right so for the matrix multiplication we know that for this

element in the place of so one a or let's call this Matrix we don't even actually need to call this anything we we can keep it simple so let's say that we are in the first draw in the First

Column so this is the first draw in the First Column for this what we need to do is we need to take the first row from the first Matrix so Matrix a and then

the First Column of the Matrix D so this one and we need to do the dot product

which means that we do basically 1 * 4 1 * 4 plus 2 *

7 so for this element which is in the second row and the First Column we need

to take the second row and First Column in here so we end up with three

times 4 so 3 * 4 and then 4 * 7 so Plus 4 * 7 The Dot product between this one and then this one so for this element which is in the

first row and then the second column of the final Matrix so first row and second column we need to pick the first row and second column and do a DOT product which

means one 1 * 6 + 2 * 7 2 *

7 and then in here in this element we got the second column and second row so second row second column which means that we need to pick

the second row and the second column the dot product of the second row of Matrix a

and the second column of Matrix D which is 3 *

6 Plus 4 * 7 4 * 7 so let's quickly calculate what this amount

is so this is the a * B+ C basically and this Matrix is 1 + 4 is 4 2 * 7 is 14 4

+ 14 is 18 1 * 6 X is 6 2 * 7 is 14 and 6 + 14

is 20 3 * 4 is 12 4 * 7 is

28 which means that we got here

40 3 * 6 is 18 4 * 7 is 28 which means we got here 36 and 46 this is our final a * b +

C let's now go ahead and calculate the second part so the second part says that we

got a * b + a * C so a * b + a * C which means that first we need to do

this calculation and then this one and then we need to add them to each other so let's quickly then calculate

what is a * B and then a * C and then add them to each other let me clean up some space in here we're going to KN

that when we write this one in a smaller format so this is equal to 18 20 40 and

46 and let me take over the second element which we still need to calculate which is AB plus a c first we will do this and then this

and then we will add them to each other so a * B is equal

to 1 2 3 4 multiplied by 5 6 7 8 5 6 7 8 now following the

same approach from the previous example when we calculate this Matrix I will then quickly calculate what is is a * B so in

here we got first row and First Column so 1 * 5 + 2 * 7 so the dot product between the first row and the First Column from

here now for this element here we got the second row and the First Column we need to take the second row in the First

Column from here and we do the dot product which means 3 * 5 + 4 * 7 in here here we got the first draw and second column so the first draw and

second column which means that we need to have 1 * 6 + 2 *

8 then here we got the second row and then second column which means 3 * 6 + 4 *

8 and then this is equal to 1 * 5 is 5 5 2 * 7 is 14 so this is

19 1 * 6 is 6 2 * 8 is 16 6 + 16 is 22 in here 3 * 5 is 15 4 * 7 is 28 so

this is then 33 and then 43 so 43 and in here we got 3 * 6 6 is 18 4 *

8 is 32 so we end up with 50 so hope I haven't made any mistakes in the calculations so this is the a *

B so a * B is then equal to 19 22 43

50 let's clean this pce and let's move ahead to the second part of the calculation which is a *

C what is a * C well a * C is 1 2 3 4 1 2 3 3 4 multiplied by

-1 0 0 -1 so here we are then getting -1 + 0 here we are

getting -3 + 0 here we have -1 + 0

so no so the first row and second column which is 0us 2 and then in here we got the second row

and the second column which is 0 - 4 which means that we end up with this Matrix and it's equal to

-1 -3 then -2 and then -4 which means that we are getting

as a final step AB plus a which

means 19 202 43 and then 50 then

plus -1 - 2 - 3 - 4 and what is this 19

- 1 is 18 43 - 3 is 40 22 - 2 is 20 50 - 4 is 46

okay so we got that this amount ab+ a c is equal to 18 20 40

46 and as you can see already here this Matrix that we got in the previous calculation from one is equal to this Matrix that we got as part of

second calculation which means that now we have proved that for this specific example indeed 1 is equal to 2 which

means that a * b + C is equal to AB plus

AC there we go so let's now look into another law which is the scalar multiplication law for matrices so the scalar multiplication

law for matrices says that if we got a scalar R and a matrix A and

B then R * a * B is equal to R * a * B and is equal to a * R *

B so here the r is just a scalar so it's a real number and then A and B are

matrices and what this low basically says is is that it doesn't matter what in which stage you will do your matrix multiplication with the scaler if you

have this external scaler you can first take the two matrices multiply them with each other so the A and then B and then multiply it with

r or you can take the scalar R multiply with your

first Matrix and then multiply with B or you can take your second Matrix multiply with the scaler and then

multiply with a it doesn't matter they will all result in the same Matrix so let us actually prove this by making use of our skills from matrix multiplication

and scalar multiplication here I've picked up bit more uh Advanced example where uh a is 2x3 and B is 3x3 in this way we will

train our multiplication skills for matrices and at the same time we will also prove that the scalar multiplication law of matrices holds so

let's go ahead and do the multiplications so first we have a matrix a what is that Matrix Matrix a

is 1 - one 2 so 1 - one and then two then we got 0

2 and then 1 which is 2 by 3 and then we got B which is equal 2 it is 3x

3 with elements 1 Z 1 1 2

Z one one and then 3 1 0 2 so it

is 3x 4 so it's 3x 4 not 3x 3 but 3x 4

Matrix now the final part that I need here is this which is R is equal to 2 the scalar value so R is equal to

2 so the first thing that I'm going to do is to calculate this amount which is R * a * B for that what I need to do is

to First calculate this a * B so let me quickly go and calculate this for us so given that the a has Dimension 2x3

and then B has a dimension 3x4 I can see that quickly that my Dimension criteria is satisfied the number of columns of a is equal to number of rows of B so

that's fine and then I know also know that the final dimension of a * B is going to be 2x4 so it's going to be a

2x4 Matrix and how do I know that well because I know that from our um all the problems that we have solved we have already seen that we always need to pick

the number of rows of the First Column and the number of columns of the second uh Matrix in order to get the final Dimension which is

2x4 so let me then go ahead and do the calculation so we are going to have a

2x4 Matrix let me write it even bigger so it's going to be a 2x4 Matrix

so for the first row and First Column I need to look in here the first row and the First Column which means I

need to take one so it's equal to

1 * 1 so + 1 * 1 is 1 - 1 * 2 is - 2 2 * 3 is 6 + 6 this is my first value

and what is this amount it is equal to 1 - 2 is - 1 and 6 - 1 is equal to 5 so this amount is five

five so what is this amount well this is my second row in the First Column so I need to make use

of second row and First Column which is equal to 0 * 1 is 0 2 * 2 is 4 and 1 * 3

is 3 4 + 3 is 7 so this value is 7 seven we are ready to go on to the next column so column number two so then this

time I need to look at the first row and second column so we are going to use this one so first we will use this first

Row 1 * 0 is 0 - 1 * 0 is 0 0 + 0 is 0 and then 2 * 1 is the only nonzero element 2 * one is two so I already know

that for my second column I got here two and what is this element well for this I need to look at the second row

and second column so this thing so 0 * 0 is 0 2 * 0 is 0 1 * 1 is one which means that here I get a

one let's not move on to on uh towards the third column so in here first I need to look at the first row so 1 - one and two and then this

time remove this I need to look at the third column because I'm here in the third

column

so 1 * 1 is 1 -1 * 1 is 1 so here I got

1 - 1 and then 2 * 0 is 0+ 0 1 - 1 + 0 is 0 because those two cancel out this means that here in this element

I got a zero and what about this element where I need to look here in the second row and here I need to look at the third

column 0 * 1 is 0 2 * 1 is 2 1 * 0 is 0 so 0 + 2 + 0 is equal to 2 so this is

2 and now we are left with the fourth column so for that I need to look in here so for the first row which means

first row in here and then the fourth column in here so first row in here and fourth column here 1 * 1 is 1 - 1 * 1 is

1 and then 2 * 2 is 4 which means that I end up with 1 - one and then + 4 and what is this this two cancel out I end up with four which means that here I

need to fill in four and what is this final element it is the second row in the fourth column

so the second row in the fourth column 0 * 1 is 0 2 * 1 is 2 1 * 2 is 2 0 + 2 + 2

is = 4 so now we obtained that a * B is this

2 * 4 Matrix as we have

expected so this is then equal to 5 2 04 and then 7 1 2 4

so then the next step would be to take the scaler R and multiply it with a *

B let me actually keep the colors consistent so a * B this is a * B so the

only thing that I need to do is to take that in here and multiply this two with

each of those elements so I will end up with the same size Matrix so 2x 4 only all these elements need to be multiplied

with the scaler which means that I will get 5 * 2 is 10 2 * 2 is 4 0 * 2 is 0 4

* 2 is 8 and then 7 * 2 is 14 1 * 2 is 2 2 * 2 is 4 and then 4 * 2 is 8 so this

is the result of the multiplication so this is the first part this is what we are referring as one so we have then

checked in here that the r times actually we have already in here

so I won't be writing again so as part of the first section we have already seen that R * a * B is this Matrix let's now move on to the next one which

is calculating the second part so this is the first part this is the second and this is the third we have this already

let's now move on and calculate this one so for this second case so the second case what we want to calculate is R * a

* B so it is R * a and then * B this is what we need to calculate so the first thing that we will do is to calculate this

part and then to calculate the entire thing the second point so let's go ahead and do that first we will take the A and then

we will multiply all its elements by scaler two to get the r and then a this amount is equal

2 1 * 2 is = 2 - 1 1 * 2 is - 2 2 * 2 is = 4 0 * 2 is = 0 2 * 2 is = 4 2 * 1 is

equal to 2 this is R * a now in The Next Step so this was one the next step we need to take this amount this

Matrix to minus 2 4 042 and multiply it with

1 2 3 0 0 1 and then 1 1 Zer and then 1 one 2 so basically the Matrix

B let's now move and work our way out with that one actually let me remove this from here and keep the space bit

more clean R time a and I will multiplying this with the Matrix 1 2 3

and then 0 0 1 and then 1 1 1 1 and then 0 2 well I know that this one is 2x3 and

this one is 3x 4 which means that the result will be 2x 4 let's now go ahead and calculate that Matrix which is equal

with a dimension of 2x 4 well for the first row and First Column let me actually go and quickly do those calculations let's now go ahead and do

those calculations so we are going to have four columns as previously the dimension is going to be 2x4 so let's do it column by column in

here it means that we are in the row one and then column 1 so 2 * 1 is equal to 2

- 2 * 2 is - 4 and then here we got 4 so 4 * 3 is 12 so we got 2 - 4 and then +

12 and what is this amount 2 - 4 is - 2 + 12 is 10 so here we got 10 let me remove

this 10 this is the second row and the First

Column which means we got 0 * 1 is 0 4 *

2 is 8 and 2 * 3 is 6 so 8 + 6 is equal to 14 so here we got 14 this is the first row and second

column which means that we are looking at this row and second column this time so 2 * 0 is 0 - 2 * 0 is 0 the only

thing that we care about is this one and this element which is 4 * 1 so this should be four let's now do the same for the second row 0 * 0 is 0 4 * 0 is 0 0 +

0 is 0 which means we are left with 2 * 1 so here it comes two let's now do the third column so for

the third column we got First Row 2 * 2 2 * 1 is 2 - 2 * 1 is - 2 and then 4 * 0

is 0 which means that here we get 0 because 2 - 2 + 0 is 0 then we got the

second row and third column which is this row and then third Comm so 0 * 1 is

0 4 * 1 is 4 2 * 0 is 0 0 + 4 + 0 is four so this is four and then for the first row and then fourth coln so it

means that we need to look at this specific column the first row is 2 * 1

it is 2 - 2 * 1 is - 2 and then 4 * 2 is 8 so 2 - 2 + 8 is

8 and then finally for the second row and the fourth column 0 * 1 is 0 4 * 1

is 4 2 * 2 is 4 4 + 4 is 8 this is the final Matrix which means that this entire amount that we just

calculated step by step this is equal to this Matrix in here okay so this is the second element let's check whether the first element is

equal to the first one so we see here 10 48 142 248 as you can see we are dealing with exactly the same Matrix which proves

that indeed R * a * B is equal to R * a * B so this part we have already proven because we have seen that 1 is equal to

2 perfect so the only thing that is remaining is to calculate this third part and to see whether this is equal to this matrices because we have seen that

the two of those are equal so the remaining thing that is left to prove this theorem is to calculate this third part let's go ahead and do that that so the third element says let's

first calculate the r * B and then multiply it by a so we need to calculate b r *

B * a this is what we need to calculate which means first we need to calculate this and then we need to can calculate the entire thing all right so let's go ahead and do that

so R * B is equal to so we need to multiply each of the elements of B by two so we end up with

this Matrix 2 0 2 2 and then 2 * 2 is 4 2 * 0 is 0

and then 2 2 and then 2 * 3 is 6 2 * 1 is 2 2 * 0 is 0 2 * 2 is four this is

that first Matrix let's now go ahead and calculate the second part which is a *

Matrix a so it

is 1 - 1 2 and then 0 2 and then 1

multiplied by this Matrix which is 2 4 6 0 0 2 and then 2 two 0 and then 2 2

4 okay perfect so this is then what we need to calculate well this is 3 * 4 this is 2 *

3 which means the result should be 2 * 4 let's go ahead and do those calcul ations this first amount will be the first row and the First Column

dotproduct of those which means 1 * 2 is 2 -1 * 4 is - 4 and then 2 * 6 is 12 so

here we got 2 - 4 + 12 and what is this amount well 2 - 4 is - 2 12 - 2 is = to

10 so this one this element is 10 then for the second show we need to look in here so 0 2 one and the dotproduct of the one with the First

Column so this thing and that is 0 * 2 is 0 2 * 4 is 8

and then 1 * 6 is 6 so what is 8 + 6 that is 14 and then for the first row and then

the second column we need to look to the first row in here and then the second column in here and the dotproduct

of the two well 1 * 0 is 0 Min - 1 * 0 is 0 and then 2 * 2 is four so that's what we are left with four

and then for the second row and then the second column so this element we

got 0 * 0 is 0 2 * 0 is 0 1 * 2 is 2 for the first row and the third column so we need to look in

here 1 * 2 is 2 - 1 * 2 is - 2 and then 2 * 0 is 0 so we are left with

zero 0 and then once we do the calculation for the second row we will see that we end up with 0 * 2 is 0 2 * 2

is 4 and then 1 * 0 is 0 so we end up

with four and then here for the final column 1 * 2 is 2 - 1 * 2 is - 2 and

then 2 * 4 is 8 the first two cancel out and we end up with 8 and then for the second row and the fourth column we look

into here again this time the second row 0 * 2 is 0 2 * 2 is 4 and 1 * 4 is 4 and then 4 + 4 is

8 so if we look in here this is our third amount we will quickly see that again we have the same Matrix with

exactly the same elements so now we have also proved this third part and we have seen that in all cases the r * a * B is

equal to R * a * B is equal to a * R * B now we are ready to move on towards the second module in this unit which is about the determinants and their

properties we are going to look into the uh determinants at high level we are going to Define them and going to understand what why they matter and why they are important then we are going to

see how we can calculate the determinants we are going to see the calculation for 2x two Matrix then 3x3 Matrix and then just in general how we

can do it and then we are going to see the properties of determinants one by one and then finally we are going to see the determinants interpretation from the geometric perspective so when we

visualize it using python so by definition the determinant is a scalar value that can be computed

from the elements of a square Matrix so this important Square Matrix and encodes certain properties of the Matrix so the determinant provides a

critical information about The Matrix such as whether it's invertible and the volume scaling factor for the linear transformation it

represents so we see that the uh concept of theer detent is highly related to many other concept that we have seen before so first here it's talking about the square Matrix then it's talking

about encoding certain properties so having the determinant it contains certain information that um is related to the properties of the system that

that Matrix is representing and then it provides critical information about the underlying metrix because the determinant is calculated from Matrix we

say the determinant of a matrix so it contains a critical information about that Matrix such as whether it's invertible or not and this goes back to the concept of inverse we will see this

once we learn the concept of determinant because the inverse calculation is dependent on the determinant but keep this thing in mind that the determinant contains

information whether we can get um inverse from a matrix or not we will see this concept over also in detail in the next section but for now we can remember

that the determinant contains important information related to the invertibility of the Matrix so having an inverse or not and then it also contains

information about the volume scaling factor for the linear transformation it represents so here we then go back to this concept of a x is equal to B and

then knowing the determinant we can then comment on this volume scaling factor for this linear transformation that it

represents so let's go uh on to the next slide to find out bit more about the determinants and specifically how we can calculate the determinant in the

mathematical terms when it comes to the 2x two Matrix because the uh determinant of a 2X two Matrix is quite

straightforward so for 2x2 matrix a with this elements where a b c and d they are all real

numbers the determinant which we Define by this de a so that is a short way of saying determinant and then in here we always

write the Matrix of which we are Computing the determinant is then equal to and then we are taking this a * D so

we are taking this diagonal elements a * d so they are on the diagonal and then we are subtracting from this this other

two the remaining two elements of the diagonal so B * C and this gives us the determinant of 2x2

matrix this is just a formula that you need to uh remember whenever you want to calculate the determinant of a matric by hand

manually so the calculation for larger matrices it involves bit more uh difficult uh calculation we will see also in a bit the uh determinant of a

3X3 Matrix It relies on the determinant of a 2X two Matrix and the idea is that every time we uh increase the dimension of our problem so let's say we are in R4

then we will go back to the R3 and then given that R3 relies on the determinant of the underlying 2x two matrices anytime we increase the dimension we

again go back to this IDE of using 2 by two matrices that form the entire Matrix in order to compute the determinant only when it is R4 R5 Etc so it becomes much

more difficult to describe and to do it manually therefore there are other algorithms which we will see at the end of this course like uh the composition algorithms and factorization algorithms

that can be used in order to calculate the determent of a matrix that has higher Dimension higher than the tree for instance but in this specific unit we are going to discuss both the

calculation of the 2x two matrices determinant and the determinant of a 3X3 matrices and we will also see detailed examples of them so without further Ado let's then

go ahead and calculate the determinant of this 2x2 matrix so let's now look into this specific example where we are calculating the determinant of this 2x2

matrix so this is the A and let's keep in mind that this is the um uh a this is the

uh B in this not in this uh way of writing the Matrix a so the uh letters corresponding of the uh elements of this

Matrix a so this is the a this is the B and then this is the C this is the D and we said that the determinant

that of a is equal to the diagonal elements so 1 * 4

minus the of diagonal Elements which is 2x3 because we said that the definition of

this determinant is that is equal to a * D and then minus B * C which is exactly what we are doing in here so if we

calculate 1 * 4 is = to 4 and then 2 * 3 is = 6 4 - 6 is = to - 2 therefore we say that the

determinant of Matrix a is equal to Min - 2 let's now go ahead and uh practice with calculation of determinants on two

other matrices so in this case we are still in the two dimensional space so we have 2 by two matrices we'll first calculate the the determinant for Matrix

a so we see that we got this element 5061 and we know that by definition the determinant of the 2x two Matrix so that

of Matrix is equal

to a * d - B * C where the Matrix has the following form so we got a and then D in here and then B and the

C in here so we see that this is basic Bally our a this is our D this is our B

and this our C the way you can also said is that those are the diagonal elements and those are the of diagonal

elements so therefore it means that we can calculate the determinant of Matrix

a by taking the five multiplying with one so it is 5 * 1 minus the off

diagonal element which is 6 * 0 and this amount is equal to 5 - 0 and is equal to 5 let's go ahead and also

calculate the determinant of Matrix B we see here that on the diagonal we have this two elements one one and of the

diagonal elements are both zero those two therefore we can calculate the determinant of this 2x2 matrix which is

also sometimes referred as I2 so it is the identity Matrix because we got here the E1 and then E2 in the two dimensional

space and the determinant of the Matrix B using this definition is then equal to

1 * 1 - 0 * 0 so 1 * 1 - 0 * 0 and this is equal to 1 and this is actually a special case of determinant

and later on we will see why it is so important to uh have this relationship of identity Matrix having a determinant

and having it equal to one um and this relationship between determinant identity Matrix is something that we see so uh in the upcoming lesson so keep this one in mind so now when we are

clear on how we can calculate the determinant for 2 by2 Matrix so this is quite simple and straightforward calculation by taking the diagonal elements a and then D and then

subtracting from that from that product a * C we are subtracting the off diagonal elements products B * C we can then get our determinant and now when we

are clear on that we are ready to go on to bit more Advanced calculations which is calculating the determinant this time

for the 3X3 Matrix so now we increase the um the dimension size and we go from R2 to

R3 because now we have a 3X3 Matrix and by definition given a 3X3 Matrix a which has the following elements so a11 a21

a31 and then a12 a32 a 32 so we have already seen this coefficient labeling this should look very familiar this is

3x3 Matrix 2 and the determinant of a matrix a denoted as that a is calculated

using the formula and here we see the formula we are basically using the 2x

two matrices that form this Matrix a in order to calculate the determinant of the 3X3 Matrix and how we are doing that

well we are using this element and then this element and this element and every time we are

hiding part of the Matrix so when we have for instance this a11 so for this

first part we are saying well let's hide the row and the column corresponding to this

element which means that we need to hide this this row and this column and what is left is this 2x two

Matrix we will calculate the determinant of this 2x2 matrix and we will multiply this with this element that we use in order to remove the corresponding row

and column this will form the first element in here so you can see a11 which is a simple

value so this is the um uh entry volume which is in the first draw and First Column a11 multiplied by the determinant

of this Matrix so this Matrix so once we have that and we already know how we can calculate a determinant of a 2X two Matrix because

this 2X two so taking the diagonal elements and then multiplying them together subtracting from that the of diagonal elements product now we are ready to go on to the next part of the

calculation which is this time adding a minus here so you can see here this here is plus and then here is

minus so we do here minus and for this second step what we need to do is kind of similar only this time

the element that we will be using to understand how we can remove the row and the column so we will then dark it out it is this one

a12 so then we will need to remove this column and this row and then the remaining Matrix which

is this one this 2x two and here I mean a 21 a 31 and then a a

23 and then a33 this is the Matrix that you can see in here remaining which means remove this

one and then this one and then the remaining 2 by two Matrix is what you need to use in order to do your calculations so you can see that I got

exactly the same in here and once again we are Computing the determinant of this Matrix we are multiplying this with this a want to element so this

element and now we have also the second element in our calculation and then we go on to the next step which is a plus sign here let

me use the same colors plus sign here and then we are using this time our final third element to understand which row and

which car we need to dark out which is this element so we then remove the first row and the last column and this is then the

Matrix the 2x two Matrix that we use in order to do our calculation so deter the determinant of this Matrix multiplied by the

a13 so we could also use in the same manner this row or this row it really depends on the kind of values the the tip that um I will provide to you or the

trick is that to always look for these z z values wherever I see Z zos or I see one one I'm thinking that hey those s u

values that um give me the more straightforward and easy calculations because if I have zeros in my Cal in my

entry so if I got a zero here for instance 0 times any determinant is zero I don't even then need to calculate the determinant right because then I know

that I'm multiplying that determinant with zero therefore if I know that that entry for instance this row contains the majority uh of zero so it is 0 01 then

of course it's a great uh row to pick to use these Target elements so in that way I will then know that this is the row that I need to

Target but if it is like that that for instance I got a matrix 10 3 4 and here I got 0 1 Zer and here I have 100 three

and four of of course the easiest thing would be to not use this row but instead use this one so in that case I will then

have this zero and zero as my target values which means that I will only need to calculate the determinant of a 2x2 matrix this uh for this one for the two

cases I don't need to do it because I know that the corresponding Target values the target elements from my Matrix will be zero so let me show you

what I mean by that so if for instance I go for this second row and not the first one what I need to do is that I can

calculate the determinant of a by taking the a21 this then will be my target I will

then need to remove this uh column and this row then I will need to do the

determinant of A1 2 A1 3 a 3 2 a33 this is what then I need to do then the next thing I need to do is of course here I have a plus here I need to do

minus because we always need to Interchange the values so here is a plus here is a minus here is a plus so I do PL a minus in here then I

do the next element in my row which is this one let me use red color so a22 so I'm

then doing a 22 multiply the determinant of so I'm re removing this row and this column A1 1

a13 and then a31 a33 and then the final part is of course as you might have already guessed is to

look into this element so it is plus a23 multiplied the determinant of let me

actually write it down in here the determinant of a11 a12 and then a31 and then

a32 so in this way basically independent of what row I will take as my leading row that I will

do my calculations and I will just need to pick one row I can always get the same value for determinant of a but choosing intelligently which row to

pick it will save you a lot of time and headache in terms of calculations because if you are dealing with a row that contains many zeros for instance

you have 0 0 one or one 0 0 or even better 00 0 then you know know automatically that you will need to calculate your DET the determinant once

here also once and here you don't even need to calculate it you know that you got zero here Z here zero here so it's automatically equal to zero so I hope this makes sense because this is a trick

that usually you will not come across but this just helps you to save a lot of time uh when it comes to calculation of your determinants in a tree by3

settings so in this case uh we have this um we now we have this definition and we know the tricks that we can use but I think it's really uh helpful to go ahead

and to solve a problem so basically this is the higher level summary of the steps that we just discussed um so the determinant of a 3X3 Matrix it simply

involves multiplying the a11 by the determinant of the 2x2 matrix that that remains after excluding the row and

column of a11 so what we did in here by doing this and subtracting the product of A1 2 and the determinant of its

respective 2 two Matrix so this part and then adding the product of a13 and the determinant of its respective 2x

two Matrix so this part and the signs alternate so it means first you always got the

plus then you always is get the minus and then the plus so they interchange you start with plus then you do the minus and then the

plus so let's go ahead and calculate the determinant of this Matrix so before even looking at the answer let's actually go ahead and do

that on this page paper so we got a matrix a which is equal to 1 2 3 4 so

basically from 1 till 9 1 2 3 4 6 4 5 6 and then 7 8 9 and for this 3x3

Matrix we need to calculate the determinant so the determinant of a the first thing that I'm seeing is that there are no no rows with zeros or

columns which means that I cannot use my uh trick and instead I will just need to go with let's say the first dra and it's also convenient given that I got as

scaler this values this much smaller values relatively to the other ones all right so first things first

let's go ahead and write down that formula so the determinant of a is equal to first we are going to take this one so our one

one times and then we got determinant of and then we

have this Matrix which is 5 6 8 and 9 this is our remaining Matrix then the next thing we need to do is to

Interchange the size uh the the sign which is minus and then we got so the remaining Matrix is then determinant

of 4 6 79 and then finally Plus plus three

times and then determinant of what do we have well this is the target so it is

4 5 7 8 right so let's go and do those calculations quickly this is equal to 1 * the

determinant of this is the diagonal element so 5

* it is 5 * 9 - 8 * 6 - 2 * 4 * 6 - 7 *

6 Sorry 4 * 9 so the diagonal elements 4 * 9 - 7 * 6

and plus three * and then 4 * 8 - 5 *

7 this is equal to so 9 * 5 is = 45 8 * 6 is =

48 then - 2 * 4 * 9 is 36 7 * 6 is =

49 7 * 6 is = to 42 + 3 * 4 * 8 is = 32 - 5 * 7 is = 35

so this is equal to 1 * - 3 - 2 * and then here we got 36 - 42 so that's - 6 +

3 * - 3 which is that = to - 3 + 12 - 9 which

which is equal to zero so let's check it indeed we got the right answer perfect so now when we are clear on how we can do this calculation

let's now go ahead and calculate yet another determinant of a 3X3 Matrix and this time I want to show you this uh simplified version by you making use of

this trick that I uh specified so instead of using this first dra as an indicator I will be using the uh second row as my indicator one thing to keep in

mind when making use of this trick is that when you start from the second row so from the even

rows even rows second fourth or sixth then in those cases you need to flip the signs

that you will be using so while in here you had Des Sign Plus in the beginning then you got a minus and then a plus when doing all these

calculations so you remember here we got plus minus plus when you start from the second row instead of first one you need to flip the order of this so you need to

start with minus you have minus you got plus and then minus so knowing this trick it also means that you go One Step Beyond and

you know how you need to intelligent ently uh reduce the time that you are spending on calculation calculation of the determinant but it also means that

you need to be careful on knowing what kind of signs you need to use because if you start from the first row you start with plus and then you do minus plus

minus plus so knowing how to start you already know how you can go on but when it comes to the second row so the even rows you need to start with a minus so

you need to do minus plus minus plus dot dot dot all right so let's now go ahead and use that technique in

here so here I see that my first row doesn't contain zeros but my second row does so this gives me indication that I can reduce the time that I spent on

calculating the determinant at least one time because I didn't no longer need to calculate that determinant so the determinant of B is

then equals you I will then start with minus given that I'm going to use this rope and then I have zero times

so because this is my element the determinant the determinant of 2306 and then

plus then this time the second element Target element is this one so it's four

times and then we

got determinant of 1 3 1 6 and then minus the five times so this

five determinant of 1 2 1 0 and what is this amount it is equal to this I don't need to calculate because I

got a zero in here this this trick is all about this to not calculate the determinant too often and then this equals you four

times four times determinant of this is 6 - 3 so 1 * 6 - 1 * 3 which is equal to 3 and then

minus 5

* determinant of 1 2 1 0 which is 0 - 2

so this equal to 4 * 32 - 5 * - 2 which is equal to 12 + 10 and this is equal to 22 let's go ahead and check this and

this is the more detailed and formal derivation so one uh interesting thing is that I calculated with my second row and in here in this slides you can see

calculation with the first dra this is just a nice way of seeing the difference that you can do and here uh in this solution what we have is that we have manually calculated this first

determinant too so in total three determinants but we again in end up with the same determinant so independ what kind of row you will use in order to

calculate your uh determinant of Matrix B you will all always end up with the um with the same similar volume unless you have made a mistake in your calculations

so you just need to keep track of the uh rows that contain many zeros and you need to um be careful in terms of the signs that you need to use and the sign

that you will need to start if you start with the first row then start with plus if you start with the second row then it is minus and then plus Etc so as you can see here it's a plus and then minus and

then Plus in my case I did with my second row therefore I started with minus all right so let's now move on to

the properties of determinants so the determinant of an identity Matrix is one that's something that we have also seen when doing our calculations because we

are so that in one specific case when we had this example so this Matrix B and the Matrix B was the identity 2 in the two dimensional space we have calculated

its determinant and we saw that it's equal to one and this was not a coincidence because the determinant of identity matrices is always equal to

one then the second property is that swapping two rows or Columns of a matrix changes the sign of its determinant so if you swap rows or

columns in your Matrix so if you end up with Matrix A and B they are exactly the same only one swaps the two columns or two rows then you are changing the

determinant of that Matrix uh the sign of that determinant but not devalue itself it means that if you got a and

you got B and your a is equal to let's say uh A1 and then uh A2 and then A3 so

it contains these columns and then Matrix B is equal to um let's say

A2 and then A1 A3 then the

determinant determinant of a will be equal to minus of the determinant of B

you can also say determinant of B will then be equal to the minus determinant of a this is basically the idea of this

property let's now move on to the third property which says that if a matrix has a row or a column of zeros its

determinant is zero so if you got a matrix a that contains this different values a11 H1 dot dot dot

a and uh M1 and then here you got suddenly um column that contains all zeros and then the rest are nonzero even

so in that case you know that your determinant is equal to

zero so for a specific example if you got for instance Matrix 1 2 0 0 0 3 13

then the determinant of this Matrix is equal to zero and

otherwise if you got a matrix B that has a column of zeros so column that is entirely of

zeros so let's say here we have 1 one one and we have a zero Vector here so we

got in here 0 0 0 and then 3 4 five then given that we have here this zero Vector

then the determinant of Matrix B is equal to zero and this actually straightforward to be seen from this

calculations that we saw because if you do the uh if you pick this specific row and then you do zero times the ter DET minant of the remaining Matrix 0 times

determinant of the other Matrix and then plus so PL and then so minus and then plus and then minus 0 * determinant of the third

Matrix it is obvious that 0 * a determinant is 0 0 * determinant is z 0 * determinant is zero which means that you got a whole bunch of zeros to be

added to each other or subtracted from each other this means that if you have a row or a Col with zeros this already gives you an idea that your determinant is equal to zero you don't even need to

do calculations so the final property of determinants is that if a determinant of a product of matrices equals the product

of their determinants so the determinant determinant of a b is equal to determinant of a multip by

determinant of B this is basically what this property is about so let's quickly go through examples to ensure that we are at the same page with all these properties and

we can prove them so let's say we have an identity Matrix n by n which is we are dealing with I in now according to this first

property when we calculate the determinant of this Matrix

so determinant of i n is equal to 1 Let's actually look at a specific example so here we got um identity um

Matrix in the two dimensional space in the R2 and we can quickly calculate the determinant of this I2 and we can see

that it is equal to this diagonal element so 1 by one - 0 * U it's actually something that we did as part

of my previous examples so this equal to 1 - Z and is equal to 1 one thing that I wanted to show you before moving on to

the next example about the swapping rows is that when we are swapping some of the rows or some of the comms or two rows or two cars of Matrix a we are referring to

this matrix by this notation so we add this nod in here and we say that that this is basically the manipulated version of Matrix a so if we have for

instance Matrix a equals u a b and and c and those are vectors and then we are

swapping two of The Columns let's say we are swapping this two we get B and then a and then C then this Matrix will are referring as

a not this is just an a matter of notation and we just learned that as part of the properties that the determinant of this new

Matrix is equal to minus the determinant of a so if a matrix a has a row or column of zeros then the determinant of it is zero so let's actually quickly look at

this specific example example in here we got a which is uh having a column of zeros and another column of B and D where B and D are real

numbers so let's prove that this determinant is actually equal to zero so the determinant of a 2X two Matrix we have already seen is equal to the

diagonal elements so 0 * D minus the of diagonal Elements which is 0 * B 0 * B

and what is number * 0 is equal to 0 0 - 0 * B is also Z it's equal to Z therefore the determinant of a is equal to

Z so when it comes to the uh determinant of a product of a matrices let's prove that the determinant of a * B is equal

to the determinant of a times the determinant of B so therefore the first thing we need to do is to calculate this

a * B let's quickly go ahead and do that so let me add here this um blank

file so a is equal to 1 2 3 and 4 B is equal to 5

6 78 and I I want to prove that the determinant of a b is equal to

determinant of a Time determinant of B first I will be calculating this and then I will be calculating

this so for the first one what I need to do is that first I need to

calculate d a * B which is equal to 1 2 3 4

times 5 6 7 8 and then this is equal 2 should be 2

by two so first I take this 1 * 5 is 5 2 * 7 is 14 14 + 5 is 19 then for this one I need to pick this

row so 3 * 5 is 15 and then 4 * 7 is 28

so 15 + 28 is so there we have 33 43 so I got here 43 then I'm going on to the next column

which is in this case 1 * 6 is 6 6 + 6 16 is

22 and now the second column 3 * 6 is 18 4 * 8 is 32 and this gives me

50 all right so now I have the a * B then as the next step what I need to do is to calculate the

determinant of this a * B which is equal to the determinant of this Matrix 19 22 4350 that I just

calculated and what is this amount the diagonal elements 19 * 50 - 43 * 22 19 *

50 is then equal to 95 and 43 * 22 is 94 6 which means that

we end up with four this means that the determinant of the a * B is equal to 4 let's quickly check what are the parts

of the second amount so for that I need to calculate determinant of a which is

equal to 1 * 4 - 2 * 3 1 * 4 is 4 3 * 2 is 6 so 4 - 6 is = - 2 determinant of B

is equal to 5 * 8 which is equal to 4T and then 7

* 6 is equal to 42 and this is equal to - 2 and determinant of a *

determinant of B is equal to - 2 * -2 which is equal to 4 so we can see that now we just provve that the determinant

of a * B is equal to 4 so we have seen that determinant of a is equal to 4 and we see that that's exactly the same as

determinant a * determinant of B which is equal to 4 so we have just proven that the this equation indeed

holds so the determinants they are not just um some calculations or some amounts but they are actually uh important concept and their

interpretation um is highly relevant from geometric perspective so the terminant have a geometric interpretation and the for example the

terent of a 2X two uh Matrix or 3x3 Matrix they represent the area in case of 2x two or the volume in case of 3x3

Matrix uh of the parallelogram that they are forming so uh this is often referred as

a parallel uh piped um I hope I'm pronouncing this correctly and it's formed by the con vectors of the Matrix

so if we have for instance this uh Matrix a and then we have a b and then C and D we have this A and C which is the first vector and then B and D which is

the second vector and the uh the shoe vectors they actually form a parallelogram um when it comes to the uh

two dimensional space and the area that this uh parallelogram um is forming that is equal to the determinant

of this Matrix so the determinant ofer this scalar value that summarizes this linear transformation that we describe by this

Matrix because we saw that we had this a x is equal to B linear system that we were describing using this coefficient Matrix and this

was our unknowns this was our variable and then this B was the um amount that we were uh putting this as equal to if B was equal to zero then we were solving

the homogeneous system otherwise we had this non-homogeneous system and in the geometric terms the determinant of this Matrix

a so the determinant of a um in case of 2x two space so in R2 um when we got two vectors

basically in our Matrix a this is equal to the area that is spent by these vectors in the two dimensional space in a bit I will also show you specific

example such that um we will be on the same page when it comes to this concept of parallelogram the deter determinant and those vectors that form the column

um uh space of the uh Matrix a uh when it comes to the three dimensional space when we have R3 so we got 3x3 Matrix of

a then the determinant of this Matrix a is the volume that is um formed by these uh threedimensional vectors because

unlike the 2D in R3 we got the three vectors that form the a let's say this one this one and then this one and then here we can

create this area covered by this Tre vectors and the area that is formed by the tree vectors

from a it is equal to the determinant of that Matrix a so in terms of the 3D it's bit harder to uh visualize it but in uh

case of the two-dimensional space I think this will help uh to improve our understanding of the determinants and make this interpretation uh from

geometry uh from geometrical perspective so given the two vectors A and B in the two dimensional space the determinant of this Matrix uh is then equal to the um

diagonal elements we already know minus the of diagonal elements right so we are also saying we have seen this notation already very

often you will see this volume this is the absolute we already know this from high school this is the absolute volume because the determinant can also be a

negative number we have seen minus 20 or minus 2 and we know that the area cannot be a negative number therefore we are

adding this absolute term here so knowing for example that we have this m matx a which consists of the elements 3

2 and then 1 14 we know that the determinant of this a is equal to 3 * 4

12 - 1 * 2 it is 10 and the absolute value of it so absolute value of 10 is equal to 10 given that is positive and

this is exactly what we have here and this is referred as the area of the parallelogram that the two vectors are

forming and how does that look like in uh the uh coordinate space so this is the parallelogram that we were referring by and

this area that is formed by this parallelogram is equal to the determinant of the a The Matrix

a so so one thing that we need to keep in mind is the definition of parallelogram which means that those two are parallel and they are the same so this and this lines those two are the

same and then of course the same holes for those two they are parallel and they have the same um length therefore this

figure in here this is what we are referring as parallelogram and those two vectors that we can see in here

this one and this one they form this parallelogram and they are the two vectors that are part of the Matrix

a hence if we got two vectors that the uh that come from The Matrix a so Matrix a and we got here this two vectors in a

2X two Matrix then the determinant of this Matrix is then describing the area that

these two vectors are using or are spanning when creating this parallelogram so the determinants they

play an important Ro in understanding the geometric properties of the spaces that uh spent uh by these vectors they provide valuable insights when it comes

to the scaling effect effect of linear transformation the or orientation and the um the locations of them in the cordan system as well as the Practical

applications in calculating areas in calculating volumes welcome to another unit in our fundamentals to linear algebra course where we are going to talk about Advanced linear algebra

Concepts so uh in the first module we are going to talk about Vector spaces and the projections we are going to define the bases in a couple of examples

of them we have already touched upon this concept briefly when we are calculating the basis of a no space and the basis of a comp space we are going

to do a similar example in this case and then we are going to uh look into this concept of the uh standard bases for uh different spaces including the R2 we're

going to introduce the concept of projections what is the definition of projections what is a Formula how we can calculate it we are going to look into detailed examples of that

then we are going to talk about the concept of uton normal basis in this module we are going to introduce this concept and we are going to understand the orog gonality normalization we are

going to then discuss a very important topic in linear algebra which is a gramme process we're going to Define it we are going to see the overview the

step-by-step process of applying grme uh algorithm then we are going to see an example of it and the calculations step by step and then we are going to talk about

applications of auton normal bases the application of gram Smiths process and the importance of this auton normal

basis this is the module one of this part so let's first Define the basis a basis of a vector space is a set of of

linearly independent vectors that spend the entire Vector space every Vector in the space can be expressed as a unique

linear combination of the basis vectors so there are a couple of parts in this definition they are really important and first thing that we need

to uh mention here is this Vector space that says it is a set of linearly independent vectors that spend the

entire Vector space this is very important because um here we are with

the basis is simply this Vector space that is a set of linearly independent vectors which means that one of these

vectors cannot be Rewritten as a linear combination of the other one so we have a linearly independent vectors and they

span the entire Vector space so for instance if we are in R2 then the basis of a vector space is

then a set of linearly independent vectors that span this entire R2 so we do we then need to

have for vectors forming a basis so let's say we have a basis of vector

space for us to say that this is the basis of this Vector space let's say in R2 we need to First

say we need to First prove that these vectors this

vectors are linearly independent and two they

span the entire R2 which means that span of this vectors is equal to R2 we can actually be even more spefic

specific in a example of let's say having a vectors A and

B we can say that this set that we have here consisting of vectors A and B in R2 form the

bases of a vector space if the first criteria is that a and b are

linearly independent and the second criteria is that those two vectors together they

spend the entire Vector space of R2 which means that span of a and b Vector space is equal to

R2 on more specific example and then the second part of this definition says that every Vector in the

space can be expressed as a unique linear combination of the basis factors which means in our specific example when we had this A and B forming the bases of

a vector space this means that if we prove that this is indeed the basis of this Vector space then any

combination every Vector let's say a vector

C that consists of this C1 and C2 elements that this Vector this random Vector from R2

C can be represented as a linear combination of these vectors A and B so

let's say we have a coefficient K1 * a plus K2 *

B then here we are representing this random Vector c as a linear combination of these vectors A and B which form the

bases of a vector space of this Vector space so we have previously spoken about

the no space and Comm space so let's now go ahead and do one more example when we are calculating the new space and the Comm space and then we are again

calculating this concept of basis of Vector space and a b basis of the com space and then we will be uh finding the

basis of a vector space uh with um R2 example so given that we have already looked into this concept the basis of Comm space and based of no space I will

try to uh go through this example bit more quickly to save time on more complex Concepts so let's say we have an example

Le of a matrix and that Matrix is a is equal to 1 2

36 this is our 2x two Matrix a and the first thing that I want to do is to understand look into my Matrix and

understand whether I'm dealing with unique vectors or not and by unique I mean whether I'm dealing with two vectors that are linearly dependent or

linearly independent this kind of inspection always helps us to save time when we are doing our calculation for the no space and for the Comm space and for the basis of no space and base of

Comm space now here we can see that this is our A1 the first Vector the first com Vector forming the Matrix a and this

Vector is the A2 another thing that uh we can notice here is that we can easily take the

First Column A1 multiply it by two and get the A2 because 1 * 2 is 2 3 * 2 is 6 that is that 2

* A1 is equal to A2 which means that we can say that A1 and

H2 are linearly dependent okay so seeing this and knowing this this can help us to quickly

go through our calculations of the bases of the co space and the bases of a no space so let's go ahead and first

calculate what is the basis of no space of

a so we have already learned that the um basis of a no space can be calculated when looking into the first no space so

we C we need to calculate the no space and then we need to calculate the basis of that no space so this means that we need to get the

na and we have learned that in order to get DNA we for that need to solve the a x is equal to zero

problem and this x will give us the no space of a we have also learned that the no space of a is equal

to the no space of r r EF of a which means that using gausian reduction or gausian elimination we can quickly find

the solution to this problem of a x is equal to Z and find this x this is simply solving a similar problem only in

this case the B so this is equal to zero because we are dealing with the homogeneous case I want do the calculation for this

we have done a ton of examples when we were doing this step by-step calculation getting the uh argumented Matrix of a and then uh doing all these different

draw operations normalizations and then eliminations in order to uh get this uh complex Matrix a to the point of uh

basic representation from which either we can visibly see the solution to the problem or we can at least simplify it and describe it as a linear combination of vectors

in this case if you go ahead and solve this problem you will find that the x that

solves the a x is equal to Z problem is unique and this x is equal to minus

0.894 as the first element and then 0.447 as a second element this can be a good practice also to refresh um the memory when it comes to the gaion

elimination and reduction the example itself is quite simple the a is just a 2x2 matrix um and um by performing couple of operations uh in terms of

normalization and elimination you can find this X for your a is equal to zero given that now we know what is the

solution to a is equal to Z problem now we know what no space is because in this case

the all this help us to understand that the no space of a is then equal to the

set the vector set where as part of this we got just single column which is - 0.894 and

0.447 this is the no space this is the first part I will say it 1.1 and then 1.2 will

be to get the basis basis of this Na and we have just seen what is the

definition of the bases so the basis of vector space is a set of linearly independent vectors that spend the entire Vector space therefore given that we got just

this single Vector as a solution to our problem we can see then very quickly that the new space of a is based on this

and then the basis of the no space is simply this entire set so knowing what the solution is to our homogeneous problem a is equal to

zero so let me also write down in here then we know that the no space the N A is then equal

2D Vector minus 0.894 and 0.447 this is my Vector X that solve this ax isal to zero

problem and this is simply the no space of a and given that we have calculated and we have got this

unique solution to our problem we can say that any Vector in R2 can be represented as a linear combination of this

Vector so 1.2 any Vector in R2 can be represented

as linear combination combination of this Vector

X therefore we are saying that the bases of of no space of

a is this entire set consisting of the single Vector so this is about the basis of a no space let's Now quickly look into the concept

of the basis of a calm space so the first thing we need to then uh get is the column space

so to get the basis of Comm space we need to get the ca first which

is the Comm space of a and what is the Comm c space of a the Comm space of a is the uh setle and the space of the

vectors that we can see in here in this A1 and A2 is it's quite straightforward so this two vectors they

form the Comm space of this Matrix a so then the ca is

simply the set of one three and then two six vectors this is A1 this is

A2 now we have just seen in the beginning before even starting our calculations that A1 and A2 are linearly dependent because A2 can be right

written as 2 * A1 so one of these vectors can be written as a linear combination of the other one this means that we got just a single linearly

independent vectors and why is this important because we have seen in the definition of the basis that for us to

have a basis we need to have a linearly independent vectors so the basis of vector space in this case the Comm space is a set of linearly independent vectors

that need to spend the entire Vector space in this case R2 so therefore we need to look into the ca

that we got in here and select one of these two vectors that can be considered as linearly independent let's say we pick

one three now we know that we can then write any Vector in R2 as a linear combination

of this Vector 1 3 so we can scale this Vector one Tre and get a new Vector in R2 therefore or we are saying that the

basis of Comm space basis of Comm space

space of a is then the set of one Tre because one Tre so A1 is then

linearly independent and the span of A1 is

R2 now when it comes to the uh basis of the entire R2 one thing that we can notice is that this A1 so one

three it's not forming it's not spanning the entire R2 because because we cannot uh write any random Vector in r two as a linear

combination of this two therefore we are saying that this is the basis of Comm space but we are not saying that this is the basis of R2 and the final element in this definition that I want you to uh

focus on is that every Vector in the space can be expressed as a unique linear combination of the basis vectors so in here we have looked into

this idea of bases of a new space and the base of Comm space and we saw that we are talking about specifically the new space and comp space but when it

comes to the entire space for instance the basis for R2 then the basis of Comm

space for instance is no longer um helping us because the basis of Comm space it consists of this Vector one tree and this one tree alone is not

satisfying the second criteria that says that this Vector needs to spend the entire Vector space because this one Tre

vector it's a single vector and this Vector it is not forming the entire R2

it's not um the basis for R2 it's not spinning the entire uh R2 so

given that the one tree is not spinning the entire

R2 because of that we know that the one tree is not the set of one Tre is not the

basis of R2 so this distinguishing of the basis of R2 basis of Comm space basis of no

space is really important because basis for R2 it means that we need to find set of linearly independent vectors that

they together form the entire R2 they span the R2 which means any random Vector that we can see in R2 we can

represent as a linear combination of the vectors in this space so in here let me also prove that this one tree alone is

actually not forming the R2 it's not spinning the R2 which then uh concludes that they are not the it is not the

basis of R2 CU and after this I will then provide you an example where we have a set of vectors that span R2 and

are linearly independent which means that they are the bases of the entire R2 so first I want to show you why this

single Vector one Tre is not the basis of R2 so being the base of R2 we have the criteria that the vectors need to be

linearly dependent so let me actually clear up some space here so I want to see and find the basis of

R2 first I want to prove that this set which is the base of Comm

space I want to prove that this is not the basis of

R2 then I will also as part of the second part of this proof look in look into the case when we do have vectors and the set of vectors it forms the base

of R2 so the first thing the first criteria of the basis of R2

says that quote 1.1 the first criteria says that this Vector in this Vector space it need to be they need to be

linearly independent well that criteria is valid given that one3 is linearly

independent this means that criteria one is satisfied so whenever you got just one vector this criteria is automatically

satisfied so then you have the 1.2 which says that we need to have this spin of these vectors equal to R2

so is the span of one3 the R2

well no and how we can prove that because the idea is that any Vector including an example where I have for

instance uh let's say four and five this Vector that I need to be able to find a scalar that will help me to create a

linear combination let's say C linear combination using this Vector 13 which will then set this amount this

to be equal to this which means that I need to be able to write my random Vector 45 as a linear combination of

this Vector that forms my uh Vector space so let's see whether that is even possible well here I got four and

five if I do this multiplication in the right hand side I get C and here I got 3

C because C * 1 is C and 3 * C is C and this means that I have an equation 4 is equal

to C and 5 is equal to 3 * C

from this I get that the C is equal to 4 and C is equal to 5 / to 3 but that is impossible because 4 is

not equal to 5 / to 3 which means that I'm proving in here and I got to prove that the uh any random chosen Vector 45

cannot be written as a linear combination of this Vector that 4 forms this uh space

therefore as random Vector from

R2 can't be written as linear

combination of one three criteria two is not satisfied because for that we had to say

that this pen of One Tree is equal to R2 which we saw that it's not the case because then we would have been able to represent this four five as a linear combination of the one Tre Vector okay

so now we have proven that the one Tre is not forming the bases of R2 let's now look into what then does form the basis of R2 an example of

it so we are familiar with the unit vectors E1 and E2 into

R2 which form the identity Matrix I and this is 1 0 and this is 0 1

also 1 0 0 1 in the form of a matrix so in this example we have a set consisting of E1 and

E2 where this is this E1 this is the E2 and the set corresponding to this Vector space is then

1 Z and then 01 and now I will be proving that this space this Vector space does

indeed equal to the bases of R2 so this is the basis of

R2 so the first criteria of the bases is that these two vectors should be linearly independent now we can quickly uh

remember from our previous theory that the two unit vectors one z01 are actually linearly independent that's

something that we have proven and you can easily see it also from here there is no way that you can find um scalar

C that you can multiply this Vector we and get a vector 0 one because for that for this one to become a zero you need

to multiply this with zero but then 0 * 0 is not equal to 1 which means that there is no way that you can find a

scaler C to multiply this E1 to get the E2 so let me write this down

E1 and E2 are linearly independent because there is

no scaler C which is a real number such that such

that c * E1 is equal to Ich so this

means you can't write hu as linear

combination of A1 or vice versa this means that E1 and E2 are linearly

independent and this satisfies our first criteria so criteria one is satisfied

what we have also learned is that any Vector in R2 can be actually written as a linear combination of a unit vectors

that form that um R2 in this case 1 0 and

01 so let's assume that this random Vector is C1 C2 so this is C vector

and what we want to prove is that we can always write this C in terms of linear combination of these two vectors and how can we do that

well let's say here we got a K1 K1 which is a real number and we multiply this by one

Z and then we add K2 K2 and then here 01 so this is our E1 this is our E2 can

we do this well what is this this is equal to K1 0 plus 0

K2 and and what does this give us well this means this

amount let me write it over K1 * 1 which is the E1 plus K2 * 01

which are which is our second Vector E2 this is equal to K1

0+ 0 K2 and this is equal to K1

K2 so I got on one hand this Vector C1

C2 which I want to write as a linear combination of K1 E1 plus K2

E2 if I take D K1 equal to C K1 and

K2 K2 equal to C2 well then in that case I can prove so

this is basically equal to C1 and C2 which means if I take this K1 and K2 equal to C1 and C2 respectively and

those numbers are given then I can represent this vector c as a linear combination of

E1 and E2 which is what I had to prove in order to say that the

Spen of one Z which is the E1 and 01 which is

E2 is equal to R2 because any random Vector that will be provided to me with an element C1 and C2 and those are just real numbers can be

written as a linear combination of these two vectors this means that the spend of these two vectors is equal to

R2 and this is basically the second criteria so criteria to

satisfied and if the criteria one and criteria 2 are both satisfied it means

that this Vector space of 1 Z and 01 this is the

basis of the entire R2 so let's now talk about the concept of projections by definition a projection of a vector a onto another

Vector B is the orthogonal projection of a along B it's denoted by approach and then B underneath here we see the index

and then a so projection of a onto B so here is the A and here is the B and represents the component of a in the

direction of B so component of a in the direction of B all right so in order to properly

understand this concept the intuition of it let's actually make use of the R2 space so let's first start by picturing

in our flat world the R2 coordinate so the Cartesian coordinate system so let's say here we got our y AIS here we got our

x-axis so this is the X this is the Y and uh here we of course we need to keep in mind this is just an example when it comes to projections we can

always go beyond R2 but for keep it simple and truly understand this Concepts and this intuition behind the projection I want to simplify this and

do the example in R2 so here uh imagine that we got this line

and this is our a line that goes through the center that let's call this line

B so B is line in R2 let's say this is that line and now that imagine that we have

this Vector which is part of this line let's say this is this line

and this line is the representing by uh on this line we got this Vector B and this Vector is basically part of that

line as you can see this is the vector B on this line B so we know from this concept of the line

spanning the R2 and then vectors we know that in this case independent what is the magnitude of this Vector what is the direction of this Vector we can

represent this line B by this linear combination based on this Vector so linear combination of this Vector which

is in this case D

set set then here we got some C where C is a real number multiplied by this

Vector B knowing that this C is just a real number so let's make it actually green so we can basically say that this entire

line B can be represented as this set of this linear combinations of these vectors so for instance if this

is one and we do the C is equal to two then we can get this part of so we can get this Vector otherwise this is equal to three we can

get this vector or C is equal to for this vector and then and so on which means that we can always come up with a linear combination forming a part of this line therefore we are seeing that

this line can be represented as all these linear combinations uh of this Vector B which is part of this

line and here the C is just a scaler so a number which is a real number so this C * Vector B represents this uh entire

line so we will knit this in a bit but for now imagine this line and part of this line which is this Vector

B so imagine then that we got yet another Vector which is let's say in

here again going from the center but this time in this different direction so in here this

is Vector a we call this Vector an A so you can see that this Vector a is

actually much longer than the vector B and we see that Vector a is not lying on the same line as B so B is lying on the

line b and a is not lying on the line B now let's say we want to project this Vector a onto this Vector B

which means that we want to project this a in this direction so we want

to bring this Vector a onto this line let me actually use a different color and the word of the projection

actually does make sense in here as you might notice because we're trying to cast the shadow of a onto this line

of B and how can we do that we can only do that if we connect this Vector

a like this with this orthogonal line let's Say by using a different color of

this so with this perpendicular line we then will be connecting the vector a to the line B because we want

to project our Vector a onto this direction so this perpendicular line that you see in here that

goes from Vector a to the line B where on line B we have the vector B so here

is the line a line B and this perpendicular line it goes from a to line B and on line V we have the

vector B that is represented like this then the projection of a onto Line B

is this Shadow Vector that you see in here and the word projection or the name projection actually does make sense

because we are projecting this Vector a onto this line and it creates this Shadow so we are casting this Shadow on here and this Vector is what we are

referring as projection of vector a onto l line B notice that we don't say projection of B

on Vector B but instead we are saying projection of a on the line B then another thing we can notice is that we are getting this

projection of a on B so this

vector by taking the vector a so Vector a and subtracting from that

projection of a on B that is the formula for this

Vector that we refer as a perpendicular that goes from a to line B so when drawing

this perpendicular line from a to line B we are referring this as a minus projection of a b because you can see that this Vector is simply this Vector

minus this Vector that is the um mathematical expression for this perpendicular

line so how we can then find out what is this C that we got in here because we understand that to get this exact

formula for the projection of a on the line B we need to understand what

is the scaler specifically what value are we using to multiply this Vector B to get to this point so what is that

c what is C what is C such that c times

a is then equal to projection of a on the line B because we can have

different sorts of a linear combination of vector B on this line uh B and in fact B this line B is the set of all linear

combinations of this Vector B and I want to know specifically what is the vector that we see in here what is the shadow Vector

because this is the projection of a on the line B what we see in here now how can we do

that well let's first formally Define on this specific case what is the projection of a on this line B so projection

of a on line B is some Vector that is also

on line B where a minus projection

of a on B is per pendicular or

ortogonal to this is basically the definition of the projection of a on

line B under this specific example so in this case the way we can find this

projection is by looking into this C so this is what we are interested this specific specific C

* B vector and knowing C and knowing B we already know what is B what B is knowing C

we can then describe this specific projection so one thing that we can know is the condition under which we say two vectors are autal that's something that we already have learned as part of the

previous lessons so let's go ahead and find that amount so now what we need to do is to calculate this value of C because value of C calculation will then

lead us to the exact uh Vector that we are interested in which is this projection so our end goal is to find

out what is this projection of a on B this is what we want and for that we need to calculate this C because we already know the vector B so let me

quickly remove this part cuz here we will then do our calculation so one thing that we need to make use of is this part when it says orthogonal because we know that if two

vectors are orthogonal then they dot product is equal to zero so we know that this Vector is orthogonal to this target

Vector which means that we can say that the vector a and then minus

projection of a on B multiplied with Vector

B that this is equal to zero this is something that we know by definition of orthogonality two vectors are orthogonal it means that their

dotproduct is then equal to zero now let's make use of that part so this means that we need to describe

this projection of A and B we need to make use of the fact that we know that this projection of a onto B is actually

some linear combination of vector B so let me actually go ahead and remove this part we already know the definition

so let us go ahead and calculate that c that we need in order to find out what is this entire projection so few things that we need to clear out is those formulas because then we can make use of

them to find the C so we know that by definition the projection of a on the line B it is this Vector that we get where we draw this

perpendicular line from Vector a onto Line B and we said that this line is equal to this amount this is simply the vector a minus this Vector the shadow

Vector which we said it's defined by projection of A and B this thing so we can make use of that because we also see in here that this we are saying isogonal

to this Vector so given that this uh Vector a minus projection a b is orthogonal to

line B that is also orthogonal on this specific Vector which is the projection itself so from this we can make use of the fact that two vectors

when they are autal their dotproduct is equal to zero in order to find this uh

value of C so firstly we just set that the a minus projection of a on the line

B that this multiplied by this Vector B is equal to zero because those two lines they should be

perpendicular but at the same time we know that this is simply the linear combination of this Vector because this

line is perpendicular to this one and this line is some linear combin a of

this Vector B because if I have here a vector and then I have the longer version of that Vector on the same line which is then a linear combination of

this original Vector let's say this is my Vector B then this second Vector that I have in here is then equal to some C *

Vector B this is also exactly what we said in here here we said any Vector on line B can be represented as a linear

combination of vector B and this is exactly what we are seeing in here so this

projection is simply that c times Vector B this is something that we have already

said so we are just making use of that to fill in that volum so this then results in a minus this C

* B multiplied by this Vector B is equal to zero formula so here we are simply making use

of the fact that the projection of a onto B is the shadow Vector which is then equal to some

linear combination of this original Vector B which is on this line B then I can easily find the scaler C

from here because we know how we can easily calculate this dot product so let us actually go ahead and do that let's first multiply

this a by B and then my minus so I'm simply

opening the parenthesis C * then I got B by B and this equal to zero so C

* B time B is then equal

to a and b which means that

c is equal to a * B / to B * B now when we have the C we can easily

derive the formula for the projection of a on the line B so this is the first part this is the second

part so then the projection of a on B so

projection of a on B is equal to this c

c times the B and we just found out that this is equal to the C was equal

to a * B / to B * B and now we need to take this C and then

multiply by Vector B this is then the projection of a on B this Vector so projection

of a on line B so you will notice that this is the same that we just got so whether you

compute the projection of a on the entire line b or projection of a on the specific Vector b as we are using the vector b as a source for drawing our

line this is the same as the project rection of vector a on Vector

B and this is the same formula as we SE in here so this is the projection formula that we have just uh found out so projection of a onto B is given by

this formula a * B so the dot product of the vector A and B divided to the dot product of the bay withd itself and

multiply with the vector B and this is the in here this is something that we have calculated time and time again in our examples so if we

go back to our example then here we can see that this

is our Vector B this is our Vector a and we are saying if we take the vector a and we project it onto this Vector B

then we can calculate this Pro projection which is in here the formula for this entire Vector which we are calling projection

of a on b or projection of a on B this can be find out so the the length

of that Vector we can find by using this formula so the dot product of vector A and B divided to the dotproduct of B with itself and then multiplied with

Vector B so again a DOT product produ and this is of course something that we get as a vector so this is a vector something that is equal to this

entire Vector in here this Vector so uh I know that this uh might look bit messy because it contains many

moving Parts but I wanted to provide this detailed explanation and the step by-step process even if it is bit confusing and bit messy um in the

beginning because this help us to understand what this uh formula is about and what is the intuition behind it because what we are doing is that we are

making use of the fact that the line can be represented as a linear combination of all the

um vectors that we use in here so this is Vector B and this entire line B is a linear combination of this vector B and we can make use of that in order to find

that scalar that we are multiplying to create this single linear combination that will end up giving us this Vector that we see in here which is the

projection the projection that we are interested which is this line This is the projection that we are defining by this projection a on to

B and we can get that by making use of the fact that this this perpendicular line that we are creating in here which is simply the vector a minus this

projection this is this Vector this projection Vector that this is perpendicular to this line

B and if the vector B is part of this line B this means also that this line a minus projection a is also perpendicular to that vector vector B making use of

that formula we can then uh make use use of the product of the two we know that the dot product of two perpendicular vectors is equal to zero making use of that we can then obtain this specific

scaler C that we need in order to get the final formula for our projection we are interested in

this C because knowing C we can then multiply with this Vector B to get our final projection and we have found that

that projection a on B is is defined as the dotproduct of the A and B divided to the dotproduct of the B with the B and multiply with the vector B and this is again a

vector now let's look into a couple of numeric examples to clarify this topic and practice with it so given vectors A and vectors B find the projection of a

on to B so without looking into answer I will quickly go onto that example itself so Vector a

is this Vector 3 4 can also represent this by our more common notation which

is three and four and then Vector B is one and zero so let's quickly draw

our coordinate system this our xaxis this our y axis and then what is the a

the a is three and four three and four so this is our

a and what is the B the B is one and zero which which means that our line B is

then C times the vector B given that the C is a real number and one thing that you can notice is that the line B is

actually our x-axis it is this line this is our line B this is our line l b

so the projection is then this line this is our projection because we can know that by drawing a perpendicular

line in here from a to the line B we can get then the connection between

our Vector a and Vector B and create our projection so this is then the A minus

projection of a on line B and this part is then this is then this

projection a on B and how we can get this projection

well we just learned that the projection of a on B is equal to dotproduct

of a with B divide it to dotproduct of B with B itself and multiply it by

B this is the formula that we can use and even if you don't remember the formula by heart you can make use of this visualization to figure out what that formula is because we know that if

this line is perpendicular to this one then a minus projection of a on B multiplied by this projection a on B should be equal to zero and this

projection of a on B is equal to some scalar C multiplied by Vector B that's something that we see in here the first thing we need to do to

compute the dot product between a and b a * B is equal

to 34 multiplied by 1 0 this is the dot product which is then equal to 3 * 1 + 0

* 4 and this is equal to 3 the next thing we need to to do is to compute the dotproduct between B itself

so B * B and what's that that is 1 0 with 1 Z multiplied this is equal to 1 1

* 1 + 0 plus 0 * 0 is equal to 1 then the third thing that we can do

then is to obtain the final value which is projection of a on

B is then equal

to three / 2 1 multiplied by the vector B which is 1 0 which is equal to 3

0 and this actually makes sense visually too as you can see in here this is the tree for the xaxis and here we have the

center Z so this projection is then the vector 3 0 so even without calculation we could see just from plotting the uh on the coordinate system the vectors A

and B that the projection of a on B will be this Vector 3 but we have followed the formula in order to do calculation step by step which is something that you can see

in this answer too so the projection of this Vector a onto B is then this Vector of a length tree in the direction of B so you can see

that it is of the length of three so this is the tree on the direction of B so on the line B let's now move ahead and look into a

different example but this time we will do the calculation in a quicker way so we got two vectors 4 three and B is equal to 20 and we need to find this projection

of a on to B so the first thing we need to do is to calculate the a * B which is

equal to 43 multili 2 0 and that's equal to 4 * 2 + 3 * 0 and it's equal to 8 the second thing we need to calculate is the

B do product with B which is equal to 2 0 2 0 this is then equal to four and the final part is to take and uh from this

one and two this values and then bring them all together so then the projection

of a on B is equal to H ided to

4 multiplied by the vector to0 and this is equal 2 8 / 2 4 is 2 2 *

2 is 2 2 * 0 is 0 so we are getting this two Vector so projection of a on B is then this Vector 20 which is actually on

this xaxis similar to what we had before only with the length of t uh towards the direction of which is then equal to 4

and0 and this is again similar to what we had before uh where we got the projection of a on B on that end up on the x axis but now with the length of

four so now our projection has the following Vector so the uh following magnitude and Direction so this is the step by-step

process that I just followed if if you want to do it bit slowly and this is the final result so uh the interpretation of this

projection is that this projection a onto B is simply this 4 zero this means that the A's component in the direction

of B it spends uh four units along this x-axis that we saw in here because this is the value X this is the value of y

so this projection shows us that A's influence in the direction of B is completely horizontal with this magnitude of four because we saw that we

end up with the projection on the x-axis again so this was four this was our projection vector and if you plot this

entire Vector a and Vector B on this x-axis and y axis then you can clearly see that the uh horizontal line that we

end up with the uh projection of a n b is very similar to what we had

before in here let's now talk about a concept of auton normal bases so let's now Define what the auton normal bases are so by

definition auton normal basis for a vector space is a basis where all vector vors are orthogonal or perpendicular to

each other and each Vector is of unit length so as you can notice here here we have a special type of basis it's called

auton normal basis because in the beginning of this section of this module we defined formally this concept of bases we talked about the concept of

colal uh space and then the uh basis of a comp space the no space the basis of a n space and then we talked about the

concept of the bases of the entire space for instance the R2 and now we are defining a special type of bases which we are referring as auton normal bases

and this auton normal basis as you can see from this definition it contains two criteria for it to be auton normal so an auton basis for a vectory space is a

basis where a all vector are orthogonal or perpendicular to each other and B each Vector is of unit length we already

have learned that when we have vectors let's say Vector A and B perpendicular it means that A and B their dot product is equal to zero

that's the first criteria that we need for calling our basis an auton normal basis then the second criteria is that

each of these vectors they need to have a length of one if we have this condition satisfied

then we are saying that our vectors they help us to form this auton normal basis if we got three vectors forming this

Vector space it means that we need to have the a * B = to

0 a * C = 0 and then B * C = 0 this is if we are in in case we are using three

different vectors that Define our Vector space in this

case let me make this part smaller so let's put the length of B in here in this case

the second criteria becomes that the length of a is equal to the length of B and then is

equal to the length of c and is equal to one so depending on the number of vectors that you use to form your vector space the proof that you are dealing

with auton normal bases will be different here we got just two vectors here we got three vectors but in both case we first need to prove that we are

dealing with uh vectors Each of which are set of orthogonal perpendicular vectors and all of them pairwise they need to be perpendicular and at the same

time the second criteria says that they all need to have a unit length so their length should be equal to one we need this auton normal bases in

order to simplify our calculations including the calculations of projections and Transformations that we just so before when we were discussing this concept of projecting a vector onto

a line or projecting a vector onto not a vector because we were in this basic case when we had just two vectors in R2 and calculating projection in R2 is

very easy because we can make use of this formula um a and then B uh the dot product of them and then divided two product of the B and then times the B

this was quite straightforward right but when it came so this is the projection of a on B

but when it comes to projection in higher dimensional space let's say you have R5 or you have R 100 or R th000 then it becomes much more difficult to

do those projections and to calculate the projections and for those cases we can make use of this concept of auton normal basis to simplify our calculations and we will see that in a

bit so let's first understand this orthogonality and the normalization part so orthogonality refers then to the part of uh when we are saying that the

vectors should be orthogonal to each other and the normalization refers to the fact uh to the fact that the length should be one this is basically the set

of two criteria that I just discussed this is uh the summary slide that will give you an indication what is meant by that so if we have two vectors V and W

then we say that the first criteria is that those two vectors are orthogonal which means their dot product is equal to zero and we are saying that their length is equal to one which we are

referring as a normalized vector so if the vector has a length of one then we are calling a vector v

normalized so if both of this criteria of normalization and orthogonality is satisfied that we are saying that we are dealing with an uton normal basis so now where we have

learned this idea of projections also this idea of autog colonization and the uh concept of auton normal basis we are ready to discuss the concept of the

grade process so the grade process is this method for orthogonalizing a set of vectors in an inner product space and turning them into an auton normal

set so let's say we have a set of vectors we want to uh bring and transform all these vectors onto this

auton normal set of vectors which means that we want them to be aized so we want them to be perpendicular and we want them to be normalized because we know

that the two criteria were specified right so the first criteria was that we need to have vectors ortogonal hence we are doing

orthogonalization and the second criteria was that they need to be normalized because we want the vectors to

have length one so we are doing normalization this process of turning this set of

vectors onto this uton normal set by using this method of orthogonalization which is something that we are referring as a grme

process this is something that we can use in order to simplify later this different sorts of Transformations which we need in order to perform bit more

advanced uh Transformations like Matrix uh factorization different decomposition techniques so given this set of linearly independent vectors this process which

we are referring as grme process produces this auton normal set that is spinning the same Subspace so we have the same

Subspace it's just that we are turning the set of vectors into an auton normal set of vectors that is spanning the same

Subspace so the gr Street process step by step looks like something like this so given the vectors A1 A2 up to a n the

first thing we need to do is to start with the vector V1 which is equal to our first Vector A1 and first we need to

normalize this vector and how we can normalize this Vector well we need to take this vector and we need to divide it to its length so the grme process

step by step will look like like something like this so in the first step what we need to do when starting with these vectors of A1 A2 up to a n so in RN we need to First Take the first

vector and we need to normalize it and how we can normalize the vector and ensure that its length is equal to this length of P1 well we need to take that

vector and we need to divide it to this length because when we take the vector the 1 and we divide it to its

length of V1 then we will ensure that the length of that Vector is equal to one we can actually prove that very

easily but I won't do it in here uh feel free to go through the process assuming that the length of the vector what what you want

to achieve at the end is that the length of a vector v is equal to one this is something that we want to achieve and this normalization process

can be done if we find a way to ensure that we uh get this E1 because E1 means that we

end up with this Vector y 000000 0 this will be for first Vector so V1 this is E1 so the one is really

important here so we want to normalize this Vector View 1 by uh ensuring that

we get the E1 so we go from V1 to E1 and the way we do that is that we take the V1 and we divide it to the length of

V1 and in this way we get the E1 so the normalized version of P1 is E1 so then for each subsequent Vector a

k which means A2 A3 A4 up to a n we need to subtract its projection on all the previously computed orthogonal

vectors in this way by using this tab two we are ensuring that all these different each pair wise set of A1 A2

and then A2 A3 Etc they are all orthogonal to each other and we know that this projection is something that we got when we had

this two perpendicular lines so we had this Vector we're projecting onto this Vector we got that by finding this perpendicular line and making use of

that using this property we are then making use of that in order to see how we can ensure that the subsequent Vector that we have is always perpendicular to

this one so let me actually write down what is in this

formula so here VK is equal to a minus the sum of all the projections so then we need to normalize the VK to get the

EK and then we need to repeat this step two and three for all vectors which means that first here we apply this

normalization on the vector A1 so V1 is to A1 and then we get the normalization

by getting this E1 so E1 is normalized version and then we need to apply a bit

different tactique for our V2 V3 up to VN and then let me actually write down

this for this General case so what this processed this the GR let me ensure that I'm not making a typo

Schmid process step by step means step number one for

vectors A1 A2 A3 dot dot dot a n so we are in the RN then step number one basically says take

the V1 and set it equal to this first element V1 this is

A1 then what we need to do is to normalize it to get the E1 so normalize

normalized V1 to get E1 which is equal to 1 0 0 0

and then dot dot dot zero and the size of this n by one and how we can do

that by taking this Vector V1 and divided it to the length of V1 which basically means in this specific case A1

divided to the length of A1 this will then give us our A1 this Vector this is basically what

the step one entails then in the step number two we have for each subsequent a where K is just an index

referring to whether we are dealing with K is equal to 2 so uh A2 A3 and then dot dot dot a

n this is what basically the K is used for to refer to which Vector we are dealing

with we need to subtract its projection on all previously computed orthogonal

vectors by using this formula so let's actually do a couple of those case to see what is going on for instance for K is equal to

2 so K is equal to 2 and here is the formula by the way so

VK VK is equal to AK minus some K is equal to K starts with

one and then let me use a different

index so I is = to 1 till K minus one and then projection

of a k a k on E1 or eii so the EI that we have just computed

because every time you are then normalizing and normalizing every time your vectors and then you are uh finding out what is the projection of your

vector a onto that EI and then you are substract in that from your vector so what this means in

Practical terms when for instance your K

is equal to 2 it means that V 2 is equal

to a 2 minus sum of I is = 1 and then K is = 2 K - 1 this means this is

one projection of a a and then 2 on

A1 given that this is one this is simply equal to A2 minus

projection of A1 that's normalized version of E1 and

then A2 so projection of A2 on E1 and then in the step number three we need to do we need to go from VK to get EK so basically we are ensuring with the

step number two the orthogonally uh orthogonality condition and with step number three I

me add some space in here so in the step number three step number three we then saying let's

normalize normalized this VK that we have just computed in here because we remember that the second

criteria after tonality is normalization that the unit or the length of the vector should be equal to

one so then VK in this case for K is equal to 2 for K is equal to 2 means that we need to go from V2 to E2

and the way we can do that is by taking the V2 by V2 and then divide it to the

length of V2 this will then give us the E2 this is the normalization part and the step number four basically

means repeat repat step two

entry for all case which means that if we go back so we are done with V2 so we have obtained

V2 and then we have obtained normaliz normalized version of V2 by getting this E2 we are ready to come back and do the

same for K is equal to three and for K is equal to three in Step number two we got

V3 is equal to A3 minus making use of this

formula sum overall I is = to 1 K - 1 is = to 2 and then

projection of this time a Tre see three k is equal to three and then on E2

actually it says EI let me remove this this otherwise we would have made a mistake this should be I because an I will change per K this is the entire

idea we need to re um subtract all the um projections what this basically means is that we need to take A3 and this time given that here we have two instead of

one in here we need you have an extra step which means A3 minus and then what this formula

basically says this is the sum of the projections of A3 on e i where I goes from one till two so

projection of a Tre on a one when K so when I this is the I is

equal to one case plus projection of a Tre on a 2 this is the I

equal to 2 case this is basically what this summation says this is this element and we have seen this as part of the high

school but also the pre-algebra course okay so now when we are clear on how we can calculate the V3 in the step number two for K is equal to 3 we are

ready to go onto the step number three and what was step number three the step number three

for K is equal to 3 was saying let's take the V3 and normalize it to go from V3 to E3 and how we can do that by

taking the V3 and dividing it to the length of V tree to get on to E

tree and this cycle goes on and on until we cover all the case so all the vectors

so the idea is that we first for our initial step we set the V1 equal to

A1 we normalize it then starting from the K is equal to two we don't go first

on and on WE autog it by formula in here by using this we can ensure that each of these vectors is then orthogonal to all

the other vectors so for K is equal to 2 we ensure that this uh Vector that we get is orthogonal to all the other ones and the case equal to treat that the

third Vector isogonal to all the other ones and we are doing that in Step number two so for each case for each K we

basically are ensuring that in this case we have an a vector that is orthogonal to all the other vectors in this set and for each Vector we are also

normalizing it to satisfy the second criteria because we had these two criterias to create this auton normal set so we are doing this in subsequent

uh way so first for K isal to 1 so basically for A1 and then we are doing this for K is equal to 2 so A2 and then

until K is equal to n so a n what we are doing every time is that we are obtaining this V1 and then we go from V1

to E1 to normalize it and then here we are getting the V2 here to go to E2 by normalizing it so this basically

the step two and step three and then we do this every time up until to the point of obtaining VN and then from VN we go

to en n to normalize it so this is the idea of this entire process step by step to start with V1 as

part of the step number one and then as part of Step number two for each subsequent Vector a k so K is equal to two obtain the VK and then normalize it

for K is equal to 3 obtain the V3 and then normalize it to get E3 up to the point of the last Vector which is a n

the vector a n we compute the VN and then we normalize it to get the and this is what this part is which

is the St number two that says repeat steps 2 and three for all vectors it means that every time when you increase your K when you go to the next Vector we

first compute the V so VK and then you normalize it you get the EK and then you go back to the step number two at three because you then again need to calculate

the VK and then EK and then for the next case so this is something that you will see also a lot when you are writing the code for

your uh algorithms because in many cases you need to do this reputation of the steps so uh you for one Vector you do something or for one iteration you do

process and then you uh go back and do for the next one and for the next one this process is what we are referring by repeat step number two and three for all

vectors so let's now look into an example let's apply this grumme process to vectors A1 and A2 where A1 is 1 1 0

and A2 is 101 so let's go ahead and do that so A1 is equal to 1 1

0 A2 is equal to 1 0 1 we want to apply this gret process to

create this Aon normal basis for the Subspace that is Pinn by A1 and A2 so now we have this set 1 1

0 and one 1 and what we want is to create an auton noral basis so

creating creating or to normal normal basis

which CR meet process so here we got only two vectors so obviously it's this and it's a very

simplified version of it what was the first step in our case uh in our algorithm it was to set the V1 equal to

A1 what we need to do step number one we need to set the V1 equal to A1 and we need to

normalize normalize the V1 to get E1 that's what our goal is so

let's go ahead and do that V1 is equal to A1 and is equal

to 1 1 0er that's our A1 so 1 1 Z and in order to normalize V1 and get the

E1 we know that this is equal to V1 / to the length of

V1 which is then equal to take the V1 so that is 1 1 0 and then divide it to the

length of V1 and you can very quickly see that given

V1 is equal to V1 * V1 that's something that we learned in the very beginning of our fundamentals to linear algebra course that the length of V1 is simply the dotproduct between uh V1 and V1 and

it's equal to 1 1 0 * 1 1 0

which is equal to 1 + 1 so two so this is then equal to 1 1 0 / 2 which is

equal to 1 / 2 1 / to 2 and then 0 this is our E1 so we are done with our step number one because now we have

V1 and we got E1 so what was the step number two in the step number

two we need to set the k equal to 2 this is the next K so for A2 what we need to do is we want to

get V2 and normalize V2 by getting E2 and how can

we get that well first let's find what is the V2 well V2 was and using that formula that

we saw before which was this formula so it's equal to a k minus and the sum I is equal to 1 K minus one and then

projection of a onto EI I so let's take this formula

over this is equal to a 2 because K is equal to 2 a a k minus Su and then I is

equal to one till K -1 and then K -1 which is equal to basically 1 given that K is equal to

2 and then projection of A2 onto e i and this is equal to A2 minus given that we got k - Y is

equal to 1 so the limit for our summation is equal to 1 so this one this means that like

before we got just one part as part of our summation so minus and then

projection of let me actually keep the same color I want it to be consistent so projection of

A2 on d e 1 so you see here the i i is equal to 1 and the limit of the I is K minus one

which is equal to 1 so we got here just E1 so we got the V2 formula we can then now calculate it because we know

A2 and the A2 is this so one one 1 0 1 but now we got a problem we don't

know what this is so let's quickly go and calculate this part so

projection of A2 on a

1 and we learned from the projection formula that this is equal to H 2 * E1 /

2 E1 * E1 so the dot product multip by E1 and what is this this is equal

to 1 1 multiplied by and what is the E1 E1 we just calculate in here so it is 1

/ to 2 1 / to 2 and then zero here / two and then 1 / 2 1 / 2 and then 0 0

multiplied by 1 / 2 1 / to 2 and then Z here multiplied by the same Vector so

E1 so this two cancel out this two also cancel out and as you can see we are getting that

the projection of A2 on E1 is equal to this vector we can also manually check that actually so let's let's do that so let's see we

are not canceling out these vectors and instead we are manually calculating

this so here we got 101 ultip by 0.5 and 0.50 this is equal to 1 *

1 / 2 is 1 / 2 0 * 1 / 2 is 0 1 * 0 is 1 so + 1 / 2 this amount is

1/4 + 1/4 this multiplied by the vector 1 / 2

1 / 2 and then 0 in here this is equal

to 1 / 2 1 + 1 / 2 is = to 3 / to 2 and

then 1 1/4 + 1/4 is equal to 1 / 2 2 multiplied by 1 2 and then 1 2 and then

zero what is this amount well those two cancel out so we end up with three * and

then 1 / 2 2 and then 1 / 2 and then zero this is then the projection 3 / 2 3

/ to two and then zero so let me remove all these calculations and then we can take over the projection

value which is 3 / to 2 3 / to 2 and then zero to

get our Vector

V2 which is equal to 1 - 3 / to 2 0 - 3 / to 2 and then 1 - 0 and this is equal

to here it is 1 here it is - 3 / 2 and here is minus and then 1 / 2

2 because 3 / 2 is minus uh it is 1.5 and then 1 - 1.5 is simply minus 0.5 so this is then the vector

V2 then what we need to do is to normalize this Vector to get D

E2 which is then equal to V2 ided to V2 length which is simply equal to

V2 / to V2 * V2 so the dot product and this is equal to let's take the V2 which

is -1 / 2 and then - 3 / 2 and then 1 and then divided 2 and this amount let's

quickly calculate that it is equal to so the length of V2 is equal

to - 1 / 2^

2 + - 3 / 2^ 2 + 1 this is equal to 1/4

+ 9/ to 4 + 1 which is 4 / to 4 and then this is equal to 1 + 9 is 10 10 + 4 is

14 so 14 / 2 4 this is the length of it so 14 / 2 4 so then this is equal to this Vector

to this threedimensional Vector min-1 * 14 - 1 / 2 * 14 / 4 is

equal to this is 7 so minus 7 / 2 4 and then - 3 /

2 think I just made a mistake here actually so Min - 1 / to 2 so the first

element and then ided 24 / 4 is actually actually equal to this multipli by four ided to 14 so you take this element then

divide it to this one and we know that a / to B * C / 2 D is equal to a *

D and then B * C so we are basically flipping this side this is from

pre-algebra and then here this is equal to 2 and then is equal to -1 /

27 then let's do the second one twoo so we got minus 3 / to 2 / to 14 /

to 4 is actually equal to - 3 / to 2 * 4 / to 14 and then if we remove this this is

then 2 this is 7 this cancel out this equal to - 3 / to 7 - 3 / to 7 and then

finally we got 1 / 2 14 / 2 4 which is equal to 4 / 2 14 this

equal to 2 / to 7 so 2 / to 7 and this is our A2 and given that we got just two vectors so we have already reached the

end of our solution so now when we have already the V1 and the V2 the E1 and the E2 We have basically completed the

process of this grummet uh procedure because we have already uh only two vectors that means that we need to have

V1 and V2 and then uh E1 and E2 and this is all that you need in case you got two vectors if you have three vectors of course the process will include um the

same process of Step number two and three so the V2 and the normalization of it two times for your K is equal to 2 and K is equal to 3 and then if you have

more vectors than every time you will have more of the steps but at the end what we want to have is the set of vectors that are orthogonal and at the

same time they are normalized in this case we say that this vectors form this oron normal bases now why is this important the applications of

orthonormal bases well firstly it simplifies a complex Vector operations and uh this is the basis of

many uh more difficult mathematical Concepts uh like foror series or quantum mechanics it's used also um when when it

comes to this auton normal basis uh also signal processing and it's a critical uh process in numerical methods especially in machine learning algorithms and in

data compression so we will see this process to be used also as part of uh decomposition techniques which is really important when it comes to different

algorithms uh whether it's optimization algorithms but also um algorithms that are used for recommender systems for example and those uh Concepts they all

come together and we will see later on when we will be discussing the concepts of the compositions and metrics factorization so this Aon normal basis

and this grum process they are really foundationally in linear algebra they provide tools for simplifying and also solving these higher dimensional problems efficiently their application

include different fields of science engineering demonstrating their versatility and utility let's now talk about the special matrices and their properties so we are

going to talk about special matrices like symmetric matrices and their example diagonal matrices and their corresponding example but also the ortogonal matrices with the corresponding

example so when it comes to the special matrices special matrices have unique properties such as being symmetric or all nonzero elements on the diagonal

like diagonal matrices or orthogonality uh in matrices which means that we have orthogonal matrices so when it comes to the

symmetric Matrix it means that uh the a The Matrix a is equal to its transpose the a so a is equal to a in this case we

can confirm and say that the Matrix a is symmetric so in this case we have Matrix a and we know that the way we need to to

transpose this Matrix is to taking this rows and making them The Columns of our transpose Matrix so a is then equal

to 2 - 1 and then zero then the second row which is min -1 and then 2 and then Min -1 and then the third row which is 0

- one and two so the third row then becomes my third column so as you can see those two are the same so I'm using then the definition of the transpose of

the um Matrix and then here then we end up with two matrices they are actually the same so we can see that the A and

the a in both the First Column they got 2 minus one and then zero the second column minus one 2 and minus one the third column 0 - 1 and two so their

columns and their rows they are the same which means that we are dealing with a symmetric Matrix so whenever we want to check whether the Matrix is symmetric we just need to take the transpose of it

and see where the the Matrix is equal to its transpose in that case we are dealing with symmetric Matrix do also note therefore for uh

Matrix to be symmetric it needs to be a square Matrix so it needs to be 2x two into two dimensional space or 3 by3 in the three dimensional space or n by N in

N dimensional space which means that the number of rows should be equal to number of columns because otherwise when you flip

your number of rows with number of columns on in case there is no um uh Square version of that Matrix so m is

not equal to n in that case a will have a dimension of M by n and then then a t

so a t will have a dimension of n by m which means that there is no way

that a can be equal to a this is not then possible therefore we need to have a square Matrix for them to be

symmetric let's now talk about diagonal matrix so a diagonal Matrix has a nonzero element only on its diagonal

which means that in this case we have this nonzero elements on the diagonal so let's call it

d11 d22 and then d33 this equal to three this equal to 5 this equal to 7 and all the other elements as you can see in here they are

zeros so the concept of diagonal matrices is very uh simple therefore we will then go through the next example which is about orthogonal Matrix now

this is a concept that we haven't yet seen and we spoken about so let's Cod read through this bit slowly so an orthogonal Matrix is a square Matrix

whose columns and rows are orthogonal unit vectors so oron normal vectors and its transpose equals its

inverse so there are two two part of this elements so firstly it says that for the Matrix to be octogonal

Matrix it should be a square Matrix so Square Matrix and then its

columns and rows are orthogonal unit vectors so columns and rows

are orthogonal unit vectors which means they need to be normalized so

normalized so um we have seen when forming this orthonormal basis that we had this process of uh this condition of

orthogonality the vectors had to be AAL and they had to have a length of one which means that they they had to be normal I we can see exactly the same in

here so hence the name oron normal vectors so they are utal and they are unit vectors which means they are

normalized so then the final condition is added in here which actually is not so much a condition but rather than a property something that we can prove

that once we have all this we can also say that if we are dealing with toal Matrix then

Q T * Q so the dotproduct of the

transpose with that Matrix Q is equal to

the Q * QT is equal to I why because the QT is equal to the Q minus one because

the transpose of that Matrix Q is actually equal to its inverse and given that we learned that the Q minus one so the inverse time Q is

= to Q inverse * Q is equal to I and given that here we are learning

that Q T is = to Q minus one we are then making use of this to claim

this so instead of minus On's that we are used to when we are dealing with inverses here we have t the

transpose so in this case this orthogonal Matrix that we have just learned about this is the square Matrix whose columns and rows are orthogonal and they are also normalized meaning

that we are dealing with QT QT sh Q minus one it will look like this so this q1 you can see that here we

got the first row here we got the second row and if we calculate the dot product between this row and

this row we can quickly see that we are getting a value of zero so we can prove that they are actually autogo those two rows let's go ahead and actually prove

that so let's call this R1 let's call this R2 this is Row one and row two and I will leave the uh column version so q1

* Q2 that's do product on you to prove that the columns are perpendicular I will work with the rows so R1 * R2 for

me to prove that they are orthogonal I need to prove that this equal to

zero can we do that well let's try so 1 / 2 < of 2 1 / 2 of 2 ultied by 1 / 2 <

of 2 needs some bigger space in here so 1 / 2 of 2 and then

minus 1 / 2 of 2 that's how I can calculate the dot product between R1 and R2 R2 and

R1 you can see that the elements in here are the same and here the elements are also the

same so then this is equal to 1 / 2 < of 2 * 1 / 2 of 2 - 1 / to of

2 * 1 / 2 of 2 I'm simply taking this minus and given that the dotproduct is basically Plus and then this amount I'm just taking

this and bringing up in here to a avoid one more step uh given the space is quite limited now what do we see in

here this value is the same as this value which means that this is equal to zero and we know that the two vectors to

be or token they need to have a DOT product equal to zero so here we have proven that dotproduct of R1 and R2 is equal to

zero so this proves that R1 and R2 so the two rows of this Matrix so R1 and

R2 are orogo we can also prove that the second criteria of auton normal vectors is also satisfied in here we can prove that when

we look at the length of this vector and of this one then they are of the unit one so let's

actually go ahead and do for one of them so let's prove that

for 1 / 2 of 2 and then 1 / 2 Ro of 2 this is a vector that the length of it this is let's say

our first row so this is R1 then the R1 length is equal

to 1 / 2 of 2 2 + 1 / 2 of 2 2 this is

equal to 1 / 2 + 1 / 2 and what is 1 / 2 + 1 / 2 it's equal to 1 so we have

proven that the length of R1 is equal to 1 you can quickly and easily also compute that for the second row and you will then also prove that the R2 the

length of it is also one which is then the second criter area which said that for the vectors to form this auton normal bases so to be auton normal

vectors they uh also had to have a length of one so they had to be a unit vectors in this case then we can make use of the property that the

Q 2 transpose is equal to Q inverse and this then result in Q2

transpose * Q2 Q2 which is equal to Q2 * Q2 transpose which is equal to the identity Matrix and specifically I2 because we are in the

R2 so both this Q2 and the previous example those are orthogonal matrices and in here we have proven that

the rows are indeed oroginal and we have also seen that the length of them are unit vectors meaning that we have automatically got

this I will leave this one to you to do those proofs so to uh ensure that the row one and row two are orthogonal so they are perpendicular which means that

the product of their uh the dot product of these two vectors is equal to zero and also that they are normalized which means the length of them is equal to one

and this means that then this holds you can actually even go ahead and uh practice the material that we are uh

learned as part of the previous units by calculating the inverse of this Matrix and checking that the inverse of this Matrix is indeed equal to the transpose

of the Matrix so that QT is equal to Q minus 1 because we learned how we can compute the inverse of a matrix because

the inverse of a matrix was equal to 1 / the determinant of this Matrix times and then the manipulated version

of it which was in this case 0 0 and then we need to have here 1 so -1 * 1 and then 1 * - one so we have to

multiply this and this by minus so one and then here minus one so in this way you can also prove that this inverse is actually equal to

the Q to transpose because then you can prove that indeed and you can see for yourself

that this formula is in equal to the Q2 transpose because then you can prove that indeed and you can see for yourself

that this formula is indeed true in this module we are going to talk about Matrix factorization we are going to discuss the significance of Matrix factorization we are going to Define Matrix

factorization we are going also to discuss the common applications of Matrix factorization across different fields and then we are going to see detailed examples of metrix

factorization so let's talk about why Matrix factorization matters so metrix factorization techniques they are essential for various reasons they are used for simplifying metrix operations

like solving linear systems or when we have this um many uh matrices but we want to to um simplify these operations that we apply to these matrices and we

want to solve the problem then we can make this uh complex Matrix operations more manageable and make these uh calculations more manageable by using

Matrix factorization techniques we can also use Matrix factorization directly to solve system all linear equations efficiently we can also use Matrix factorization to perform igon value de

composition singular value de de composition or called SVD and other operations which are crucial in machine learning and data analysis so ion values

and igon vectors you might have heard already they are part of also PCA which is the principal component analysis and this comes from uh fundamentals of

statistics and the uh PCA is used as a dimensionality technique and in fact it's one of the most popular damage s techniques that you will find in the

industry used in the data science using data analytics machine learning even in the Deep learning so Matrix factorization can also be used to reduce

the computational complexity by making use of this factorization we can then simplify the process and also make it more efficient for computation and it's especially important when we are dealing

with this High dimensional data when we have many features or we have a very large model and complex model then this uh meing factorization technique can

make a huge difference in our data processing process so this techniques underpin many algorithms in numeric

analysis in optimizations and Beyond so whenever it comes to machine learning or data science or many other fields you will see this uh process and this term

metrix authorization appearing a lot even um in the example of a streaming company Netflix which I'm sure that you are aware of netrix is using uh metrix

uh factorization to build a recommender system and uh metrix authorization usage in building recommender algorithm for personalized recommendations is actually

one of the most popular applications of metric factorization therefore I wanted to specifically discuss this topic as part of our Advanced linear algebra course and some of the concepts might

seem bit more complex than the ones that we have discussed as part of the previous units but once we go through them step by step and I will give you all the details in all these examples

this entire process of this different metrix factorization techniques should become much more clear and straightforward so we will be discussing not just one but multiple fundamental

metrix factorization techniques beside of talking the high level where they are used and how you can choose for what type of applications so we are going to

demes this entire concept of metrix factorization and we are going to uh start from high level then we are going to go into the deepest details let's now

formally Define the metrix factorization so metrix factorization refers to decomposing a matrix into product of two

or more matrices revealing its structure and simplifying further analysis so what is this idea behind metric factorization the idea is that if

we have a matrix a and we want to simplify our process of calculation or multiplication anything

that's related to this a but this a in itself it contains this weird numbers or it is just too complex you know it contains this ton of different numbers you don't recognize whether the columns

are linearly independent it's not very readable from the first View and you just want to make your life easier when performing this calculations well for that you you can make use of this Matrix

factorization to write this a in terms of some other matrices let's say um and I'm calling here randomly Q or t so it's

equal to for instance the dotproduct of these two matrices Q * T where Q is much simpler and the t is also much simpler

so those may contain vectors that are um for instance this can be a diagonal matrix or it can be a matrix uh with specific properties when using those you

will feel much more comfortable so it will be easier for you to use them in order to multiply uh with other matri matrices it can be easier for you to solve this problem but of course if you

are in the two-dimensional space let's say you are in R2 or in R3 then most likely it will be quite straightforward for you to use the a itself but if you

are in the r 100 or R 1,000 then of course this uh entire computations they become super complex it will be

difficult to understand and compute this linear combinations find out whether you are dealing with a linearly independent

columns find out um the um new space the calm space the basis of the new space and calm space and all these they might seem uh much more difficult when you are

in high dimensional space for in those cases we can then make use of metric factorization to make the entire process much more

simplified and also more efficient this entire calculation process so common types of Matrix factorization include lower upper uh Matrix factorization or

in short L QR factorization an Infamous type of factorization which is called orthogonal triangular

factorization and then we have SVD singular value de compos ition yet another in famous metrix factorization and then finally the igon

de composition also another Super popular metrix factorization technique so uh the QR SVD and igod

composition are in fact highly popular the composition and Metric factorization techniques that you will see appearing

in the 90% of all the statistics related and machine learning related ated books so this just comes to prove how important these concepts are when it

comes to properly learning and mastering these more applied uh science related fields like machine learn so if you want to go beyond the level of knowing

algorithms but rather than to also be able to edit the algorithms tweak them adjust them be able to understand machine learning algorithms deep

learning algorithms data science and at its core and in order to become a professional well-rounded professional then this I composition the singular

valid composition and the QR metrix authorization techniques are techniques that you want to know and you want to understand at least higher level such

that you can easier grasp more advanced concepts that come from the applied sciences like machine learning and AI so let's first discuss at high level

what this C q r DEC composition is so what the QR DEC composition does is that it decomposes a matrix into an orthogonal Matrix which we are referring

by q and then an upper triangular Matrix R so in here you can see that we have this two different matrices so we are

basically saying a is equal to this product of this Matrix q and r R where

the first one this Matrix Q this one should be orthogonal Matrix so this part is really important and we

have learned as part of the previous module the definition of orthogonal Matrix we learned that the rows or columns they had to be orthogonal to

each other and we also learned that they need to have a length of one they need to um be normalized and we learned that this means that the

transpose of those matrices is equal to the inverse of the matrices so this was just the last part of the previous module and this is exactly what this

Matrix Q is about so we are saying that we will decompose I into this two matrices as a product of these two matrices Q andr one of which this Matrix

Q should uh be uh an autal Matrix which means the rows and the columns they should be or toal to each other so their uh dot product each of them should be

equal to zero and they need to be normalized so the length of them should be one for each of those rows and vectors and then the second part of this

U the composition is this Matrix R which says that the Matrix R should be an upper triangular Matrix and what is the definition of upper

triangular well in this case you can think of this r as this Matrix where we have here all zeros and then here on the diagonal you

have numbers nonzero numbers and then here let's say 1 2 3 4 5 and then here in the upper part you will also have numbers so unlike in the lower part of

this Matrix R where you will have zeros in here you will have also noner numbers numbers let's say seven uh 10 uh

8 and I'm just writing these numbers randomly so of course in a real case when we have this Matrix a and we go through this process of QR de composition of course we will have an

appropriate q and appropriate R where these numbers will be different and they will be specific numbers that we will be calculating but the idea is that we need

to get this upper triangular Matrix R for this calculation to make sense so we will then be using this qard

composition for solving linear um linear Le squares problems for instance which is part of the linear regression too

because linear regression from machine learning and from statistics uh it is based on the least Square technique the estimation technique that we are using

for linear regression in machine learning um to solve this linear regression problem is called Ordinary leas squares so the algorithm is based on this idea of Le squares which is

trying to minimize a squared uh residuales of the morel and that can be done by using this idea of QR decomposition so it helps us to provide

numerically stable solutions for this type of problems too and C de composition is used extensively in Signal processing and statistical

analysis let's now briefly talk about the L decomposition so L decomposition decomposes a matrix into lower triangular Matrix so this is the

opposite of what we had before we can have an upper triangular triangular

Matrix like we had in the QR the composition we can also have a lower triangular Matrix so you might have already guessed how it will look like I

want go into that very soon in the QR composition example you will see the idea of the upper triangular I will also show the idea of a lower

triang so the composition in case of Lu uh is done by decomposing a matrix into lower triangular Matrix l and an upper

triangular Matrix U so basically the difference between the QR de composition and L de composition is that in the QR DEC composition we are decomposing a

matrix into orthogonal Matrix and an upper triangular Matrix while in case of L DEC composition we are decomposing a matrix into lower triangular Matrix and

an upper triangular Matrix so here you can see that we no longer have this idea of orthogonal matrix but instead of that we are talking about lower triangle

Matrix so in that aspect uh L U the composition is different from QR DEC composition so what the L de composition does is that it facilitates the solving

of linear equations and Matrix inversions it is common in engineering and in physical sciences for systems of this linear equation to be solved by

using lud de composition and in fact if you are learning Quantum uh mechanics that this L decomposition can definitely help you to better understand many

Concepts but if your target fields are machine learning deep learning or artificial intelligence then for those

using QR composition will be uh much more often a case than using this lud de composition let's now talk about the

singular value decomposition so what the SVD does is that it decomposes a matrix into three matrices so first one is the

orthogonal Matrix C the second one is a diagonal matrix and then the third one is this V which is the conjugate transpose of an

orthogonal Matrix so for now this might s bit complex and you can see that unlike the QR or Lu de composition where we got

just uh two uh matrices as a result of our decomposition in case of SD we got three matrices like the name suggests

two by the way so three parts and this might seem bit complex but we are going to go through this process step by step and I'm going to

provide you detailed example such that this will all make sense but for now let's focus at the high level usage of SVD so singular value decomposition is

one of the most popular decomposition techniques and it is also Direct Al used as part of machine learning algorithms to form uh those machine learning

algorithms it is also used in the data compression in the noise reduction so when we are trying to clean our data and remove the Noise by using SVD because

SVD can help us to identify those outliers and then remove them from the data by performing noise reduction and it is also used in the

principal component analysis the PCA the uh same dimensionality reduction technique that I just uh mentioned related to the ion de composition

because SVD and the igon de composition are highly related to each other so this SVD is used as part of this PCA algorithm and PCA is the most popular

the infamous dimensionality reduction technique that is used both in the advanced statistical studies in the statistics in general also in finance

and is also used as part of many machine learning and deep learning applications so knowing PCA is a must if you want to get into uh data analytics or data

science machine learning and AI but also uh it helped you it will help you also to uh understand uh many other

Concepts when it comes to this Fields so the SVD provides insight into the structure but also the rank of the Matrix so we are going to see this as

part of our example too let's now also briefly talk about the igen dec composition so igen DEC composition which is highly related to this concept of igen values and ion

vectors it decomposes a matrix into this ion values and ion vectors which then shows the matrix's fundamental properties which are related to this

idea of correlation what kind of information does this uh Matrix contain what is the variation in what direction is the variation the largest and this

icon that composite which is then related to also this idea of SVD and in general this dimensions and correlations is critical for understanding linear

Transformations the stability analysis and systems of differential equations but beside this uh mathematical side of uh Concepts and

understanding these mathematical topics the ion de composition is also the basis for many algorithms in numerical deor algebra

uh but also many applied linear algebra topics like in the data science in machine learning and it's used heavily in artificial intelligence for feature

extraction for dimensionality reduction related again to the concept of PCA because PCA is based entirely on this

concept of IG de composition PCA is the direct result of computing the igon values and igon vectors

so without knowing what are Dion values and ion vectors you cannot perform PCA because the first step of the PCA is the computation of the icon values and icon

vectors and then using different rules which we are referring uh as the elbow rule or Kaiser rule we can then use this icon values and icon vectors to

understand what are the features in our data that contain the most variation so the most information and then we can use that in order to

understand what are the most important features in our data and reduce the dimension of our model by selecting this most important features because what PCA

basically does is that it it uses these icon values and icon vectors to understand how we can uh create a linear combination out of our features and

understand the the amount of those linear combinations that contain the most variation and then select those and uh select the largest amount of

information in the data and then Skip and drop those uninformative being your combinations while still keeping the most information and this definition of

the most will then be decided by this differentials this is just higher level Insight background information on what you can expect when you are talking

about applying this highly technical linear algebra concept of Icon de composition into an applied science Fields like data science or machine

learning or AI but we will see this later and I'll also make comments regarding this and though PCA won't be discussed as part of this course because

here we are talking about linear algebra but PCA is part of the fundamentals statistics course and in there we are no longer providing all these different

details on how you can uh perform this I composition therefore knowing how to perform I composition will then set you

for success to actually understand the mathematics behind the statistical Concepts like PCA and also later on

understand how you can use that PCA in a machine learning Concepts and in AI Concepts like outo encoders and how you can relate your um out to encoders to

this concept of PCA how they are related what are their commonalities and what are their differences so everything is about the choice and choosing the right tool for

your problem when it comes to the decomposition tools metrix authorization tools we have seen that there are many options and the question is which one should we pick in what cases so choosing

the right tool is really important when it comes to this different metrix factorization techniques because they are many choices and each of them they can be used for different sorts of

problems so therefore in order to understand which one you need to pick in what kind of cases what kind of requirements you have and what kind of solve uh problem you are trying to solve

that in those cases you will need to have this knowledge that you will learn as part of this course in order to make that right choice of the

twool so the choice among Q are the composition the L de composition the SVD and IG de composition it really depends on your specific problems requirements and the

data characteristics so are you dealing with a complex data are you dealing with a simple data with low Dimensions what is the goal that you uh want to uh

achieve what is the problem that you are trying to solve is it to uh reduce the dimension of your feature space is it to

solve a problem with linear equations is it to solve a quantum mechanics problem or is it to um

incorporate this as part of your machine learning algorithm so QR and LU DEC compositions are usually preferred for solving linear systems while SVD and I

Anda compositions they help us for deeper insights when it comes to the data and what kind of information it contains how we can reduce the dimension of the data or how we can use it as part

of machine learning algorithm for uh noise reduction identifying outliers Etc so um this type of algorithms like SVD

and ion de composition it helps us to also uh intuitively using geometry and our knowledge of geometry to um visualize the data for instance the PCA

helps us to visualize this High dimensional data using just couple of principal components let's say we have 10 features in our

model so we have a dimension of 10 we are in r10 but we want to visualize our data by using PCA we can then reduce the dimension and come up with uh three

principal components which are a linear combination of our original 10 vectors and then we can use the three principal components to visualize our data in

3D and this basically helps us to geometrically visualize our data and then make presentation s make much more sense of our story so to do uh

storytelling for our data and uh much more and these two uh models and tools they are invaluable when it comes to uh

applications in machine learning in deep learning in data science and artificial intelligence so meing factorization techniques they are super important when it comes to

computational mathematics they are also directly affecting the data science Ai and many other algorithms so they are not only important in terms of the problem that they are trying to solve

but also in order to make the computation process so when coding in python or in other programming languages to make that process much more efficient

they also help us to uh make these computations efficient and provide insights into different properties that we have in our

data as part of this course we are not only going to discuss one but actually three of these four the composition techniques and this metrix factorization

techniques in detail we are going to talk about the qard de composition we are going to not just discuss it but uh also to learn it step by step and we are

going to do a detail example with all the steps involved such that you will feel confident doing a QR decomposition all by yourself then we are also going

to do an SVD de comp position as well as ion the composition and then we are again going to discuss them in terms of their mathematical formulation the

definition but also the application step-by-step process and a detailed example such that you can conduct each of those metric factorization techniques and these decomposition techniques by

yourself manually doing all these calculations this understanding and this examples and this Concepts will help you to not just be able to formulate what

these techniques are about but really and truly understand and then use them later on whether when doing your own research writing scientific papers or

tweaking the algorithm all by yourself when inventing new algorithms I won't be discussing this L de composition technique because we

already know uh that the QR and LU they are both used for similar type of problems therefore to save us time I have selected carefully the uh most

important the composition techniques and Metric factorization techniques that you will most likely be dealing with will be dealing with in your future career in applied sciences