PBA Bali 2025 Module 1 - Fundamentals | Scale
By Polkadot Blockchain Academy
Summary
Topics Covered
- Serialization Demystified
- Bytes Over Bits for Memory Access
- Little Endian Enables Free Casting
- Compact Encodes Variable Integers
- Scale Vectors Leverage Compact
Full Transcript
Um, well, first off, I want to introduce myself. Uh, although Nikos already did.
myself. Uh, although Nikos already did.
Um, my name is Joseph. I'm the team lead of the Papy team. Um,
I'll be teaching at the Dubstra with my colleagues, um, Victor and Carlo. Um but
I will also be teaching here uh for the fundamentals.
Uh yeah, I've been in the ecosystem since 2021 and I was at the point like I did the the
I I did the uh the PVA edition in Singapore. So you
know maybe in a couple of years some of you will be teaching as well. This is
just to say that we're always learning and at some point we start learning. We
get hooked into this, we learn more and uh and and sometimes we end up teaching ourselves. So, okay, what are we going
ourselves. So, okay, what are we going to be learning today? We're going to be the learning scale. Um first off, I want to know I want to get a sense of how
many of you know a little bit what this is about. So, does this term scales or
is about. So, does this term scales or skill encoding and decoding ring a bell to any of you? raise your hand.
Okay. So, a few of you already are more or less familiar with scale. Okay. Um
it's also very important that for the duration of this session, if at some point in time uh I'm going too fast, I'm explaining something in a way that it's
confusing or whatever, raise your hand.
We'll go over it. It's a lot more important that you learn than me just like going through the slides like nothing happened. Yeah. Okay.
nothing happened. Yeah. Okay.
So what is what is scale? We'll get into that in a moment. Before we go there, we're going to talk about serialization.
Serialization is the process of translating a data structure or an object in any programming language um into a format that it can be stored,
right? Like so if you use JavaScript
right? Like so if you use JavaScript like uh and you do JSON stringify that is serializing in a very naive way, but it's actually serializing, right?
serializing like whatever JavaScript object or or stuff that you have into a string and then you do JavaScript pars and then you des serialize it right um
it's that's a trivial way of serializing um there are different kinds of serialization right like there's self-describing and non-selfcribing the example that I gave you before about
like the JavaScript JSON that one is self is self-describing because um every like when you deciialize it. Well, when
you are when you you're serializing meta information, not only uh the actual values but also like the how these data
structures are are being uh are are being um presented. Right?
Then another concept that is important to uh to remember is codeex. Um we're
going to be talking about codex all the time in reality. A codec. The only thing that a codec is is the junction of two things of an encoder
and a decoder. An encoder is you take some sort of type you encode it into the kind of output that you want normally
binary data and a decoder takes that output and it makes into the input. So
um a codec is just a data structure if you if you want that consists in two functions one for encoding and one for decoding and if you think about the types the type of the input of the
encoder is the same as the type of the output of the decoder. Yeah. Okay.
the decoder. Yeah. Okay.
We're going to do a quick recap on binary data because it's important that we are all on the same page um on on these things. So, first off, what is a
these things. So, first off, what is a bite? Who can tell me what is a bite?
bite? Who can tell me what is a bite?
Who knows what what a bite is? I know
it's can seem very basic. Okay. So, what
is it? Don't worry. Don't worry. Just
just yell.
>> It's eight bits. Why? Why eight bits?
It's true. like he say it's 8 bits in 99.9% of of uh of computer architectures it is 8
bits why is it why why why eight bits of all numbers why why not six why not four why not three >> that's a good question I I actually
never thought about that uh probably out of convenience >> it's it's 8 bits as we as as we just
discussed it's because is the smallest addressable unit in memory of memory right so when we try to access uh a position in in memory in a computer like
again yeah uh in almost all architectures that's the smallest uh the smallest unit of memory so there is no way like sorry yes
>> uh that still doesn't answer like why it's eight >> oh no what I was trying to explain is that why it's eight because that's the
that's the that the architectures have been done for um because it start with with a bit but um yes but what I'm what
the the the point that I'm trying to convey here is that the reason why we work with byes and and the reason why this is like kind of the the
like not working directly with like uh just zeros and ones is because when we try to access the memory of of a computer it always gives us a chunk of
eight bits right which is a bite at minimum right so when we're manipulating data we're manipulating it in a bite not in something smaller than that yeah
um and something is going to be very useful is the exodimal representation right so I guess everyone is more or less familiar with exodimal values and
excess representation we're going to just uh go over it again um excessimal representation is uh B 16 number
representation as you know we generally speaking uh we normally use base 10 numbers right there only one zero uh types of people the ones that understand
binary and the ones that doesn't do you get that joke so what I mean here is that there are many different bases right like we use 10 probably because we have 10 fingers
like if we had had 12 fingers we'll have been using base 12 which is Bish superior actually but that's not the point. The point is that B16 is very
point. The point is that B16 is very useful for dealing with binary data precisely because the properties that it has which we will see in a moment. So B
16 is we are using 16 symbols from zero to f right. Um
it's worth noting that the whenever we see zero x that's a very common um notation that expresses that what comes
after is expressed in b 16. Okay. So the
reason why B1 16 is very useful for dealing with binary data um especially with bytes is because um
each symbol accounts for what is called a nibble which is four bits right that way um whenever we are trying to
get like this is a bite right and these are like the inside the bite we can split it in two parts like one nibble which is four bits and the other nibble which are the other
four bits. So when we see 6 A we know
four bits. So when we see 6 A we know that there is a bite right in which
the fir like the first four bits right are a six and the other ones are a 10 right like this is as you can see this is two and this is eight so all together
do 10 and in excessimal that's an eight right like um so it excessimal is a is a very useful notation
for visualizing how what are the values inside inside this device.
Did is that clear enough? Like do you understand why why we use very often uh when we're dealing with binary data uh accessible? We could be using B 64 which
accessible? We could be using B 64 which makes things more compressed but then we wouldn't be able to distinguish in the first character whether that's one bite
or two, right? Um okay. So now that we have done a recap of like the most basic basic stuff um let's go directly into
scale. Scale stands for simple
scale. Scale stands for simple concatenated aggregate. It's the same
concatenated aggregate. It's the same just like they were trying to make it sound fancier because still it wouldn't
sound so good. Um and little Indian who knows what little Indian is.
Okay.
Um we we don't worry about it. We will I will explain it in depth in a moment. I
just wanted to get like a Okay. So
is designed for high performance copy free data encoding and decoding in resource constraint environments.
Meaning that whenever we have to whenever we have to pick uh
a a serial a method of serialization, we have to choose between different trade-offs, right? Um, we could have
trade-offs, right? Um, we could have something that is not so performant but really good for debugging or we could
have something that is highly performant but maybe not so that you know not so great for humans to
decode, right? So scale has picked the
decode, right? So scale has picked the path of being highly performance, right?
um and has also picked the path of being very uh succinct in in the amount of memory that it takes um and it works really well in in
resource constraint environments. Sorry,
there's a question there.
>> Uh Ben, could you expand on the phrase copy free data?
>> Yes, actually I'm going to do that just in a moment. I was going to do that now.
So um this has to do I will expand on that later when I explain the properties of little Indianness but just so that you
get an idea um what it means is that um it's designed so that if you want to cast um for instance you have uh U32 right I
will I'll expand later when we talk about little Indianness but basically using the same the same memory if you want to cast it into a U8, you don't need to recop it and reshuffle it.
That's one of the advantage of of of using redundiness, but we'll get deeper later. I just wanted you to know um that
later. I just wanted you to know um that these are like basically the targets, right? Like when scale was designed, it's like what we want to
avoid um having to uh use too much memory and and and and do and to do things um
as cheaply as possible when decoding this information, right? And also when uh zing it around. So scale is non non
self-describing, right? What means is
self-describing, right? What means is that when you get like scale binary data, unless you know what it is that you're going to decode, you won't be
able to decode it, right? Like it unless you know the definitions of what you're looking at, you won't be able to decode it. As opposed to the celia jumble that
it. As opposed to the celia jumble that I gave before where you have like um a stringified like JSON. Well, the the stringified JSON itself is self-describing. you have in there like
self-describing. you have in there like like the name of each fields and like whether this is an array or this is an object or this this is a number or a
boolean value. When when things are
boolean value. When when things are scale incputed, the the one that's going to decode it
needs to know what are what the definitions are otherwise go figure what it means. Okay.
it means. Okay.
Um little Indianness has some performing benefits like free casting. Now when we see little Indianness you'll see um and
this is related to copy free in coin. Um
we'll we're you'll see what I mean indeed. Okay. So little Indian
indeed. Okay. So little Indian um little Indian has to do with in which order we put the bytes, right?
So if you if you see these two different stacks, this represent like memory addresses, right? So here we have byte
addresses, right? So here we have byte zero, by one, by two, by three. Imagine
this. This will be a good This could be for instance U32, right? Like there or I32 or whatever. So imagine a U32.
Does everyone know what a U32 is? Is a
an unsigned 32 um number. So an unsigned 32 number is a is is is a number that takes four bytes. Right? Now the
question is how do we put these bytes? In which
order we put them? um the least significant the least significant um uh bike goes first or goes last. Well,
depending which order we we choose is going to be big Indian or little Indian.
Let's see some examples. Okay,
how do we encode the value 15? Like this
15, this is a decimal value like this is like 15. Okay. Um as a U32, right? Um
like 15. Okay. Um as a U32, right? Um
how do we do it? Imagine that we want to do it big Indian, right? So what would that be? Okay, there's someone wants to
that be? Okay, there's someone wants to >> Hi Martin. Uh so the big Indian will be like 0000005.
>> Yes.
>> Yeah.
>> So it will be zero. Exactly. So we see like the order of the byes of big Indian here we have 15. Yeah. And then we have all these are padded because they're not
being used. And then this one goes here,
being used. And then this one goes here, right? Like that's for for most people
right? Like that's for for most people that that write left to right. This kind
of makes sense, right? Like what about little Indian? What will be little
little Indian? What will be little Indian?
>> Go for So it will be zero f 000000.
>> Mhm. Yes. So it's zero f. One thing that I want you to note is that it's zero f.
I'm I'm I'm not as smart as as he is. So
when I when I first learned about Indianness, I assumed that it was going to be like a mirror. So I thought it was going to be f0.
But it's not f. Why? Because the bits inside the bite are left alone like Indianness only
applies to the bytes right the inside inside inside the bite the bits are just placed however like the the and that's because really the computer for the
computer like the bite is is is a black box just like a bunch of of of of zeros and ones right so just keep in mind that endness this is can be a little bit
confusing when we see like things as as as as binary data as pure binary data zeros and ones. Just keep in mind that endness applies to bytes not bits. Yeah.
Is that clear?
Does everyone understand how um how niness works? Oh, so since we are here
niness works? Oh, so since we are here just notice one thing I think they I I I'm going to explain that later but like I feel like explaining now. Imagine that
you want to cast this value.
You it's a U32.
Imagine that you want to cast it as a you know that it's expressed as a U32 but because it comes from a system that it's using U32s but you know that this
value is always going to be able to fit in one bite. So you want to cast it to a U8, right? Okay. So what do you do when
U8, right? Okay. So what do you do when you want to cast it as a U8? If you have a big Indian, you you have to basically copy all these
take this and and and and put in another place of memory to grab this. What
happens with little engine? You can use the same address of memory because when you cast it, you're going to take just the CF. So you don't need to recop it.
the CF. So you don't need to recop it.
So that is Oh, shoot.
Oh, sorry.
Do you understand what that means? This
is one of the little advantages of of using uh little Indian which is like uh casting is is kind of free. Oh shoot.
Okay.
I'm f.
All right. So yeah, what I was explaining is is that when we cast it, we can use the same address in memory.
Just it's going to take until here.
We'll need to recopy it here. If we try to cast it and we try to pass the same memory, the same memory accesses address. It won't that property doesn't
address. It won't that property doesn't apply. So you will need to um you will
apply. So you will need to um you will need to recopy, shuffle it, put it somewhere else and pass it. Uh there are questions. So let's answer the
questions. So let's answer the questions. Yeah.
questions. Yeah.
>> Uh SP is there a way to two questions.
Is there a way to convert either from big engine to little engine and little engine to big engine? And if it's just focused on byes, what happens in
situations where we're trying to account for storage of bits? Maybe the next thing is like there's a bit not complete by the case maybe is is are there
scenarios like that?
So hold on let's go let's step back for a moment like can we so basically can we convert something from little
Indian to to be Indian sure like whenever you decide to see like normally you're working with binary data like if you are working with type or whatever you're going to be using uh you intate
array with a data view whatever if you do ras like each language is going to give you different tools bas basically
to access uh uh buffers of of binary data and then express the binary data there however you want. Right? So those
are blobs of binary data. So you can definitely take some binary data that was encoded in in in little and translate to Vania.
Sure you can do that. It has a it has a cost but yeah you can like basically you can have uh in the same way that you
have a U32 uh encoder you can have one for lieutenant one for bean and yeah you can like basically you read it and then you put it yes and the other question was sorry
>> because because the the the example we're considering I don't it's unique to this example it says how to encode the 15 decimal value so I'm just I'm just looking at if there are scenarios where
we're not dealing with bytes then but we're dealing with bits and if >> I understand sorry >> so if are there scenarios where we're not dealing with with bytes itself but
we're just dealing with bits itself >> no no no no like the minimal the the minimal unit of measure for binary data as I explained before is the bite right
sometimes as we'll see later we do what it's called I don't want to confuse you but like sometimes like What you do is like inside the bite you do what it's called
bitbox boxing box that's how I call it I don't know what the proper term is but like inside the bite you express something more than than just a number and then you just look at the at the
bits and there is actually a really good case that that that's about to come that that is part of scale but for now I'm just explaining Indianness let's not get ahead of ourselves um there was another
question >> maybe I'm going a little too far but is the The reason that little Indian is more useful is that you can easily find
the extra zeros and wasted space or does using little Indian make it any more efficient and more compressed?
>> Go ahead.
>> So with big Indian with the six zeros up front I don't know if it's the compiler that has to identify it but >> so this these these six zeros these are like these are three bytes. Yeah.
>> Yeah.
>> There are three empty the three empty bytes here. Yes. And then this one.
bytes here. Yes. And then this one.
Yeah. Yeah.
>> Is it easier to identify the unused bytes by using little NDN first for the machine?
>> But you're using all the such thing as unused bytes. It's a it's a U32. Like you're using all the bytes.
a U32. Like you're using all the bytes.
It's just like the value contained inside is just 15. But you're using all the bytes. It's it's a it's a U32.
the bytes. It's it's a it's a U32.
>> I see. So it's easier to read the little Indian because the >> zero like little Indian is a little bit more difficult to read for people because like I gave you 15 but when you see a number like bigger that has like
other things here it's a little bit confusing while this one it's like you just keep going concatenating. Uh so for humans like little engine is a little
bit confusing but for computers for programs is more efficient basically because of the pre-casting that I explained. Oh Is there a way to
explained. Oh Is there a way to change the setting or >> Cool. Thank you.
>> Cool. Thank you.
>> Sorry. Um, sorry.
>> I want to see myself again. I want to go.
>> All right.
Did I answer your question? I'm not sure if I did.
>> Yeah.
>> Okay.
>> I can try to elaborate on the answer if I can.
>> No, no, no. Because there are questions like >> but but I don't have a question. I I
just want to elaborate why little Indian could be easier to read like uh in big Indian you need to read this first three
uh uh let's three bytes of zeros and then you have the 15 but if you expecting to to lead you 32 like you're expecting there to be 15 you can just
lead read one bite and skip the rest.
>> Uhhuh.
>> Yeah. So that's that's makes it easier to read because you don't need to read all of them to >> memory lang is easier to read >> like for the for the computers like >> all right then will there will be just access later that we'll see if that's
true >> okay there were other questions >> I was just wondering are there any benefits to using big Indian >> are there benefits of using big engine >> yeah just like maybe not in this case
but just in general >> everything is about trade-offs yes uh like I I so I'm sure like there must be a trade-off where like a big Indian is is
is is a good option. Ethereum uses a big Indian.
>> Um I think yeah I'm building on that question. Is it primarily because of
question. Is it primarily because of historical reasons that x86, ARM and all these architectures use little engine uh and it's only like Java or I don't know maybe
>> no I mean like the reality is that like uh another big reason why scale uses little engine is because in general by default uses little engine and and as
you know as you probably know uh polka dot until recently well even today it compiles to wasam later it will compile to PBM the PBM that you will compile it
also is is going to be plutonium based.
Uh but yeah like there was quite a bit of research and and when they developed was they realized that for the use case of wasam lithanium was a lot more
powerful. Yeah.
powerful. Yeah.
Uh but yeah like it's like like there are trade-offs right like again like uh in Ethereum I don't want to talk too much but like
the minimum word is 256 uh so it's like waste of memory non-stop but like there the bination is the same so there are trade-offs like um yeah
anyways I don't want to um yeah let's let's let's let's keep going okay so Now we'll look at the
scale basic primitives right like these are the simple the most easy easy simple types that you can imagine and and then we'll explain like the most complex ones
right so unsigned unsigned uh numbers right uh you sorry these are U8 to 256
I'm not sure if they are considering adding more than that but like for now uh let's say that scale supports from UA to U266.
There is a there's another primitives that allows us to have even largest numbers, but we'll get there later. Um
so for instance in these examples, this is how 20 is represented, right? Like
this is a U8.
Um it takes it takes one one bite. This
is a U16 takes two bytes. um as you can and here you can again see uh the free casting right that comes with it because it's the same just starting um two zeros
here uh and then 256 um this is a little bit confusing right because like normally you will see this right and and and you'll be like oh
that's one no that's 256 right because we 255 is FF but now when we add one or it's one in the in the in in the next
bite. So it becomes like this. That's
bite. So it becomes like this. That's
why I said that little onion it's a little bit difficult for humans to it's not difficult. It's unintuitive at
not difficult. It's unintuitive at first. Once you got used to it like you
first. Once you got used to it like you start you start seeing the patterns.
Does that make sense? Yeah. Okay. So
these are unsigned numbers. These are
sign numbers. Um please tell me that you are like it's not super important but how many people are familiar with like
do you understand why minus one is fff everyone understands to's complement is anyone here that doesn't understand to's complement
don't be shy raise your hand okay super quick um because I don't have a slide for is negative values
every uh negative values pretty much everywhere. Uh
everywhere. Uh they they are represented using what it's called.
My handwriting is beautiful to to complain which means that
if you have value it starts with zero like we are going to looking at the binary data right like so imagine
that we have one one bite and this bite inside has eight different flags, right?
So if we look at zero, it will be 0 0 0, right? Like these are different the the
right? Like these are different the the the the eight different beats, right?
Now if you want to represent five, right? So five is
five, right? So five is what?
one right zero one zero 0 0 0 yes like this is
this is positive values now what if we want to do minus five right so if we just use like there are different
ways of representing uh positive and negative values if one possible way is let's just use the the the bit at the at the at the most significant bit it's
called this one is the the one at the left um for expressing the sign right so if we did that
we could do this and that will be minus five right and that seems intuitive it's like okay this flag kind of let us know whether it's a positive value or a negative value the problem with this is
there are two big problems with with doing with using that meth method for the first one is that then we have two different zeros. We
have minus 0 and and positive 0. We
don't want to have minus 0 because 0 is zero, right? And the other big problem
zero, right? And the other big problem is that um when doing the x4 operation that does addition, this doesn't work.
So to kind of fix those problems there was this thing that was imple in invented that I think is pretty cool that's two complement and it's it's the the
standard way of of representing uh negative values in general the so if you want to if you want to have if you want
to express uh five as a as as a negative five um basically what you will do is like you will flip all these bits right like you will do the opposite and then
you will add one and then that will give you uh that will give you the two's complement value which is this which is if I recall correctly is the same one as
going it will be uh minus 5 will be da I think it will be
uh one zero one one one so this will be minus 5. Uh if you had eight for instance, imagine that you have eight and you want to do minus No,
that's not eight.
Uh imagine that you have eight and you and you want to and you want to convert this into minus 8 in complement, right? So
what you would do is like you will flip all these values. So one one one zero one one one one and then you add one and basically what ends up happening
when you add when you add one is that it becomes 0 0 0 1 0 0 0 Yes.
Uh yeah, but they do another trick which is like you just keep this until you find one one and then you flip all the others which is the same. It's it's it's
exactly the same. So you go like that until you go from right to left until you find one one. So you go from right to left and you keep those intact and the when you find the first one you
leave that one and the the rest you flip it. So minus 8 in complement two will be
it. So minus 8 in complement two will be one one one one these are bits obviously and then
uh one 0 0 0 right so this is two's complement that's why so and the one of the nice things uh one of the nice properties about to's
complement is that um on one end in one bite we can express we can use all the possible permutations for all poss possible values. Meaning that we can go
possible values. Meaning that we can go from minus 128 to positive 128, right?
Because if we didn't do that, then we'll be wasting at least uh minus 0 and zero.
So that that would be pretty bad. And
the other one is that when we do mathematical operations at the CPU, um we treat it like any other number and and and it works out well. like we do
the exor and and and uh for doing an addition for instance and and it's the same yes questions. I didn't explain this part very well but like hopefully it's not super important because we're
not going to be using yes.
So how would you know that's negative eight and not just a really big positive number?
>> Because this is what this is telling us.
>> Oh okay.
>> Yeah. like inside it because if it's an there are assigned and unsigned uh values, right? Like we need to know if
values, right? Like we need to know if we're working with a signed or an unsigned value. If it's a signed value,
unsigned value. If it's a signed value, the bit at the very at at at the the most significant bit is going to tell us if this is positive or this is negative.
If it's one, it's negative. If it's
zero, it's positive.
Yes. Uh sorry.
I'm listening. I'm listening. Oh.
>> I have a question in the meantime.
>> Yeah.
>> Yeah. Go ahead.
>> Okay. Yeah. Adrian here. Um, so this applies for Little Indian. Do I just believe that for big Indian?
>> No, hold on. Like uh like uh to complement >> yes >> is orthogonal to Indianness.
>> Okay.
>> Because to complement works at works at like two compliment is orthogonal.
>> Sure.
>> It's it's it's >> but the leftmost bit is reserved for the negative.
>> This that this one this one bite right like remember that inside the bite there is no endness inside the bite. These are
the bits inside the bite.
>> Okay.
>> Does that make sense?
>> Yes. Okay.
>> Remember endness applies at the bite level. So it's like I always talk about
level. So it's like I always talk about boxes inside the boxes. So like like inside the box of of the bite um is where we have these bits >> and end like and and two compliment
applies to the bits uh inside the bike.
>> So it doesn't matter if it's big Indian or little Indian that that space is reserved for indicating negative value for sign.
>> That is that is correct like like basically like so it's a good it's a good question. Um uh so minus one minus
good question. Um uh so minus one minus one is going to always be the same in little onion and in and in big onion.
Why? Because minus one is always going to be like all the beats one. So it
doesn't matter, right? Um but
but minus2 for instance, right? Then
what's going to happen if it's a U8?
Well, if it's a U8, it's a U8. Like the
engine is cannot apply to just one. But
if it's a U16, yeah, then we're going to see it flip. Does that make sense?
>> Yeah. You're padding the zeros. Okay.
>> Yeah. Yeah. Thank you.
>> Yeah. Two compliment is just is just a way to to to represent like the the binary behind it uh on each on each bite. Um yeah,
bite. Um yeah, >> I I was going to remove it. Are we done?
Like I I don't I didn't want to spend too much time with with this. I just
want you to know that also we're not you negative uh signed signed integers are rarely used in blockchain. Well,
yeah, rarely used.
Um but yeah, if you use them then then you should be familiar with um with those compliment right. So again
now that we know in this example we can see clearly right like um or for instance this one this one is a better one right like check it out
right like minus 256 right like in in little Indian the FF is here in bigness
will be just complement two but the the bites will be will be flipped Let's go. Okay. Uh booleans,
Let's go. Okay. Uh booleans,
it's another basic primitive. Uh we take we use one bite for expressing booleans, which means that we're wasting a we're wasting seven bits um
that are completely unused such as life um and they they get encoded and decoded like that. Yes. Cool. Okay. Scale. scale
like that. Yes. Cool. Okay. Scale. scale
is the coolest part of the coolest and the coolest part of
of oh sorry scale compact is the coolest part of scale it's like uh it's this kind of very unique primitive that uh
kind of it's the plaque signature of scale if you want okay so let's dig into it it's it's it's a little bit it's pretty Okay, so scale
is a primitive that allows us to optimally store any integer from zero to two at power 536
without wasting memory or using memory in the most efficient way possible. Right? So think about it.
way possible. Right? So think about it.
There are many times that there there are many times that we that we have to deal with a situation where um we know that a value can potentially be very
very very very big but that most of the times is going to be small. So what do we do? Do we reserve like uh a U 256
we do? Do we reserve like uh a U 256 wasting a lot of bytes um when in reality most of the times we could be using less. So this is a little bit one
using less. So this is a little bit one of the what the problem that that compact solves like it's it can dynamically know how many bytes we have
to use for each number. So the way that it works this trick is like this. The
two least significant bits of the first bite are going to tell us how many bytes we're going to use next. Because
remember that in scale everything is concatenated and we know we have to know the definition beforehand and as we as we decode
we know what's next right so we now we're about to decode uh compact. So
when we decode the compact we take the first bite and out of the first bite we look at the two least significant bits right okay so for instance
there are four different modes because these two bits are going to give us four different permutations if these two bits are set to zero it
means that the value is self-containing this bite right so we just write one bite we know this is going that we know that we're reading a compact value. And
now when we look into that bite, we look at the two least significant bits. If they are set to zero, that means that we need to
use the other six bits of this bite to to to read the value. That means that in this modality, we can only have values.
We can only express from 0 to 63.
Meaning that if the number is more than 64, we're only going to be using one bite. Awesome.
bite. Awesome.
If the flag is set to one, by the way, this zero V here, remember that before we explained that like there's this convention that is 0x, there's another
one maybe not so known that it's zero V is is kind of it's a convention for saying that what comes next is expressed in base two. It's like zeros and ones.
Okay. So
if it's set if it's set to to one, if the flag is set to one, it means that we're going to be using two bytes. The
bite that we're currently reading and the next one, right? So we can express up to to 14 because we have six bits in this bite plus the eight bytes of the
next one. That's 14. That's that's 14
next one. That's 14. That's that's 14 bits, right? In total. So this is the
bits, right? In total. So this is the maximum value that we can express in this mode. The third mode it's telling
this mode. The third mode it's telling us hey again you're using four bytes like the bite that you're reading plus
the next three that come afterwards.
And last but the most powerful one is when it's set when it's set to three.
When this when these two bits are set to one one or they are both set um then what this is telling us is the remainder six bits
of this of this bite that you're reading are going to tell us how many more bytes are you going to need to read. Right?
So if it ends with one one, we're going to take these six bits that we have and we're going to compute we're going to see like okay, how many bytes are they telling us and to the number
that they give us, we're going to add it four, right? So that's why
four, right? So that's why compact allows us to to to express a to to to to use a lot of bytes when we need them and very few when we don't need
them. So if the number is very small,
them. So if the number is very small, we're going to be using just one bite.
If the number is very large, we're going to be using uh uh an insane amount of of bytes. Does that make sense? Let's see
bytes. Does that make sense? Let's see
some examples because this is a little bit it's a little bit complex.
Okay. So how do we Okay. So let's let's encode together one
Okay. So let's let's encode together one as a compact value.
So these are bits of these are bits, right? So
right? So it's we only need one bite, right?
Because it's it's less than 64.
So these two bits we set them to zero to indicate that it's in mode zero. Like we
only need one bite. And then in the in the remainder six bits that we have we set it to one. Right? Therefore this is
in extra decimals. Who can tell me what this is in extra decimal?
No one for real. Oh my gosh. 04 right?
Like this is this is four, right? Like
this is zero. This is two. This is four.
So this is four. So what about two?
Two in X decimal is >> nice and three. What will be three?
>> I don't have three. Oh, yeah. No, I
actually have three.
>> Okay, I have three.
>> Okay, so this one this one will be uh I believe D >> C >> C. Okay.
>> C. Okay.
>> But yeah, close enough like the basically it's 12. So one thing to notice is that as you can see like one two three like it's like one tricky way
to to increase like they keep going like four + 4 + 4. Why? Because these two bits are taken. So in order to in order to increase it by respecting those those
two bits the next value is going to be 4 + 4 plus 4 going to be a power of four for for for variant zero. So and so on
so forth. So like C is is
so forth. So like C is is C is 12, right?
Ergo the the the next one like that I don't have four here but four will be 0 X one zero because it's 16, right? Um
but anyways like so but you see the point right like in scale we look at the first bite and the first bite is telling us hey we look at these bits it's like you only need to use this bite then we
look at the rest of these ones and they are telling us these are three and we express it like that in excess decimal 63 obviously is the maximum that we can
have in in in this mode zero which is expressed like that what about 64. So
for 64 as you can see we don't have enough bits here. So we need to move to the next variant which is saying like hey you need to use the next bite. So in
Exodimal that will be 01 01. So because
scale is concatenated remember like we start reading the first the first bite and when we read the first bite it's like aha I need to use the next one. We
go we read the next one we compute it and it gives us 64.
Yeah.
65. Who can tell me? Who can guess how 65 is expressed in Exodimal? Take into account that this
in Exodimal? Take into account that this is 64.
What will that be?
>> 0501.
>> 0 what?
>> 05.
>> 05.
>> Yeah. 0501.
>> Yes.
Okay. a little exercise for you.
>> Sure.
>> He has a question.
>> Oh yes.
>> Sorry. Uh I guess I didn't quite understand like the first example because I thought like the last two significant like the least significant digits just represented how many bits it was taking or how many bytes it was
taking and then everything after that just read normally. So why wouldn't the first example just be 0x01? I'm a bit confused.
>> Sorry. What's what's confusing? So I
understand that the last two by like the last two bits the last two bits. Yeah.
Yeah. Yeah. So that represents >> the the last the the least significant bits of the first bite. Yes. Are are are flags letting us
bite. Yes. Are are are flags letting us know the modes. And there are four modes. Let's go back one sec. There are
modes. Let's go back one sec. There are
like these four modes, right? Like
when the last two when the last two bits are set to zero, they are telling us just use the other bits in this in the same bite. Does that make sense?
same bite. Does that make sense?
>> Yeah, like the the six digits before it, right?
>> The what?
>> Oh, sorry. Uh okay, maybe I didn't understand. Okay, can you explain that
understand. Okay, can you explain that again? So, just use what?
again? So, just use what?
>> Let's let's recap because this is important. Okay. So
important. Okay. So
scale compact. So remember scale is is concatenated but like so we read bytes and as we read they are telling us what
do we need to do next right? So
we read the first bite. When we read the first bite we look at the at the last two bits of that first bite. Yeah. And
there are four different permutations. 0
1 2 3. Yeah. If the permutation is zero, what it's telling us is like you don't need to look any further. Just like the value is self-contained in this bite,
right? And for that you have to use the
right? And for that you have to use the remainder six bits that you have. Yeah.
Therefore in mode zero we can express at until 63 because there are two bits that are that are being used for the flag. We
cannot reuse them for for the value of the number. Right?
the number. Right?
If it ends with Z1, it's like, oh, on top of this on top of this bite, you need to use the next one. But obviously,
in the next one, there are no flags anymore, right? So for the next one, we
anymore, right? So for the next one, we can use the eight bits in there, right?
So in total, we have six six bits available in the first bite and eight bits in the next one. Total we have 14 bits. So the maximum value is two power
bits. So the maximum value is two power 14.
Mode four tells us that we need to use four bytes, right? Like it goes like okay, we need to use four bytes. Same
deal. And the last the last mode this is this is mode two, sorry. And and mode three like or
sorry. And and mode three like or depends how you look at it. Um when the two flags are being set, what this is telling us is like okay
like the six bits that you have in here that the six bits that that were for the value they are going to tell you the
number of bytes that you need to grab plus four, right? Okay. So
at first I was going to I was showing just like the basic examp 4. Yeah. What about two? Well, two
4. Yeah. What about two? Well, two
is this one here. So in extra decimal that's eight. Three
that's eight. Three inex decimal is 12. Well, it's C is C is 12 basically. Yeah, because it's it's
12 basically. Yeah, because it's it's it's this guy. Yeah.
>> Okay. I I understand. Now
>> 63.
>> Yeah. What about 65? So for 65 ah 65 doesn't fit in here. So, we're going to need one extra bite. Therefore, the plug is going to be set to 01, letting us
know, hey, go look at the next bite, too, right? So, it's set to to to one
too, right? So, it's set to to to one and then all this is zero and this one is one because remember little Indian.
Does that make sense?
>> Yeah. Yeah. I for some reason I wasn't I like I was separating the first six bits and the last two bits and I wasn't realizing that you made them all into the same number.
>> Okay. No, it's it's good that we go through it again like it it doesn't that like you see in this case like this is 64 like as a compact value the compact
value 64 is 0 1 01 yeah and 65 is who can tell me what 65 is we just saw it someone say before
thank you okay so now I want you to do an exercise like don't use AI Don't use don't you can use a computer if you want but don't use a library like you have
five minutes stops to to solve this um but try to manually decode with what you've learned um try to decode these two these two things yes
uh so why hi sit here so the first two bits will always store information about how much the system has to read so unless the information itself is less
than like two bits. I'm always there this is an edge case where I'm always wasting two bits to tell the system that okay you have to read these two bits. So
in that case scale encoding is always like has more memory usage than just normal.
So my first two bits like the le least significant bits will always contain information about the actual uh data I want to send right. So unless it's le
like more than two bits the actual information I'm going to waste like two bits of my sto like memory to tell the system okay you have to only read this much
is that right understanding then like this is an edge case we're talking about >> sure I mean like so if you have like if
you have the guarantee that the value that you're going to read will always be um less than 200 than 56 then don't use
compact use a U8 right but if you don't have that guarantee right in many cases in some cases it makes sense to use compact because if it's
going to be a variable number most of the times when it's small you're going to use so yes I guess the point you're trying to make is that you like we're
wasting we're we're investing two bits um for expressing uh for expressing um how many bytes we're going to use and and in some cases
because of these two bits we end up using more. Yes. In in in some cases if
using more. Yes. In in in some cases if the number is very small. Yes.
>> Yeah. just I want to point an example that to me was clarifying is that for example when you want to set a counter so for example how many accounts do I
have in my application or whatever this is a good um fit for compact because you know that numbers are preferably small so you
start by one two three four and let's say you want to have at max I don't know two million accounts or whatever then you don't need to reserve the whole
memory that makes it fit 2 million you can use compact so that in the early stages uh the number is small right that does that make sense >> yeah so if the allocation is very big
but the probability the realistic numbers are small then I can always use this >> exactly because it's preferably the numbers are preferably small >> okay >> so like account balance so
>> a balance a balance for example is is a is a good fit as well >> we'll see a more concrete example like when we look at the complex types of of of scale. We will get into vectors and
of scale. We will get into vectors and vectors I'm already like giving you a a a spoiler alert. Um vectors the first bite
spoiler alert. Um vectors the first bite like sorry the first bites are going to let us know the length. Right? So most
of the time the length of a vector is not that big. Sometimes it's very very d it's very long. Right? because sometimes
we're expressing vectors of bytes or sometimes we're expressing vectors of complex things. So most of the times the
complex things. So most of the times the number of the dynamic number of elements is small. That's why compact is very
is small. That's why compact is very useful because we use the we use just a few bytes um over like overall we always will be
saving a lot of memory, right?
>> Okay.
>> The guy in the front there. I like have you guys had time have you guys and girls like uh room everyone >> there are two more questions. Yes the
guy in the front >> uh how do we get the maximum number? Why
is it two two power of 536?
>> Go ahead.
>> Uh how do we get the maximum number? How
is the why is the two to power of 536?
Oh, the maximum number.
>> Yes, >> because 53.
>> Yeah. Yeah. So, the maximum number.
Let's let's let's think about it, right?
The maximum number is in this mode. This
mode is telling us use the reminder six bits of this bite to let you know how many bytes we have.
Right? Okay. So if the six bits are set right plus four yeah so the question is
um let's do it let's work it together.
Yeah, it's one one one one one.
So this is for the right. Now
this is uh 63.
This is 63. Right? Now it's 63 + 4 + 4.
Okay. So 63 + 4 equals 67.
So 67 * 8 because we are looking at at the bits is whatever. But you will realize
that 67* 8 is two at the power of 67* 8 minus one is that. Does that make sense?
>> Yes. So the maximum the maximum number of bytes that um the maximum number of bytes that compact can
tell us to use for a number is uh 67 which is insane. Like it's like it it really is insane. Like I've never like
never seen anything like I mean normally very rarely you will need to use something above U256, right? So, but
yeah, like that's that's how you that's how you get the the maximum one. Does
that make sense?
Well, last one.
Uh, hi, I'm Narin. Uh, that still leads to a max of uh 2 to the^ of 51 and two, right? And not to the power of 563 or 5
right? And not to the power of 563 or 5 whatever number it was.
Sorry, what? What was that?
>> Um, even if we turn use the whole bite for uh representing u the length of uh you know how much are we using, how many
bytes are we using, it leads to what? Um
if all the bits are turned on, all the six bits are turned on that will say we use we have 64 bytes which effectively
leads to uh up to 64 + 1 that's 65 bytes usage right it's not 2 to the^ of 536 it will be to the^ of 512 isn't it? No, no,
it it's not right. The thing is that um there's a trick and the last mode that that was yeah the last mode tells us to uh read the length. So the maxi in the
rest of the bite so the maximum length that we can read is 63 but we uh we do plus four because we already know that
if the answer was in so if the number fits in four bytes we would be using the mode above. So we add four bytes to the
mode above. So we add four bytes to the to the length. So it's 67 actually.
>> Yeah. I I explain that. Yeah. Yeah. I
>> how do we get 67?
>> So the the thing is that um that we can encode 63 in in six bits, right? So the maximum length that we can
right? So the maximum length that we can encode is 63 bytes. But we add four as well because if we use even those two bits.
>> So in mode in mode four in mode four which is like with the byes it's so this one is telling us use this bite. This
one is telling us this bite and the next one. This one is telling us this bite
one. This one is telling us this bite and the next three. And this one is telling okay the rest six bits are going to tell you how many bytes you need to get plus four. Why? Because there is
this here like all it's it's too is less. That's why I I mentioned plus4.
less. That's why I I mentioned plus4.
Yeah.
Um anyways, like can we Yes. Uh has
anyone solved the problem that I gave you manually?
>> Yes. Uh but I have a question. So you
can take four bytes in two cases when you have >> No, no, no, no, no, no, no. That's what
we're going to see in a moment. No, you
cannot.
>> Okay. Okay.
>> No, you cannot. I I know you're going.
No, you cannot. That that's that's that's that that's the trick here. Yeah.
Okay. So who can solve that? But that
that's a really good question. Okay. So
who can solve these two? How do we how do we um how do we decode this um as compact?
>> So the can I say the answer? So the
first one is uh 16,320 because it's like 25 time 63 I believe or 64* 64.
This is how this is how I there are thousand ways of doing it. This is how I will have done it because I'm I'm mostly JavaScript developer. This is how I will
JavaScript developer. This is how I will have solved this um in uh if if if it was up to me. Um and maybe I will mess
up but like I will have done 0x FF 0 um
two.
Is this your result?
>> Okay. So this is what did I do here?
It's like okay it's it's I pass it to beandian and then there are two bits that I don't want out. Basically, it's
like it's uh but there are thousand ways of doing it, right? But yes, if you got this number, you you got it right. This
is the this is the right answer.
Now, what about this one? Who can who has been able to who has been able to to to decode this one as a compact? You
>> uh so the other one is I believe one.
>> No, this is a this is a trap.
This is this value is if you try to decode that as a compact, you're going to get a panic. You're going to get a throw. You're going to get something bad
throw. You're going to get something bad because that is not a valid compact value. And this is something that is
value. And this is something that is very important and it's related to what he was saying before which is that certain permutations with compact are
not valid. Why?
not valid. Why?
Because of what we explained here. Um
okay. So why why is this one not valid?
Because here this 05 what this is telling us is this is telling us it's like we're using just one bite, right? Like there's nothing else.
right? Like there's nothing else.
There's no more memory, right? And this
05 is one zero one zero and four. Right?
Like this is five, right? So okay, this flag is telling what is this telling us?
Is you need to use the next bite. But
where is the next bite? It's not here.
So when we try to get the next bite, what's going to happen? Like if it drastic, we're going to get a panic. If
we Yes.
What?
>> Uh what what if you uh fill it and add zeros at the end? It's still one >> at the end of what?
>> At the end of this. If you add another bite, >> you add another bite then a different story.
>> Yes. Yes. Yes. So z 05 0 0 is a valid compact value.
This is a valid compact value. Yes.
Well, oh no no no. It's still not because because it will it the reason No, no, no, no. It's not. It's it's not the reason why it's not uh hold on a
second.
No 0.00 is is also not a valid compact value. Why not? because the value that
value. Why not? because the value that it's trying to convey fits in just one bite. So it's not that's not the that's
bite. So it's not that's not the that's not the proper way of encoding it, right? Like even if the bite is there,
right? Like even if the bite is there, even if we put it this two zeros, this will still not be a valid compact value because sure we'll grab the next bite,
but this is this is not the right way of expressing it because of its own value.
what what the the point that I'm trying to convey here is that not all possible uh binary permutations are valid compact values right precisely because the
because of the nature of this right does that make sense in the same way that you could a more a more concrete yes this is a this is actually a very good example
because here we have one right and it's little Indian so it's already here so if we just put zeros no no no like because it's one for one the flat to be 0 0.
Yes.
So yeah, that was like I I did that on purpose just to to convey the point that um uh in compact not all permutations
are valid. Does that does that make
are valid. Does that does that make sense? Question. Sorry.
sense? Question. Sorry.
>> So does that apply to all odd numbers under 64?
>> Go again.
>> Is that true then? as in not valid for all odd numbers under 64.
>> I can understand you, sir.
>> Like five is an odd number under 64.
Does this problem apply to all odd numbers under 64?
>> Yes, I guess.
>> Not just five, but seven, nine, 11.
>> Oh, yeah. Yeah, yeah, yeah.
>> The point is like there are many Yeah.
If like Yeah. So, shall we take a break?
Can I mean >> All right, let's speed it up. Okay, so
complex types complex types are complex types. Um now until now we've
complex types. Um now until now we've seen like um the primitives right like so unsigned numbers signed numbers booleans and compact compact is like
this very powerful primitive. Now let's
see the complex uh the complex types um there are doubles simply the concatenation of different types of codecs. So a complex type is basically a
codecs. So a complex type is basically a codec. It's it's a type that references
codec. It's it's a type that references other types, right? Toples, strrus,
vectors, arrays. A vector is when the size is dynamic. An array is when the size is static. An enum um we'll see them in we'll see them at dep in a
moment if we have time. Um an enome is a discriminated uh union. And then we have what I call specialized enums which are
like uh t specialized tag unions. One is
option and the other one is result. And
then specialized vector anopic uh which are kind of meta complex types. Not that
important for now. Let's move on. Okay.
The enum the enum is a is a complex type in which we express for instance we express enum and it's like okay an enom
uh where like the t fu is a u6 the t bar is a boolean and the t b is nothing so it's just like so the first bite the
first bite is going to tell us which which variant are we going to read next.
So if the first bite is telling us is zero, if the first t is zero, we know that we have to read two bytes next
which are U16. If the T is one, then we need to read one bite which is going to be a boolean. If the T is two, like if
the value of the first bite is two, we don't need to read anything else because it's just bus, right? So
uh uh an enum or attack union is is this uh complex type that um it's very cool because it it it it allows us to it's
like okay now you need to decode you're going it this variant is the is the winner is the winning one. This is the value that comes and then as as we read
it we know what we need to um go into the code. Okay. So let's see some
the code. Okay. So let's see some examples for this. Let's imagine uh that we are
um uh that we're trying to decode this value. So in this case this will be
value. So in this case this will be this zero here.
Sorry this zero here is the one letting us know that we need to read four bytes and these four bytes are for one.
This is the same thing for this is telling us okay this is a this is a one so it's bar so the next the next bite is
a boolean which is false and this one is just okay it's there's nothing else here
so and it's two so it's fast yeah um some observations about enoms enums make it possible to have circle
codec definition. So we can have some
codec definition. So we can have some codex that have circular uh circular references between them like like but because if they're expressing an enome the enome
can break the circularity eventually right so you can have you could have an an enome where one of the inner variants is uh is a vector of that same enum but
there's going to always be a variant that is going to break that circularity right so enomes among other things make it possible to have secular type definitions otherwise we wouldn't be
able to because uh we wouldn't be able to get off the side. Um
another observation is that the result and option result and option are are are two special types. Um an option is basically telling us whether there is a
value or not. If the is set to zero, it means like there's nothing else for you to read. And if each one is like that
to read. And if each one is like that value, it's present. And a result is is basically an a type for when things go well and a type for when things don't go
well. Um these are just specialized
well. Um these are just specialized denoms. There are they are still um t unions variants whatever you want to
call. Um, and the other observation
call. Um, and the other observation is that because the first it's the first bite is the one that indicates the it's the flag that indicates which variant
we're going to be using. There cannot be more than 256 options unless we do some tricks with like you could have some sort of convention that like oh if I
need more than 256 then I'm going to use FF. I'm going to use 255 as um as as I'm
FF. I'm going to use 255 as um as as I'm going to put I'm going to do a nested genome in there. Unless you do something like that. Um a genome cannot express in
like that. Um a genome cannot express in its own more than 256 options. Okay.
Um okay. Uh this is I already explained this uh an option is a specialized genome where zero means that there's no
value one there's a value.
So if we have an option of a U16 and we read Z 0 that means this is no because it's zero like so there's
nothing here uh an option of U16 where it's set to one this is what what does this decode to an option
of U16 to this what does this decode to who can tell me what this is number num number one for sure
256 because it's little Indian you see little Indian always fixes yeah like but that's that that's that's a point like we start reading and then here it looks like it's going to be one
but if you think about it it goes like that and it becomes to 56 um okay result uh result is a specialized
kind of uh variant where 0 means that it went well and 01 one means they didn't go well. So we can have a result that's
go well. So we can have a result that's like okay when it when it goes well it's a boolean and when it's an error it's a U8. I'm going to move fast because it's
U8. I'm going to move fast because it's just a specialized thing. It's it's more of the same and we don't have a lot of time. Okay. The struggles
time. Okay. The struggles
struggles are conceptually the exact same thing because remember the for scale scale is not is is is not self-describing. So you need to know up
self-describing. So you need to know up front the types that you're reading.
So a str and a tapole it is the same but with strrus the fields have labels they have names and in a tapel they don't like you know
by index what what is what they are but they don't. Um so for example
they don't. Um so for example these two things
this this double of tpples right and this strct are going to produce the same this the same output and the same that
they're going to produce the same binary data. uh is just that when it decodes,
data. uh is just that when it decodes, one is going to decode with name uh labels in the fields and the other one it won't. Okay. So
it won't. Okay. So
let's do a quick exercise together. Um
let's decode this let's decode this strct. So just so that we see that it's
strct. So just so that we see that it's just concatenation. Let's decode this
just concatenation. Let's decode this truck together. Okay. So um this FF
truck together. Okay. So um this FF belongs to what?
to the color red. Right? So this means that red is going to be 256, green is going to be
zero, blue is going to be 16.
Is it going to be ready?
Yes, it's boolean is going to be true.
And then the price, which is an option, it's set to true. So the value is there.
So, we're going to read the next the the next two bytes because it's a U16 and that's going to produce the value 256.
Yeah, it goes like that. Another
example, uh, it goes like this. I wanted
to show we're very short on time. I'm not going to Oh, Um, okay.
Vectors. Um, vectors is another complex type. And now here we can see the one of
type. And now here we can see the one of the places where um compact is actually very useful. A vector is a complex type
very useful. A vector is a complex type where the encoded value.
So the first thing that we read is the number of instances that we're going to find. Right? So for example a vector of
find. Right? So for example a vector of U8. This is just bytes right?
U8. This is just bytes right?
If it's an empty vector, what is it going to be?
Zero.
If it's a vector with one item, what is it going to be?
The first bite is the first bite is 04 and then is 0401. But it's correct. Like the
is 0401. But it's correct. Like the
first bite is telling us the length which is 04 and then we have the value inside which is 01. Yes, it's 04 because remember it's compact encoded. So in
compact encoded uh one was now two remember two it was 08 because
we go 4x4 so it's 08. Yeah. And then we have two values which are 01 and zero.
Yeah. So this goes like that. Okay, this
let's make it funner. We can also have a vector of compacts like this was was this one was easier because it was just like what about the vector of compacts?
We can have a vector of a vector of a vector of a vector like we could have a these are complex types, right? But
let's see a vector of compacts. Let's
see if we can if we can decode them.
Okay so how is it going to be encoded? There are
two elements which means that you will start with 08 right
and then what are we going to find 04 and zero 08 04 0 yes does that make sense
and last but not least let's try it um three is 12 which is is C 0 C
Um 0 C 0 1 0 0 and oh no no no it's compact. Sorry sorry
sorry. All right we don't have too much time to think about it but like yes 0 C is telling us that there are three
04 is one 0 0 and 64 is 0 1 01 arrays. arrays are the ex are
arrays. arrays are the ex are when the length is known um when the length is known uh at compile time then we don't need to
read the size first like we just we just put it there right so in that case these ones are a lot easier we it's it's basically like a
like a like a tpple of of the same thing but with a length of it known beforehand basically. So here are a few examples um
basically. So here are a few examples um of how to encode and decode and decode um arrays like it's basically like vector but without the compact at the
beginning because the one who is reading already knows how many iterations of that of that type um go in there. Yeah.
Okay. And
maybe we'll have time to five minutes.
Okay. So,
all right. So, am I going too fast? Was
that clear? More or less. Yeah. Okay.
Opaque. Opaque is kind of um so sometimes remember like with scale one thing that you may have noticed is that we um we have to in order we have
to always go in order to decode. So we
need to decode everything. So sometimes
we want to be like hey like the next n bytes please keep them and inside these n bytes maybe we know that it's a scale
encoded data but you don't need to you don't need to decode it because it's for someone else or for later evaluation right um that's when we use opaque
opaque is basically a a way of saying for instance the the the the body of block right is a vector of opaque extrinsic. So what what does
that mean? Extrinsic is a codec that is
that mean? Extrinsic is a codec that is a quite complex codec that it generates some binary data but many times the node
is not interested in in in decoding this extrinsic because it's the runtime that that will actually apply. So when it imports it it's going to import it as as
opaque. It's going there's going to be a
opaque. It's going there's going to be a vector there's going to be a vector um of opaque extrinsics. So basically
opaque means that there's going to be a compact value that is going to tell us how many bytes come next that need to be treated as a blob of binary data, right?
Even if this blob of binary data is actually a scale encoded but for someone else to deal with, right? So a good
example is the block of of the body. Um
there are other examples like it's it's just quite a bit. Um
yeah and then it turns out that I went too fast.
And then here we have uh here we have some implementations of if you want to use it with your favorite programming languages. Um these are some
programming languages. Um these are some implementations of of of scale encoders and decoders. Okay. Um
this the the the most important one is the one from parity uh for RAS because scale is mostly thought for RS for sure.
Um, there's one pretty cool one uh for Typescript that we use it. And then I I also want to show you some some tools
that you can use to fiddle around and kind of learn a bit more uh about about scale. So the first thing is like
about scale. So the first thing is like if you go to the puppy uh death console right and and you try to create an extrinsic or whatever
we'll talk about like how how where the encoders and the colors for these things are uh when we talk about the metadata but like if you do like uh oh
What?
What in the world? It's not picking it up.
Sorry. One second.
One thing that you can see here is that um one cool thing that you can see in this in the dev tools of pap is that whenever there's a scale encoded data as
you hover the different values here you can it highlights each part of it.
How is it in in the what is is in the in the binary data. Um there is this other tool. It's it's uh it's it's very alpha
tool. It's it's uh it's it's very alpha but it's scale.pappyhow.
And then in here you can use it for trying different things and you put a compact value here and then you see it here. You can put a variant here. So you
here. You can put a variant here. So you
can create your own little definitions in in here.
And you see this is telling us, oh, this is not good because we said that it was an I16. So if it had been a U16 instead, that would have been a valid
value.
And then what? Ah, I'm wait. Ouch.
what? Ah, I'm wait. Ouch.
>> Joseph, we need to wrap up now.
>> Okay. Um so try it on your own with scale of how. Uh
thank you for your time. I hope uh this was useful.
Loading video analysis...