AWS re:Invent 2022 - Application networking foundations (NET204)

- Hey, welcome to Application Networking Foundations.

My name is Tom.

I'm joined today by Jamie.

We're both principal solution architects.

So we deal with customer network designs day in, day out.

And in today's session what we're gonna do is we're gonna provide a definition of application networking, what it actually is, and then we'll talk about the building blocks that make up application networking.

So we'll give you a really 10,000-level view of some of the services that fit under that umbrella.

And the goal for us here is to help you find a direction.

It's not gonna be a deep dive, it's a foundational level session, but it should give you a direction on which service to select for which use case.

The services will be split into two categories.

We'll have service discovery, as well as service connectivity and once we introduce the building blocks, I'll hand off to Jamie to show you some example architectures that are made out of those different building blocks.

So what we're not gonna cover today are the traditional networking foundations.

So VPC, transit gateway peering.

If you are interested in diving deeper into those components, we have a really good session from last year's re:Invent that's available on demand.

You can just catch up on it whenever you can.

Okay, so to better illustrate application networking, I wanna use TCP/IP layers model.

If you're from networking, you might be familiar with the OSI model.

This is just a simplified version.

It shows you the different layers that participate in forwarding traffic between two different systems on a network.

So we have a client on the left hand side and we have a web server on the right hand side and we have four different layers.

And at the application layer, the client will formulate the request, so get a particular host name, a particular path.

Once the request is formulated, it's passed down the stack to the transport layer.

Transport layer is taking care of TCP, UDP, port allocation.

So allocate a source port, it will know that it's going to a destination, will have Port 80.

So those ports will be added to the packet.

And then that segment goes down the stack to the IP layer where we add IP addresses, source and destination IP addresses so that the systems can communicate.

I'm kind of gonna ignore the network access layer, it's not really relevant for this talk, but traffic is gonna use the IP addresses to get to the other side.

And then the web server will strip the IP layer, look at the TCP ports and pass it on to the transport layer, strip the TCP ports source and destination, and then pass the request to the web server.

So when we're talking about application networking, we'll be staying in the transport layer and the application layer.

So that's where we're making intelligent routing decisions or forwarding decisions in your network based on transport or application layer.

We're not gonna be looking at forwarding traffic in a more traditional way using just IP addresses and sending traffic from A to B.

And we do have a application networking site.

So this is a public website where it gives you a bit of a definition of what application networking is and gives you some example services.

We actually believe there's a large number of services that really fall under the application networking umbrella that are not even covered on that page and we'll touch on each one today.

But before we do that, I wanna do a quick recap on some of the compute options we have in AWS because we'll refer to that.

So if you're running an application and you wanna use some of our compute options, you can either run a virtual machine using Amazon Elastic Compute Cloud EC2, you can run a container and we'll talk about what options exist for that or you can bring your own code.

We have a serverless runtime with Lambda, so you can bring your Python code, a Go code and run it in Lambda.

If you're running containers, we have two orchestration options.

One is relying on the open source Kubernetes framework and the other one is Elastic Container Service, which is more proprietary to AWS.

And in each one of those environments, you don't actually run an individual container.

The construct that you are running is either pod in Kubernetes and we'll spend more time on that today or task in the case of Elastic Container Service.

And a pod can have one or more containers, but just know that all of those containers are sharing the same networking stack.

And often it could be one container that's a web application and another container that's performing some additional function sitting within the same pod, maybe grabbing logs from the web server.

Okay, so if we split our application networking services into two categories, we have the service discovery section and service connectivity and we'll start off with service discovery.

So service discovery is gonna be all about finding where the service that you're trying to communicate with lives.

And the example I'm gonna use here today is a monolithic application.

So what that means is that is an application that's running on a single compute, it's using the same run time, so maybe it's a Java application and all the services running on that application are effectively different libraries as part of the same application.

So if you wanna make a change to a single service, you have to redeploy the whole app.

And with a monolithic application, say this is a virtual machine hosting it, you have to have multiple instantiations for high availability.

And then what you would typically do is put a load balancer in front of it to help you distribute the traffic.

And we'll talk about load balancers a little bit later, but here I wanna focus purely on the service discovery.

So what we're using here for surface discovery is DNS.

So DNS allows you to create a zone, in this case, example.com, and then configure records within that zone to point to your resource.

So you can set up myapp.example.com

and point it to your load balancer.

So the client sends a DNS query, gets the IP address of the load balancer back and can establish the connection.

Now we have a managed service in AWS that takes care of DNS resolution and DNS hosting, Amazon Route 53.

So you can use it for domain registration, you can use it for hosting domains, whether they're public or private.

You can do things like health checking, intelligent request routing based on geography for example.

And it also takes care of in-VPC DNS resolution.

So slightly different use case, but that also falls under the Route 53 umbrella.

And we actually had a session on Route 53 happening this week, it's already happened, NET 206.

So I encourage you to check it out on demand.

Okay, so let's have a look at a more complicated scenario.

So in this case, we still have multiple services, but now the microservices.

So maybe each one is running a different runtime, one is Java, one is Go, one is Python.

They're running on different compute.

So they're running on containers or maybe also on VMs and they will also have multiple instantiations for high availability.

But now this time, each compute hosting each service has to have multiple endpoints.

Now if you're exposing your service through load balancer, you might say, okay, service A is gonna be my front-end service, so maybe that's hosting my front-end of my application.

So that can still use the load balancer and clients from outside can still use DNS to get access to the front-end of the application.

But where it gets more complicated is that service-to-service communication, right?

So now these services have to use the network to communicate with another.

They have different IP addresses and they need to figure out where each one is.

So how do we solve it?

We can use a service registry and have each one of our services effectively register with the service registry and say, hey, this is me here, here are my attributes, here are my endpoints and if each service does it, then services can use that service registry to actually discover each other.

So that's something that you would have to include in your application, that registration piece.

And we do have a managed service in AWS that takes care of service discovery, that is AWS Cloud Map.

And AWS Cloud Map operates slightly similar to how you would look at DNS.

There is still a namespace, in this case, it's example.com.

But underneath a namespace, you register specific services and under each service, you register each instance of that particular service and you can decide on your own attribute.

So you can specify any information you want about that individual service.

So if our client or some other service that you deploy an SDK for tries to discover our services, it can still use DNS.

So I can still send DNS request to Cloud Map.

So I'm showing you an example with effectively doing a dig, which is looking up DNS information from a particular host name.

And if I'm using DNS, I'm gonna get an IP address back, so it's not super useful and it's very similar to doing discovery of DNS.

Where Cloud Map really shines is if you're doing your service discovery through API calls.

So I'm showing you here an API call using AWS CLI, but in reality in production, you would be including the discovery in your SDK in your application.

And then what you get back is all of the information that you've defined in Cloud Map.

So you get all of the additional attributes that you're interested in.

So an example here we have a project name or we have a version ID.

So as a client, I can decide that I'm only interested in endpoints of service A that are running version 1.1.

So you have all the control on the client side.

So how does it differ from DNS?

I'm not gonna go through all of these points, I'm just gonna highlight a couple here.

The big one is, we already talked about it, it's the ability to include custom attributes.

So you can include whatever information you want whenever you register a service.

And the second benefit is, with DNS, when you do DNS resolution, there's caching.

So queries and any changes that are happening might fall under the fact that some systems will cache the answers and you have to wait for the time to live for that cache to expire before the changes take effect.

With Cloud Map using APIs, so changes propagate really fast.

So within five seconds, you can make changes.

Now, how does AWS Cloud Map fit with Kubernetes environments?

So here we have an example Kubernetes cluster and we're running some service, service A, and within the same cluster, there is an inbuilt service discovery using DNS within a Kubernetes cluster.

For example, CoreDNS is one option.

And then when services in the same cluster try to communicate, they would talk to CoreDNS to find out where the other service is.

So you don't really need Cloud Map in that scenario.

Now when Cloud Map really helps, if you have two clusters and each one knows about only services that live within its own cluster, but you can use Cloud Map to share information between those two clusters and help them communicate.

So now service A can discover service B and vice versa.

Now one thing I wanna call out with service discovery is that it takes care of the service registration piece and finding out where the end points that I'm trying to talk to actually are, but it's not participating in the service communication.

There's nothing in the path of the traffic between your services.

So for that, we need some kind of service connectivity component that could be sitting in the middle and adding some additional functions for us.

So what I wanna introduce is a proxy and proxies sometimes get a bad name, but I feel like you should hug your proxy cause they bring a lot of value for you.

So in a typical communication, if we have a source and destination and we have traffic going back and forth, a proxy would be something that is deployed in the middle and a source connects to a proxy and then proxy connects to the destination on behalf of the source.

And in some more complicated scenarios, we have a proxy on the source site and then another proxy on the client side.

There's different naming conventions for them.

Sometimes they're called forwarded reverse or egress and ingress proxy.

A reverse proxy could be a load balancer for example and a forward proxy could be a proxy that you have within your corporation that's doing some kind of filtering.

So why use of proxy in the first place, right?

Proxies give you the ability to have a bunch of additional functionality that you don't have to develop in your own application.

So you can do things like TLS termination, you can do health checking, load balancing, tracing, all of those additional functions that you now don't have to think about putting into your application code.

And if we look at our AWS services with this proxy-based lens, we have a structure here with different types of services that operate at different levels of the TCP/IP layer that we talked about earlier.

So we have services like Network Load Balancer or Global Accelerator that sit in the transport layer, so they only focus on TCP/UDP.

We have services that operate in the application layer that are effectively Application Load Balancer, API Gateway, CloudFront.

And then finally we have services that also live in the application space, but they're not really load balancers, they're not really reverse proxies and examples here are AWS App Mesh and the service we launched on Wednesday, I believe, Amazon VPC Lattice.

And we'll touch on each one today, give you kind of a high-level overview.

There's not enough time in this session to really dive deep into each one of them.

What we'll do is we'll take our networking lens and we'll look at each service from a network lens perspective.

And then Jamie's gonna try and put them together into an architecture.

Okay, so I'm gonna start with services that operate as reverse proxies.

So they would usually sit in front of your destinations and receive traffic from the sources.

So the first example of a reverse proxy is a network load balancer, NLB.

You deploy a network load balancer inside of a VPC, so that's your private environment in AWS.

And it will point to some targets that you will navigate traffic to cause it's a load balancer and it's presented inside the VPC through individual network interface per availability zone.

It will get a static IP address per availability zone as well.

So if we look at it from a network perspective, it operates that transport layer, but it allows you to do TLS termination, which is kind of a little bit outside of transport layer.

It has that static IP address capability.

It also allows you to preserve the client IP address.

So clients coming in from outside, their IP address can be preserved all the way to the target application.

And then from a target perspective, you can point it to EC2 instances, you can point it to IP addresses, you can point it to an application load balancer and I'll explain why you would do that.

And then finally, if you're running containers, network load balancer operates as a load balancer service in the Kubernetes space.

So as traffic comes in from the outside or from inside the VPC, it would hit one of those network interfaces.

The network load balancer only looks at the TCP and UDP layer.

It will do some hashing based on the ports and protocol and decide on which target to forward the traffic to.

Network Load Balancer also has this very cool capability to allow you to present its listener, so the port it's listening on to another VPC.

So effectively you're sharing a service to a consumer in another VPC.

So those consumers can connect to your network load balancer without you having to create a full-on network path using VPC peering or transit gateway.

And there's a bunch of benefits here.

So first of all, you don't have to care about overlapping IP space.

Even if the service consumer VPC overlaps with the provider VPC, private link is still going to work.

Only the consumers can initiate connections, so it's one way communication.

If I'm the provider, I cannot initiate connections to the consumer VPC.

We are exposing only the individual port that the network load balancer is listening on.

So if the network load balancer is presenting a application on Port 80, that's the only port clients will be able to connect to.

And finally it's using AWS backbone to interconnect those VPCs and it's all happening within the same region.

Okay, so that was a single region, single VPC.

Say we have multiple regions and we want to navigate traffic to them from users on the outside world, users on the internet.

Now if you're using the traditional internet, the further away you are from the regions you're trying to get to, the more likely you're gonna hit some internet weather conditions and it's also pretty tricky to load balance traffic between different destinations.

So we have a global load balancer service called AWS Global Accelerator.

It operates on the premise of edge locations.

So we have hundreds of edge locations around the world.

They present your application that you define through in anycast IP address.

Anycast just means that it's the same IP address advertised from every location.

So your users always see the same IP address and they always connect to the same IP address regardless of where they are.

And Global Accelerator also has the ability for you to control which regions you wanna use using traffic dials.

So you have a lot of ability to define how the traffic is gonna flow.

So maybe we want to take the region on the left hand side out of action, maybe doing some maintenance.

We can take the traffic dial down to zero and then traffic from the clients will immediately move to the next available region that's closest to where the clients are.

Okay, so summarizing Global Accelerator, operates still at the transport layer, so it doesn't do anything on the application layer.

You get this anycast IP address, you actually get two anycast IP addresses.

We support both IPv4 as well as IPv6.

Supports client IP preservation.

So similar to network load balancer, traffic from those clients coming in all the way to the region will preserve that client IP address.

We do some magic there, it's pretty hard to do, but it works.

And well, you can target our network load balancers, application load balancers, EC2 instances, but they have to live inside AWS region.

Okay, so let's move up the stack, let's go into the application layer now.

So we'll start off with application load balancer.

Similar to network load balancer, it's a load balancer, so it lives inside of the VPC, but this time it's not presented through a single network interface.

There will be one or more network interfaces.

So application load balancer doesn't have a static IP address.

The IP addresses will change.

So that's why if you need a static IP address, you may wanna put a network load balancer in front of an application load balancer.

It's focusing primarily on HTTP and HTTPs applications, so it has a lot of functionality with helping you make decisions based on information within HTTP.

So headers, path, query string, TLS termination, all of that is supported.

It doesn't preserve the client IP address.

So the remote users connecting to the application, the IP address will not be preserved in the packet, but it will be moved to a HTTP header.

So you can still see it, just not in the packet.

And then from a target point of view, you can target EC2 instances, IP addresses, an interesting call out here, you can target Lambda function as well as you can target containers.

So in the Kubernetes space, application load balancer takes care of the ingress object.

Okay, so as the traffic comes in, it hits one of those interfaces, application load balancer this time makes the decision on the application layer to decide which target to forward the traffic to.

So you can forward traffic to different target groups maybe depending on the path.

So we really are scraping the surface here with the capabilities of our load balancers.

So if you wanna dig deeper into the differences between different types of load balancers, we actually have a really good documentation page that goes into those details.

Okay, so staying still in the application layer, but moving away from load balancing, more into organizing your APIs.

So if you are looking at aggregating and simplifying how you manage your APIs, you can use Amazon API Gateway.

And you'll notice that API Gateway doesn't live inside of VPC.

It's a service that's deployed into the region where you will define your APIs and decide how they should be consumed and what they should be mapped to on the backend.

So if you're first defining how they're consumed, they can be consumed publicly with a public IP address using regional endpoint.

You can also use an edge optimized endpoint, which is making use out of our content delivery network service that we'll talk about in a second.

And then if you wanna present it privately into a VPC, so I have someone inside a private VPC that I wanna expose my API to, you can do that with a private endpoint.

And then on the backend, you can integrate your API with a bunch of different backend integration.

So it could be Lambda function, could be DynamoDB, a simple storage service, or you can even point it to a load balancer that's in another VPC.

So API Gateway is focusing on two different types of APIs, REST and HTTP APIs.

There will be a static IP address if you're trying to consume it, but only if you present it into the VPC using the private endpoint.

Similar to ALB, it's not preserving the client IP address in the packet, but it will move it into the HTTP header.

And then from a backend point of view, you can pretty much point it to anything, AWS services, load balancers, as well as any HTTP/HTTPs endpoints that you might have even on the public internet.

So one big call out here and comparison that we often get is when would I use application load balancer versus when will I use API Gateway if I'm pointing to Lambda function, right?

Both support Lambda function as the backend.

So we'll use an example here.

We have POST action and we're sending some JSON with first name, last name and some age, and we're sending it either to ALB and API Gateway to then pass onto Lambda function.

If this is all we're doing, we should be using application load balancer.

From a cost standpoint, it's gonna be more cost effective than using API gateway.

So if you're not doing anything more complex, use ALB.

Now as soon as you start doing something that's a little bit more complex, maybe instead of sending the JSON to a Lambda function, you're sending it directly to a NoSQL database, DynamoDB.

Or you want to integrate with any other service or you want to do things like request validation.

So maybe you wanna check the JSON that's being sent and validate that first name and last name that are required actually exist in the JSON.

If not, it will be rejected.

Or if you want do mutual TLS or if you wanna transform the JSON that's coming in before it gets to your backend.

If you wanna do things like request throttling per API, per method, or if you wanna do advanced authentication.

In all of those use cases, you should be looking at API Gateway.

Okay, so we touched on CloudFront a little bit, but I wanna spend more time and explain what it is.

So CloudFront is our content delivery network.

So it's focusing on optimizing the experience of applications running on the web.

It's focusing on HTTP/HTTPS, as well as some streaming applications.

And the functionality that it brings is very focused on optimizing experience.

So it has an edge cache that will allow you to cache static content.

It supports DLS termination at the edge.

It supports simple CloudFront function, so you can run a little bit of code at the edge location.

If you wanna do more advanced serverless functionality on CloudFront, you can do Lambda at the edge, which will run the Lambda in the closest region to the edge location that the traffic is coming in on.

And then you can also do things like AWS WAF to do some additional security.

So as the traffic is coming in, the traffic that is cached can be returned from the cache.

The dynamic content is gonna be going over our global backbone, which is congestionless connection all the way to the region.

One thing I want to call out here is that the backend can be on AWS, but you can also have the backend for CloudFront anywhere on the internet.

It could be running in your data center on premise.

So another common question we get is what's the difference between Global Accelerator and CloudFront?

They both have edges, they both utilize the global network, they both have some performance improvements.

When should I use which one?

The simplest answer is if you have a HTTP/HTTPS application that will use caching, use CloudFront.

For anything else, you can look at Global Accelerator.

But I wanna compare side by side.

So first thing is Global Accelerator supports any TCP or UDP applications, whereas CloudFront is purely focused on HTTP/HTTPS.

Global Accelerator has anycast IP addresses.

So your users are connecting to the same IP address regardless of where they are, whereas with CloudFront, the IP addresses change.

So we always have to use DNS name.

The functionality on the edges with Global Accelerator is pretty simple.

Receives TCP connection or UDP connection, sends it to the closest region, that's it.

With CloudFront, we have all of the bells and whistles, caching, TLS termination, CloudFront function, Lambda, the edge.

Finally on the back end, Global Accelerator is really good at load balancing to different AWS destinations.

So if you wanna have active-active scenarios, active-passive, make decisions on how to forward traffic, Global Accelerator is excellent for that.

If you're just sending traffic to an origin, then you can use CloudFront and we support both origins that are running on AWS as well as origins that are running on premise.

Okay, so we talked about scenarios where we have clearly defined sources and clearly defined destinations and then we have a reverse proxy that adds some functionality in the middle.

So what about a scenario where we have our sources and destination mixed up?

So some of them could be sources one time then destinations, so it gets a little bit more complex.

So the pattern that's available here is to set up proxies that are running really close to the sources of destination.

This is really common with container environments where you would run a proxy as a site car next to your actual application and then the proxies establish communication to one another and effectively create a mesh.

So you might have heard the term service mesh and we'll spend a little bit more time on that.

So we'll go back to the anatomy of the proxy.

We remember the source and forward proxy, reverse proxy and destination.

Let's replace the source and destination with some arbitrary microservices and let's put them inside of a container.

So now we have a service inside of a container and a proxy inside of a container.

And then if we're running Kubernetes, we can put them together into a pod, so they're sharing the same network infrastructure.

So the pod is now our compute unit, the application lives or the service lives within the service pod.

And then the proxy is taking care of all of the communication in and out of the pod.

So if this is now getting scaled across a larger deployment, you would have multiple pods deployed on your worker nodes, which effectively are in fact virtual machines and you'll have a lot of them.

The proxies would register with some control plane to say, hey, this is my service that I'm supporting and the control plane is now gonna take care of the service discovery.

And then the proxies themselves will take care of service-to-service communication based on that discovery parameters.

So that altogether really makes up a service mesh.

And at AWS, we have a managed service that helps with running your service mesh called AWS App Mesh.

It takes care of the control plane side of your service mesh.

So we don't manage the proxies, the proxies still live in the containers.

In that scenario, they're Envoy proxies, you still have to manage them, but we're taking care of the control plane that takes care of the service discovery even within that service mesh.

And there's a lot of benefits of running a service mesh, right?

We don't have to go to the central location and then spread out, anything can talk to anything and use the proxies to do a lot of additional functionality.

So we can do things like traffic shaping, load balance to different targets, do blue/green deployments, weighted distribution.

We can do path-based, header-based, the delivery, we can support protocols like HTTPS, TCP, gRPC.

And then from visibility standpoint, those proxies give us the richness of information of what's actually going in your environment, from just logging to metrics and actually tracing a particular request through the service.

So there's a lot of cool features with service meshes.

Now, if you run a service mesh, whether it's an AWS App Mesh managed one or one of the open source ones, the thing that you still need to think about is the network.

So maybe the Kubernetes pods run on different clusters, then you need to put those clusters into VPCs.

And if you want them to be able to communicate, you need to set up some kind of connectivity between those VPCs.

So maybe run a transit gateway, maybe set up peering, but it's up to you to make sure that there's IP layer connectivity so that the service mesh can establish across all the network.

So a lot of customers were coming to us and saying, okay, we like the benefits of service mesh, but could AWS manage a little bit more of that environment, right?

Could AWS take care of managing the proxies?

We don't wanna run the proxies and we also don't want to be focused on building the network, right?

Why do our developers have to know what kind of network is running underneath and involve that in the communication?

So we did exactly that.

We moved the proxies out and deployed them as a centralized AWS managed proxy that all the services can communicate to and from and the proxy will authenticate all the communication.

And we'll go for an example and I'll tell you what that service is.

So in a traditional deployment, say we have three VPCs, two Kubernetes clusters, everything is connected through some kind of transit gateway.

Maybe we have a service mesh across those two clusters, but then we have something running on EC2, some other application that's not part of our Kubernetes deployment.

So maybe we expose that for an application load balancer to everything else.

Then maybe we have some serverless code running on AWS Lambda.

So it's getting a little bit more complex.

It's actually really hard, if not impossible, to bring those things under the service mesh.

So with yesterday's launch of Amazon VPC Lattice, it's currently in public preview, we're simplifying that environment quite significantly.

So there's no longer transit gateway, there's no longer an application load balancer you have to deploy.

There's a service discovery part of Amazon VPC Lattice.

It's a registry where you define your services.

So all of the services living on different compute platforms, whether it's Lambda, whether it's EKS, whether it's EC2, can be registered with that service discovery.

And then for communication, we have the centralized proxy, application layer proxy that allows you to connect all the VPCs and enable communication between different services running on different compute platforms. And we're using DNS for service discovery, so when those services need to communicate, they will do a DNS lookup, Route 53 Resolver which takes care of DNS resolution inside of VPC,

will respond with Amazon VPC Lattice as the destination for the services that are registered with it and traffic is just gonna flow to VPC Lattice and as long as you set up the right permissions using identity and access management, so it's using IM instead of NTLS, you can establish that communication.

So if you wanna learn more about VPC Lattice, I actually got good news.

This session hasn't happened yet, so it's happening tomorrow.

So if you're still here tomorrow and you have energy after the party, NET 215, you can still watch it tomorrow.

If not, it's still gonna be available on demand within a week.

Okay and the final comparison I want to do is compare some of the services that are often used with microservices.

So with API Gateway, the common use case is that where you have the clients and destinations kind of clearly defined and you're exposing your backends through APIs that are running on API Gateway.

As soon as we're talking about any-to-any communication where each service might need to communicate with a mesh of other services, then you can start looking at things like the service mesh with App Mesh or VPC Lattice.

So they're both for those East/West use cases where the biggest difference is between Amazon VPC Lattice and App Mesh is how much of that service is being managed for you.

With App Mesh, we're managing the control plane, with VPC Lattice, we're managing the control plane as well as the data plane.

So we're setting up the communication between VPCs so your developers don't have to care about setting up transit gateways or VPC peering.

Now from a set of supported services, API Gateway supports any AWS-based services.

So you can use it as a frontend to S3 or DynamoDB, whereas App Mesh and Lattice focus purely on services that you manage with the big difference that actually you can bring Lambda-based functions into VPC Lattice, whereas you can't present Lambda with App Mesh.

All right, so these were the building blocks.

I'm gonna hand off to Jamie to show you some example scenarios where we're using those building blocks and putting them together.

- Thank you.

Thanks Tom.

Hi, my name is Jamie and I'm going to now play with all the toys that Tom just described.

So I've got the easier job.

Tom had the harder job of actually trying to go into each of those pieces, but he did stress on one thing that I wanna also make very clear, is that this is a 200-level session.

So we're here to show you these individual pieces.

I'm gonna walk through an architecture that's gonna grow and become a little bit more modern, if you will, from a monolithic application to something a little bit more of a microservice or a modern application as I mentioned before.

And the idea is to plug in some of these things that Tom mentioned, and then you can later go and figure out all the other good features that we didn't have time to mention today.

So as I mentioned, vertically scaled application here.

We've got our monolithic compute.

This might have your web frontend, your portal, all of that fun stuff in one big machine or one big EC2 instance.

We have our service discovery, which we use in Route 53.

And then we have an ELB of some kind to bring in our traffic and distribute it between one or more of our monolithic compute pieces.

So we have options as we saw earlier from Tom.

We have our ALB, which does layer seven-path based routing.

It has other features too like WAF integration, you can do authentication offload with it, anything pretty much at the layer seven option that you're going to be using, ALB is your go-to.

It has a lot of features that you can dial in and with the multiple target groups facing our backend storage, which we're using for databases.

Our second that we have and you can go to multiple target groups as well, is the NLB, the network load balancer.

This has the long live connections.

This is what a lot of our gaming friends like to use so that you don't get cut off when you have a new autoscaled EC2 instance that gets added in the backend and then it reshuffles.

It'll actually keep those connections long and connected.

And then of course we have private link.

Private link is, as Tom mentioned before and I'm wearing the T-shirt today, see, get private link connection.

So private link is...

I had to put in that plug for our PM because he was saying, "Mention more private link."

So private link gives you that connectivity between multiple VPCs, unidirectional stateful flow, right?

Those endpoints that Tom had showed before that could be distributed around your environment, this is the thing that you would use with an NLB to offer that service.

And then again, we still have our storage in the backend.

Again, these are databases.

So now let's take this logical diagram and apply it to something that we are more used to seeing within our Amazon infrastructure.

So I've got my Route 53, it's doing my name to IP address translation for us.

I've got an ALB sitting in front of my standard EC2 compute, right?

And I've got my databases sitting the backend for my standard storage.

So now we're going along in our journey and we wanna start adding a little bit more capabilities to our application and our infrastructure.

So we have a requirement to bring in API calls.

Let's say we needed to do some data transform.

Maybe one of the entry ways into our application could be something like receiving a text.

But we have to make sure that that text data gets normalized so that our application can handle it and we'll add maybe a little bit of cache, like a Redis cache or something like that.

So it could be stored in there for our application to refer to.

So I'm bringing up Lambda and API Gateway for these two jobs.

Now, it doesn't necessarily mean that API Gateway sits in front.

Not everything is a direct replacement for what we had before.

So they work just fine together.

Tom showed a comparison between ALB and API Gateway and I'm gonna show how they can work together in the same workload.

So we have our requester, we have our short term data store such as Redis, we're adding microservices such as Lambda and then we add our API gateway.

Our API gateway is talking to our Lambda, we're getting our texts in from our input, it's sending it to our short-term data store.

Our monolithic compute can then read from that, do what it needs to do, store anything that the user is doing, everything is working just fine with this three-tiered web app.

So let's take a look at what that looks like in motion.

So here we are with what you've seen before.

I've now added my service requester.

It's going through our API gateway, which lives in the region.

I'm using Lambda, the function and features that Tom had mentioned about where it can actually live inside your VPC.

And then we're sending it to our ALB.

Our ALB is doing our load balancing for us.

Maybe we have, again, some offloading or we have our WAF integration for some statistics and it's connecting to our standard RDS or our standard databases.

Everything's going well now.

But now our application needs to be deployed in our CI/CD in a faster fashion.

So we're gonna look at using some kind of containerization.

Anybody here use Kubernetes?

Yeah, we've got quite a few.

Containers just in general?

Yap, so we all know the benefits of what those are.

You have the ability to do your development in a similar environment, you can put it in a registry, there's lots of goodness that we can use.

So we wanna use that in our infrastructure.

And we've chosen Kubernetes because we can't choose all of the things or you'd be sitting here forever or you'd all be walking out the door as I still am talking.

So here we are, we have our deployment.

I've changed it over to Kubernetes.

We're just gonna call it app one, it's gonna represent one app.

There's pods inside, all of that fun stuff.

We've got our Redis, our long term data store, we're doing good.

But we need to bring data into those containers and we have an option for that.

We have load balancer controller options.

There's two, right?

I'm not gonna read everything off on the slide, I'll just go and highlight it a little bit.

But the native one comes with CLB or classic load balancer.

We're not getting rid of it right away, but we're not actively developing on classic load balancer.

So all of those features that you get at layer seven from the classic load balancer, you're not gonna have as rich of an environment as, say, an ALB.

And you can use an NLB in what's known as instance mode.

So then you can connect your instances targets and you can send traffic.

What we recommend is that you install the AWS Load Balancer controller.

It needs to be installed, but it gives you a couple of other options.

You can share your ALB between services and you could also use it for IP as a target.

ALB is there and NLB is there, no CLB.

So if I'm gonna run down this thing, let's take our Kubernetes cluster, we're gonna populate it with our pods.

We have our ports exposed.

We're going to use our ALB as the ingress controller.

We can get our updates, we can send back feedback to our API server and it's easily provisioned after you install it.

So then I mentioned before that you can share a load balancer, so you don't have to spawn a load balancer for every single pod or every single application that's coming up.

Again, we're choosing the application load balancer and I have /app-a, I'm gonna send it to pod A that's representing that.

I have /b, same thing.

And then as I mentioned, we have IP as a target.

IP as a target is extremely fast.

It is low latent, so you can actually separate out your pods and your infrastructure based on what you need for speed or for growth and ease of configurations.

If you wanna know a little bit more about application or the load balancer controller, here is the QR card.

I'm sorry, I should have put that up earlier as you guys were taking pictures, you might have some duplicates now.

So now let's just change over our compute to pods, right?

Nothing really new and groundbreaking here.

It's still a compute architecture, but now our CI/CD is a little better.

So let's move on to breaking up into multiple services.

Now that we have containers, we can distribute and have different teams working on different things.

Our deployment is a lot quicker.

So here we go.

We have app one, we have our containers.

We add, say, another microservice to do something that we wanna do on the fly.

We add our second application, we add our third and you notice that you have to maintain all of those connections.

You have to maintain the connections to your storage and you have to maintain your connections to the other parts of your compute and your app.

We're still using our Route 53 for service discovery, we have our API Gateway and our ALB for our ingress into our application, so all is good.

But what happens if you start doing this more and more?

So I'll do the same thing, I'll add my microservices B.

We've already seen our app two, we add app three and then we add app four and then app five and now you're maintaining all these individual connections and you're also maintaining all the connections to your data store.

So who can tell me what this kind of looks like?

Spaghetti (laughs).

A mess, right?

And yes, I took a picture of this spaghetti, so you won't find it anywhere on the internet.

You're more than welcome to copy it.

So let's clean this a little bit, right?

We've got our cool toys, App Mesh.

So here we are.

I'm using App Mesh.

Everything is connecting to the App Mesh in terms of our compute, all of our containers.

For our API Gateway to talk, we need an NLB for our ingress.

And then our microservices are still hanging off our API gateway and our ALB.

They don't connect into the App Mesh, but this looks a little bit better, I think.

When I add another container now, all I need to do, I'm sorry, not a container, but when I add another cluster, all I need to do is point what I need to to the App Mesh.

The App Mesh can then figure out where everything goes and connect it to my data store.

So if we're gonna dig into App Mesh a little bit, here we are with App Mesh just kind of focused in.

You'll see that I have my three different paths.

I have my virtual services, my virtual routers showing my protocols, and then my nodes that connect up to my services.

Everything is running well.

I have my API Gateway going through my NLB, I have my ALB, it's going into the App Mesh, all is good.

Everybody knows where everything is.

So Tom had mentioned one of the cool features you can do with a service mesh is something like blue/green.

So what does that look like?

So if I'm gonna take a look, I've got all of my ingress coming in and I come out with a second version of app two, I can go ahead and wait what I want my secondary paths to be, add app two in and then go ahead and start sending traffic to it.

The other thing you can do with a service mesh that Tom had also mentioned is tracing.

How many of you have gone through the scenario of something being wrong with your application?

And now that your application is so distributed, you have to find out actually what part is causing that, right?

It's very difficult to look at it on a networking standpoint because networking wise, everything seems to be going to the same thing or everywhere at once.

So if I look at this environment and say, okay, I've got my service discovery running, I've got my API Gateway sending traffic in, I'm going in through my load balancers and then I notice that there's an issue, but I don't know what the issue is.

Some of you might have already seen it on the screen, but let's start our trace.

Everything's good, we go along and then we find we have an issue.

The issue is that they deployed app two over app three and it's not responding correctly and that's where I know to look for my problem.

So we're halfway there.

We've discussed what we're doing with our application.

We're discussing about our compute connecting into a single service such as a mesh, but our storage story still needs a little help, right?

Everything still has to do all their interconnectivities to storage.

So Cloud Map is a good example that Tom had brought up before that can help us with this.

Our service to key pairs for Cloud Map will allow us to actually have our compute figure out where things are.

So now we're essentially just connecting all of our compute to Cloud Map to find out where the storage is.

And then if we wanna make any adds with some changes, we can do it there.

So what does that look like?

So here we are, we're running along just fine like we were before.

We have all of our storage neatly laid out throughout our environment, it's very distributed through different AZs, we're doing all the best practices.

We install our Cloud Map and then we go ahead and we set up our key value pairs of where our schemas are, our databases, and then we distribute them within the rest of our environment.

Now it becomes much easier for us because as I mentioned, what do we do when we add a new DB or schema?

Instead of going everywhere to every single machine, we wanna have to go ahead and make our configuration changes, we'll add our database, we'll add our schema or our change or our version, we'll go ahead and make that manifest and then deploy that manifest to the machines that care about it.

So now we've got more of a singular set, a more single place to go for our storage connectivity.

And again, this isn't just for storage, you can use it for a myriad of other things for your service discovery.

So this is looking a little bit better, but not necessarily super easy, right?

We've got our spaghetti straightened out, but we're still managing all the strains.

That's just the way it is.

There's gotta be something that we can do that makes this a little easier.

And of course, because Tom went before me and he stole my thunder, we're talking about VPC Lattice, right?

VPC Lattice will allow us to scale and it will allow us to facilitate communications in a more managed way.

With the App Mesh before, you were still configuring a bunch of pieces that you really had to get in there in the weeds and make sure that you knew what your traffic flow was going to be.

I'm not saying you can ignore your traffic flow entirely, but this will make it a bit easier.

So now we're gonna add VPC Lattice to our logical diagram, right?

And then you're gonna say, "Well Jamie, that kind of looks the same.

How's that special?"

Well, here's the new infrastructure that we actually deployed with an AWS.

You'll notice that I am separating my ingress or my frontend with my backend and then I've turned on VPC Lattice, much like you would turn on a S3 gateway endpoint or DynamoDB gateway endpoint, and it'll now allow my applications to find the rest of the other applications without the need of setting up

any other type of crazy guardrails to get information to go through.

The other change you should notice is that now I am no longer needing an NLB and my Lambdas are connecting directly to VPC Lattice.

So Lattice can handle Lambdas as well directly.

So let's take this a little bit further and say, well, I'm now separating my environment so that I am using multiple VPCs.

Maybe I'm getting around some limits, I want to do some form of small micro or cell-based architecture so that I can go ahead and visualize my logs, whatever it may be.

But you'll notice that Lattice is still allowing communications between one VPC and the other, my frontend and my backend.

And you can go even further.

Now we're gonna segment even more, right?

We're gonna realize and utilize our multiple VPCs for a boundary for our application development.

And there could be all sorts of lambdas involved, other things that we wanna deploy throughout this environment.

But there is one thing that I'm not showing in this diagram with three VPCs that you would normally see and almost every other presentation here.

Can anybody guess what they don't see between these VPCs?

Transit gateway, VPC peering, exactly.

We don't need those, right?

Because VPC Lattice actually will do this for us in the background.

Now it's not necessarily at this point a direct replacement for all of your other traffic that doesn't need Lattice, but this could be very helpful if you think about it.

I know somewhere, some people, nobody in this room might have an environment that might be a little messy and is hard to make changes when you add applications.

But if you had something like VPC Lattice and you pointed all of those to Lattice, enabling it in your VPCs, if you made a change in your transit gateway or you made a change with your peering, your applications are still gonna talk.

Now, yes, your security groups are still important, yes, you're gonna still gonna have to pay attention to your VPC routing.

You don't want to kill all that and say, "Jamie said I could just do this," and your application goes down.

But in general, it makes your ability to upgrade and move a little bit easier and you still get to enjoy path-based routing, you still get to enjoy setting up your listeners for where things go for load balancing, you still get to enjoy waiting, everything you had before with App Mesh, but now a lot of it is more managed for you.

So now that we've got this all taken care of, our application, our internals of our application are looking great, it's a lot easier to manage, we're scaling well, what does our global reach look like?

Now, Tom had given us a couple of toys for those as well.

The first one that he had talked about we're gonna highlight today or here.

So we have a presence in the US, we have customers who have a presence in Europe and customers in Asia Pacific and they're all going after our backend because we're so popular.

Everything's running so well, we're pretty low latent, but the thing that we notice is that, one, the distance, right?

Between our customers and our backend or our application could be different for each one of them, giving them a different experience, right?

Internet weather is what Tom had mentioned.

Plus, now that we have so many people banging on our front door, we might have some congestion or more heavy demands on our infrastructure, which could increase prices.

So one of the things that we can do is Amazon CloudFront.

As was mentioned before, CloudFront will actually add points of presence as close as possible to your customers and allow that communication to go as quick as possible and travel a shorter distance instead of to your entire region.

Anytime you need to make a change to your CloudFront infrastructure, you can go ahead and make that change and distribute it out and your customers don't know the wiser and those changes go over the AWS backbone.

Now, yes, there might be some things that you have in your infrastructure or have in your application where CloudFront can't handle.

And I didn't really represent those dots because at this point, we're hoping that it's not so much anymore where it's something you need to be concerned about.

So the other thing that we mentioned too is availability, high availability and global reach and Tom gave us another clue with that as well with Global Accelerator or AGA.

So let's take the scenario that I'm in US East, everything's working well, we're really happy with our architecture and then we are told that we're going to install something new into Ireland maybe or Europe for our European presence.

And because we want HA, we want to have everything in sync, we bring up another infrastructure that looks just the same as ours.

We were smart, we use CloudFormation or we use Terraform or some form of infrastructures code, so this is fairly easy to do.

You'll also notice that our Lattice is region bound.

It's a regional construct, so we do the same configurations we did before over there.

And then we go ahead and we turn on our Global Accelerator.

Cool thing about that Global Accelerator as was highlighted earlier is you get a pair of anycast IP addresses.

I'm only representing one here, but we do have a pair.

You get two separate ones because we're always that whole, everything fails all the time and we wanna make sure you're most protected.

And that is installed on points of presence around the globe with two separate pieces of infrastructure that represents those IP addresses.

And then the global AWS backbone, fast performant 100-gig dual backbone that we control, we don't have anybody else involved in that, will handle the traffic from the Global Accelerator, which is close as possible to your customer, to your backend and then traffic flows, right?

So everything is looking good here.

We have our environment.

But one question you might be asking is, "Hey Jamie, how is that highly available?

I just taught saw two streams. Sure, you're sharing one IP address, but how does that make it better for me?"

Well, I'm gonna answer you.

We have our environment, everything is working well, and then somebody somewhere unfortunately makes a mistake.

It takes down your application and now your users in Europe can no longer talk to that region.

Global Accelerator sees this because it's inheriting the health checks from those ELB end points, those ALBs and knows that it's down.

So it'll go ahead and divert the traffic to the next closest available region.

Your customers never know the difference.

It's still up.

So here we are.

Now you go in and you fix it.

You make whatever changes that are needed, it comes online, your health checking now registers that everything is okay.

Then there you go, traffic starts going anew.

So now we've gone and addressed our internal application, managing that internal application and our global presence and managing that global presence.

No more spaghetti, everybody's happy and now we're done.

Thank you.

(audience applauds)