Multi-Cloud Kubernetes Series Part 3

Parker Smits • January 3, 2024

Multi-Cloud Kubernetes:

A Guide to the Galaxy

It’s been a busy month for us at CirrusNorth, so this blog post is a bit delayed. Apologies! But we will try our best to continue our insights on challenges and solutions to multi-cloud Kubernetes setups.

This week, we dive into one of the most interesting topics: Networking.

When aiming for multi-cloud solutions in Kubernetes, it's crucial to distinguish between two common multi-cloud setups and discuss the challenges and solutions with each. Here, I am going to coin two new terms.

Mirrored Cloud Architecture (MCA):

A mirrored cloud is one where we have identical k8s installations in two different clouds. Generally, the traffic doesn’t have to span across clouds, as each cluster has a full set of applications available to it. This can be done for disaster recovery or HA reasons.

Collaborative Cloud Architecture (CCA):

A collaborative cloud is one where we have single applications deployed to one cloud but not another. This could be to leverage cloud-specific features or collaborate closely with teams that have their own infrastructure that you don’t directly control.

---

Note: We’ll cover service discovery, localizing requests, and security options for both MCA and CCA. Although some challenges won’t be applicable in all scenarios.

First, let’s dive deeper into Mirrored Cloud Architecture.

Service discovery is an important networking concern in multi-cloud architectures. In a mirrored cloud architecture, if not careful, you can route requests across the clouds for each and every request unnecessarily. This will affect latency and networking transfer costs between clouds. Here are two diagrams depicting both an example of allowing requests to travel between clouds and keeping subsequent network calls isolated to the original cluster.

Here you can and should leverage Kubernetes and cloud-native solutions for load balancing and service discovery (DNS, Cloud load balancers, ingress object, k8s services). But you will have to have duplicated service discovery and network paths for both the original external request and then another network path for subsequent API requests. This will keep your latency low and data transfer costs low as well.

This approach also keeps security in mind. Ensure no more than necessary network hops use the public internet. One security concern to be aware of is this assumes all APIs have an ingress object. If you understand your app architecture, you can/may eliminate the ingress object for APIs that won’t be hit externally, further improving security posture.

Next let's chat about Collaborative Cloud Architectures

CCA can either be a lot simpler but incur networking costs, or much more complicated and can mitigate some networking challenges. I’ll explore both here.

The first option for CCA is nothing. All these k8s clusters function independently. Nothing is an option, and each cluster can continue to route requests between the clusters at a slight cost in latency and data transfer costs. In most web applications, this is totally acceptable, and only strong arguments should be considered to explore other options. Remember my original statement: every advantage comes with its own set of disadvantages. We are trading off slight costs in latency and cloud costs for preventing capital expenses of building a more complicated solution and the operational overhead that comes with it.

Now, if we are going to pursue a more intelligent approach to bridge networking between the clusters and perform more intelligent service discovery, our best option is to go with Istio as a service mesh. Istio is the most common service mesh in production today and has the widest array of features across networking, SSL, service discovery, etc. Istio will help the security implications of all network calls going across the internet from cloud to cloud. It will perform pod-to-pod TLS termination with a sidecar. Also, service discovery is very powerful for Istio. They excel in cross-cluster communication and can prefer local services over remote or can use topology-based routing to route to the nearest pod if you have many clusters.

Next up in the series will be the most difficult endeavor: data.

< Older Post Newer Post >