Getting Started with Confluent Cloud Networking

Networking is a core technology skill set that affects practitioners at all levels and parts of a business, from developers and users who often benefit from learning foundational networking concepts to infrastructure and network architects who live and breathe networking. As an example, developers who understand networking fundamentals are better equipped to troubleshoot and identify connectivity issues when they understand the path their data takes. This understanding also helps ease communication with infrastructure and operations teams, making everybody’s job easier.

The free Confluent Cloud Networking course on Confluent Developer targets both the specialist and the non-specialist, beginning with a review of networking and cloud networking fundamentals, then continuing on to present all of the available networking options for Confluent Cloud. Hands-on exercises throughout the course solidify its concepts. At its end, you will have the knowledge you need to consider issues like security, implementation time, and cost, in order to understand which networking options are appropriate for your architecture.

Introduction to Confluent Cloud networking

The course begins with a review of the building blocks of networks: public and private IP addresses, DNS, and CIDR ranges. Next, these fundamentals are applied to the cloud infrastructure provided by AWS, Azure, and Google. In particular, this section discusses logical private networks, known as VPCs (Virtual Private Clouds) on AWS and Google and VNets (Virtual Nets) on Azure, as well as the infrastructure–virtual machines and other services—that run on these networks.

Confluent Cloud Overview

In the Confluent Cloud overview module, you’ll learn the questions you need to answer during the initial design phase of your architecture with Confluent Cloud—i.e., what you are connecting (the services, whether self hosted, third-party, or multiregion or multicloud); where you are connecting from (on premises or in the cloud, from a corporate network, or from a home office); and how you want to connect (for example, requirements for private networking). It also presents an overview of commonly used enterprise topologies and the Confluent Cloud networking options that are available to answer the requirements of these topologies: secure public endpoints, VPC/VNet peering, AWS transit gateway, and AWS/Azure private link. The course continues with modules that cover each of these networking options in detail.

Connect to Confluent Cloud with secure public endpoints

Connecting to Confluent Cloud over a secure public endpoint is a production-grade option that is the simplest to implement and could be the only one you’ll ever need—particularly if you don’t have specific InfoSec or compliance requirements. Additionally, it is the sole networking option for Basic and Standard-level clusters on Confluent Cloud.

Hands on: Configuring a cluster with public endpoints

In this exercise, you will learn to create a Confluent Cloud cluster with a public endpoint, populate it with data using a fully managed Datagen connector, then consume the data over the public internet from an AWS EC2 instance running in a VPC.

VPC peering

The first private networking option for Confluent Cloud covered in the course is a peering connection, the most easily understood private option, and one that all of the public cloud providers let you create. You will learn how to set up this type of connection with Confluent Cloud, its requirements (for example, you must specify a non-overlapping /16 CIDR range for use by the Confluent Cloud network), as well as its limitations.

Hands on: Configuring a VPC peered cluster

In this exercise, you will learn to provision a cluster in Confluent Cloud and peer it to an AWS VPC. You will then produce data to a topic in your Confluent Cloud cluster and consume data from that topic from an EC2 instance in the AWS VPC.

AWS Transit Gateway

Next, the course covers how to connect to Confluent Cloud using AWS Transit Gateway, illustrating how it functions like a router for your VPCs, enabling you to overcome the limitations of the cloud’s usual non-transitive VPC peering. Instead of requiring each VPC to have a separate connection with Confluent Cloud, each of your VPCs can simply connect to Transit Gateway to access Confluent Cloud.

AWS/Azure PrivateLink

The fourth networking option the course covers is PrivateLink, available for AWS and Azure, which allows you to access your Confluent Cloud cluster through an endpoint in your virtual network. It’s one of the most secure options from a cloud networking perspective, and it’s also easy to set up with respect to IP addresses, only requiring up to three IPs, rather than a /16 CIDR range as with VPC/VNet peering. You will learn how to set up PrivateLink as well as its benefits and limitations.

Hands on: Configuring a PrivateLink cluster

In the final exercise, you will establish a PrivateLink connection between a Confluent Cloud cluster and an AWS VPC, produce data to a topic in the cluster and consume data from that topic from an EC2 instance in the AWS VPC.

Which networking option best fits your requirements?

The course concludes by revisiting the what, where, and how questions that need to be answered in the initial Confluent Cloud architecture design phase, which you will be much better prepared to answer this time. It then summarizes the available connectivity options and their tradeoffs. It ends with a descriptive overview of the components that make up the Confluent Cloud control plane and data plane.

To learn much more about the networking topics we have covered in this post, make sure to work your way through the full course on Confluent Developer. Additionally, if you’re using a free trial of Confluent Cloud to access the infrastructure you’ll need in the exercises, make sure to use the promo code CL60BLOG for an additional $60 of free usage.*

Start the Course