Apache Kafka as a Service with Confluent Cloud Now Available on Azure Marketplace

Less than six months ago, we announced support for Microsoft Azure in Confluent Cloud, which allows developers using Azure as a public cloud to build event streaming applications with Apache Kafka^®. Today, we are taking another step forward, and we are delighted to share that Confluent Cloud is now available on Azure Marketplace. This blog post is a part of the Applied Cloud Stories initiative from Microsoft.

The integrated experience and billing model makes it easier for developers to get started with Confluent Cloud, our fully managed service for Apache Kafka, by leveraging their existing billing service on Azure.

Why is this announcement important?

There has been tremendous adoption of Apache Kafka throughout the years, and increasingly developers are using Kafka as the foundation for their event streaming applications. For this reason, it is important for developers to have access to a fully managed Apache Kafka service that frees them from operational complexities, so they don’t need to be pros in order to use the technology.

This is where Confluent Cloud comes in. Built as a cloud-native service, Confluent Cloud offers developers a serverless experience with elastic scaling and pricing that charges only for what you stream.

While Confluent Cloud has been available on Azure for some time, it previously required developers to sign up and provide a credit card as a form of payment to get started. This meant having to manage an additional bill in addition to their Azure bill. This could be a point of friction since developers familiar with Azure are not used to the concept of swiping a credit card for every service or resource they spin up for usage.

We are happy to say that having two separate bills is no longer necessary, and you can now use Confluent Cloud with your existing Azure credits. And, any consumption of Confluent Cloud is part of your monthly Azure bill.

But having a managed service for Apache Kafka through a single integrated billing model is not the only advantage. Confluent Cloud is a complete platform that offers you the tools to build event streaming applications with ease. Also available as managed services, tools like Confluent Schema Registry, connectors such as Azure Blob Storage Sink, and ksqlDB allow developers to focus on creating and inventing things rather than handling plumbing.

For instance, if you are developing a use case that is all about event ingestion into Azure Synapse (formerly Azure SQL Data Warehouse), and events need to be curated (e.g., filtered, partitioned, adjusted, aggregated, or enriched) before getting into an Azure Synapse table, Confluent Cloud can help you cut the implementation complexity by more than half since it gives you all the tools required to do just that.

What about Azure Event Hubs?

Confluent Cloud delivers capabilities—notably high-throughput messaging, storage, and stream processing—that would require Event Hubs and Stream Analytics to be combined in order to deliver the same. It has a much simpler and richer development model. But Confluent Cloud does not have to be a replacement for existing Azure technologies such as Event Hubs and Stream Analytics. You can continue to use them with any existing application and use Confluent Cloud for new applications.

Confluent Cloud is a natural option for developers coming from the open source world and also for anyone familiar with Apache Kafka, providing options for Azure developers.

Getting started with Confluent Cloud on Azure Marketplace

In honor of Linus Torvalds, the creator of Linux kernel who once said, “Talk is cheap, show me the code,” let’s see how Azure developers can start streaming with Confluent Cloud.

Step 1: Spin up Confluent Cloud

First, log into your account to spin up Confluent Cloud. Then click on the “Marketplace” option, as shown in Figure 1.

Figure 1. Azure portal showing all the services that can be accessed with your account

Step 2: Select Confluent Cloud in Marketplace

In Marketplace, simply use this page or search for “Apache Kafka” or “Confluent Cloud.” Select the option shown in Figure 2.

Figure 2. Selecting Confluent Cloud in Marketplace

Step 3: Purchase Confluent Cloud

After selecting Confluent Cloud, you will be served with a page that allows you to purchase the service as a SaaS resource. Simply click on the button “Create” as shown in Figure 3.

Figure 3. Purchasing Confluent Cloud

To pay for Confluent Cloud, Azure has enabled Confluent Consumption Units (CCUs). Each CCU equals $0.01. Pricing in Azure Marketplace is no different than our direct purchase pricing. For example, if you stream 100 GB of data into a Basic cluster in region westus2, you would pay $12.00 or the equivalent of 1,200 CCUs.

Name your SaaS resource and select which subscription you want to associate this service to. Read the terms of use, and then click on the “Subscribe” button to complete the purchase.

Figure 4. Completing your Confluent Cloud purchase

It may take several seconds to complete your purchase of Confluent Cloud. You can view the status using the notifications bar in the Azure portal, as shown in Figure 5.

Figure 5. Viewing the progress of your Confluent Cloud deployment

Step 4: Configure Confluent Cloud on Azure

Once Azure shows that Confluent Cloud has been successfully purchased, you can start using the service. Note that up to this point, there are absolutely no charges since you haven’t configured Confluent Cloud yet. An email will be sent asking you to configure your Confluent Cloud subscription, as shown in Figure 6.

Figure 6. Email from Azure Marketplace about configuring Confluent Cloud

After clicking the “Configure now >” button, you will be brought to the Confluent Cloud site where you will have the chance to register the admin user.

Step 5: Register the admin user

When registering the admin user, make sure to use the same email you used while purchasing Confluent Cloud, which should be the same email associated with your Azure account. Fill in all the fields required and then click on the “Submit” button as shown in Figure 7.

Figure 7. Registering the administrator for Confluent Cloud

Step 6: Verify your email

You should receive an email asking you to verify your email address. This step is required for security purposes, so Confluent can verify if you are the person that initiated the registration process. Click on the “Verify your email address” link as shown in Figure 8.

Figure 8. Verifying your email address

After verification, you will be automatically logged into Confluent Cloud with your new account (which happens to be the administrator of the account), and you can start playing with the service immediately. Note that as admin, you can register one or more users who will receive email invitations to join your account and be able to create resources as well.

For new users, the first $50.00 of usage will be free each month for the first three months. That means if you have a steady flow of events that on average computes ~50 KB per second (an estimate for region westus2 with a single zone and topic retention policy configured to seven days), you won’t pay anything because the total bill would be ~$42.80, and the $50 allowance covers that. In other words, you can try the service for free.

Figure 9. Now the fun begins. Create as many clusters you need.

When it comes to creating clusters, you have some decisions to make, but none that are as scary as they would be if you were managing Apache Kafka by yourself. Instead, you just have to name your cluster (it can be any string of your choice), choose which region from Azure to spin up the cluster from (it should be as close as possible to your apps to minimize latency), and choose the availability of your cluster. Figure 10 shows an example of cluster creation.

Figure 10. Creating a new cluster in Confluent Cloud

Creating a cluster in Confluent Cloud enabled by Azure is no different than if users were to sign up directly with Confluent, except for the fact that any cluster you create won’t ask for credit card details. The Confluent Cloud documentation provides more details on how to use the service.

Summary

Confluent Cloud makes it easy and fun to develop event streaming applications. Now, we’re taking another step towards putting Apache Kafka at the heart of every organization by making Confluent Cloud easier to use through availability on Azure Marketplace, so you can use the service with your existing Azure billing account.

If you haven’t already, sign up for Confluent Cloud today!