Redpanda Cloud - Annual Commits (BYOC & Dedicated/Serverless)

I use Redpanda as a provider for my personal project. I create a tool to manage Redpanda, to do observability on it, to do development, to do testing, to do dashboarding, and monitoring on it. I am creating a tool to do almost most of the features as a developer, DevOps, and test person.

If I have to summarize Redpanda, it is a competitor of Kafka. The good part about Redpanda is that there is a free part, which is great. It is embedding most Kafka features. You have a number of features similar to Confluent, but for free, which is great. With Confluent, almost everything is not free. You have a dashboard for free, but it is limited. In Redpanda, you have many features, but some of them are not free, of course. However, you have more features for free than Confluent, for instance. I put Redpanda in the Kafka family, along with Confluent and Kafka.

I think Redpanda needs to increase the connector options. Kafka is well known for Kafka Connect. I think Redpanda lacks this. Redpanda really focused on many things such as contract types in Kafka, Avro, Protobuf, and JSON Schema. They manage it automatically. You do not even have to specify it in the schema registry. It can detect automatically the JSON Schema, the Protobuf, the Avro, the zip, and so on. This is great. However, I think for the connectors, they are still young, so they need to enhance the connectors with anything such as MongoDB, cloud, big data, Elasticsearch, Datadog, Splunk, MySQL, databases, SGBDR, flat file, anything. I think connectors need to be more announced. And also performance reviews. Kafka was great, but there is nothing around Kafka. You need to implement or find tools by yourself, such as Conductor. This is why I created my own tool where I manage Redpanda and eleven other brokers.

I have been using Redpanda for six months.

I have not experienced any lagging, crashing, or downtime in Redpanda.

I think the scalability of Redpanda is great because you can use Kubernetes or OpenShift because it is container-ready. If you want to have it scalable, I would recommend installing it on containers for high availability. When you have a container failing, you just kill the container and you rerun a new container with replicas. This is great.

I have not used the technical support that Redpanda provides because I use the free version. I develop alone on this. This was one of my previous employment and skills before. I continue by my side. I do not have any subscription and I never had one. I had a subscription on IBM MQ a lot. I have subscriptions on TIBCO EMS and Rendezvous, subscriptions on SonicMQ, and so on, but never on Redpanda. I only use free features, so I do everything by myself. Unfortunately for the support management, I have no knowledge.

The initial deployment of Redpanda is fairly as easy as installing ActiveMQ or RabbitMQ, which is very easy. In complexity, I think the most complex is IBM MQ, really. Mostly, of course, when you do clustering, it is very difficult and high availability, and so on. This one is very easy. I put eight on ten. The premium version has an analysis tool for when you have a license, saying this has to be enhanced, this is too tiny, you need to put it bigger, this is too big, you need to put it smaller for messages, for topics. It explains your ISR is not well configured and so on. So it goes further but it is cheaper than Confluent because Confluent is so expensive. I think it is a good competitor but still young, so it needs more experience to be at the level of Confluent.

Redpanda requires maintenance on my end. Of course, you need to check that you do not explode the size of the disk space. You need to check that the performance is still good with the ISR repartition. It is doing that, but you still need to check that everything is okay on the hardware level of your machine because it consumes resources, of course. I think this needs alerting for the disk space, for the rotation of the partitions in the topics, and so on, just as Kafka does. You need to check that it is still based on the ISR, of course. You need to check that there is no latency. You need to do performance tests sometimes to check that everything is still fast, mostly when you use streams.

I also used the Docker version. I used the installer first and then I migrated to Docker.

It is a difficult question to give a score to Redpanda overall. Unfortunately, you cannot give a score objectively. I will try to be objective. It is difficult to give a score based on what you need. If you want reliability on messages, I would recommend IBM MQ. If you need something that is compared to Kafka for streams, for performance, for failover over the regions, then I would say you can use Redpanda. I would give Redpanda an overall rating of seven out of ten.

We used Redpanda as our main message hub for processing different messages asynchronously and with regards to the consumer's throughput.

The biggest main feature that Redpanda offers is the same Kafka API, so you could always use the already-built libraries and the big environment that is around Apache Kafka as a message broker. There are a ton of libraries in every possible language, and then you could seamlessly plug Redpanda instead of Apache Kafka and use that instead.

Through a lot of benchmarks and whatnot, we found out that on a larger scale, when the latency of the messages is important, not talking about throughput, the latency is important, Redpanda offers a way better latency profile than Apache Kafka. So, we used to have Apache Kafka and then we switched to Redpanda.

When you claim that you are Kafka-compatible, and in our product we use a ton of libraries built for Kafka, and then in some really, really specific niche cases, they had different ideas on how to do something, and that would make the library kind of useless or not work as it's intended. Because in some cases, especially when it was about the way a leader broker is selected and things related to the consensus of your cluster of Redpanda, it was way different than the behavior of Kafka.

One of the reasons that we migrated was back when Zookeeper was a different thing that you had to manage and configure to have a Kafka cluster, but Redpanda was plug-and-play, ready to go. You could have one node or ten nodes and then the consensus and leadership, everything was obvious and ready to go from the start. So deployment-wise, it is a really good product.

The most important thing that Redpanda could do better is, first of all, define the way that they are different from Kafka, and then try to fork some libraries so that we could use those specific libraries built for Redpanda rather than us using the Kafka ones. I would assume that because the differences are very minor but very niche, they could have essentially the same library and the same amount of code that is already there for Kafka and then change it a bit so that you could use the Redpanda client instead of the Kafka client.

In my career, in total, I was at a company for three and a half years and it was a main stack for message brokers, so we can say three and a half years. But me personally interacting with it was one and a half years into the company, so two years.

We had one incident that we concluded was a case of memory leak. Back then there was a similar issue in their GitHub regarding this specific bug that we encountered. After a while, we just patched it with the new version and everything was fine. I do not know about systems with ten thousand microservices and how they would react in that situation, but in our system where the latency and the throughput were way more important with less amount of things integrated with Redpanda, it was fine. We only had this major issue.

It is pretty scalable. We have used the scalability features, and we have added different nodes to the already up and running cluster. As I mentioned earlier about the consensus and the leadership thing that is kind of different from Kafka, we encountered some specific bugs related to that when we wanted to add a new node to the cluster. Overall, I would rate them a seven out of ten. I do not say that is the most scalable message broker there is, because I guess Kafka is way better in scalability, but it is not terrible either.

I do not have technical support. However, they have a Slack community and Slack channel, and we used to communicate with them a lot. In so many cases, the developers themselves answered us because we were heavily benchmarking what they said and what they claimed that they were. At one point, about three years ago, we contacted them and we had this Google Meets meeting that we discussed what we found and what they developed. Not the technical support as in the usual way, but the community and the development support was great.

I have used Apache Kafka as an alternative.

It was very easy. That is one of the reasons that we migrated. Back when Zookeeper was a different thing that you had to manage and configure to have a Kafka cluster, but Redpanda was plug-and-play, ready to go. You could have one node or ten nodes and then the consensus and leadership, everything was obvious and ready to go from the start.

Since we were self-hosting the open-source version, we had to maintain it, but I do not think that the maintenance was a burden. I was more into developing things and integrating things with Redpanda. The maintenance team of infrastructure was a whole separate team, but as far as I know, it was not a big hassle.

Unfortunately, when we were using Redpanda, we were self-hosting the open-source version, so I do not really know about the pricing.

I have used Apache Kafka as an alternative.

We used to use Apache Kafka and then we switched to Redpanda. I kind of appreciate the way they are going with their documentation more than Apache Kafka. However, Apache Kafka, because it is used in a lot of environments with a thousand or ten thousand pods and ten thousand moving parts and all of those things, in those cases I guess Apache Kafka would still surpass Redpanda. But in case of low-latency systems, Redpanda is way better. This review has an overall rating of eight point five.

For my current field, I process data, but for Redpanda specifically, it is more for data that occurs live. For example, in my project, I use Redpanda to get MTA subway feeds live so that I can process them later with another tool such as Spark for partitioning and saving it into S3 buckets for storage.

I do want to mention that there was a part of Redpanda where you can configure and create containers with Docker. That is one of the valuable features. The sub part of it is that you can configure the retention size to a specific number of megabytes, and that way, when you go to the producer or the consumer part, you can see how much messages are being retained. Even if you configure it, you can have the messages there for a week or so, even if you don't have it running.

I would say Redpanda is pretty stable overall. However, in terms of the topics and the retention, it could definitely be improved. For me, I was trying to get a lot of data, but I would only get one day's worth of data, even though I asked how I could increase the retention size. I was told it would be seven days worth, but in practice, it would just be one day's worth. The way it cleans up the old data meant that I would basically have to run the consumer job 24/7. I would say the biggest scalability improvement could be the retention.

I was looking at other features on the UI, but I am not familiar with them. I don't know if there is documentation for them, but there are other tabs on the UI that I am not really familiar with. Maybe because it is hosted locally, I am not really sure. It would be helpful if there was some sort of documentation or an instruction set for how to integrate other software to make the workflow much easier.

I want to say over two years, but this year specifically, I have gained way more insight compared to the previous years.

I have not experienced any stability issues.

I would say Redpanda is pretty stable. In terms of the topics and the retention, scalability could definitely be improved. For me, I was trying to get a lot of data, but I would only get one day's worth of data, even though I asked how I could increase the retention size. I was told it would be seven days worth, but in practice, it would just be one day's worth. The way it cleans up the old data meant that I would basically have to run the consumer job 24/7. I would say the biggest scalability improvement could be the retention.

I have not really thought about it because I am not familiar with other similar products.

It was a little bit difficult because when I was creating the containers through Docker, part of it was trying to figure out whether I wanted the whole workflow to run locally. For me, that consumes a lot of power. So I had a virtual machine run Redpanda exclusively, but the configuration for that, for the local host and all that, was a little bit confusing. Maybe that is just the virtual machine aspect and not really a Redpanda thing. But other than that, I think setting it up was not too complicated. It is just the virtual machine portion of it.

Locally, I would say it was around an hour. Not too much.

In terms of maintenance, not necessarily. I would say it was pretty straightforward. The only nuance to it was making sure the JSON data was compiling properly, and then setting up the retention size.

That is kind of the issue right now. I don't have a company verified email. I am mostly using it for personal purposes with my personal email for everything. I just never got the hang of how to create a company email.

My overall review rating for Redpanda is 7 out of 10.

My main use case for Redpanda is primarily for streaming, and I am currently focusing on building data lakehouses because I find them really interesting.

Redpanda fits into my data lakehouse setup by normally fetching a source of data and mapping and transforming the data, so there are many ways to do that. I am exploring the possibilities of Redpanda, such as mapping the data with Redpanda Connect, or as an alternative, implementing my own solution. There are several ways to approach this. Additionally, I can sync the transformed data into any other destination, which really helps. Redpanda, through Redpanda Connect for instance, makes that really easy, and I am exploring those possibilities.

I have many areas to add about my use case or how I am experimenting with Redpanda. It is not about the streaming itself. It is about configurations, schemas, schema evolution, schema drift, how to map data in many ways, as well as configuration for governance, deployment, security, and control access. There are many areas that I would like to deep dive into and to control and to learn that I find really interesting in general with Redpanda. I think it is a really good software. As an alternative to Kafka, I think it is an amazing drop-in, and with one binary. That is a real advantage because other alternatives are really heavy. I see Redpanda as really light in that sense, so it is an amazing product.

The best features Redpanda offers start with its single binary, which helps a lot. Managing Redpanda through the command-line interface is a really big advantage. Having one binary, a UI, and a dashboard with the configuration helps a lot as well. The command-line interface and the UI are amazing, apart from the configuration regarding streaming, topics, and queues. That is for sure the strongest feature, but for me, it is not about the topics themselves. It is about other areas where I can control, view, and manage.

The command-line interface and the UI have made my work easier by allowing me to deal with topics or with configurations really easily, issuing commands. When I am dealing with CI/CD pipelines, I can issue the commands easily and configure anything. That is really helpful and is one use case, for instance.

Redpanda can be improved in several ways, and the more I experiment with the product, the more limitations I find. I understand that this is about business, and the product should grow, and they have to make money. That is certain. However, I would like to have more freedom in terms of having plugins to operate in other fields. For instance, if I am learning Apache Iceberg and I want to have a connector to map data to or to sync data to Apache Iceberg, that is a paid connector. That is a limitation. I could pay for a subscription, but I am under a testing phase to be sure that this is the product I want. There are many paid connectors and operators that it would be great to have as free alternatives. I can understand it is the business model.

I see positive impacts from Redpanda as it makes all the processes that streaming and mapping have easier. In general, I think it is really easy to install, to manage, and to deal with. It is a really great product.

I could not find a way to query the topics in Redpanda, something similar to what Apache Kafka has with ksqlDB or something equivalent. I think Redpanda added that feature recently, but I am not sure if that is paid or free. I would like to test that as well, just to have topics with data and query for visibility and for other purposes. I would like to explore that, but I am not sure if that is a paid feature or a free one.

I have been working in my current field for more than 15 years.

Redpanda is stable in my experience.

Regarding Redpanda's scalability, I could not test that, so I do not have any opinion on that. I read that it is pretty stable. The performance is amazing. I read about the experiences of other companies using it, and their experiences are great and awesome. However, I could not test that on my own. I hope to do that soon, but so far I could not.

I think Redpanda has really amazing customer support based on my experience and from what I have read. I find the AI guide that Redpanda's website has really awesome. That is really helpful. By asking a question, I can solve many issues, doubts, or any comments that I might have. That is a pretty amazing solution.

I previously used Apache Kafka, as everyone is focused on it, and recently, all the changes that Apache Kafka had regarding Zookeeper and Kraft are amazing. It is always a good option to learn alternatives, to broaden my experience with other solutions, and to experiment with other solutions, just to confirm and state my feelings about the product and to discover new features. There is always an edge to using other products. My motivation was to check an alternative and to learn from this alternative because I had my vision and my experience with Apache Kafka.

I have seen a return on investment and personal gains since I started using Redpanda. For instance, for local projects and to experiment, I find the investment is really amazing because it is really quick to install and operate with. I can experiment in a minute. There are tons of examples.

Regarding my experience with pricing, setup cost, and licensing for Redpanda, I am exploring the product. If I am convinced about the product and the capabilities, and I am sure I will be because I find the product really amazing, for sure I will acquire a license to operate on other paid features if I like them. That is something that I strongly consider for the future, even for a personal project, not for a professional enterprise project. So far, I cannot state any opinion on pricing or setup cost.

I evaluated other options before choosing Redpanda, but mostly queues and not publish-subscriber options such as RabbitMQ or Azure Queues, but not as publish-subscriber. For publish-subscriber, I only know Apache Kafka and Redpanda.

The first advice I would give to others looking into using Redpanda is that it is a binary, so it is just installing, and it is ready to use. That is amazing to start with. Redpanda's page contains amazing information. The AI question-answer solution is amazing as well. I can have a solution with one click to whatever concern I have. That is amazing as well. There is plenty of documentation with examples. I think it is really quick to operate on Redpanda in minutes.

I have additional thoughts about Redpanda. I want to say continue this work. Redpanda is amazing. I love it. Please continue with new features and improvements. I think Redpanda will dominate the market in the future, and it will be a really strong contender regarding streaming and AI. I hope so. You are really doing a great job. Continue with this, please. I would rate my overall experience with Redpanda as a nine out of ten.

I have been using Redpanda at Model Cover for around three years, but I believe you should rely on better information. Redpanda is not the best technology available on the market. Agent technology is currently the trend. Redpanda helps my organization solve problems related to big data pipelines and microservices mostly.

I feel Redpanda performs properly for big data pipelines and microservices. It can handle most of the requirements that are given, but it is still a little bit outdated. There are better open-source alternatives available.

I would like to add that Redpanda is mostly very good for streaming data. This is excellent for streaming data and it is faster than most alternatives, and without JVM, which is beneficial.

Redpanda has positively impacted my organization by enabling MVP fast to production. I can share that it has made things three weeks faster.

I have seen positive outcomes including improved efficiency, cost savings, and better performance metrics because it is open-source, fast, and good. There is also cost reduction.

I think that Redpanda's accuracy and reliability of output are good because it is Apache Kafka compatible, making it very scalable.

I think Redpanda fits enough for what it is made for, so there is nothing to improve at the moment. I chose 8.5 to 9 because I think maybe building something from scratch in my case would make more sense because this is what we ended up doing. However, it takes considerable effort. It depends on the available resources. If you do not have many resources, you can go for it.

I have been using Redpanda for two years.

In my experience, Redpanda is stable.

Redpanda's scalability is proper. It is properly scalable and you can simply put it on a Kubernetes Pod or Docker Swarm and scale horizontally or vertically.

Customer support for Redpanda is open source. I did not have any interactions with their team, which may be an issue.

I did not use a different solution before Redpanda.

My experience with pricing, setup cost, and licensing for Redpanda is that it is straightforward with fast deployment.

I have seen a return on investment with Redpanda because we saved some resources.

My experience with pricing, setup cost, and licensing for Redpanda is that it is straightforward with fast deployment.

I did not evaluate other options before choosing Redpanda. Alternatives such as building from scratch or Kafka standalone were not considered.

Regarding Redpanda's AI capabilities, if your infrastructure is secure, you are fine and good to go. It is really good.

My advice for others looking into using Redpanda is to go for it.

I have additional thoughts about Redpanda: there is an over-focus on the topic, and that is the only feedback I have.

I have given this review a rating of 8.5 out of 10.

My main use case for Redpanda is for event management in infrastructures that are not so robust or oversized that they would need a more voluminous Apache Kafka environment, such as an on-demand retail event management system to register and receive sales lines in a retail flow.

I implemented this retail event management flow using Redpanda to build the most minimalist infrastructure possible to avoid oversizing with Apache Kafka, removing unnecessary artifacts such as Zookeeper management and more complex management related to systems like brokers. Since it was only a simple management of sending and consuming events from consumers and event receivers, Redpanda provided a quick and scalable solution in a reasonable time.

The best features that Redpanda offers include the absence of Zookeeper and self-management of the infrastructure to avoid determining which is the parent node and all resizing in case of a failure, resulting in much more transparent fault tolerance to the user.

I have noticed this transparency in daily use by abstracting through Docker containers or infrastructure such as Kubernetes, where management is very simple: I simply provision the infrastructure and register it, allowing quick scalability without the need for extensive technical knowledge of the infrastructure.

Redpanda has positively impacted my organization by being easy to implement quickly for use cases such as the implementation of MVPs and PoCs, making scalability, tolerance, and availability very interesting and facilitating the provisioning of infrastructure efficiently.

Redpanda needs more visibility and primarily greater access to documentation. Although the user-friendly documentation provided is fairly good, making it easier and better known is important, along with breaking the hegemony of Kafka.

I have been using Redpanda for a few months over the last few years.

I consider Redpanda to be a stable solution.

I would rate the scalability of Redpanda as good, but it is not as complex and customizable an infrastructure as Apache.

I previously used Apache Kafka before implementing Redpanda.

I decided to switch from Apache Kafka to Redpanda not as a complete replacement, as it still coexists in the organization, but it has been optimized in some use cases where Redpanda could be more interesting.

Before choosing Redpanda, I evaluated options such as RabbitMQ, Kafka itself, and some other alternatives that had been considered.

My advice to other professionals considering using Redpanda is to give it a chance, as it is an interesting solution that does not always require opting for more common solutions, being easy to use with a not overly complex onboarding process and a manageable learning curve. I am very happy to have used, tried, and experienced Redpanda. I would rate this review a seven out of ten.

My main use case for Redpanda is streaming once data. I use Redpanda for streaming data by supporting streaming to various data providers, putting it in Parquet format, structuring it, and sending it to the streaming infrastructure.

Redpanda helps us solve Iceberg and data storage with very low latency. The best features Redpanda offers are low latency, Iceberg, and a low infrastructure footprint.

Iceberg makes the biggest difference for my team, and it impacts our daily work by managing the Iceberg process. Regarding the features, it is very scalable and very easy to integrate into the ecosystem.

Redpanda has positively impacted my organization because the process is now easy and we have not used another streaming platform for comparison, but it has been better than Kafka. The process is easier now because the infrastructure footprint is less since it is written in C++, so it needs fewer resources.

Redpanda can be improved as the support they provide for BYOC could be better. They should support the REST catalog for Iceberg better since it is only partially supported in most cloud or hyperscalers.

I have been using Redpanda for a year.

I would rate Redpanda a nine out of ten. I chose nine out of ten because the support and the REST catalog not being there or not fully supported for Iceberg kept it from being a perfect score.

I have never used Redpanda's AI capabilities regarding its governance and security. I have never used Redpanda's AI capabilities to evaluate its accuracy and reliability of output either.

I do not have anything else I think Redpanda could improve or any other challenges I have faced with it. My advice to others looking into using Redpanda is to consider it. I recommend that you do your own analysis with other streaming providers to see if it meets your needs.

Redpanda is a good product that works well and I had no issues. My overall rating for this product is nine out of ten.

My main use case for Redpanda is streaming to deduplicate my jobs.

To give you a specific example of how I use Redpanda to deduplicate my jobs, we connect to vendors, grab files that are on their system, and pass it through Redpanda, which will deduplicate and indicate whether we have already seen this file. If we have, we do not start the job; otherwise, it will proceed to start the job to download those files.

The best features Redpanda offers include the ease of building inputs and outputs.

The ease of building inputs and outputs has made my work easier because I can quickly set up a new input or output without having to write extensive code; I can simply set up the configuration and it runs.

Redpanda has positively impacted my organization by allowing us to move from a batch approach to a more streaming approach for our jobs, which cuts down on our delivery time and allows us to better meet our SLAs for our clients.

One issue I had with Redpanda was debugging an issue where our API would stall whenever we output to our API, and I was not seeing anything in the logs explaining why it was taking a long time. It would be helpful to see trace logging in Redpanda waiting for the output response, which would help debug that issue.

I have been using Redpanda for about a year and a half.

Redpanda is stable.

I have not had any issues with Redpanda's scalability.

I was previously using Dolphin Scheduler, but I switched because I wanted streaming as opposed to batch or micro-batch.

I use Redpanda Connect, and I have not had any issues with the setup cost.

Before choosing Redpanda, I looked at Kafka as well, but I had experience with Redpanda, so I went with that.

My advice for others looking into using Redpanda is to rely heavily on the documentation, as it is very useful. I rate this product 7 out of 10.