I use Apache NiFi for data ingestion from multiple sources into our data lake.

Can't say many positive impacts apart from ease of use for simpler use cases. The rest is mostly tech debt and negative impacts.

What stood out the most about Apache NiFi was its ease of use when it comes to smaller datasets or when you have multiple streaming data sets that need to be ingested in multiple steps. When you don't have a team with a lot of coding background, Apache NiFi comes in very handy.

The ease of use in Apache NiFi has helped my team because anyone can learn how to use it in a short amount of time, so we were able to get a lot of work done. You don't have to build anything from scratch as you do with Spark or any other tool sets because you already have blocks created within Apache NiFi, so you save time. When we talk about how much time Apache NiFi has saved in development, you save considerable time.

There are a lot of challenges with Apache NiFi. I believe it is very limited when it comes to scalability and version control. Various improvements are needed for Apache NiFi; the way the tool is structured needs to improve.

Having version control and metadata access is essential. If Apache NiFi compute could scale with workload through ephemeral scaling, that could help; but there is not much we can extract from the Apache NiFi API. It is a huge pain when it comes to reading Apache NiFi configuration metadata. Apache NiFi needs to be faster, and having version control, metadata access, and some improvements would be beneficial. However, it is a good solution with room for improvement.

I have been using Apache NiFi for most of my career, but over the past two years, I have used it quite a lot.

Apache NiFi is stable in most cases. Downtime with Apache NiFi is not unheard of, so we do run into downtimes every now and then, but for the most part, it is stable.

Scalability for Apache NiFi is a problem; it does not scale as well as other Spark solutions. Scalability is one of the reasons downtime is still a thing on Apache NiFi.

I have had to reach out for help with Apache NiFi, but since I get Apache NiFi from Cloudera, they have their premium support, so it is not that big an issue.

We used an older version of Apache NiFi before Cloudera; we chose to continue with Apache NiFi because we were coming from Apache NiFi.

Complex

We host it on AWS and get it from Cloudera

I haven't seen a return on investment. There may be return on investment based on the technology and easily moving our workloads onto Apache NiFi from our previous system. However, the tech debt that is created from Apache NiFi is not a good addition. It saves time on development, but then you have to cater to some custom solutions.

It is expensive with little to show for it. But I personally don't deal with billing specifics

We evaluated using Spark for all of our ingestions before choosing Apache NiFi, and at that point, we had to build a framework entirely from scratch. That is why we didn't go there, and we went for Apache NiFi.

The process and cost consumption, I believe, is actually not that good when working with Apache NiFi. Apache NiFi is one of the most expensive tools we use. I do believe it is an obsolete technology and the world is moving away from it. I have given this review a rating of five out of ten.