My main use case for Upsolver is during an IT consulting project for a large enterprise running a cloud-native data platform on AWS. I used Upsolver to ingest and process high-volume stream data from web, mobile, and microservices sources from Amazon Kinesis with semi-structured JSON and frequent schema changes. The goal was to deliver near-real-time analytics on S3 and Redshift while reducing the complexity and fragility of existing custom Spark pipelines.
A specific example of how I used Upsolver in that project is that it handled the schema changes seamlessly. A new or modified JSON field did not break pipelines, which significantly improved stability in an agile environment. I used Upsolver for automatic schema evolution, and it was very useful for us.
One of the best features Upsolver offers is the automatic schema evolution. Another good feature is SQL-based streaming transformations. Complex streaming transformations such as cleansing, deduplication, and enrichment were implemented using SQL and drastically reduced the need for custom Spark code.
My experience with the SQL-based streaming transformations in Upsolver is that it had a significant positive impact on the overall data engineering workflow. By replacing custom Spark streaming jobs with declarative SQL logic, I simplified development, review, and deployment processes. Data transformations such as parsing, filtering, enrichment, and deduplication could be implemented and modified quickly without rebuilding or redeploying complex code-based pipelines.
Upsolver has impacted my organization positively because it brings many benefits. The first one is faster onboarding of new data sources. Another one is more reliable streaming pipelines. Another one is near-real-time data availability, which is very important for us. It also reduced operational effort for data engineering teams.
A specific outcome that highlights these benefits is that the time to onboard new sources is reduced from weeks to days. Custom Spark code reduction reached 50 to 40 percent. Pipeline failures are reduced by 70 to 80 percent. Data latency is improved from hours to minutes.
I think that Upsolver can be improved in orchestration because it is not a full orchestration tool. I believe it could be better in this regard. The cost needs attention at a very large scale. I think improvements regarding cost are important. Upsolver may be less suitable for very complex batch transformation, which could be an area to improve in the future.
I have been using Upsolver since 2023.
In my opinion, Upsolver is stable.
Upsolver's scalability is good because it demonstrated strong scalability in production environments. As a fully managed cloud-native platform, it automatically scaled to handle increasing data volume and throughput without requiring manual intervention or infrastructure tuning. During peak loads, the system was able to process higher event rates while maintaining stable latency.
I did not need the customer support, so I do not have experience with it.
I did not previously use a different solution. This is the first time for me that I use this kind of solution.
My company does not have a business relationship with this vendor other than being a customer.
I have seen a return on investment because we reduced a lot of Spark code, and the time on onboarding new sources was reduced from weeks to days. The data engineering operational effort decreased by 30 to 40 percent.
My experience with pricing, setup cost, and licensing was a very good experience, but it is not a direct experience because it was not my responsibility. It was in charge of the customer. However, in general, it was a very good experience.
Before choosing Upsolver, I did not evaluate other options.
My advice to others looking into using Upsolver is that I would recommend it to organizations working with streaming and semi-structured data in the cloud. I would rate this product an 8.5 out of 10.
