Databricks Data Intelligence Platform

What do you like best about the product?

As a Data Scientist working for a mid-size company, my main use case for Databricks is as the central engine for all of our data processing and predictive modeling pipeline. I use it every day to pull raw dirty data from our cloud storage, explore it with complicated SQL queries and then create and train machine learning models with PySpark and Python. Basically it gives our data engineering and data science teams a common place to play on the same huge data sets at the same time without having to endlessly exchange files or credentials.From a day-to-day workflow perspective, I love the fluidity of the collaborative notebook environment. The ability to work with different languages in the same workplace is a great advantage. I can perform an optimized SQL query to pull in a hefty data set in one cell, then process it in the next using PySpark, and visualize it with Python libraries straight after. This fully removes the need to constantly bounce between different tools or IDEs. Another big victory for my daily work is the out-of-the-box connection with MLflow. It makes it very easy to roll back to a previous version, automatically tracks hyperparameter tuning, compares several model runs, and manages the full lifespan of a model. I really enjoy how Databricks takes away the effort of managing Spark clusters, you can spin up a distributed cluster with a few clicks, and focus on writing algorithms vs playing DevOps.

What do you dislike about the product?

And despite all its potential, working with Databricks does come with certain daily difficulties. What is most important for a mid-sized company like us is the aggressive pricing model for compute costs. The monthly payment can get out of control very rapidly, if you’re not compulsively watching your cluster configurations and auto-termination settings especially if a high-memory cluster is unintentionally left operating over the weekend. Another major pain point is the built-in Git integration. Databricks Repos has been helpful however managing complicated merge conflicts or branch management still feels unexpectedly clumsy compared to a regular local IDE like VS Code. Lastly, the learning curve is rather severe for new employees. The user interface might be complicated and debugging distributed computing failures can be a major bottleneck for young data scientists getting up to speed.

What problems is the product solving and how is that benefiting you?

The largest basic problem that Databricks tackled for our business was breaking down the separate silos between our data engineers and data science team. We saw this effect in the real world recently when we were working on a project to build a fraud detection algorithm. In the prior approach, I would have to submit a ticket to data engineering, wait days for them to extract and clean the data, and then try to train the model locally. I would get out of date data by the time I got it, and my machine would crash all the time owing to memory constraints. I could immediately connect to our Delta Lake, utilize PySpark to process the huge data size without any memory issues and train the model on a scalable cluster, all in the same ecosystem using Databricks. This one-stop-shop decreased our model deployment duration from about a month to a couple of days, dramatically enhancing how fast we offer actionable business value.

What do you like best about the product?

Databricks provides a unified platform and is very efficient working with large scale terabytes level data. I also like the integration with various cloud services which is seamless and very helpful. Also, the inbuilt Apache spark and very efficient AI/ML workflow orchestration stands out from others. And the databricks support has been outstanding in case of any issues.

What do you dislike about the product?

With features comes cost and using databricks at a scale we use it (terrabytes data, multi customer, multi environment) becomes cost challenging. Also, learning curve can be bit steep for new beginners.

What problems is the product solving and how is that benefiting you?

Our primary challenge was managing large volume data for multiple customers and across different regions. Databricks very efficiently resolved that challenge with it unified platform and very good cloud integration. Our data pipelines are much faster and more orchestrated than ever.

What do you like best about the product?

I like how Databricks helps process and analyze large volumes of data without a lot of Complexity. which saves time and improves productivity.

What do you dislike about the product?

The only downside with Databricks is that it gets expensive with time.

What problems is the product solving and how is that benefiting you?

Auto-scaling clusters have saved us a lot of time. We don't have to worry about managing infrastructure while processing large datasets.

What do you like best about the product?

I run a data science team at a mid-sized company where we handle everything from messy data pipelines to heavy-duty machine learning. Databricks is the core engine of our stack. We use it to ingest raw customer telemetry, clean it up, and run massive PySpark jobs to train our predictive models. We also rely heavily on its MLflow integration to manage our model registry and handle deployments. Essentially, it's the infrastructure playground where all our heavy data lifting happens.The sheer raw performance is unmatched. If you are dealing with massive, bloated datasets that choke local machines or standard cloud instances, Databricks handles them like a beast. The managed Spark environment takes away a massive chunk of the infrastructure headaches involved in setting up clusters from scratch. From a pure data science perspective, having collaborative notebooks where my team can jump in, write Python or SQL concurrently, and instantly visualize data without switching tools is a massive plus. The MLflow integration is also fantastic; being able to track hyperparameters, log artifacts, and register models in the exact same workspace where the data actually lives saves us from fragmented tool sprawl and keeps our MLOps pipelines incredibly tight.

What do you dislike about the product?

The user experience can be deeply frustrating, and the platform often feels like a collection of entirely different tools taped together. The UI is clunky, unintuitive, and constantly changing, which means you waste time just trying to navigate the workspace. Debugging a failed Spark job is also an absolute nightmare—you have to dig through endless layers of convoluted driver and executor logs just to find a simple syntax or out-of-memory error. But my absolute biggest issue is the pricing structure. The billing is completely opaque. They charge you Databricks Units (DBUs) on top of your standard cloud provider's compute costs, and if a junior dev accidentally leaves a high-concurrency cluster running over the weekend without auto-termination strictly configured, you will face an eye-watering bill on Monday.

What problems is the product solving and how is that benefiting you?

Before moving to Databricks, our data engineering and data science teams were completely siloed. Engineers would dump files into cloud storage, and we would struggle to pull that data, map schemas, and train models without running out of memory. Databricks fundamentally solved this fragmentation. For instance, we recently built a real-time recommendation engine where we needed to process millions of daily user events. With Databricks, we built an end-to-end pipeline that handles the data engineering, trains the model, and exposes the model registry to our production environment under one roof. It cut our time-to-production from months to days, which, despite the UX headaches and high costs, makes it a necessary evil for a company handling data at our scale.

What do you like best about the product?

As a researcher of AI, it seems like infrastructure is the number one problem, especially setting up clusters, building drivers, and scaling distributed training. Databricks takes care of all that by itself. I can easily and quickly deploy a cluster of nodes for GPUs with PyTorch and DeepSpeed preconfigured in a few clicks. This built-in MLflow is a lifesaver to keep track of experiments. All the hyperparameters or architecture changes with respect to an embedding model are automatically being tracked every time. ESSENTIAL: I no longer have to struggle to get clean and versioned datasets from data engineers for training purposes when working with Delta Lake. Getting around those feature stores is also very easy with the Unity Catalog.

What do you dislike about the product?

First, it's really expensive, brother. On an extremely large A100 GPU cluster, if you, or someone on your team, forget to configure the auto-terminate, you are going to have a very bleak day with finance tomorrow. Expenses can add up quickly. Additionally, although they are too lightweight to be an ideal platform for distributed deep learning, the debugging workflow may be tedious. The intersection of the computing nodes makes it difficult to find the exact PyTorch-Out-Of-Memory or CUDA-Out-Of-Memory error occurring in the Spark logs. I also feel like the native MLflow UI in Databricks isn't as advanced and specialized as some of the tools like Weights & Biases.

What problems is the product solving and how is that benefiting you?

It fills the oceanic yawning void between research in AI and data engineering. To get terabytes of unstructured text data pre-trained in the past was a multi-step nightmare in different environments. I can make heavy data preparation using Spark and immediately switch to Python for training my model in the same ecosystem. It helps to communicate goodwill amongst the entire team. Everything is in one workspace, so my transition of raw data to experiment tracking to finally registering the model in the registry is done in one, unified, pipeline.

What do you like best about the product?

I work as a Data Analyst and every day, I use Databricks to complete my data tasks. The best thing I like is the processing speed. We were loading large tables and it was taking too long before we could load big tables using normal databases. My rich SQL queries are very fast in Databricks due to the use of Apache Spark backend.

In addition, the Notebook feature is quite useful to me. I can create SQL code in a cell and in the next cell, I can write Python or Pandas code to do some particular data cleaning. It is also easy to connect Databricks to our Power BI dashboards.

What do you dislike about the product?

There are some things which I am facing issues with. First is the cluster starting time. In the morning, it takes 5-10 minutes to boot but once I log in. When the management requests urgent report, then I must make myself sit and wait till the cluster turns green.

What problems is the product solving and how is that benefiting you?

Its primary issue that it is resolving is the ability to process large volumes of company data without system freezing. Previously, it was a pain to deal with millions of rows. I can now easily query, filter and aggregate large datasets.

It is helping my team since data engineers and data analysts are sharing the same workspace. In case data engineers make a new table, I can see it right away and can query it directly in my notebook. Using notebooks to share with other team members to have it reviewed is similar to using Google Doc, which makes my everyday reporting work incredibly quick.

What do you like best about the product?

We have a very big dealer and distributor network throughout India in our sanitaryware business. Big sales data are being generated everyday. Speed is my favorite thing about Databricks. Whenever we use simple excel or old software before, it is always hanging. And now our company data team is churning through the millions of rows in a short time. As a Senior Sales Specialist, I am making sure that I get my territory dashboard and forecasting reports at least daily in the morning. It is bringing all disperse data together in a good manner.

What do you dislike about the product?

Worst thing is that it is highly technical software. As a sales person, I cannot apply it in locating data directly. I need to request data engineering or IT team to code or make query every time I desire some new custom report. User-non technical interface is becoming very complicated. And my management is continually saying this costs a great deal.

What problems is the product solving and how is that benefiting you?

We are solving target tracking and inventory matching problem. We have a wide range of products, such as tiles, faucets, and washbasins, in Cera. Databricks is assisting our company to understand what region is selling what product more and the trend in the market. My advantage is due to this, because I can advise my local dealers accordingly, as to next month order. It is providing highly precise sales forecast and saving me my manual reporting time and I am closing my sales targets with ease.

What do you like best about the product?

Databricks delivers excellent performance when working with large volumes of data. The Databricks SQL environment also makes it easier for business users and analysts to explore insights without having to rely so heavily on engineering teams. Features such as Unity Catalog strengthen governance and simplify managing access across departments.

What do you dislike about the product?

The learning curve can feel steep, especially for users coming from traditional data warehouse solutions. In my experience, query optimization can also take some extra effort, since certain workloads require additional tuning to get the best results.

What problems is the product solving and how is that benefiting you?

We needed a solution that could handle both data warehousing and advanced analytics without creating new data silos. Databricks has helped us centralize our data assets while still maintaining strong governance. As a result, teams can access trusted data more quickly and build reports with fewer delays, which has improved decision-making across the organization.

What do you like best about the product?

What I like most is how Databricks brings data engineering, analytics, and machine learning together in one environment. Our teams no longer need to jump between multiple tools to build pipelines, analyze data, and train models, which keeps work more consistent and streamlined. The notebook experience is genuinely collaborative and helps us move from exploration to development much faster. Integration with Spark and Delta Lake also makes it easier for us to process large datasets efficiently and stay organized as projects grow.

What do you dislike about the product?

The platform can feel overwhelming for new users due to the sheer number of features available. Some of the more advanced configurations also require a solid understanding of cloud infrastructure and cluster management, which can add to the learning curve. Cost monitoring needs close attention as well, especially for teams that run large workloads on a frequent basis.

What problems is the product solving and how is that benefiting you?

Databricks has helped us modernize our data platform and replace several disconnected tools. We now use it as a single place for ETL processing, analytics, and machine learning workloads. As a result, our operational complexity has gone down, and collaboration between data engineers and analysts has improved.

What do you like best about the product?

For me and my team Databricks brings data engineering as well as analytics and machine learning into one platform . Genuinely speaking it makes our work easy with large datasets . their pricing is also good

What do you dislike about the product?

Honestly saying some advanced features of Databricks can be difficult to configure initially or simply I should say the initial setup can be complex for beginners

What problems is the product solving and how is that benefiting you?

For our team it helps manage and analyze large datasets efficiently I must say It improves collaboration between teams and makes it easier to generate insights for business decisions .