dbt Platform

What do you like best about the product?

What I like most about dbt is that it brings software engineering best practices to SQL-based data transformations, making our SQL code base maintainable at scale. It has a clear model structure like staging, intermediate, and reporting layers. It provides macros and ref macros that make logic reusable, and the dependencies are really easy to understand. I appreciate its good collaboration with Git and integration with version control. Dbt has a strong documentation background, providing an auto-generated documentation site, so everyone is aware of what's happening in the project. The initial setup of dbt is really easy thanks to its great documentation, and it's available for almost all major data warehouses.

What do you dislike about the product?

One of the pain points is debugging and error troubleshooting. Error messages can really be vague, making it difficult to pinpoint which part of the core caused the failure. Also, large models are painful to debug. Query plan visibility inside dbt would be really helpful. Step by step execution for failed models would also be helpful.

What problems is the product solving and how is that benefiting you?

dbt provides a standard structure for our code base, eases data transformation with Jinja templating, organizes SQL scattered across tools, offers version control with Git, and includes data quality tests, making transformations maintainable and dependencies clear.

What do you like best about the product?

dbt runs well on Redshift, since that is what was mentioned over and over again in the notes; however, dbt simply compiles the SQL and the warehouse itself handles the heavy lifting. Using Git and Version Control for Data Models, is nice because it keeps the data model from exploding. dbt also integrates with our AWS infrastructure without requiring tears. The speed is sufficient, as it simply passes the work to the database; although, having the transformation logic in one location is helpful.

What do you dislike about the product?

The cost is becoming increasingly expensive and considering dbt is essentially a fancy SQL Compiler. dbt also has poor performance when handling un-structured data (although this may be due to Redshift); I'm unsure, everything seems to blend together. Additionally, the learning curve is very steep if you are not familiar with Jinja and setting-up YAML files properly.

What problems is the product solving and how is that benefiting you?

dbt allows us to scale the analytics engineering work so we are not running ad-hoc SQL scripts on a laptop. dbt separates the compute and storage logic, allowing us to define the "what", while it determines the "how". dbt automatically manages the dependency graphs, which is great, as I cannot handle tracking those manually.

dbt  is used for data transformation and data engineering with multiple data transformations and engineering functions. It is also used for orchestrating data engineering pipelines. An example of this is ingesting data from Azure  Blob or S3  sources and then transforming it into different layers in the data platform.

The best features of dbt  include lineage and Jinja templating languages that make it easy for creating pipelines.

The built-in lineage feature provides a good understanding of the several layers where data is being loaded in dbt, allowing visibility from different layers into the end product.

dbt has positively impacted version controlling as it has different version control steps involved. The specific improvements seen with version control in dbt are that it has helped trace the data lineage, enabled faster trace and rollbacks, and enabled safe collaboration at every scale, which has improved data quality.

A return on investment has been seen from using dbt as the time has reduced while utilizing dbt in the form of data pipelines and ETL scripting. There is operational efficiency achieved, and data quality and governance have also been achieved with modular SQL and version controlling, which reduced duplication of data and data errors.

dbt is not as stable as preferred, as it has had a few outages in the current year itself, so improvement should be made in the outages section as it is not stable.

The copilot in dbt is not very comfortable for users, and my team has already tried using it but opted to move off from the dbt copilot to other copilots such as GitHub .

Improvement is needed in the tool itself in terms of the copilot, in terms of covering outages, in terms of testing, and in terms of quality reasons related to governance and collaboration.

dbt has been used for about a year.

dbt is not as stable as preferred, as it has had a few outages in the current year itself, so improvement should be made in the outages section. Overall, dbt is stable.

In terms of scalability, dbt has improved the scalability of the organization depending on different dimensions for team size, data, and complexity of transformations.

The customer support from dbt was good and was identified and resolved by the customer support team when reached out to.

Neutral

Initially, multiple solutions such as Talend Studio and Informatica were utilized for different projects before switching to dbt.

The experience with pricing, setup cost, and licensing was that it was straightforward for the pricing setup and also on the licensing part for dbt.

A return on investment has been seen from using dbt as the time has reduced while utilizing dbt in the form of data pipelines and ETL scripting. There is operational efficiency achieved, and data quality and governance have also been achieved with modular SQL and version controlling, which reduced duplication of data and data errors.

dbt was purchased through the AWS Marketplace .

Before choosing dbt, other options were evaluated, but dbt was the preferred choice as it was an open-source solution that was already on the track.

My advice to others looking into using dbt is that it is a good tool for having ETL or ELT transformations done. To begin with, a pilot project can be added with modular SQL or modeling, Git  workflows, and a standardized project structure from source, staging, intermediate, to the mart layers, which will optimize performance. I would rate this solution a seven out of ten.

Public Cloud

Amazon Web Services  (AWS )

What do you like best about the product?

DBT is great for organizing data models. It is user friendly, integrates well with other tools, and they had a great onboarding process.

What do you dislike about the product?

In dbt Cloud, I cant work on two different branches at the same time in different browsers.

What problems is the product solving and how is that benefiting you?

DBT allows us to take raw data from many sources and output it in clean, easy to use output tables that are used in our bi tool.

What do you like best about the product?

The interesting fact about dbt is that it simplifies the process of managing data pipelines. It was implemented successfully and I depend on it on a daily basis and hence my frequency of use is high. The amount of features such as model testing, documentation, and version control is especially appreciated by me. It has minimized errors in our conversion processes and has simplified the process of teamwork a lot and has helped the team maintain pipelines which are uniform and structured across projects.

What do you dislike about the product?

The thing I dislike with dbt is that it may be difficult to troubleshoot model errors. The features are good, and error messages are not always helpful in disclosing the problem. High frequency of use implies that such moments have the capacity of derailing workflows since I use it frequently. There is responsive customer support but edge-case fixes are not always immediately available, so the team occasionally has to check outputs before proceeding.

What problems is the product solving and how is that benefiting you?

Dbt has resolved the problem of inaccurate or inconsistent transformations within our workflows. It has simple implementation and I use it frequently hence my usage frequency is also high. It has many features that can be used to test and keep track of the version that helps in uncovering errors at the earlier stages. It is lean cooperation throughout the team, reduced manual checks that have to be done multiple times, and ensures our data is reliable and can be used in reporting and business decisions.