DataHub

My main use case for Data Hub is to catalog the dataset across my company and to get the lineage of data in the my company pipeline.

To give an example of how I use Data Hub in my day-to-day work, suppose the data is flowing from a source to Kafka and then to some data storages. If some cross-team wants to use the data but there is a problem at the Kafka level, we are not sure who all are consuming that data. Data Hub is very useful for us in this scenario. It can generate the lineage from source to destination, and when there is an issue at the Kafka side, we will get to know what the end results and impacted data sources are.

I would add that sometimes when we do not want to share the data or when the customer or another team wants to consume the data, we are not sure what kind of data is there. We have to look at the schema. Data Hub is useful for us as we are doing the cataloging of all the datasets across my company, allowing us to later use and see the table information and schema information so that the team can identify what data is PII or non-PII.

The best features Data Hub offers include support for cataloging and lineage very well, as we are getting all the different types of connectors to consume and use across the my company dataset pipeline. Apart from that, the GraphQL APIs provided by Data Hub are very good, allowing us to get all the information we need programmatically whenever we need it.

Regarding how the GraphQL APIs help my team in day-to-day tasks, we sometimes use custom logic to check whether the data has PII or non-PII. We have some AI model running on top of it, which requires classification. Based on the dataset URL, we are getting information about the dataset using the GraphQL APIs. GraphQL APIs are very handy, allowing us to customize properties and pass on the necessary information. For example, if we need a structured property, we can get those structured properties. If we need tags or owners, we can retrieve that as well.

Data Hub positively impacts my organization by enhancing collaboration as previously, we had to ask the team to provide the schema information. my company operates in a cross-region environment, so a person in India could wait a day to receive information about the schema from someone in the US. However, with Data Hub, we have a centralized place where we can access all the schema of the datasets, making it very helpful. Additionally, whenever there is a problem, using the lineage helps us quickly identify the impacted team or dataset.

Whenever there is an incident, we first go to Data Hub to see the downstream teams impacted and stop any jobs running on those datasets. It helps us save around eighty percent of time, as we previously had to track down information manually to find the owners, but using Data Hub, we can tag the owners of the datasets directly in the tool.

For improvements to Data Hub, I feel the security is a bit on the weaker side. We have ingestion jobs that require exact permissions for different owners, but this setup does not align with the my company grouping system. We need to create some custom grouping to manage those permissions. I would appreciate it if there were a method to consolidate all the information on a single page, which would simplify sharing permissions for running ingestion jobs.

Additionally, I do feel that the metadata test we run daily takes too long. Initially, it takes one day, which I find excessive. Ideally, we should get information within one hour. These are the two main issues that would benefit from improvement for our use case.

I have been using Data Hub for one and a half years.

Data Hub is stable in my experience. However, there are times when we attempt to upgrade it, and it may go down for a couple of minutes, but not more than that.

Data Hub handles scalability effectively, accommodating growing data and users.

I have had to reach out to Data Hub customer support multiple times. For example, when we were setting up a private link to connect to Data Hub GraphQL APIs, we required our account to be whitelisted. I have also requested some future features for our use cases. For instance, when working with a metadata test scenario, I needed to have a range date column, which was not available. I requested the Data Hub team to make it public so we could use it.

I have seen a return on investment with Data Hub. For instance, I have noticed time savings during incidents and while looking up schemas. In terms of resources, Data Hub centralizes data cataloging and classification, saving us from having to disclose PII column information to teams not utilizing it. Regarding financial metrics, I do not have specific metrics available.

Before choosing Data Hub, we looked into Unity Catalog from Databricks, but we ultimately decided to stick with Data Hub.

My advice for others looking into using Data Hub is to use it for cataloging, classification, and centralizing all your schema. Data Hub supports a variety of connectors and has excellent lineage options. Additionally, make sure to utilize the well-written documentation that can guide you in building your product solutions. I would rate this product a nine out of ten.

My main use case for Data Hub is that we use it as a library for all the data assets that we generate. It serves as an internal data mart where people can search for whatever data they need, and they can search by tags, by roles, and then add more metadata to it. This provides visibility to the data.

A specific example of how my team uses Data Hub in a real-world scenario is that we collect and manipulate a bunch of data layers. Because we have huge teams, the exposure to data that we have already manipulated can sometimes be hindered when using traditional systems. Data Hub acts as a search engine for all of the data. One example would be when the marketing team was looking for specific data around marketing. They discovered that once they searched it on Data Hub, it was easily visible. They did not have to retrieve it from the raw layer and manipulate it for their usage because another team had already built it.

Regarding how my teams interact with Data Hub, we use Data Hub with a self-hosted system. We have connectors which look into multiple data sources, manipulation engines, and orchestration layers to gather the metadata, and then that is pulled into Data Hub. This is how we get data assets in Data Hub.

The best features that Data Hub offers include primarily data discovery and data governance. Data Hub has data catalogs, which helps with the business glossary, ownership tracking, and lineage. Lineage is something that we are strongly using at this point in time. It helps us understand the impact analysis, such as what breaks if I change this column. Data Hub also provides data observability, helping us understand what data is fresh, what is not, and what has changed schema recently. Additionally, it makes our system AI and LLM ready.

The lineage feature has changed the way my team works and collaborates significantly. Because we now have data lineage through Data Hub, if we have a really huge dependent pipeline with multiple layers of upstream and downstream dependency, and something breaks in the downstream system, we can exactly pinpoint what all data assets would be affected. Having that lineage functionality helps us drill down what needs to be debugged and fixed and what exact part is breaking. It saves us time in remedying the issue.

I really like the integrations that Data Hub provides. Data Hub has a very large set of integrations that we can do with Snowflake, Databricks, BigQuery, Redshift, DBT, and Airflow.

Data Hub has positively impacted my organization as teams can now be directly dependent on one source of truth for all their data needs. The time spent finding information has become significantly smaller, which is the real productivity improvement that I have seen, impacting multiple teams throughout the organization. I estimate that we save about thirty to forty percent of the time now since we do not have to read documents or message people for specific data assets. This results in a productivity increase of around thirty to forty percent in terms of time and efficiency.

I think Data Hub can be improved by supporting the open source version better. Many features have moved to the paid version now, making it difficult for small-scale companies to operate on Data Hub because we are required to pay, even though it started as an open source project that is now essentially behind a paywall.

One needed improvement for Data Hub would be stronger AI-powered metadata discovery. I understand Data Hub has been investing in AI, but the natural language processing power on Data Hub search is not that good. The search itself is not accurate many times. Another improvement could be enhancing the DBT developer experience, such as surfacing DBT test failures directly in lineage. Additionally, when we change schema, if it could provide a risk scoring of some sort, that would also be beneficial. Lastly, automated cleanup recommendations would help because managing orphan data assets on Data Hub currently takes a lot of manual time.

I have been using Data Hub for a year.

Data Hub is pretty stable in my experience with no downtime or issues.

Data Hub's scalability has been effective, handling our organization's growth and data volume well.

I have not had to reach out to customer support.

I did not previously use a different solution before Data Hub.

My experience with pricing, setup cost, and licensing has been pleasant, and I have no complaints.

Before choosing Data Hub, we evaluated Atlan and decided on Data Hub because it has a cleaner UI and also a decent open source community to support it.

Data Hub does most of the job it is designed to do, but there could still be improvement as the industry progresses, particularly around metadata discovery. Regarding Data Hub's AI capabilities, its governance and security do the job really well as of right now. I do not have any complaints, especially around data classification, as it allows us to have control over whatever data we are displaying, including customization for PII, sensitive, and financial data. Data Hub has met our expectations regarding its accuracy and reliability of output, and there have not been any issues.

My advice to others looking into using Data Hub is that it is a pretty nice product right now with easy integration. The pricing model could be negotiated, so it is essential to keep that in mind. I would rate Data Hub a solid eight on a scale of one to ten.

My main use case for Data Hub is to enrich the metadata to classify for PII data. As an administrator, I crawl a number of data sources and bring the metadata into a single place, then assign the ownership, such as a data owner or steward, for all the data assets. With their help, we classify the data into PII direct and indirect, sensitive, non-sensitive, and so on. We add tags and glossary terms onto the data elements. The main use case is for DSAR compliance; for GDPR DSAR compliance, we try to identify the PII data in the catalog so that we know where the PII data is in our data inventory.

The catalog helps with metadata discovery and to find the owners/ stewards of data sources. Without a data catalog, we scramble around and speak to multiple teams, which is time consuming.

The best features that Data Hub offers include the management of ownership, with standard out-of-the-box ownership such as business owner, data steward, or technical owner, which is relevant for us. It also integrates with Active Directory. In our Active Directory, we maintain certain roles based on the scrum teams related to a team member, and by integrating with Active Directory, we are able to bring the same roles and map them to the corresponding ownership roles within Data Hub. Data Hub has integrations with Slack, Snowflake, BigQuery, and so on, which we use.

Data Hub has positively impacted our organization by bringing the tribal knowledge that resides with team members into a single place where users can discover and understand the data elements before they make use of it. Users can ask questions via Slack to understand how a data element is defined and get the answers back. This definitely saves time; without a data catalog in place, users need to ask around to find out what a particular data element means and to find out the owners. Now, with the data catalog, searching and discovering data elements and the corresponding owners is easier, saving approximately thirty to forty percent of the time that would have been spent finding out the owners and definitions of the data elements.

Data Hub can be improved with more automation; there are some inbuilt automations, such as documenting definitions of data elements using AI, which is useful. I wonder if it can automate the classification exercise, possibly using AI to auto-classify PII direct and indirect items.

Just started using it.

Data Hub is stable.

Data Hub shows scalability in terms of the number of users and the number of new databases and data elements.

We have not gone that far with customer support; as far as the POC is concerned, we received good support from the team and the sales team that helped us evaluate the tool.

Neutral

We previously used OvalEdge as our data catalog before switching to find a tool that has more AI capability and allows extension of usage to non-technical users, seeking a tool that is less clunky and more intuitive.

slightly techical. But there is enough documentation available.

No

I have not yet seen a return on investment, and I do not have that information to share.

Regarding experience with pricing, setup cost, and licensing, I think if we have a budget of one hundred thousand US dollars, we will be able to deploy a reasonable version and connect to a number of data sources.

Before choosing Data Hub, we compared it with Atlan and Alation.

I chose seven out of ten because there are better catalogs available in the market that offer more features. The UI, especially when setting up new data sources and crawling them, is a little cumbersome, but it is a one-time activity, so it is manageable; however, the UI could be improved concerning administration.

My advice to others looking into using Data Hub, also known as Acryl, is that it is a reasonably stable product that satisfies most data catalog use cases; however, Atlan appears to be the closest competitor, while Alation is the market leader among the three. Data Hub has an open-source version I believe, and it may be worth considering that option as well.

I rated this review seven out of ten.

We adopted Data Hub in the context of a large enterprise customer operating in a regulated industry with a strong focus on data governance, data discoverability, and ownership clarity across multiple cloud-native platforms. The solution was deployed on AWS, and the main business problem was the lack of a centralized, reliable view of data assets, including poor data discoverability, unclear data ownership and stewardship, limited lineage visibility across ingestion and transformation layers, and high dependency on tribal knowledge held by a few individuals. Data Hub was selected as an enterprise data catalog and metadata backbone with the goal of enabling both technical teams and business users to easily understand, trust, and reuse data.

We used Data Hub to create very good data discoverability, assign data ownership and stewardship, improve data quality processes, and establish good data governance for our customer in terms of data catalog, data lineage, and metadata management in general.

Our key benefits that we achieved include centralized metadata management across multiple AWS services and data platforms and improved data discoverability, significantly reducing the time required to find relevant data sets. Clear data ownership and stewardship improved accountability and collaboration between teams. End-to-end lineage visibility enabled faster impact analysis and safer changes, and faster onboarding of new data users through self-service access to documentation and metadata. From a governance perspective, Data Hub became a single source of truth for metadata, supporting both compliance requirements that are very important in a data governance environment and day-to-day operational needs.

The main strengths we experienced with Data Hub are a strong metadata model and its extensibility because Data Hub offers a rich and flexible metadata model that adapts well to complex enterprise scenarios. Excellent lineage capabilities are provided because the lineage visualization is clear, actionable, and extremely useful for impact analysis and governance workflow. The open source foundation with enterprise readiness is significant because the open architecture avoids vendor lock-in while still being suitable for production-grade environments.

Data Hub is very effective for us because we build the data lineage from the beginning, from origination to visualization, to the final use of the data. We follow and track a path of the data, which improves analysis and enables us to find where data is used and the impact of deleting data. This is also very important in a regulatory environment.

The impact is very positive, and there are many benefits for us using Data Hub because it was easier to make data governance, create centralized metadata management, improve data discoverability, and manage data in general. The areas for improvement, in my opinion, are the initial setup and configuration that can be complex without prior experience, especially in large-scale environments. User experience for non-technical users could be further simplified, particularly around advanced metadata concepts. The out-of-the-box governance workflow, for example, approvals and certification, could be more prescriptive for customers at early maturity stages.

Data Hub can be improved in the initial setup and configuration that is somewhat complex, and also in operational monitoring that could benefit from more native dashboards and alerts. However, these are not blockers, but areas where additional guidance or product enhancement would further accelerate adoption.

I have been using Data Hub since 2023.

Based on internal measurement and feedback from the data teams, there are many impacts. Time to locate and understand a data set was reduced by approximately 40-50 percent. Manual documentation effort was reduced by around 40 percent. Dependency on senior data engineers for data explanation dropped significantly. Data onboarding time for new team members decreased from weeks to days.

I would rate this product a 9 out of 10. I chose nine because Data Hub proved to be a robust, scalable, enterprise-ready data catalog that is well-suited for AWS-based architecture and complex organizational environments. It is always possible to improve and useful to maintain space for further optimization.

My advice is to use Data Hub to move from fragmented metadata and manual processes to a modern, governed, and self-service data ecosystem, delivering clear value in terms of efficiency, cost saving, and data trust. We would confidently recommend Data Hub to organizations looking to improve data governance, data discovery, and metadata management on AWS.

My main use case for Acryl Data is analytics.

Acryl Data helps with processing large amounts of data as it is a very good tool that gives good flexibility to store a huge amount of data and is easier to use. The UI is good.

The best features Acryl Data offers include storage. When I mention storage, I refer to its scalability.

The positive impact of Acryl Data is that it has increased efficiency.

I do not have comments on how Acryl Data can be improved.

I have been using Acryl Data for two years.

Acryl Data is stable.

Acryl Data's scalability is good.

The customer support is good.

I did not previously use a different solution.

My experience with pricing and setup was good.

I have seen a return on investment as it has saved time.

Before choosing Acryl Data, I did not evaluate other options.

My advice to others looking into using Acryl Data is that they can use it. I gave this product a rating of 9.

My main use case for Acryl Data is to extract insights from customer data. I use Acryl Data for a project in order to identify all the customers and find out which customer buys a lot of items.

The best feature Acryl Data offers is the simplicity of the UI. The UI is simple for me because it is easy to navigate. Acryl Data has positively impacted my organization by speeding up all the development. It sped up development because the team can access data faster, improving speed by approximately 50%.

The product cannot be improved in just one area. There are no points in support or documentation that require improvement. There are no improvements needed for Acryl Data that I have not mentioned yet.

I have been using Acryl Data for five months.

Acryl Data is stable.

I think the scalability of Acryl Data is a good point.

The customer support is fine; we do not need any customer support, but I think it was fine.

I did not previously use a different solution; I have no experience with any other solutions.

I have seen a return on investment through time saved and also money saved. I do not have specific numbers or examples about the time or money saved.

I did not evaluate other options before choosing Acryl Data; I evaluated only this option.

My advice to others looking into using Acryl Data is to start faster with the analytic insights. I would rate this product a 10.