DataHub

My main use case for Data Hub is data lineage tracking. With Data Hub, we track multiple sources, ingestion sources, and different sources where the data resides in S3 . We bring all that metadata into Data Hub to track lineage on the data ingestion patterns that we perform or transformations that we do, and how they move from different tables or assets or the data pipelines. Whatever transformations we do with Spark and S3 , Snowflake , all those are being tracked via Data Hub. We have S3 buckets and Snowflake  tables, and all those lineage tracking is managed through the platform.

My main use case is mostly covered as we used Data Hub for metadata tracking and lineage for whatever transformations that we do so that we can track each transformation down the line.

In my experience, the best features Data Hub offers include lineage tracking, which is mostly on the asset level, a good glossary, and good connector support.

Regarding asset level and the good glossary, we need the glossary of our products so that it is easy to track which product, what went at what time on that particular product, how many assets are related, and so on. For asset integrations, Data Hub makes it easy to ingest all that metadata of those particular assets from S3 via connectors, which is quite easy. It has good connector support, although limited in some cases.

Overall, Data Hub is a good tool. If we talk about lineage, metadata, and observability on some high level, including domain descriptions, PII classification, datasets, and keeping datasets in one place along with policies, it is good in that particular sense. We do have a plan based on project-to-project usage, but in some of the projects, we do use Data Hub as well.

I would like to add that for the connectors, there is sometimes limited support for using wildcards to get the items or assets ingested from sources like S3; it does not support very good wildcard filters. Additionally, Data Hub has a problem with column-level lineage support, especially regarding non-pro users or those without any plans. If I talk about the free features of Data Hub open source, those two I found could be improved during my use case.

Regarding improvements needed for Data Hub, I have already mentioned the limitations on the usage of wildcards in the ingestion or connectors; that can be worked upon, especially regarding the open-source part of Data Hub. The rest is that I hope the UI is quite good.

I used Data Hub for one and a half years.

My advice for others looking into using Data Hub is that it is a good tool if you want to capture all that metadata, lineage, keep track of governance, security, and observability. It just depends on how you want to use it; you can choose the open-source version or the paid version and subscription-based model. The paid versions have more features, but open-source Data Hub, which most people will try to go for, has some limitations, such as the missing column-level lineage with Spark. You need to consider those points, but overall, it is good. I would rate this product an 8 out of 10.

My main use case for Data Hub involves integrating our HR system or Active Directory, which automatically pulls in all 2,000 workers and groups them into their respective project squads and R&D teams. Each team gets its own team profile page in Data Hub, which helps solve the classic corporate headache of determining who to ask for specific information.

When a team builds a data pipeline, a Kafka topic for telecom signals, or a dashboard, it is tagged explicitly with their team profile as the owner in Data Hub. This means that if a developer in Split , working in the same company, needs to find a specific network dataset, they do not waste days spamming Slack channels; they can simply look it up in Data Hub and find the team profile that owns it along with the direct contact info or Slack channel.

Additionally, it enables us to run a data mesh model with 2,000 people, allowing one central IT team to manage everything while Data Hub facilitates splitting the company into logical domains such as electronic health, telecom networks, IoT, or smart cities.

The best features that Data Hub offers include the ability to centralize everything in one platform, such as creating profiles and organizing them into separate domains like engineering, health teams, supporting teams, and HR teams. This allows information to be shared across different domains.

Utilizing the data mesh model enables the company to maximize functionality using a single solution. Data Hub supports collaboration between different teams and departments significantly, as evidenced when we created various data mesh modules and established different domains such as E-Health, telecom networks, and IoT. This allowed us to share datasets effectively, and with authenticated users, the communication and responses were much quicker.

Among those features, I find the collaborative aspects the most valuable in my work because it has greatly improved our operations over the past year. We evaluated various licenses and methods to integrate data catalog platforms, ultimately deciding to move forward with Data Hub since it was more compatible with our company's security requirements. Compared to other tools, it received better support from the community, which is updated daily, allowing us to collaborate effectively through contact sharing.

Data Hub has positively impacted my organization by functioning as an all-in-one solution. It uses data mesh and separates domains to manage privileged access based on user validation, allowing us to share data sets across the company, which informs everyone about internal regulations. Furthermore, it significantly aids new joiners in understanding the operations and knowing who works on specific projects, while also providing updates on changes occurring within various sectors and domains.

The frequency and quality of updates or new features released for Data Hub have been impressive. This extensive community support was a key factor for us at Ericsson Nikola Tesla to choose Data Hub as our data catalog.

Regarding how Data Hub can be improved, I believe they should focus on enhancing their marketing efforts. Within our company, we were unaware of the Data Hub platform while searching for data catalog options that offered strong security and collaboration. Better marketing would help other companies learn about this effective solution.

My rating of eight rather than a nine or ten pertains to the connections with different systems. Specifically, the integration with Slack and Azure , as well as how we link our HR system to Data Hub, could be improved for better compatibility.

Integrating Data Hub with our existing tools and systems was not very easy, which is why my rating is an eight. We attempted to incorporate our HR system with Data Hub, aiming to set governance status for the 2,000 employees in our organization, but I did not complete this aspect before leaving the organization.

I have been using Data Hub for at least six months at the company called Ericsson Nikola Tesla in Zagreb, which has a massive operation with an entire ICT and R&D division of around 2,000 workers.

In terms of scalability, I believe Data Hub performs exceptionally well as more teams come on board, making it efficient for large organizations with approximately 2,000 employees. It adequately supports the scalability of data sets and the implementation of data mesh models.

During implementation, the documentation and support resources from Data Hub were very helpful. I followed the guidelines, accessed each section, and understood the platform effectively, which made the initial setup easy.

Data Hub is flexible, optimistic, and user-friendly in terms of its interface and experience. I rate Data Hub an eight on a scale of one to ten.

The learning curve for new users adopting Data Hub is addressed through their learning section that guides users on how to navigate the platform. I found it quite simple and effective to follow.

We purchased Data Hub through the AWS Marketplace .

As for specific outcomes or metrics, I currently do not possess numbers since we are still in the early stages of implementing Data Hub within our company. However, the HR department reported significant time savings in completing tasks before and after adopting Data Hub, which has resulted in faster completion and better collaboration without interrupting others.

Data Hub has worked for me personally, as I noticed that after we began ingesting Data Hub into our Ericsson Nikola Tesla company network, it proved to be incredibly helpful for easier access to information. By positioning team profiles at the center of Data Hub, it prevents the duplication of data sets, accelerates onboarding for new engineers, and fosters more connected and collaborative teams within our large employee base. Personally, it has helped me specify tasks and has contributed to the company's progress with the data catalog we chose.

My advice for others considering using Data Hub is to understand how it works and explore its integration potential within their organization. Engaging with community support can also be beneficial, as the team's collaborative approach is impressive.

My main use case for Data Hub is to build data products inside Natura; primarily, I built data products for CRM , which is customer relationship management, and also for some data products for the product field, such as analytic fields. I used it by dividing the company into domains, and each domain has its own functionality and its own structure. With that approach, I used it extensively for building domains. I also used it to build data lineage across the entire data journey, from the ingestion of the data to the use of the data in the final part, such as in a data product or in a dashboard.

A specific example of how I used Data Hub for building domains and data lineage is the domain called GenAI, which is primarily built for products based on AI, mainly generative AI. To accomplish this, I used Data Hub to track the data from the ingestion field. I used some CDP tools such as Segment  and I also have data in an S3  bucket that was ingested to Databricks  using Airflow . With that setup, I track this lineage from the origin system. After that, I performed a lot of transformation of the data inside Databricks  to clean the data and conduct some data augmentation. After that, this data is used to train some models using Databricks LLM. With that, I ingest all this metadata into Data Hub and I can see from where the data is coming from and to where the data is going. This is primarily for LLMs to help consultants at the end of the product.

The best features that Data Hub offers include the capability to make conversational questions inside the platform, which I believe is the best thing that they built in the past year. It is also easy to connect different data sources. Since data lakes, I have connectors to some databases and also to some business analyst tools and other tools. I can connect many types of data inside Data Hub and see what is going on and how we govern the data. Data Hub is a pretty good tool for that. I also value very much the open-source version because it is free and everyone can use it.

I do not have much experience using the conversational questions feature, but I do not need to go to the asset to see from where the data is coming from and where it is going. I can simply ask, 'How can we calculate the sales in this month?' and Data Hub will identify which table will be used and from where this data is coming from and where this data is going. This is very effective.

Data Hub has impacted my organization positively by helping us build a data governance environment and share the knowledge about the data for the entire company. As we used the open-source version, we have no limitation in how many people can use the tool, which is excellent. I can conduct many tests and test as quickly as possible. It is very good for building POCs, for example. Data Hub also helps to give this understanding of the data for the entire company. Everyone in the company can see the data and know where the data is coming from and where it is going. I believe it is very effective and all the people in the organization, not only the data field personnel, can understand more about the data and also help to build better products.

I believe Data Hub could provide more functionalities in the free version. I understand that we have to pay the persons who build the platform, but the free version has some limitations. Some capabilities of the paid version being included in the free version would be beneficial. Another improvement that is needed in Data Hub is how I can get data from Data Hub to build some metrics. I know that I have the API and the GraphQL API, but I believe it could be better. If this is improved, it would be very helpful.

I used Data Hub for over two years, mostly in the open-source version.

I do not have actual metrics to provide currently; I only have some metrics. I certainly improved the data discovery part of building a data product because it is really fast to know if the data product already exists or does not exist. In the past, I had many products that were the same, and with that, I had a lot of work doing it twice or three times in different parts of the process. This is very good. I do not actually know the exact number of time saved, but I certainly saved time. I have a metric that before Data Hub, I believe 20 to 30 persons used and had knowledge about the data. Currently, I have almost 250 persons using Data Hub.

I did not use many AI features in Data Hub, as I stopped using Data Hub before it started offering these functionalities.

My advice to others looking into using Data Hub is to start by trying the tool using the free version. If it is sufficient and you already understand how to use it, you can transition to the paid version. However, you can accomplish everything in the free version. I would rate this review an eight out of ten.