DataHub

My main use case for Data Hub involves integrating our HR system or Active Directory, which automatically pulls in all 2,000 workers and groups them into their respective project squads and R&D teams. Each team gets its own team profile page in Data Hub, which helps solve the classic corporate headache of determining who to ask for specific information.

When a team builds a data pipeline, a Kafka topic for telecom signals, or a dashboard, it is tagged explicitly with their team profile as the owner in Data Hub. This means that if a developer in Split , working in the same company, needs to find a specific network dataset, they do not waste days spamming Slack channels; they can simply look it up in Data Hub and find the team profile that owns it along with the direct contact info or Slack channel.

Additionally, it enables us to run a data mesh model with 2,000 people, allowing one central IT team to manage everything while Data Hub facilitates splitting the company into logical domains such as electronic health, telecom networks, IoT, or smart cities.

The best features that Data Hub offers include the ability to centralize everything in one platform, such as creating profiles and organizing them into separate domains like engineering, health teams, supporting teams, and HR teams. This allows information to be shared across different domains.

Utilizing the data mesh model enables the company to maximize functionality using a single solution. Data Hub supports collaboration between different teams and departments significantly, as evidenced when we created various data mesh modules and established different domains such as E-Health, telecom networks, and IoT. This allowed us to share datasets effectively, and with authenticated users, the communication and responses were much quicker.

Among those features, I find the collaborative aspects the most valuable in my work because it has greatly improved our operations over the past year. We evaluated various licenses and methods to integrate data catalog platforms, ultimately deciding to move forward with Data Hub since it was more compatible with our company's security requirements. Compared to other tools, it received better support from the community, which is updated daily, allowing us to collaborate effectively through contact sharing.

Data Hub has positively impacted my organization by functioning as an all-in-one solution. It uses data mesh and separates domains to manage privileged access based on user validation, allowing us to share data sets across the company, which informs everyone about internal regulations. Furthermore, it significantly aids new joiners in understanding the operations and knowing who works on specific projects, while also providing updates on changes occurring within various sectors and domains.

The frequency and quality of updates or new features released for Data Hub have been impressive. This extensive community support was a key factor for us at Ericsson Nikola Tesla to choose Data Hub as our data catalog.

Regarding how Data Hub can be improved, I believe they should focus on enhancing their marketing efforts. Within our company, we were unaware of the Data Hub platform while searching for data catalog options that offered strong security and collaboration. Better marketing would help other companies learn about this effective solution.

My rating of eight rather than a nine or ten pertains to the connections with different systems. Specifically, the integration with Slack and Azure , as well as how we link our HR system to Data Hub, could be improved for better compatibility.

Integrating Data Hub with our existing tools and systems was not very easy, which is why my rating is an eight. We attempted to incorporate our HR system with Data Hub, aiming to set governance status for the 2,000 employees in our organization, but I did not complete this aspect before leaving the organization.

I have been using Data Hub for at least six months at the company called Ericsson Nikola Tesla in Zagreb, which has a massive operation with an entire ICT and R&D division of around 2,000 workers.

In terms of scalability, I believe Data Hub performs exceptionally well as more teams come on board, making it efficient for large organizations with approximately 2,000 employees. It adequately supports the scalability of data sets and the implementation of data mesh models.

During implementation, the documentation and support resources from Data Hub were very helpful. I followed the guidelines, accessed each section, and understood the platform effectively, which made the initial setup easy.

Data Hub is flexible, optimistic, and user-friendly in terms of its interface and experience. I rate Data Hub an eight on a scale of one to ten.

The learning curve for new users adopting Data Hub is addressed through their learning section that guides users on how to navigate the platform. I found it quite simple and effective to follow.

We purchased Data Hub through the AWS Marketplace .

As for specific outcomes or metrics, I currently do not possess numbers since we are still in the early stages of implementing Data Hub within our company. However, the HR department reported significant time savings in completing tasks before and after adopting Data Hub, which has resulted in faster completion and better collaboration without interrupting others.

Data Hub has worked for me personally, as I noticed that after we began ingesting Data Hub into our Ericsson Nikola Tesla company network, it proved to be incredibly helpful for easier access to information. By positioning team profiles at the center of Data Hub, it prevents the duplication of data sets, accelerates onboarding for new engineers, and fosters more connected and collaborative teams within our large employee base. Personally, it has helped me specify tasks and has contributed to the company's progress with the data catalog we chose.

My advice for others considering using Data Hub is to understand how it works and explore its integration potential within their organization. Engaging with community support can also be beneficial, as the team's collaborative approach is impressive.

My main use case for Data Hub is to build data products inside Natura; primarily, I built data products for CRM , which is customer relationship management, and also for some data products for the product field, such as analytic fields. I used it by dividing the company into domains, and each domain has its own functionality and its own structure. With that approach, I used it extensively for building domains. I also used it to build data lineage across the entire data journey, from the ingestion of the data to the use of the data in the final part, such as in a data product or in a dashboard.

A specific example of how I used Data Hub for building domains and data lineage is the domain called GenAI, which is primarily built for products based on AI, mainly generative AI. To accomplish this, I used Data Hub to track the data from the ingestion field. I used some CDP tools such as Segment  and I also have data in an S3  bucket that was ingested to Databricks  using Airflow . With that setup, I track this lineage from the origin system. After that, I performed a lot of transformation of the data inside Databricks  to clean the data and conduct some data augmentation. After that, this data is used to train some models using Databricks LLM. With that, I ingest all this metadata into Data Hub and I can see from where the data is coming from and to where the data is going. This is primarily for LLMs to help consultants at the end of the product.

The best features that Data Hub offers include the capability to make conversational questions inside the platform, which I believe is the best thing that they built in the past year. It is also easy to connect different data sources. Since data lakes, I have connectors to some databases and also to some business analyst tools and other tools. I can connect many types of data inside Data Hub and see what is going on and how we govern the data. Data Hub is a pretty good tool for that. I also value very much the open-source version because it is free and everyone can use it.

I do not have much experience using the conversational questions feature, but I do not need to go to the asset to see from where the data is coming from and where it is going. I can simply ask, 'How can we calculate the sales in this month?' and Data Hub will identify which table will be used and from where this data is coming from and where this data is going. This is very effective.

Data Hub has impacted my organization positively by helping us build a data governance environment and share the knowledge about the data for the entire company. As we used the open-source version, we have no limitation in how many people can use the tool, which is excellent. I can conduct many tests and test as quickly as possible. It is very good for building POCs, for example. Data Hub also helps to give this understanding of the data for the entire company. Everyone in the company can see the data and know where the data is coming from and where it is going. I believe it is very effective and all the people in the organization, not only the data field personnel, can understand more about the data and also help to build better products.

I believe Data Hub could provide more functionalities in the free version. I understand that we have to pay the persons who build the platform, but the free version has some limitations. Some capabilities of the paid version being included in the free version would be beneficial. Another improvement that is needed in Data Hub is how I can get data from Data Hub to build some metrics. I know that I have the API and the GraphQL API, but I believe it could be better. If this is improved, it would be very helpful.

I used Data Hub for over two years, mostly in the open-source version.

I do not have actual metrics to provide currently; I only have some metrics. I certainly improved the data discovery part of building a data product because it is really fast to know if the data product already exists or does not exist. In the past, I had many products that were the same, and with that, I had a lot of work doing it twice or three times in different parts of the process. This is very good. I do not actually know the exact number of time saved, but I certainly saved time. I have a metric that before Data Hub, I believe 20 to 30 persons used and had knowledge about the data. Currently, I have almost 250 persons using Data Hub.

I did not use many AI features in Data Hub, as I stopped using Data Hub before it started offering these functionalities.

My advice to others looking into using Data Hub is to start by trying the tool using the free version. If it is sufficient and you already understand how to use it, you can transition to the paid version. However, you can accomplish everything in the free version. I would rate this review an eight out of ten.

My main use case for Data Hub is to catalog the dataset across my company and to get the lineage of data in the my company pipeline.

To give an example of how I use Data Hub in my day-to-day work, suppose the data is flowing from a source to Kafka and then to some data storages. If some cross-team wants to use the data but there is a problem at the Kafka level, we are not sure who all are consuming that data. Data Hub is very useful for us in this scenario. It can generate the lineage from source to destination, and when there is an issue at the Kafka side, we will get to know what the end results and impacted data sources are.

I would add that sometimes when we do not want to share the data or when the customer or another team wants to consume the data, we are not sure what kind of data is there. We have to look at the schema. Data Hub is useful for us as we are doing the cataloging of all the datasets across my company, allowing us to later use and see the table information and schema information so that the team can identify what data is PII or non-PII.

The best features Data Hub offers include support for cataloging and lineage very well, as we are getting all the different types of connectors to consume and use across the my company dataset pipeline. Apart from that, the GraphQL APIs provided by Data Hub are very good, allowing us to get all the information we need programmatically whenever we need it.

Regarding how the GraphQL APIs help my team in day-to-day tasks, we sometimes use custom logic to check whether the data has PII or non-PII. We have some AI model running on top of it, which requires classification. Based on the dataset URL, we are getting information about the dataset using the GraphQL APIs. GraphQL APIs are very handy, allowing us to customize properties and pass on the necessary information. For example, if we need a structured property, we can get those structured properties. If we need tags or owners, we can retrieve that as well.

Data Hub positively impacts my organization by enhancing collaboration as previously, we had to ask the team to provide the schema information. my company operates in a cross-region environment, so a person in India could wait a day to receive information about the schema from someone in the US. However, with Data Hub, we have a centralized place where we can access all the schema of the datasets, making it very helpful. Additionally, whenever there is a problem, using the lineage helps us quickly identify the impacted team or dataset.

Whenever there is an incident, we first go to Data Hub to see the downstream teams impacted and stop any jobs running on those datasets. It helps us save around eighty percent of time, as we previously had to track down information manually to find the owners, but using Data Hub, we can tag the owners of the datasets directly in the tool.

For improvements to Data Hub, I feel the security is a bit on the weaker side. We have ingestion jobs that require exact permissions for different owners, but this setup does not align with the my company grouping system. We need to create some custom grouping to manage those permissions. I would appreciate it if there were a method to consolidate all the information on a single page, which would simplify sharing permissions for running ingestion jobs.

Additionally, I do feel that the metadata test we run daily takes too long. Initially, it takes one day, which I find excessive. Ideally, we should get information within one hour. These are the two main issues that would benefit from improvement for our use case.

I have been using Data Hub for one and a half years.

Data Hub is stable in my experience. However, there are times when we attempt to upgrade it, and it may go down for a couple of minutes, but not more than that.

Data Hub handles scalability effectively, accommodating growing data and users.

I have had to reach out to Data Hub customer support multiple times. For example, when we were setting up a private link to connect to Data Hub GraphQL APIs, we required our account to be whitelisted. I have also requested some future features for our use cases. For instance, when working with a metadata test scenario, I needed to have a range date column, which was not available. I requested the Data Hub team to make it public so we could use it.

I have seen a return on investment with Data Hub. For instance, I have noticed time savings during incidents and while looking up schemas. In terms of resources, Data Hub centralizes data cataloging and classification, saving us from having to disclose PII column information to teams not utilizing it. Regarding financial metrics, I do not have specific metrics available.

Before choosing Data Hub, we looked into Unity Catalog from Databricks , but we ultimately decided to stick with Data Hub.

My advice for others looking into using Data Hub is to use it for cataloging, classification, and centralizing all your schema. Data Hub supports a variety of connectors and has excellent lineage options. Additionally, make sure to utilize the well-written documentation that can guide you in building your product solutions. I would rate this product a nine out of ten.