24 May 2021
Metadata meets the Data Mesh
- hosted by LinkedIn, Acryl Data and the DataHub community.
https://datahubproject.io
https://acryl.io
THE DATA MESH PROMISE
As data explodes in size and complexity inside organizations, and as organizations move to become even more data driven, we are uncovering new bottlenecks wherever centralization exists. The central data engineering or platform team shows up as a common choke point that is slowing down the time it takes to unlock value from data with quality.
A new attempt to break out of this pattern — data mesh — has emerged as a theme dominating the data conversations these days. Data Mesh encourages business units within an organization to think about managing their data assets as “data products”. Key tenets of this include domain-oriented modeling and ownership; which distributes ownership of data from its production all the way to its analytics form.
Turning this promise into reality still needs a lot of open questions to be answered by the current generation of technologists.
THE BURNING QUESTIONS
- What is the role of metadata and data discovery in a data mesh implementation?
- Are knowledge graphs orthogonal or complementary to data mesh implementations?
- Should we apply all practices from micro-service implementations towards best practices in data mesh?
- Can we learn from the rich database literature around database modeling (such as the Kimball methods), or are those patterns obsolete in a data mesh implementation?
- Does data mesh require physical isolation of data or can you implement it even on top of a centralized data warehouse or data lake?
- hosted by LinkedIn, Acryl Data and the DataHub community.
https://datahubproject.io
https://acryl.io
THE DATA MESH PROMISE
As data explodes in size and complexity inside organizations, and as organizations move to become even more data driven, we are uncovering new bottlenecks wherever centralization exists. The central data engineering or platform team shows up as a common choke point that is slowing down the time it takes to unlock value from data with quality.
A new attempt to break out of this pattern — data mesh — has emerged as a theme dominating the data conversations these days. Data Mesh encourages business units within an organization to think about managing their data assets as “data products”. Key tenets of this include domain-oriented modeling and ownership; which distributes ownership of data from its production all the way to its analytics form.
Turning this promise into reality still needs a lot of open questions to be answered by the current generation of technologists.
THE BURNING QUESTIONS
- What is the role of metadata and data discovery in a data mesh implementation?
- Are knowledge graphs orthogonal or complementary to data mesh implementations?
- Should we apply all practices from micro-service implementations towards best practices in data mesh?
- Can we learn from the rich database literature around database modeling (such as the Kimball methods), or are those patterns obsolete in a data mesh implementation?
- Does data mesh require physical isolation of data or can you implement it even on top of a centralized data warehouse or data lake?
- 11 participants
- 1:20 hours
