GitLab / Data Team

In this workshop we discussed:

Mapping a Sisense query to Tableau Datasource

Building a Datasource using Tableau Data Modeling with Relationships and Joins

Published Datasource Best Practices:
Removing Columns
Renaming Columns
Creating Calculated Fields
Grouping Columns
Datasource Filters
Connection Credentials
Datasource Extracts

No description provided.

Session number 5: Chris Shar, Radovan Bacovic and Dennis Van Rooijen - pillars of successful DBT sharding

Data Engineer Internship - DBT sharding, Chris Sharp and Radovan Bacovic discussing how to shard DBT models per provider

Radovan and Chris discuss dbt DAG sharding

No description provided.

Chris and Radovan talking about DBT sharing for Salesforce data

Chris and Radovan discussing about next steps of how to decouple dbt DAG in Airflow

Introduction on Handover of the mechanism to the PI team for the new app created by the Data team.

Justin W. and Radovan Bacovic discussed about Restructure data being delivered by Automated SQL instance service ping, development and testing process for Python and DBT changes.

Marcus, Dennis and Radovan discussing of how to approach the PostHog backfilling process in order to overcome low performance with the PostHog ingestion.

Monthly catch up meeting for Data Platform , PI and Infra team

No description provided.

No description provided.

Sushma, Chris, and Radovan discussed a solution for Instance Redis metrics fix.

Discussing the solution will explore in order to find the optimal solution for the Postgress extraction part.

Chris, Mathieu and Radovan discuss usage_ping details as a part of knowledge transfer.

No description provided.

Carolyn, Dave and Mathieu are talking about Service Ping data source.

A demo from data engineering reviewing the data pump framework to discuss as a solution for a Snowflake to Salesforce integration

Relevant MRs https://gitlab.com/gitlab-data/analytics/-/merge_requests/4181 and https://gitlab.com/gitlab-data/analytics/-/merge_requests/4180

Tutorial on how to use the Data Discovery Function in Sisense.

Data Team demos towards FY21-Q3 Great Team - Self-Service Data.

https://docs.google.com/document/d/17Xjk5pS7MfARWEKAKThhLlCUnt-G3Q1aGVMWeTYwWT8/edit?usp=sharing

Preview session for the upcoming Data Team Group Conversation livestream scheduled for 2020-08-13. Hear Data Team responsibilities, objectives, and direction from Rob Parker.

https://docs.google.com/presentation/d/1N9Jq4DcLWhkJKLw9tHFr7UIwbZUUR1q_euS9Ez65moQ/edit?usp=sharing

This is a discovery and scoping discussion about how we can improve the project import process with sample data that can benefit both the quality performance tests and customer-facing demos with rich data. The quality team has rake tasks that use the Faker package to generate large quantities of sample data and the Customer Success team uses a custom built Demo Designer UI application and database that then uses the GitLab API to import the created records into a GitLab instance.

The mutual outcome of this discussion is the creation of a "demo-data-as-code" approach using YAML or JSON that is sequenced similar to how CURL calls to the API would be. This allows different ways to create the demo data with different tools but a common output that can then be parsed by GitLab's project import and future group import abilities.

Learn more in the issue.
https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/7397.

Participants:
Mek Stittri (Director of Quality Engineering)
Jeff Martin (Sr Demo Systems Engineer - Customer Success)
Kyle Wiebers (Quality Engineering Manager, Engineering Productivity)

Walkthrough with Data Analysts on how to read/update ruby and yaml code for the (K)PI and performance_indicator.yml file.

No description provided.

No description provided.

No description provided.

We walk through how to set up visual studio for the GitLab data team.


https://about.gitlab.com/handbook/business-ops/data-team/

No description provided.

No description provided.

Justin wanted to get a sense of how I work and use GitLab. So I recorded a 1 hour session and tackled the following issues.

Provision Derek https://gitlab.com/gitlab-com/access-requests/issues/2050#note_218331402
Create analyst_infrastructure role https://gitlab.com/gitlab-data/analytics/issues/2482
Upgrade to 0.14.2 https://gitlab.com/gitlab-data/analytics/issues/2461
Re-enable source freshness https://gitlab.com/gitlab-data/analytics/issues/2435
Organize Macros https://gitlab.com/gitlab-data/analytics/issues/2455

https://gitlab.com/gitlab-data/analytics/merge_requests/1494

Implemented in https://gitlab.com/gitlab-data/analytics/merge_requests/1473/ The data team can now specify which dbt models to run and test in the MR.

No description provided.

No description provided.

No description provided.

https://gitlab.com/groups/gitlab-data/-/epics/49

Relevant MR https://gitlab.com/gitlab-data/analytics/merge_requests/1239

See agenda for links

In this meeting, Emilie Schario, Data Analyst, and Sid Sijbrandij, CEO, discuss the best way to organize the Metrics pages in the company handbook. They discuss the benefits and disadvantages of the current Metrics page structure, what the next iteration of storing metrics definitions should look like, and what the next steps for addressing this problem are.

No description provided.

No description provided.

Cathy is working to implement the Zendesk dbt package from Fishtown Analytics but found some discrepancies in the expected data format. Emilie walks her through how to solve this problem.

Data Team Project: https://gitlab.com/gitlab-data/analytics
Zendesk dbt package: https://github.com/fishtown-analytics/zendesk
Cathy's MR: https://gitlab.com/gitlab-data/analytics/merge_requests/752
Docs on dbt Package Management: https://docs.getdbt.com/docs/package-management
dbt: https://www.getdbt.com