GitLab / Memory Team

No description provided.

No description provided.

No description provided.

No description provided.

Self Managed instance admin workflow:


Step 1: Configure Gitlab.com account
Step 2: Configure code suggestions on SM instance


Self Managed user workflow:
Install and configure VS Code Gitlab Workflow extension


Slides: https://docs.google.com/presentation/d/1vGh-7beMIs0GgB3C8W8LS8cNpYeikpswIPX9qOJAMf0/edit?usp=sharing

Demo for:
- Self managed instance admin workflow for Gitlab.com
- Self managed instance admin workflow for SM instance
- Self managed user workflow for configuring VS Code Gitlab Workflow extenstion

No description provided.

What we need:
- Introduce the idea of instance level authentication
- An authenticated connection between a GitLab instance and GitLab.com
- GitLab instances are responsible for user-level authentication
- GitLab.com is responsible for instance-level authentication

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

Planning issue https://gitlab.com/gitlab-org/application-performance-team/team-tasks/-/issues/120

Presented by: Changzheng Liu, EM of Application Performance

This research proposal is designed to understand the types of performance problems users encounter and the frequency in which users experience those problems in GitLab.

Kickoff for the 14.3 release for the Memory team.

Planning issue: https://gitlab.com/gitlab-org/memory-team/team-tasks/-/issues/98

Kickoff for the 14.2 release for the Memory team.

Planning issue: https://gitlab.com/gitlab-org/memory-team/team-tasks/-/issues/96

No description provided.

Kickoff for the 14.1 release for the Memory team.

Planning issue: https://gitlab.com/gitlab-org/memory-team/team-tasks/-/issues/95

Kickoff for the 13.12 release for the Memory team.

Planning issue: https://gitlab.com/gitlab-org/memory-team/team-tasks/-/issues/93

This is the corporate marketing leads call where we provide corporate marketing updates. Please see full agenda here for all updates, https://docs.google.com/document/d/1ff5yjwVkICu7Hq199DHxICbjCriiuHaDTo1QucagB0I/edit#heading=h.srdzla8j8oal

Kickoff for the 13.12 release for the Memory team.

Planning issue: https://gitlab.com/gitlab-org/memory-team/team-tasks/-/issues/89

No description provided.

Kickoff for the 13.11 release for the Memory team.

Planning issue: https://gitlab.com/gitlab-org/memory-team/team-tasks/-/issues/88

No description provided.

No description provided.

Kickoff for the 13.10 release for the memory team.

Planning issue: https://gitlab.com/gitlab-org/memory-team/team-tasks/-/issues/87

No description provided.

No description provided.

No description provided.

Kickoff for the 13.9 release for the memory team.

Clarification:

Fabian did mix up Puma's single mode a bit.
Puma in single-mode will still be multithreaded, so users will still enjoy the ability for Puma to handle requests concurrently to some extent. (threads are fairly lightweight constructs.)
"Single" here refers to it running a single process, as opposed to clustered mode where there is a primary/parent process that after startup forks off several worker processes. This is what can consume a lof of excess memory.

Planning issue: https://gitlab.com/gitlab-org/memory-team/team-tasks/-/issues/85

No description provided.

No description provided.

No description provided.

No description provided.

The Memory team is stack ranking issues using RICE to understand priorities to achieve the GitLab on 2GB RAM goal.

A quick overview how the Memory team can utilize the RICE framework to stack rank issues quickly. This can help provide clarity and help align on priorities.

It is also fun!

See https://about.gitlab.com/handbook/product/product-processes/#using-the-rice-framework

Memory issue: https://gitlab.com/gitlab-org/memory-team/team-tasks/-/issues/72

No description provided.

No description provided.

No description provided.

0:20 Looked into Puma 5.0 Experimental features
- nakayoshi_fork https://gitlab.com/gitlab-org/memory-team/memory-team-2gb-week/-/issues/6#note_450184965
- Fork_worker
11:52 Outcome - single mode is better
- Trade-offs
15:00 Puma 5.0 discussions
18:00 Reforking discussions
- Is there a way to have the refork benefit without having to refork
21:42 Looked at 20RPS in 2GB
28:00 Deduplication
36:20 Heap usage of Puma process
- Heapy
39:15 pmap Puma single mode
41:15 Understand what calls malloc and how much memory allocated
- Looked at jemalloc and jeprof
47:00 Benchmark settings discussion
- The tested settings are super aggressive
The settings still need to be tuned

1:00 Overview
- Agenda overview - https://docs.google.com/document/d/12-NOJCobK8wLShCv1Spx69l1VPc-UyvCl8QSBaGqbyM/edit#heading=h.j74stu9fgzjy
1:50 Run Puma in Single Mode recap
- https://gitlab.com/gitlab-org/memory-team/memory-team-2gb-week/-/issues/2#note_449139363
- Each worker takes about 150MB
10:50 Started to apply Puma 5.0.x
- https://gitlab.com/gitlab-org/memory-team/memory-team-2gb-week/-/issues/6#note_449815086
- Further research needed to understand the numbers
16:30 Discussed results of using running Omnibus with Spring
- https://gitlab.com/gitlab-org/memory-team/memory-team-2gb-week/-/issues/3
18:20 Discussed zswap
- https://gitlab.com/groups/gitlab-org/memory-team/-/epics/2#note_449804623
19:20 Tried running without unneeded services
- https://gitlab.com/gitlab-org/memory-team/memory-team-2gb-week/-/issues/8
21:00 Discussed sharing code across the whole codebase
- Our monolith is not structured to have a clear split of usage patterns
- Eager loading
24:55 Discuss switching to Omnibus with Ruby 2.7
- Check malloc/preallocation sizes
- Investigate ordering of .rb load for optimizations
32:30 Wrapped up looking into compaction
37:20 Discussed tooling (GDB) and visualization of memory allocation
38:50 Investigate Memory Content
- https://gitlab.com/gitlab-org/memory-team/memory-team-2gb-week/-/issues/4
45:20 String deduplication
49:30 Systemtap example
52:00 Heap analysis with different tools
1:04:20 Thoughts on the format of this discovery week

0:30 - Gather baseline performance - https://gitlab.com/gitlab-org/memory-team/team-tasks/-/issues/75
- Initial impressions and swap space needed
- Discussed using Spring with Omnibus
- Application loaded only once
- Loading Puma and Sidekiq on top of preforking fast
- Rails console instantaneous with Spring preloader
- Most of the memory is unique to a forked process
- Only 200-250MB shared - this is known
3:33 Are we loading everything?
- Next steps - print out what code is being loaded within what process
- Determine why we have so many copies in memory
- Running GPT against 2G and 2G with split to compare graphs
5:40 Investigate Memory Content - https://gitlab.com/gitlab-org/memory-team/memory-team-2gb-week/-/issues/4
- Took 3 different snapshots using rbtrace
- Compared scripts and removed common objects
- Goal - get only the long lasting objects
- Repeated steps multiple times and got different results each time
- Also ran same steps for Sidekiq worker - no discernable pattern
11:25 Discussions about heap and malloc
12:00 Topic - Investigate used gems - https://gitlab.com/gitlab-org/memory-team/memory-team-2gb-week/-/issues/5
- List of memory usage per gem
- Maximum is 200MB
- Only includes just Ruby gems
- Re run with cut-off point of zero to see volume of things in memory
- Discussed loading graphql in memory (schema file)
- Discussed not needing to load whole API in sidekiq
- Can we optimize what we load? How much memory does the API package use? We can shave this from the SideKiq process
- Memory profiler - look into retained memory
19:51 Topic - Investigate Ruby 2.7 GC Compaction - https://gitlab.com/gitlab-org/memory-team/memory-team-2gb-week/-/issues/1
- Overview of garbage compaction
- Description of fragmentation - only free when entire page is free
- Discuss HeapFrag tool
- Demo heap on our processes
27:00 Experimenting with timing of compacting
- Not sure about memory impact
36:00 Discussing the time taken for garbage compacting
- Discussed different ways to configure gc
39:40 Suggestions on when to call GC
- GC before forking
- GC when application is fully initialized
○ Perhaps within initializer
41:20 Experiment with Puma in Single Mode https://gitlab.com/gitlab-org/memory-team/memory-team-2gb-week/-/issues/2
- Initial failures on 4GB machine
- Instructions
- Why we can't use it
- Next steps
○ explain difference in PSS
○ Test different request pressure
○ Describe why we don't use a single mode
52:00 What's next

Discuss approach, identify and pick themes
Picked baseline testing configuration
2:04 Garbage collection discussion
- GC tuning more about mapping pages we are using
- How we can configure to be more aggressive?
- Is it happening?
- How many pages are half empty
3:40 Idle vs. Runtime
- Are there parts of the application holding big parts of memory
- Example number of strings/symbols
- Understand why runtime memory is increasing
- Memory fragmentation
- Build a reference tree to understand GC usage
- Can we somehow nullify some usage here
7:00 Discuss examples of how to break down into smaller topics
8:30 Topic selection and goals
- Move topics into dedicated issue
- Thought process, metrics, data
- Next day, read out results of investigation
- How it went
10:45 Discuss baseline topics
- Calibration session Tuesday
- Goals
- Consolidate and join goals on Friday
14:40 Topics
- Dive into specific topics detailed in https://gitlab.com/groups/gitlab-org/memory-team/-/epics/2
- Topic - review Gem usage, remove dependencies
- Topic - COW, compacting GC
- Topic - Puma 5.0
- Topic - review memory content
- Topic - using spring for pre-loading

No description provided.

Kickoff for the 13.7 release for the memory team.

Planning issue: https://gitlab.com/gitlab-org/memory-team/team-tasks/-/issues/69

No description provided.

No description provided.

Kickoff for the 13.6 release for the memory team.

Planning issue: https://gitlab.com/gitlab-org/memory-team/team-tasks/-/issues/66

No description provided.

No description provided.

No description provided.

https://gitlab.com/gitlab-com/Product/issues/861

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

Internal doc link https://docs.google.com/document/d/10Q4gSHTvfR-Z6LgnvwBjnCn_-JNDNRPLth3wMr2X7rU/edit#

No description provided.

No description provided.

Discussion about import/export feature and what we can do to improve resiliency, security and reduce complexity of the process

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

Topics
- Team formation ideas
- How to optimize for Puma
- Follow up items

In this meeting we discuss priorities, processes and timelines for the upcoming milestones.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.