Apache Cassandra Apache Cassandra World Party | April 2021, 28 Apr 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Raising the Bar on QA | Apache Cassandra World Party 2021

Description

Presentation from the first Cassandra World Party on April 28, 2021. Learn more about Apache Cassandra at https://cassandra.apache.org/.

Mick Weaver of DataStax presents “Raising the Bar on QA” on what's happened with Apache Cassandra's CI systems: ci-cassandra and circleci; and what's involved in the forty thousand tests we run against every commit.

About the Presenter
Mick is a principal architect at DataStax, helping Cassandra users from innovation to operations.

Apache Cassandra is an open source NoSQL distributed database trusted by thousands of companies for scalability and high availability without compromising performance. Linear scalability and proven fault-tolerance on commodity hardware or cloud infrastructure make it the perfect platform for mission-critical data. Get started: https://cassandra.apache.org/quickstart/

The Apache Cassandra World Party 4.0 was a virtual, one-day celebration on Wednesday, April 28, 2021 in preparation for the launch of 4.0, the most stable database release ever shipped.

Join the Cassandra community: https://cassandra.apache.org/community/

#Apache Cassandra #database #NoSQL

A

Mick uh very happy to have you on board. Here I mean you and I have been attending these community events for a long time, uh long time, uh pmc, member committer, um also the the manager or the the man who makes it work on the cicd board. And if you look on the, if you look on the mailing list, you'll see mick going it's green and nick, it's green, isn't it.

B

It is just like your background in a long time.

A

Yeah, well, you know I.

B

A

Hop it doesn't allow for anything else, so I think we're good katarina. Can you hear everything.

C

Okay, so I think patrick already did a good job, but let me say a few words to you. So mick is a principal architect at datastax who helps with cassandra from innovations to operations and beyond technology. He is crazy about snowboarding rock climbing trail.

B

Running skiing.

C

Surfing just anything awesome in the nature and in this session he will dive into what happened in apache cassandra, ci systems, ci cassandra and circle ci and what's involved in the 40 000 tests, we run against every commit so welcome, mick and enjoy.

B

B

All right we're ready. Let's go.

C

Let's go hello.

B

Everyone, hello, emilia hello, pacific and asia if you're still up and good morning americas as I've been introduced, my name is mick samweever, I'm a cassandra committer and on the pmc, and I've got the pleasure of your attention for the next five minutes. So yeah we've heard how awesome cassandra 4 is going to be, but I want to talk about what's been going on with testing and ci and what this means for you, I'm thinking, safety, trust stability, so whether you're breaking any snowboarding or you're pumping out cassandra's latest feature.

B

You can't be afraid we have to have an environment where we are free to make mistakes.

B

Cassandra is now used everywhere. 90 of the fortune, 100 are using it. It's considered early maturity, technology, stability and reliability are crucial and, as a community, we've gone from being driven from just one company to many companies and stakeholders that need to coordinate and collaborate and trust each other.

B

We're talking about apple netflix, amazon, bloomberg, alibaba, dynatrace, wayway, insta, cluster data stacks and more and cassandra also in 2021, we're starting to see something brand new. There is a ton of innovation happening around it from the cloud offerings to stargate to cate sandra. If you take a peek at the post, four zero roadmap, you see a strong theme of plugability developer experience and safety and we're now aiming for an always shippable trunk.

B

So ci is key today. The two systems that we use are circle ci and ci. Cassandra circle ci is the commercial solution. The main usage today of circle ci is pre-commit testing your patches before they get reviewed before they get merged. It requires a high resource profile.

B

Basically, you need to have a premium account that your company pays for as it uses hundreds or thousands of containers and lots of resources.

B

There are lots of stages to this ci and developers just trigger those stages that they feel are relevant to the patch they're working on ci cassandra, on the other hand, is our community ci. It's based on cloud b's, it's provided by the asf and it's running 40 agents or servers around the world donated by numerous different companies in our community. Its main usage is post, commit. It runs all of its stages automatically, and that includes today, forty thousand tests and for the first time in many years on the lead up to four zero.

B

We are seeing green runs of the whole pipeline. For the first time we have kind of four categories of tests in that pipeline. We have proper unit tests, unit tests that have some cassandra server component in it java distributed tests and python distributed tests. The python distributed tests are old, the java distributed tests are new. We see most people writing the java distributed test today, but testing distributed tech is hardcore. It's wild. I feel like this fella on many days.

B

Anyone who's tried to run the python upgrade dig test locally knows this pain, make this run on heterogeneous hardware around the world, trying to maximize saturation and performance on a system that itself has no ci and never runs locally. The same way yeah it's lots of fun, but it is all part-time and volunteer efforts. No one in this community is dedicated to these systems or is a built engineer.

B

Key people who have helped out with the ci systems and get us close to 4-0 or on this slide, but also generally in the history of cassandra. No release has been tested as much as 4-0.

B

Tons of work has gone into zero, close to four thousand commits over one and a half thousand euro tickets, close to a thousand bucks over twelve hundred of those tickets that test related and over two hundred of them were dedicated solely to fixing flaky tests and now, after years of work with our first rc of 4-0, we have officially lifted the feature. Freeze.

B

But there's still lots to do on the previous slide. I gave a call out for the things that we want to add um and that we need help uh so yeah go, get 4-0, run it in production, but also don't be afraid to help out jump in pick your poison and help us out. Please thank you.

C

Thank you, mick.