Ceph Cephalocon Barcelona 2019, 24 May 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Brazilian Government Case - Brenno Martinez, Serpro

Description

Brazilian Government Case - Brenno Martinez, Serpro

Developed by SERPRO, the Brazilian government IT company, the CNH Digital is a project that aims to replace traditional printed driver's license by a phone application. Ceph is responsible for storing drivers' fingerprints, signature and photo. Each license consists of 12 objects, and there are currently 600 million objects stored. There are multiple government services and systems, serving as central identification storage.

About
Brenno Martinez
Serpro
Infrastructure Engineer
Curitiba, Brazil
Websiteserpro.gov.br
SERPRO is Brazil's biggest government-owned IT services corporation. Which has grown developing software and services and was consolidated by improving technologies adopted by several federal, state and municipal public agencies, incorporated into brazilian citizen's life.
Brenno works on a team responsible for providing distributed storage based on CEPH, which supports dozens of solutions.

A

Yeah good afternoon, my name is Brenda Martinez I'm here to talk about our first first experience using self a workforce air pro, which is an IT company from Brazilian government server, provide solutions to public agencies and Brazilian citizens. We develop and run software in our own data centers. We have a cloud solution which is called estelí. Oh that's a Portuguese word for shipyard.

A

This cloud is based on OpenStack kubernetes staff and many others. Technologies.

A

Our first project using SAP, was in early 2017 when our developer team asked for a block, storage. Sorry, an object storage to keep all the Brazilian drivers licenses.

A

Nowadays, there are more than 60 million licenses in Brazil.

A

Each license has a photo signature, 10, fingerprints and a QR code, so our team challenge was to provide a reliable and scalable storage on a limited budget. Actually, at that time we had no budget for new purchases.

A

So CF, we, if you know, have no experience SEF seemed to be the ideal solution and, as we had already some servers with spinning discs in one of our data centers, we decided to start a proof-of-concept and a few weeks later, these testing cluster became our prediction environment.

A

After some time uploading objects, the cluster started showing some slow operation messages. We saw a lot of flapping OS DS when the OSD is mark.

A

Each other down, while marking themselves up to the monitor radius gateway, started to go up and down recurrently because of the the OSD slowness and also Keystone could not hold up so many connections, tokens generations and we found out that the main problem was too many objects in a single bucket. In our case, it was dozens of millions.

A

So there were many recommendations to solve this problem. One of them was to use and VMI's to index pool. But, as I said before, we had no budget for new purchases. Another recommendation could be to use in the XLS buckets, but without index buckets will not be synchronize about in a future multi-site environment.

A

So we have said rashard to the already existing buckets. We limited to 100 bucket 100,000 buckets objects per bucket. We added more Keystone servers and rewrote some Keystone code to reuse tokens and only six months later we added new OS. These servers with and VMI's to the cluster and finally, we upgraded from Java to luminous, which is a version that supports dynamically rashard.

A

A

Did SEF delivers the maximum performance that you're a hardware and network can offer you just have to set the right configuration and tuning.

A

So, as I said nowadays, SEF provides objects stored for our licensed drivers. Our drivers licenses safe, provides also block and file system storage for containers in our cloud solution and will shortly provide storage to Hadoop cluster, and we are now implementing our multi-site environment and that's it. Thank you.