Cloud Native Computing Foundation Kubernetes AI Day NA 2022, 15 Nov 2022

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Feathr, an Open Source, Battletested, Scalable and Enterprise-Grade Feature Store - Hangfei Lin

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from April 17-21, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Feathr, an Open Source, Battletested, Scalable and Enterprise-Grade Feature Store - Hangfei Lin, LinkedIn

In this presentation, we will present an open-source feature store called Feathr. Feathr is the feature store that has been used in production and battle tested in LinkedIn for over 5 years, serving all the LinkedIn machine learning feature platform. In this talk, we will cover the background of Feathr, its internal design and philosophy, and our journey on building an enterprise feature store. Some of Feathr's highlights include: Native feature transformation support, including feature aggregation, sliding window joins, feature lookup, etc.; it is 30X faster running time with bloom filters, join plan optimizer, salted join, and other optimizations. It is also cloud native with simplified and scalable architecture.

A

um Hey everyone welcome back, I hope you enjoyed the lunch um yeah from experience. I know, half of you have not been to kubecon before, but for those of you who maybe went to cubecon EU this year, the food it wouldn't have been hard to improve on the food from kubecon EU so and actually I really enjoyed my lunch so hopefully, for most of you did too um I'm actually going to hand over to Thiago who's.

A

One of the committee here um he's going to talk a little bit about like the the shirts that he brought to give out um and a bit about, like you know, trying to encourage and embrace like diversity and Global representation in in Tech.

B

uh Hi folks, uh what about playing stand up sit down once again, uh uh everybody uh stand up and sit down. uh How many of you folks were born and raised in Europe? Can you please sit down uh how many of uh how many of you folks were born and raised in Asia in Pacific.

B

uh How many of you folks were born and raised in America.

B

How many of you folks were born and raised in Latin America.

B

How many of you folks live outside Latin, America.

B

Geographical diversity matters, uh so this is a this teacher that I brought for you. Folks, as a part of the giveaway, uh is a token of appreciation. uh I know how tough it is to make to the tech industry without proper access to knowledge and resources.

B

This jersey is from the minor league soccer team from my blue-collar neighborhood. My father used to be a trigger a truck driver every time. I wear this t-shirt. It reminds me how hard it is in some regions to achieve their goals in Brazil you either become a soccer player or a soap opera star, uh it's quite the same in Latin America. Whenever you stay there, it also reminds me the importance of giving back to the community.

B

Thank you very much enjoy enjoy the rest of your day.

A

Amazing I think thank you, Tiago, um okay, so uh our next um Talk is going to be well. The title is feather an open source battle, tested, scalable and Enterprise grade feature store. I. Imagine if you wanted a feature store. Those four things are definitely things you'd be looking for, um but the uh I think that you know the cool thing with feather is this was built like at a company that already has lots of scale and has been dealing with this for five years now. So this has come out of LinkedIn and hang Faye.

A

Lin is going to talk to us about that he's a staff software engineer at LinkedIn. So let's give him a round of applause and welcome him.

C

uh Thank you. uh Can you hear me now: okay, cool? Thank you, um yeah and uh staff software engineer at the linking um today. I'm really happy to be here to share our work and the progress on AI infra on kubernetes Aid.

C

Maybe a little bit of self-introduction for myself. I joined linking about seven years in the first three years. I work on like building micro services and building data Pipelines and later on, I switched to machine learning, infra and focus on building feature store in the last four years and I like system design and product design, and I also like to hear user's feedback about our product and the users, conduct user study and understand the layer, pain points and build a product or solution for them.

C

So if you have pain points in this area, I'm happy to talk to you so with that I will maybe give a very brief introduction about. The failure. Further is an open source, Enterprise grade high performance feature store. It was built and linking about like five years ago and since then we have a adopted by a lot of majority of AI teams and AI applications at Lincoln and empowering a lot of AI applications and in this April we have make it open source, collaborating with Azure. And now it's integrated into Azure cloud.

C

And now, in this step, uh September we have, it has been accepted by Linux foundation and that AI data as a sample, a Sandbox project.

C

So um feature store is both old and new. We have been developing features so I think about five years, but they are still companies kind of just started to consider adopting feature store or thinking about the if they should use feature store, and every year there are new Solutions with different kind of angle.

C

Trying to tackle this problem and trying to tackle this problem, so I feel it's important to share our view and our view on how do we formulate this problem and the one problem we have seen and how those problems lead to this solution? We have built here so agenda today will be problem, statement, solution and architecture and roadmap and demo. If.

D

C

So I think a little bit of background here. Most people might be familiar with it, so air industry is really growing in this last decade.

C

A lot of AI applications like gbt3 and or stable diffusion to generate AI generated content and many other usage to help to people to build better AI applications and I believe AI will keep improving people's life in this century and disrupt many other Industries, and the demand is really growing exponentially. So is the demand for ML Ops and ml infrared.

C

So in the past, people has paid more attention to models but less attention to like data and the feature. Regular presenter in the first keynote mentioned, there's a shift from model Centric AI towards like data Centric AI, so for data Centric AI right.

C

Most people start to pay attention to like unstructured data like image text and so on and yeah, and also some Industries also start to pay attention to like structured and semi-structured data like user interaction, data like user user, static information data and so on, and and we also see a trend that people are shifting towards using more real-time features and building more real-time. Inferences.

C

So the goal for the goal for features is usually people want higher quality. They want the data to be featured to be more correct without any data lecture and so on, and they want the feature to be more performant like.

D

C

Know like for offline, like one high throughput or like a low latency for online inference, and they want to have like a faster iteration like AI application uh building is really iterative process, the faster you can iterate the faster you can bring more impact and, lastly, the importance of financial feature, uh it's kind of more prominent over time. A lot of applications uh requires, like more fresher feature like Netflix and tick tock. uh For example, if you somebody viewed a comedy movie, it's likely they might. They might want to view another comedy movie today.

D

It might not be very not.

C

Very reasonable to recommend a horror movie or something and.

B

C

More kind of outstanding in Tick Tock, if you view something you might be view another related stuff later on.

C

So, but how does that go fit into uh production or reality right? This is actually one of our open source user asks me: okay, I'm new to this industry. How do I actually visualize my raw data for my applications?

C

So taking a typical example like you have a website and you have user activity data coming in and usually you will have like a checking events to collect those data and, while streaming like a Kafka and send those data into offline storage, like a data like a data Lake and the part of that data will be sent into online database.

C

The typical past people usually do like for Chinese or offline influences just go through the offline database. Laws are very mature, mature kind of process in the workflow, but when it comes to like online industry, it becomes relatively harder due to the complexity of the topology and.

D

C

Usually takes quite some time for a company or team to make all the features available in different scenarios like streaming online and offline, so this is from uh that is from a technical perspective. Let's look at like a human human style human perspective. This is just a way that I'm done by Forbes asking people about how much time do you spend asking about AI engineers and the data scientists? How much time do they spend on like data cleaning, preparing the data, future training data set and cumulative algorithms, accurate, spend on data preparation and 76?

C

Actually don't really enjoy the lowest part of work? It's usually kind of a tedious.

C

So I actually also weighed like our uh internal, like a data scientists or AI engineers, and some of our, my friends from other areas or our users about a similar question like how much uh how much instrument? How much time do you spend on about future engineering related task, including feature serving feature, data cleaning, a platform model and the in average? Let's spend like 40 time on this task and most of them all don't really enjoy those type of work.

C

Essentially, they enjoy like virtual engineering itself, but they don't enjoy like maybe how to serve the feature online and how to connect data and how to make sure the data quality is good and monitoring and so on.

C

So it's really hard. It's really hard to convert raw data into features and learn serve into models. It's pretty complex in both many steps and the different tools, and it involves heterogeneous environments, offline online in the nearline, and they have different requirements actually for different use cases in the systems and all organization level, like all like Team level. It usually demands uh different, was the experience and multiple skills and different knowledge.

C

And then, when you start to build an AI team and as it scales, you will find like a lot of tribal knowledge actually for doing things around the feature related tasks.

C

So in some company, usually it's either one person like do this end-to-end like do most of the stuff or in other company. Maybe there will be a part of a team that do this together, so data scientists may just focus on very small part of things. They will just be focusing on building a prototype and after the plot that I've done, they will hand it over to other teams. This is another model, so I think this.

C

This model will sometimes leads to this kind of Meme here, okay, once they are done, they just hand it over and they don't care about what happens for the rest of it and the Land There Will Be, complaints from the data, Office Team for sure so and lastly, I think the equally like people don't like to jump from one tool to the other. This is not really fun, so this is a problem statement. We I kind of summarize here based on those observations.

C

Let's couple people usually just couple feature engineering into model because of whole big pipeline of one whole big notebook to achieve all of them is like a training and the influence, skill and training data and the influence data might not be the same and the result in a skill later on unless organization like organization problem as well, it becomes harder to reuse and share your features across student teams all across your organizations later on.

C

So we have a raw data on one side and though we wanted to make it into features and to serve into model for model inference, I feel less. Apparently, if something is missing here, there's a couple as we need to feel.

C

So, let's lead me to this question: um can we build a layer to simplify this? So let's enter our solution here.

C

So feature store, I, think official store for some people. This might be confusing, sometimes because actually I think this is not well standardized terms in some some kind of marketing material people will Define it in a narrow way. It's just me. The storage system, like a store feature data itself, for example like red, is a feature store or like Google. Has these tensor stories, mostly for all offline training purposes, people call it a feature store as well.

C

There's also like a broad definition like a feature is like a feature management system that allows you to create access industry and discover features. It usually compose of a ecosystem of a tool set. Essentially, this is also the definition used by Wikipedia.

C

So fellow feature store is more like on the broader sense. It's a feature management system, so it's better tested and linking for five years, and we are not opinionated about what feature storage system you use you can plug in like radius or aerospice or Cosmos, DB or three, and so on and right now it's open source and it's now a Linux Foundation Ai and the data sandbox project.

C

So, fellow from very high level is an abstraction layer between raw data and model. It helps you to Define features based on Raw data, raw data sources using simple apis. Okay, those features are by layer names during model training and the model inference. This will avoid a problem like for the training service skill and it allows it allows you to share features across your team and organization.

C

So I will demonstrate some use cases and examples to give you a kind of a hands-on experience of How It's like so further started. The ways like first started with like feature definitions for AI air engineers and data scientists, for example. Here you have a New York Taxi trip data that consists of like a pickup time and job of time and some trip distance information in your database in your data like in CSV format.

C

So then you can just describe what's in a path to that data and the layer timestamp. The timestamp is mostly used for, like something called a point in time join to prevent the data leak. We might not have time to call that today, but if we have time we can call that in an example later on.

C

But if you have time, uh if the data is time sensitive, you can just tell us like on the timestamp and a layer format, and then you can just tell us the key and put it give it a description of it and the line for features you can. Even the features is already kind of visualized like you can just take out the assets like for chip, distance and another example.

C

If you want to transform your raw data into another different feature, you can use that using like a SQL or python as well over here we use a SQL to say, okay, if this is a long trip distance and lastly, you just Google them into together into a feature anchor.

C

So visual angle is means like a few features like anchor to a specific source, so this is kind of where you define your first two features here for one source and the next one is about. You can do the same thing for streaming feature as well. uh It's mostly the same. The only difference is like for Kafka sauce. You have to tell us like Brokers and the topics and the schema, and then the rest of the ideology is actually pretty the same, so it can be transformed.

C

It's unified API can be transferable from one platform to the other, so after you have defined features, then you can use the defined features in different use cases and scenarios for this one. We, let's talk about the most common one like build training data set so to build a training data set. You just need to tell us an observation. Data set observation.

C

Data set usually contains the label for your training and if it's a time, sensitive observation data, you should tell us the timestamp as well, so that we can help you to join the feature in your point in time. Correct way.

C

So later on, you can just tell us the features you want. For example, here you uh I want like a location, average Fair and then lastly, you can just uh call get offline features with above metadata you provided, then we will fail. The engine will kick off the computer in the cluster you you've chosen so like we can. The features will be computed later on.

C

So after model is Trend, you may want to serve your models in server view models right, so we model offline is relatively easy right now, but sometimes serving a model in online settings is relatively harder. So to do that two sort of features here, usually the.

D

C

D

C

You just push live data into radius here, so you can just specify the table. You want to push to and then tell us what features you need and then you can just call the materialize features and we can kick off a offline computer job for you and push the features to online readys. Usually after, like a few.

D

C

You can get a data, you know write this and you can call this in your online inference class to do the online inference.

C

We also, we also provide a UI to help your team to help to discover and to share features here. You can finally all search for your features. You are interested and also we provide a lineage metadata information to understand what feature is derived from what and what is derived from water source as well.

C

So we also for Enterprise need. We know like a lot of times. The exercise control is pretty uh critical as well, so we provide access control as well. You can specify what team members or what teams see what uh project? Essentially, they can have read, write or manager, control.

C

So, lastly, we also provide a derived feature. Sometimes you want to have capability to do feature close like one feature, multiplied by the other feature or like in this example, you want to compute the embedding cosine similarity between two embeddings, so you can say, user embedding and item embedding and multiply that you can use like our function. Function called like science, email, ID to multiply by user, embedding by item and then to produce a similarity score.

C

So, oh yeah, we invest a lot of uh a lot of time. Our time on scalability, like.

A

C

Linking our scale is pretty huge, so we find actually, if we don't do any scalability optimization, uh it's pretty hard to complete the job to create a training data set.

C

So you know sometimes it runs like many many days. Sometimes it even doesn't finish. So we have invested a lot to make a processing like tens of billions of ropes and pattern by the scale data possible.

C

We build optimal native optimizations, like broom filters, drawing plan, optimizers optimizations and the sorted drawing to make lab possible.

C

Although we have Cloud friendly architecture like each individual component can scale out as it needs.

C

Security is also one thing we care about a lot, so we have like a security, credential manager and a keyword manager, a keyword to help you to store credentials to help your team to manage your credential in a secure way. So there's like a r back, we just made. We just demonstrated a layer to help you to control the access as well.

C

So, let's like revisit this diagram, okay, so this is like in price, in a real world what you have to do like there are a lot of branches. Different scenarios, if you want to visualize access or managing features in different scenarios for offline Airline or online fashion, so I highlighted these like components in Orange. If we, you want to build a kind of a mature visualization platform or pipeline, so those are other components you need to build.

C

Based on our experience, if you want to build up like a production grade, mature kind of virtualization platform or pipeline, it usually may take. Each of them may take like a few quarters of engineering hours.

C

So, and apart from that, you have to think about other like components to support your ecosystem like a discoverability, shareability and monitoring and compliance and so on.

C

So each of them may take actually quite a significant time and an investment. So with Vellum. We are trying to build a good abstraction to abstract, away loss infrastructure, details from your AI Engineers from your management management teams and so on.

C

So your data scientist team can just use our API to achieve what they want, using our elastic or Python, and also using a UI to discover the features and share their features and for some like managers or administrative, like staff, they can also use AI to achieve certain tasks. They want.

C

So here um I want to talk about the federal and linky like history present and its impacts.

C

So we started initial development in 2017. and in 2018 we started adoption within linking and in 2020.

C

We have achieved the majority of linking ml application adoption and in 2022 we make it open source and have a great Azure collaboration and also integrated into azure, and we are now a part of like linear, Foundation, AI data, simple box project, and we also have like people uh using using fellow in AWS as well right now, we fella is powering hundreds of AI models at linking thousands of features and in many kinds of entities that Empower our linking economic graph.

C

It was either pedal by the scale.

C

So these are several kind of highlights uh highlighted like impacts I want to mention here. The first one is reducing the feature, engineering time required to adding and experimenting and the serving features in the production from weeks to days in the past, it usually requires a lot of people and the time to actually build pipelines or build integration code or build applications to actually put features into production.

C

There are a lot of details that you need to worry about like a point in time, drawing and performance and latency, and how do you call a larger amount data without costing 3C problems and so on?

C

And the second is related like oh, we, we actually find actually Federal performance faster than some. Some customer build feature process in pipeline, sometimes by as much as 50 percent, and the failure also enables feature sharing between similar applications like, for example, you have a feature for your member or a feature for your job, and the other team is building similar kind of AI applications.

C

But with a slightly different motivation, you can just use failure and just call that feature name, and we can help you to orchestrate all the artifacts for you without too much hassle.

C

So next I will talk about architecture and and the roadmap, so this is the like end-to-end workflow for the Azure integration on the left.

C

Top side is the AI engineer, so data science like coming, usually they will start with interacting with the UI to understand what features has already been created in the in the system, and this is powered by rust for API, the preview and the SQL as layer data backends, and we have a python, a fellow python client to help you to create and manage and define feature definitions after you, you upload data scientists, create a lot of the land link and dispatch. Fellow engine will dispatch Those computer to the corresponding computer cluster like spark or synapse.

C

They will talk to like offline storage, Delta lake or snowflakes and so on, and we also can talk to like Kafka and event hub also for online server. We can push the data to like redis and Cosmos DB. This can later be on be used by machine learning platforms.

C

So, due to time constraint, I will not cover the roadmap in a lot of details. Maybe I will just skip it and have more time for yeah. Have some time. For summary, here is a open source feature store, which can be seen as an abstraction layer between raw data and model allows you to Define features with transformation on top of raw data source and get a feature values by uh by name during both training and influencing further also simplifies the feature for Preparation workflows and enable feature sharing across teams and even organizations.

C

So we don't have time for demo, but there's a link. If you are interested, you can open it to try it out. This is a product recommendation like in your e-commerce website and.

A

C

We still have time for Q a.

A

C

Maybe we can take some questions here: okay,.

A

Yep round of opposed foreign.

A

Yeah great talk: uh do we have any questions from the audience? Just stick up your hand and I'll come over yeah that'll be in the back.

A

Oh sorry, can I get you to sit in here, so people online.

E

Could you please talk a little bit about your compute engine? uh Is it? Is it all Spark.

C

Yeah uh the question is about a computer engine. We use I think that's a great question, a little computer engine right now. We are mostly using Spark yeah.

A

Great um any others, Rebecca yep.

D

Yeah I have a question about the our back control, so uh is the our back can be integrated with the like the companies uh uh directory or or how it's integrated with the real case or or do we just manage it within the um like the feeder contacts or not.

C

Yeah, that's a great question. uh The Arabic right now is integrated, mostly based on Azure kind of a Access Control, but we have an abstraction layer. Actually, it's just kind of a key value pair. You can replace that into different uh kind of identity management system.

C

For example, we have a user that is actually using this in AWS, so we are working with them to make this available in AWS.

A

Great um any final questions, yeah there's one more here and then we'll we'll wrap up on that.

E

uh When you did Transformations earlier, do you support geospatial type, Transformations being able to say something's within a certain radius of a point or within a certain bounding box?.

C

uh Sorry, can you repeat a question: okay,.

E

Do you support geospatial type attributes.

C

You mean geospatial, Transformations or.

A

C

The data types both.

A

C

Okay, uh we don't natively support like a real spatial, specific data types, but we support a very kind of Rich data types from like a one-dimension tensor to multiple Dimension tensors as well, and for Transformations. You can plug in the transformation you like as long as it's SQL based or is python based so I know like a lot of geospatial.

C

Libraries are python-based, you can likely to plug in python-based python-based Geo, geospatial Transformations, and what you have to do is just you need to include your dependencies and we will help you to download those dependencies when the computer runs.

A

Great right, well, um yeah, thanks again, let's have a round of applause for Hank Fey, Lin.