Cloud Native Computing Foundation / Kubernetes on Edge Day EU 2022

Empower Heterogeneous Edge AI Acceleration with K8s - Tiejun Chen & Zitong Xu, VMware

As an emerging trend in the area of edge computing, edge workloads tend to be managed and orchestrated by k8s. In the meantime, as the the top one workload of edge computing, edge AI accelerations have been enabled by different vendors' edge AI accelerators quickly, including Nvidia edge GPU series, Intel Movidius VPU, Google edge TPU, etc. Actually you can see many ASIC-based edge AI accelerators and even some high-end CPUs used in edge AI. Obviously, edge users have the challenges around empowering these heterogeneous edge AI on the edge with upstream k8s or those edge k8s versions due to missing a general unified framework on k8s. Here we'd like to introduce our unified framework as a plugin to k8s with the following key mechanisms - 1. Extend the Node Feature Discovery to detect edge AI accelerators automatically 2. Unify different vendors' device plugin to provision work nodes according to it's own edge AI accelerator 3. Introduce transparent backend accelerations to boost ML upstream frameworks such as Tensorflow, Pytorch, etc 4. Attaching remote GPU to edge In our project we provide a unified edge AI framework to help k8s empower heterogeneous AI acceleration on the edge.

Building a Multi-Node Cluster with K3s Using ARM Devices - Sergio Méndez, Universidad de San Carlos de Guatemala

Edge computing is coming as a way to improve user experience for applications, processing data near to the source. Sometimes in edge computing you share data between local machines and the cloud. If you are starting an edge computing project, you could be interested in taking advantage of containers on the edge. Here is where K3s shines to implement edge computing solutions on low resources environments that commonly use ARM devices. In this talk we are going to explore the key elements and best practices to build a K3s cluster on the edge using ARM devices. The talk will show how to build this cluster using Raspberry devices, showing how to configure your local network and a bare metal load balancer with MetalLB, and install other cloud native storage blocks such as Longhorn or OpenEBS. This to finally get a production ready K3s cluster. The talk also provides a quickstart to build edge computing environments for your needs, mentioning different use cases and challenges of this kind when building edge solutions.

Closing Remarks - Steve Wong, VMware

Edge Kubernetes and Healthcare: A Next-Gen Architecture that Scales Across Thousands of Locations - Benjamin Beeman, GE Healthcare & Jeremy Oakey, Spectro Cloud

Edge computing and K8s have the potential to transform every industry, by bringing apps, data and outcomes closer to end-users, but it can be a significant paradigm shift for most enterprises when it comes to operationalization. For the healthcare industry, requirements such as security, availability and performance across thousands of locations are demanding for a new type of architecture and efficient tooling. Join this talk, if you're interested in learning about the requirements and gotchas of running and delivering K8s applications across thousands of edge locations, with 0 tolerance for mistakes. GE Healthcare's Senior Staff Software Architect, Ben Beeman and Spectro Cloud’s CTO, Saad Malik will discuss the lessons and how we need a next generation of edge architecture for the cloud native world. Beyond the theory, we will demo live the open source building blocks of managing the complete lifecycle at the edge, such as Cluster API, Libvirt, Docker and others.

Introduction to Media Streaming Mesh - Giles Heron, Cisco

Media Streaming Mesh is a new concept for enabling real-time media applications in Kubernetes. These applications are usually based on RTP rather than TCP and hence are incompatible with web-proxy based service meshes. In addition they often use out of band TCP control channels (e.g. RTSP, SIP etc.) to negotiate UDP port numbers, and this is incompatible with the kube-proxy ClusterIP NAT. There is, however, a desire from the developers of these platforms to move away from monolithic applications towards the micro-services architecture, and to be able to leverage a cloud-native infrastructure providing load balancing, encryption and observability. Media Streaming Mesh enables this.

Keynote - Edge Native is the new Cloud Native - Modern Apps at the Edge, Marco Nicosia- VMware Tanzu

Edge computing is the next evolution of cloud. Over the last decade, we’ve learned a lot about running applications on elastic clouds. Infrastructure Operators and Developers can apply much of what we’ve learned to Edge deployments, so that you can be confident that the same app is up to date and secure at many locations simultaneously. The two concepts we’ll cover in this quick talk are disposable infrastructure and using abstractions to enable t-shirt sizing at the Edge.

Keynote: Pushing the Boundaries of Open Source for a Brand New Edge - Saad Malik, Spectro Cloud

Modern applications deployed at the edge are revolutionizing industries, turning “Kubernetes at the edge” one of the most compelling use cases for the ecosystem. While edge can be challenging to deploy and manage, the significant progress across many community projects is providing the building blocks of a brand new redefined edge that will unlock its true potential. This short keynote will cover the “now” for edge now, what’s required and the future ahead.

Lightning Talk: Connecting Klusters on the Edge with Deep Dive into Cilium Cluster Mesh - Raymond de Jong, Isovalent

Edge computing can require connecting hundreds of clusters across disparate locations and infrastructures. Without a networking solution to manage this scale and complexity, you will just have a bunch of computers talking to themselves rather than each other and your customers. Cilium is the next generation, eBPF powered open-source Cloud Native Networking solution, providing security, observability, scalability, and superior performance. Cilium has joined the CNCF as incubating project. In this session, you’ll learn how you can leverage Cilium Cluster Mesh for providing connectivity for load-balancing, observability, and security between nodes across multiple clusters, enabling simple, high-performance cross-cluster connectivity at the edge. We’ll explore how Cluster Mesh allows endpoints in connected clusters to communicate while providing full security policy enforcement. The audience will walk away with an appreciation for how eBPF can help solve their networking challenges at the edge.

Lightning Talk: Join the Cloud Native Journey to Shuttle Back and Forth Applications Across Edges - Ruoyu Ying & Qiang Ren, Intel

SD-WAN handles the network connectivity issues between distributed applications in a seamless, secure and efficient manner through replacing traditional branch routers with virtualized or appliance-based software. However, as the concept of edge prevails recently, distributed applications are usually deployed across multiple resource-constrained k8s edges. Then how to expose the services and manage the connections between in a cloud native way becomes a critical issue for users. The open-source project SD-EWAN under the Akraino community’s ICN blueprint is definitely a comprehensive solution. Its CNF, which integrated with multiple network functions based on OpenWRT, is able to serve the distributed applications with secure connections and easy management against networking and the CNF can be automatically thru the overlay controller within SD-EWAN to achieve multi-edge collaboration. This session will show the basic architecture how SDEWAN utilizing k8s to help with the network connections across applications live in different edges, and a demo to showcase its usage for cross-edge collaboration (e.g. advanced load-balancing with service mesh).

Lightning Talk: The Adaptive Edge: Balancing Cost Efficiency, Usability, and Performance Tradeoffs - Tom McCollough, Section

The network edge presents new challenges in optimizing application deployments, as the deeper an application is pushed into the Internet, the greater the associated costs to run those workloads. This talk will cover a set of optimization considerations and strategies that allow workloads to be as close to users as possible, bounded by a business value construct. At the same time, we will discuss the importance of maintaining simplicity in the usability of the system, so that developers can specify how deeply they want their application to run without overallocation of resources. Using CNCF core technologies, including Kubernetes and Prometheus as a base, this talk establishes a framework for evaluating projects within the CNCF landscape and their suitability for edge use cases. We will examine Kubernetes Federation techniques, Multi-cluster orchestrator systems, and Traffic Direction and Service Discovery strategies against a selection criteria to assist architects in making “good-fit” decisions.

MLExray: Observability for Machine Learning on the Edge - Michelle Nguyen, Stanford

Anyone who’s ever deployed on the edge has had this hope before: “It ran perfectly on my cloud environment, it’ll surely work when I deploy it across these other different environments”. Unfortunately, much of the time, this hope falls flat. This is frustratingly true for those deploying machine learning models on the edge. These models are often painstakingly trained and fine-tuned over months and days to achieve those extra few percentage points of accuracy… Only to see performance drop by over 10% once deployed to an edge device. This session will cover common problems encountered when deploying machine learning models on the edge, and how MLExray, an open-source observability framework created at Stanford, can be used to help debug these issues when they inevitably occur. *MLExray has been accepted into MLSys 2022: https://arxiv.org/pdf/2111.04779.pdf

Managing WebAssembly apps with Kubernetes - Shivay Lamba, Layer5

Developers can leverage container tools such as Kubernetes, Docker and CRI-O to deploy, manage, and run lightweight WebAssembly applications. In this talk, there will be a demonstration of how Kubernetes ecosystem tools work with WasmEdge WebAssembly applications. Compared with Linux containers, WebAssembly could be 100x faster at startup, have a much smaller memory and disk footprint, and have a better-defined safety sandbox. However, the trade-off is that WebAssembly requires its own language SDKs, and compiler toolchains, making it a more constrained developer environment than Linux containers. WebAssembly is increasingly used in Edge Computing scenarios where it is difficult to deploy Linux containers or when the application performance is vital. One of the great advantages of Linux application containers is the rich ecosystem of tools. The good news is that you can use the exact same tools to manage WebAssembly applications, enabling Linux containers and WebAssembly apps to run side-by-side in the same system.

Model Serving at the Edge Made Easier - Paul Van Eck & Animesh Singh, IBM

As edge devices consume the world, the ability to deploy AI models on these devices becomes increasingly vital. Challenges surrounding the management of numerous models across a multitude of edge hosts can be tricky. Not only that, the limited compute power that edge hosts provide makes it necessary to eliminate as much overhead as possible. These are common pain points holding users back from large scale adoption. However, with the combination of ModelMesh with technologies like K3s and MicroShift, the practicality of employing such a system has increased dramatically. As the multi-model serving backend of KServe, ModelMesh offers a small-footprint control-plane for managing model deployments on Kubernetes. Using multi-model runtimes with intelligent model loading/unloading, ModelMesh is able to make the most out of a limited set of resources while still providing the capability to serve many models for inference. Come to this talk to get the edge on edge model serving!

Opening Remarks - Steve Wong, VMware

Separation of Orchestration and Computation in KubeEdge, Understand the Why and How - Harshita Sharma, Kubermatic

With the advent of 5G, the era is going through the exciting phase of bringing Cloud Computing to Edge, with Businesses working on finding an ideal solution to meet their specific demands depending on their reality, use case, and scale.You’ve probably heard about KubeEdge - Kubernetes Native Edge Computing Framework (a CNCF Incubation project).One of the features which makes it a more flexible edge computing platform is making the separation of Orchestration capability in Cloud and Compute functionality in Edge with separate cloud and edge core modules, thus making it scalable and extendable on the cloud and at the same time, the edge can work in offline mode with optimizing resource utilization making it cost-effective.In this presentation, Harshita will talk about how the KubeEgde's Cloud and Edge Components work, communicate, and why this makes it the most compelling edge computing platform currently, based on Kubernetes!