Antrea / LIVE !!!

Lets deep dive into kube pug versioning w/ ricardo. esp. post 1.22 this is something everyone keeps an eye on.

Hey, come and join Christian Schlotter and Xinqi to explore a new feature in kube-state-metrics 2.6 that will allow for any component that uses CRDs to be observable in a cloud native way!!

Come join us where we go over the original network policy implementations, https://kubernetes.io/blog/2016/04/kubernetes-network-policy-apis/ , and some of the early work on them by our calico and other friends - and also explore the future of cluster network policies, administrative network policies...

Let's explore the krustlet --- an alternative to a golang kubelet ? And... lets see how the rust ecosystem works.

Come and join Nick Young, the tech lead of Contour and Gateway API, to learn about Kubernetes Gateway API and all the related resources - GatewayClass, Gateway, HTTPRoute, TCPRoute, Service, etc!!

Learning Kubernetes ingress and ingress controller from Nick Young, the tech lead of Contour!

Join us with andrew stoycos to go over our later addition to the KPNG family, the eBPF proxy !!!

Come and join Chris Grice to run AKO-Operator and AKO on your vSphere/Openshift/AWS/Azure clusters!!

Whats this GOLANG_PROTOBUF_REGISTRATION_CONFLICT that i keep hearing about ? Lets figure it out.

protobuf: serialization/wire format -- IDL
grpc: http framework that does RPCs using protobuf
Kubernetes
CSI: native to the specification
CNI: ? not yet 2.0 ??? exec
Kubelet:
services, messages, enum
service
endpoint
watch
…
rpc Watch() returns (stream OpItem);
containerd [exec] - /opt/cni/bin/antrea - stdout , IP parsed ?
opt cni bin antrea - ( GRPC ) - antrea-agent (on a port)
pkg kubelet cmd devicemanagerd manager.go

Impromptu Live Stream on the KPNG Windows Proxy and codebase with Microsoft

Kubernetes Enhancement Proposals are the way we get stuff approved upstream, and were behind on the KPNG kep ! Today we'll go through https://github.com/kubernetes/enhancements/pull/2094/files , and update it for sig-network to go through and approve. We'll also go through the general KEP template and look at what all it takes to itereate through to get a KEP merged.

In this episode we'll look at ways to stress test k8s clusters and, maybe, see if we can reproduce any interesting etcd race conditions... like https://github.com/kubernetes/kubernetes/issues/65517 .... (and https://github.com/kubernetes/kubernetes/issues/109399). Maybe this will result in a new Kubernetes e2e ? or a cool test we can run on the side! BTW , did you know theres a little bit of a race for CRD creation and CRD request availability in the apiserver? Maybe we can find out why !

Come and join Lan Luo and Jiajing Hu to see how antrea makes multi-cluster service happen!! The multicluster stuff has been going on a while, with different approaches (i.e. https://cloud.google.com/kubernetes-engine/docs/concepts/multi-cluster-services ). Theres also an Upstream KEP on multiservice clusters (kubernetes/enhancements/keps/sig-multicluster/1645-multi-cluster-services-api/specification.md) that you can read to learn more about this stuff.

In this show, we'll learn about how antrea approaches this problem by creating a concept of a multi-cluster service object.

From the docs (https://antrea.io/docs/main/docs/multicluster/user-guide/)

Antrea Multi-cluster implements Multi-cluster Service API, which allows users to create multi-cluster Services that can be accessed cross clusters in a ClusterSet. Antrea Multi-cluster also supports Antrea ClusterNetworkPolicy replication. Multi-cluster admins can define ClusterNetworkPolicies to be replicated across the entire ClusterSet, and enforced in all member clusters. Antrea Multi-cluster is introduced in Antrea v1.5.0, and the ClusterNetworkPolicy replication feature is supported since Antrea v1.6.0.

Lets review a few of our favorite upstream networking projects this week. As always we'll be able to answer antrea questions and topics along the way as they come up!

There was a recent etcd bug https://github.com/etcd-io/etcd/pull/13854 which brought down high density clusters and caused folks like vmware and red hat to suggest folks not to upgrade to kubernetes 1.22. We'll look at how etcds embedded stress perf-check tool along with etcd endpoint status --cluster can be used to see the raft indices of nodes and confirm wether any of your nodes has hopelessly fallen behind the leader node. Also, ricardo will show us a new way to look at the problem of securing L7 loadbalancers (even though they need some high privileges on the networking side), using hand crafterd linux jails.

And of course, another KPNG project update, this time, about how to get started on the windows kernelspace proxy.

Come and join Anlan and Yongming to explore antrea flow visibility using Grafana. It's a new feature added to antrea 1.6!

Join Scott and Jay as we look at a way to run a multi OS (windows, linux) workload cluster on VMWare tanzu - this involves having multiple MachineDeployments for Cluster API for Windows as well as Linux nodes... something not explicitly supported using the tanzu cli, but... well.. easy enough to hack together if your feeling adventurous.

Come join Yang Ding and Grayson Wu to be the first to know about the new features in Antrea 1.6!! Antrea 1.6 is coming soon!!

This week we'll explore TKG on Azure. We've previously done episodes on the CAPA networking model, and several on VSphere, NSX, and AVI, so , lets show our azure friends some love and look at how TKG (and antrea) is installed on Azure !

Egress manages external access from the Pods in a Kubernetes cluster. It'll be supported since Antrea 1.6.0. Come and join Wenqi and Jianjun to learn more about this new feature!!

Alot of folks are curious about how loadbalancers , nodeports, and so on, effect the k8s datapath.
In this show, we'll:
- Look at the datapath for pods on a Kubernetes cluster
- Look at how loadbalancers work on clouds like VMWare Tanzu (AVI) and GKE
- a quick KPNG project update
- AVI ,VRFs, and NodePortLocal vs ClusterIP vs NodePort configurations
- How GKE old services (vs newer GKE native routable services) work

Following up on Episode 16 where we explored the initial CNI plugins specification, let's learn how to build a CNI from scratch with Michael Zappa.

In this episode You'll learn about:
- How to build a CNI manually
- How the CNI specification and cnitool work
- Visualizing the overall architecture of CNI networking plugins

Lets look at the CNI Spec in detail tomorrow, in preparation for Mike Zappa's next show on how to build a CNI from scratch !!!

In this show we look at the cli 0.3 spec in antrea, and how it can evolve to the 0.4 spec and beyond, and look at the overall format of the CNI json that containerd/runc use when making new Kubernetes pods.

Containerd 1.6 - HostProcess Container support for Windows
(https://github.com/containerd/containerd/releases/tag/v1.6.0)
- [Kubernetes Policy Management Whitepaper](https://github.com/kubernetes/sig-security/blob/main/sig-security-docs/papers/policy/CNCF_Kubernetes_Policy_Management_WhitePaper_v1.pdf)
- [NetworkPolicy Status KEP](https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/2943-networkpolicy-status)
Conditions []metav1.Conditon
- antrea cni 0.3.0 ... 0.4.0 (CHECK) ?
- `kubectl delete pod -n kube-system -l "component=antrea-agent"`
- containerd ... go-cni
- runc
- containerd (go-cni) cni add
- co-cni imports. github.com/containernetworking/cni v1.0.1
- parses the cni json config
- matches 0.4.0 to the versions of its own cni config parser

Come hang out w/ Scott and Xinqi! We are gonna talk about routable pods with NSX-T as well as building a controller with Kubebuilder! Then, we're gonna take a look at a real controller "load balancer operator for kubernetes".

Join aiden obley this week to learn about IPv6 on the new VMWare Tanzu support for IPv6, and learn about how CPI node IP management and K8s ipv6 services route traffic to pods!

TKG_IP_FAMILY: "ipv6"
VSPHERE_CONTROL_PLANE_ENDPOINT: "2001:1900:2200:5f75::aaa0"
CLUSTER_CIDR: "fd00:100:96::/48"
SERVICE_CIDR: "fd00:100:64::/108"

Come hang out w/ Xinqi and Bhushan ! We're going to look again at AVI and the implementation details of the AKO operator that manages loadbalancing infrastructure for you in Tanzu. We'll also talk about AVI in AWS and Azure also.

Specifically we look at the nodeportlocal, nodeport, and cluster IP implementations of AVI loadbalancer, how it integrates with services like Route 53, and NSX, and so on.

We also spent some time talking about the contour vs envoy vs svc mesh stuff, which seems to confuse alot of folks.

Come join Zac and Fang, where we'll look at the https://github.com/K8sbykeshed/k8s-service-lb-validator test suite - and how it can be used to rapidly ascertain cluster networking health in a world of divergent service proxy implementations for K8s, and learn about sonobuoy plugin design !

Join Amim as he schools us on the sig-windows-dev-tools platform for building windows Kubernetes clusters, and testing them w/ real CNIs (antrea, calico)

Come join Scott (VRabbi) to dig into real world customer scenarios around antrea and Tanzu, and his legendary TKGM-customizations suite !
- Antrea with Tanzu 1.5, and NSX-T integration plans
- Egg nog
- vrabbi/tkgm-customizations
- packaging... everything into carvel on customer sites

Join Ricardo, and maybe some surprise xmas guests (Trevor from Avi, Kal from MSFT).... for a special Antrea-LIVE episode where we learn about what its like to live on the "user end" of the ongoing changes in the upstream Ingress community (Contour, NGINX, Avi, ...) , and talk about the future of Cloud Native Loadbalancing !

Join Sedef, Jay, and DWAYNE this week !!! We're going to dig into the internals of AWS networking, specifically with how it relates to the Kubernetes Cluster-API Provider for AWS...
NEWS
- https://kubernetes.io/blog/2021/12/08/dual-stack-networking-ga/
- https://github.com/kubernetes-sigs/kpng/issues/142 Readyness states
- https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/discovery/types.go
- CAPA
- declarative cluster / infra mgmt
- https://github.com/kubernetes-sigs/cluster-api-provider-aws
- https://cluster-api-aws.sigs.k8s.io/
- same mgmt cluster
- different credentials
- what are secondary cidrblocks for???
- classic Loadbalancers
- transient gateways
- hub and spoke peering

- AWS Networking Details
- NAT gateways
- private Epis/public subnets
- V2
- VPC-peering
- Security Groups
- IAM resources and CAPA
- Calico?Multus?Antrea - yes

Join @jayunit100 @luthermonson, and maybe a few other surprise guests for another this WEDNESDAY where we'll explore parts of the broader Antrea story. We'll dig into Antrea on GKE, Antrea Agent's startup on Windows, and look at wether or not we can install it on K3s as well.

- luther is here again !
- k3s demo
- test networkpolicys
- https://github.com/kubernetes/enhancements/pull/2975
- https://github.com/rancher/rke2/issues/2201
- https://github.com/kubernetes-sigs/sig-windows-dev-tools/blob/master/forked/0-antrea.ps1

Let's learn about Antrea's lifecycle in Tanzu, with Carvel Packages and Tanzu Addons Manager. The Carvel team will join us so we'll be sure to get into the nitty gritty details about PackageRepositorys, Kapp Controller, and PackageInstalls. In the process we'll learn alot about how VMWare tanzu views the world of custom build K8s infra.

- api deprecations
- https://kubernetes.io/blog/2021/07/14/upcoming-changes-in-kubernetes-1-22/
- https://github.com/vmware-tanzu/tanzu-framework/pull/959/
- https://github.com/vmware-tanzu/tanzu-framework/pull/959/, addon reconcilers
- How tanzu addons for CNI, etc, work
- kubetail on https://github.com/vmware-tanzu/tanzu-framework/tree/main/addons#workflow-of-tanzu-addons-manager
- https://carvel.dev/kapp-controller/docs/latest/packaging/
- PackageRepository
- PackageMetadata
- Package
- PackageInstall
- note the `cluster` field... its multicluster by default
- kubectl edit pkg cert-manager.tanzu.vmware.com.1.5.3+vmware.2-tkg.1-zshippable
- goes to custom APIServer extension
- not a CRD
- Managed by kapp controller

Join Jay and Grayson for another Antrea-LIVE episode - this time - we're going to dig into Antrea 1.4 and test it out by running AntreaProxy without kube-proxy enabled. In addition we'll look at how Antrea enables Feature Gates for new knobs.

- serviceAccount selectors for networkPolicies !
- egress IPs https://antrea.io/docs/v1.3.0/docs/egress/
- Installing Antrea 1.4
- AntreaProxy Enablement
- https://github.com/antrea-io/antrea/pull/2632 doc updates for AntreaProxy
- Running Antrea w/o kube proxy - does it work ?
- VMWare Tanzu feature gates for CNI providers and the Carvel tooling (kapp, ytt)
- kapp-controller
- BONUS ! looking at @aojeas Conntrack fix
- https://github.com/kubernetes/kubernetes/issues/105657
- https://github.com/kubernetes/kubernetes/pull/106163/files

!!! Our THIRD antrea-LIVE show !!!

This time we're focusing on multus, host-local IPAM, and whereabouts - all household terms when it comes to what telco's are up to nowadays. If you ever wondered *why* the telcos are so into having multiple networks for a single container, and how the CNI community overall is adapting to these types of lower level Kubernetes networking challenges, come hang out with us !

Hosts
- jayunit100 (Vmware)
- yashbhutwala (staked)

Guests
- Vivek Seshadari

!!! Our second antrea-LIVE show !!!
Hosts:
- @jayunit100
- @yashbutwala

Guests:
- srikar and vivek

Topics:

- K8s networking news: A new bug in kube-proxy affinity for non-ready pods
- antrea-metrics in real time
Come say hi in #antrea in Kubernetes slack!
- vivek, multus, and multus+telco networking
- https://github.com/antrea-io/antrea/blob/main/docs/network-flow-visibility.md
- Connection tracking (“conntrack”) is a core feature of the Linux kernel's networking stack. It allows the kernel to keep track of all logical network connections or flows, and thereby identify all of the packets which make up each flow so they can be handled consistently together
- conntrack is leveraged by stateful services, like stateful firewall, NAT, that require to track connections. The connections are not just TCP, and can be UDP, ICMP, SCTP, etc.
- Flow record, 2 records per connection, aggregate
- ELK Stack super cool; Antrea layer 4 service mesh
- future: policy recommendation engines + performance analytics
- prometheus installation and OVS monitoring
$ kubectl apply -f build/yamls/antrea-prometheus.yml

$ sleep 60 ; kubectl port-forward pod/prometheus-deployment-79fb7d997f-nkbjj -n monitoring --address 0.0.0.0 9090:9090 -n monitoring

!!! Our first antrea-LIVE show !!!

Focusing on upstream K8s network diagnostics with k8snetlook and FQDN policies (evolution in upstream as well as the antrea specific implementation).

- Antrea 1.3.0
- FQDN Policies
- k8snetlook

Hosts
- jayunit100 (Vmware)
- yashbhutwala (staked)
- sarun87 (Vmware)

thanks to @Arun Sriraman @Jianjun @abhiraut @Yang Ding @vrabbi @Amim Knabben @Luther (monson) for coming !

Todays Topics:
- ClusterNetworkPolicies
- Kubernetes Security
- AntreaProxy
- KubeProxy
- K8sNetLook
- Troubleshooting Kubernetes Services (ClusterIP, NodePort)
- Securing L7 traffic on Kubernetes with FQDN NetworkPolicies

Come say hi in #antrea in Kubernetes slack!