14 May 2021
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Customized Cloud Native Logging: Fluent Bit in Open Service Mesh - Sanya Kochhar, Microsoft
Fluent Bit is a powerful tool for filtering and forwarding logs, and this talk will explore how it can be easily integrated into CNCF projects in containerized environments, be customized for users’ logging needs and be the right solution for your project. It is used by Open Service Mesh (OSM), an open-source cloud native service mesh, to help users process and send logs to various destinations. So why did OSM choose this particular CNCF log forwarding solution? How can those learnings help projects in the community inform their choice? In this talk, we will go into a practical example to demonstrate how to support multiple user scenarios with a few quick toggles; more specifically, filtering on log level, custom annotating logs and running traffic behind outbound proxy servers. This will showcase how OSM leverages Fluent Bit’s functionality and plugins for its log forwarding needs.
Customized Cloud Native Logging: Fluent Bit in Open Service Mesh - Sanya Kochhar, Microsoft
Fluent Bit is a powerful tool for filtering and forwarding logs, and this talk will explore how it can be easily integrated into CNCF projects in containerized environments, be customized for users’ logging needs and be the right solution for your project. It is used by Open Service Mesh (OSM), an open-source cloud native service mesh, to help users process and send logs to various destinations. So why did OSM choose this particular CNCF log forwarding solution? How can those learnings help projects in the community inform their choice? In this talk, we will go into a practical example to demonstrate how to support multiple user scenarios with a few quick toggles; more specifically, filtering on log level, custom annotating logs and running traffic behind outbound proxy servers. This will showcase how OSM leverages Fluent Bit’s functionality and plugins for its log forwarding needs.
- 1 participant
- 23 minutes
14 May 2021
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Fluent Bit - Swiss Army Tool of Observability Data Ingestion - Michael Marshall, Neiman Marcus
In this session, Michael will show how Fluent Bit became his Swiss Army tool for observability data ingestion, how he was able to add dynamic log metrics extraction by writing a custom Golang plugin, and as an added bonus, help lower his organization’s AWS costs by selecting Fluent-Bit. Neiman Marcus needed an architecture of both open source log and metrics ingestion projects running in AWS. Michael built out an enterprise quality, fully scalable, performant, solution. All teams began lining up demanding Michael ingest their system and application logs, as soon as possible. On top of this, each team had their own formats and data schema challenges. Michael will share how he overcame those challenges using Fluent Bit. This session is complete with code examples, a brief demo, and lessons learned.
Fluent Bit - Swiss Army Tool of Observability Data Ingestion - Michael Marshall, Neiman Marcus
In this session, Michael will show how Fluent Bit became his Swiss Army tool for observability data ingestion, how he was able to add dynamic log metrics extraction by writing a custom Golang plugin, and as an added bonus, help lower his organization’s AWS costs by selecting Fluent-Bit. Neiman Marcus needed an architecture of both open source log and metrics ingestion projects running in AWS. Michael built out an enterprise quality, fully scalable, performant, solution. All teams began lining up demanding Michael ingest their system and application logs, as soon as possible. On top of this, each team had their own formats and data schema challenges. Michael will share how he overcame those challenges using Fluent Bit. This session is complete with code examples, a brief demo, and lessons learned.
- 1 participant
- 22 minutes
14 May 2021
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Keynote: DevSecOps in the US Air Force - Nicolas Chaillan, Air Force Chief Software Officer, US Air Force
In this session, Mr. Nicolas Chaillan will share how the US Department of Defense is embracing DevSecOps to drive innovation and digital modernization within the Department. This will showcase how any organization can leverage Kubernetes and Itsio to deliver value at the speed of relevance.
Keynote: DevSecOps in the US Air Force - Nicolas Chaillan, Air Force Chief Software Officer, US Air Force
In this session, Mr. Nicolas Chaillan will share how the US Department of Defense is embracing DevSecOps to drive innovation and digital modernization within the Department. This will showcase how any organization can leverage Kubernetes and Itsio to deliver value at the speed of relevance.
- 1 participant
- 18 minutes
14 May 2021
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Keynote: Fireside chat with Martin Mao, CEO of Chronosphere - Anurag Gupta, Production Manager, Calyptia; Martin Mao, CEO/Co-Founder, Chronosphere
In this session join Martin Mao, CEO of Chronosphere, in a fireside chat with Fluent maintainer Anurag to discuss the evolution of logging in relation to observability and answering questions about the broader Fluent Ecosystem.
Keynote: Fireside chat with Martin Mao, CEO of Chronosphere - Anurag Gupta, Production Manager, Calyptia; Martin Mao, CEO/Co-Founder, Chronosphere
In this session join Martin Mao, CEO of Chronosphere, in a fireside chat with Fluent maintainer Anurag to discuss the evolution of logging in relation to observability and answering questions about the broader Fluent Ecosystem.
- 2 participants
- 29 minutes
14 May 2021
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Lightning Talk: Fluentd as syslog drain for CloudFoundry - Karsten Schnitter, SAP SE
CloudFoundry can ship logs of deployed applications to user provided syslog drains. This talk will show, how to configure Fluentd to receive logs messages via syslog. Furthermore, parsing the log messages will be explained with special attention on the structured metadata from CloudFoundry and parsing of the access logs generated by the gorouter. This results in a basic configuration, that can be enriched by custom parsing rules for the log messages.
Lightning Talk: Fluentd as syslog drain for CloudFoundry - Karsten Schnitter, SAP SE
CloudFoundry can ship logs of deployed applications to user provided syslog drains. This talk will show, how to configure Fluentd to receive logs messages via syslog. Furthermore, parsing the log messages will be explained with special attention on the structured metadata from CloudFoundry and parsing of the access logs generated by the gorouter. This results in a basic configuration, that can be enriched by custom parsing rules for the log messages.
- 1 participant
- 10 minutes
14 May 2021
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Lightning Talk: Fluentd at Scale; Keys to successful Logging Syndication - Fred Moyer, Zendesk
This lighting talk will show how Zendesk uses Fluentd to manage a firehose of systems and application logging telemetry to multiple consumers. It will go into detail on configuration and operational details about how they syndicate logs to different systems for stakeholders such as security engineers, application developers, and infrastructure teams. If you find yourself dealing with an increasing firehose of logging telemetry with multiple teams consuming that telemetry, this talk is for you.
Lightning Talk: Fluentd at Scale; Keys to successful Logging Syndication - Fred Moyer, Zendesk
This lighting talk will show how Zendesk uses Fluentd to manage a firehose of systems and application logging telemetry to multiple consumers. It will go into detail on configuration and operational details about how they syndicate logs to different systems for stakeholders such as security engineers, application developers, and infrastructure teams. If you find yourself dealing with an increasing firehose of logging telemetry with multiple teams consuming that telemetry, this talk is for you.
- 1 participant
- 9 minutes
14 May 2021
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Lightning Talk: Parsing CRI JSON logs with Fluent Bit - Bart Robertson & Joseph Fultz, Microsoft
With dockerd deprecated as a Kubernetes container runtime, we moved to containerd. After the change, our Fluent Bit logging didn't parse our JSON logs correctly. containerd and CRI-O use the CRI Log format which is slightly different and requires additional parsing to parse JSON application logs. This session will detail those differences and present an end-to-end example on how we resolved the issues.
Lightning Talk: Parsing CRI JSON logs with Fluent Bit - Bart Robertson & Joseph Fultz, Microsoft
With dockerd deprecated as a Kubernetes container runtime, we moved to containerd. After the change, our Fluent Bit logging didn't parse our JSON logs correctly. containerd and CRI-O use the CRI Log format which is slightly different and requires additional parsing to parse JSON application logs. This session will detail those differences and present an end-to-end example on how we resolved the issues.
- 2 participants
- 6 minutes
14 May 2021
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Lightning Talk: Scaling the Fluent Bit Kubernetes Filter in very large clusters - Drew Zhang & Nithish Kumar Murcherla, Amazon
Fluent Bit and Fluentd users have experienced scaling issues with the Kubernetes filter in very large clusters. The Kube API Server could not handle the quantity of requests and became unresponsive. To address this issue, a new proposal for relying on the kubelet to obtain pod metadata was implemented. By this approach, Kubernetes filter could get metadata from the node and could mitigate the work load for kube API server. This session discusses the scaling issue in detail, the solution that was implemented, it’s trade-offs, and how it was tested.
Lightning Talk: Scaling the Fluent Bit Kubernetes Filter in very large clusters - Drew Zhang & Nithish Kumar Murcherla, Amazon
Fluent Bit and Fluentd users have experienced scaling issues with the Kubernetes filter in very large clusters. The Kube API Server could not handle the quantity of requests and became unresponsive. To address this issue, a new proposal for relying on the kubelet to obtain pod metadata was implemented. By this approach, Kubernetes filter could get metadata from the node and could mitigate the work load for kube API server. This session discusses the scaling issue in detail, the solution that was implemented, it’s trade-offs, and how it was tested.
- 2 participants
- 9 minutes
14 May 2021
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Logging Operator the Cloud Native Fluent Ecosystem - Sandor Guba, Cisco
The Kubernetes logging ecosystem does not support the native Kubernetes concepts. This is the reason behind the open-source project called Logging Operator. Moreover, the easy scaling of workloads results in a massive amount of logs produced. This can cause extremely huge bills on your analytics deployment (both SaaS and self-hosted). Some of the key features are namespace isolation, supporting label selectors, filtering, and using secrets for credentials. It is more than 3 years old and already deployed in production environments. Sandor will introduce the challenges of configuring a complex logging stack on Kubernetes and summarise the lessons learned of writing, supporting, and operating the Logging Operator.
Logging Operator the Cloud Native Fluent Ecosystem - Sandor Guba, Cisco
The Kubernetes logging ecosystem does not support the native Kubernetes concepts. This is the reason behind the open-source project called Logging Operator. Moreover, the easy scaling of workloads results in a massive amount of logs produced. This can cause extremely huge bills on your analytics deployment (both SaaS and self-hosted). Some of the key features are namespace isolation, supporting label selectors, filtering, and using secrets for credentials. It is more than 3 years old and already deployed in production environments. Sandor will introduce the challenges of configuring a complex logging stack on Kubernetes and summarise the lessons learned of writing, supporting, and operating the Logging Operator.
- 1 participant
- 21 minutes
14 May 2021
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Noise Cancelling Headphones for Fluent Bit - Powered by Lua - Matt Lehman & Guillermo Menjivar, GreyNoise Intelligence Inc.,
This session will demonstrate how to use Fluent Bit to rapidly prototype IP address enrichment and filtering techniques. You will learn how to write a Lua filter plugin for Fluent Bit, how to use 3rd party Lua modules, and how to add a Lua test harness. Anyone that has plugged a device directly into the internet and checked their logs has wondered “what is all this activity?”. Matt runs a global network of hundreds of sensors at GreyNoise, so he sees this “internet background noise” daily and thinks about how to deal with log volume, enrichment, and routing a lot. If you work in a security operations center, site reliability engineering, or data engineering, this session will show you how to reduce log volume within your organization by tuning out the noise.
Noise Cancelling Headphones for Fluent Bit - Powered by Lua - Matt Lehman & Guillermo Menjivar, GreyNoise Intelligence Inc.,
This session will demonstrate how to use Fluent Bit to rapidly prototype IP address enrichment and filtering techniques. You will learn how to write a Lua filter plugin for Fluent Bit, how to use 3rd party Lua modules, and how to add a Lua test harness. Anyone that has plugged a device directly into the internet and checked their logs has wondered “what is all this activity?”. Matt runs a global network of hundreds of sensors at GreyNoise, so he sees this “internet background noise” daily and thinks about how to deal with log volume, enrichment, and routing a lot. If you work in a security operations center, site reliability engineering, or data engineering, this session will show you how to reduce log volume within your organization by tuning out the noise.
- 2 participants
- 26 minutes
14 May 2021
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Scaling Log Collection with Fluent Bit - Wesley Pettit & Ugur Kira, Amazon
Ugur and Wesley will present their experience migrating customers first from Fluentd to Fluent Bit, and then to Fluent Bit with multi-workers. Benchmarking data will be shared along with recommendations on ideal settings for different use cases. Finally, they will end with a discussion of remaining scale limitations and possible solutions to solve them.
Scaling Log Collection with Fluent Bit - Wesley Pettit & Ugur Kira, Amazon
Ugur and Wesley will present their experience migrating customers first from Fluentd to Fluent Bit, and then to Fluent Bit with multi-workers. Benchmarking data will be shared along with recommendations on ideal settings for different use cases. Finally, they will end with a discussion of remaining scale limitations and possible solutions to solve them.
- 2 participants
- 27 minutes
14 May 2021
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Welcome to FluentCon! What's new with Fluentd and Fluent Bit - Anurag Gupta, Product Manager Calyptia
Welcome to FluentCon! What's new with Fluentd and Fluent Bit - Anurag Gupta, Product Manager Calyptia
- 1 participant
- 9 minutes
