17 Jan 2017
Apache Spark based applications are often comprised of many separate, interconnected components that are a good match for an orchestrated containerized platform like Kubernetes. But with the increased flexibility afforded by these technologies comes a new set of challenges for building rich data-centric applications.
In this SIG session, we will discuss techniques for building multi-component Apache Spark based applications that can be easily deployed and managed on the OpenShift platform. Building on experiences learned while developing and deploying cloud native applications, we will explore common issues that arise during the engineering process and demonstrate workflows for easing the maintenance factors associated with complex installations.
Guest Speaker: Michael McCune Senior Software Engineer, Red Hat
Michael is a software developer in Red Hat’s emerging technology group. He is a contributor to, and core reviewer for the Oshinko project, the Sahara project, the OpenStack Security Project, and the OpenStack API Working Group. For the last two years, he has been creating and deploying data-driven applications and infrastructure for the OpenStack and OpenShift platforms.
In this SIG session, we will discuss techniques for building multi-component Apache Spark based applications that can be easily deployed and managed on the OpenShift platform. Building on experiences learned while developing and deploying cloud native applications, we will explore common issues that arise during the engineering process and demonstrate workflows for easing the maintenance factors associated with complex installations.
Guest Speaker: Michael McCune Senior Software Engineer, Red Hat
Michael is a software developer in Red Hat’s emerging technology group. He is a contributor to, and core reviewer for the Oshinko project, the Sahara project, the OpenStack Security Project, and the OpenStack API Working Group. For the last two years, he has been creating and deploying data-driven applications and infrastructure for the OpenStack and OpenShift platforms.
- 2 participants
- 50 minutes
3 Nov 2016
Guest Speaker: Erik Erlandson, Senior Software Engineer at Red Hat
Apache Spark can be made natively aware of Kubernetes by implementing a Spark scheduler back-end that can run Spark application Drivers and bare Executors in kubernetes pods. In this talk , Erik will explain the design of a native Kubernetes scheduler back-end in Spark and demonstrate a Spark application submission with OpenShift.
For the latest information on OpenShift and available briefings, please visit http://commons.openshift.org or subscribe to the OpenShift Blog (https://blog.openshift.com).
The OpenShift Commons exists to provide a platform for customers, partners, developers and other open source technology initiatives to collaborate, share and accelerate the pace of innovation and adoption of OpenShift globally.
The OpenShift Commons represents a new open collaborative community model designed to facilitate communication and sharing of best practices, feedback and development across the many open source initiatives that integrate with OpenShift. The best way to get involved is to join the conversation today at http://commons.openshift.org
Apache Spark can be made natively aware of Kubernetes by implementing a Spark scheduler back-end that can run Spark application Drivers and bare Executors in kubernetes pods. In this talk , Erik will explain the design of a native Kubernetes scheduler back-end in Spark and demonstrate a Spark application submission with OpenShift.
For the latest information on OpenShift and available briefings, please visit http://commons.openshift.org or subscribe to the OpenShift Blog (https://blog.openshift.com).
The OpenShift Commons exists to provide a platform for customers, partners, developers and other open source technology initiatives to collaborate, share and accelerate the pace of innovation and adoption of OpenShift globally.
The OpenShift Commons represents a new open collaborative community model designed to facilitate communication and sharing of best practices, feedback and development across the many open source initiatives that integrate with OpenShift. The best way to get involved is to join the conversation today at http://commons.openshift.org
- 3 participants
- 19 minutes
31 Aug 2016
The first session of the Big Data Special Interest Group, expands on a a regular Commons session also titled Big Data and Apache Spark on OpenShift.
In the previous session, Red Hat’s Will Benton gave us a vocabulary for talking about data-driven applications and outlined some example architectures for building data-driven applications with microservices. In this session, he’ll provide a tutorial introduction to Apache Spark and walk through an example data-driven application. You’ll learn how to incorporate data-driven behavior into your applications and start building these capabilities with Apache Spark.
William Benton leads a data science team at Red Hat, where he has applied analytic techniques to problems ranging from forecasting cloud infrastructure costs to designing better cycling workouts. His current development focus is contributing to open-source distributed computing projects, but he has also conducted research and development in the areas of static program analysis, managed language runtimes, logic databases, cluster configuration management, and music technology. Benton holds a PhD in computer sciences from the University of Wisconsin.
For the latest information on OpenShift 3.1 and available briefings, please visit http://commons.openshift.org or subscribe to the OpenShift Blog (https://blog.openshift.com).
The OpenShift Commons exists to provide a platform for customers, partners, developers and other open source technology initiatives to collaborate, share and accelerate the pace of innovation and adoption of OpenShift globally.
The OpenShift Commons represents a new open collaborative community model designed to facilitate communication and sharing of best practices, feedback and development across the many open source initiatives that integrate with OpenShift. The best way to get involved is to join the conversation today at http://commons.openshift.org
In the previous session, Red Hat’s Will Benton gave us a vocabulary for talking about data-driven applications and outlined some example architectures for building data-driven applications with microservices. In this session, he’ll provide a tutorial introduction to Apache Spark and walk through an example data-driven application. You’ll learn how to incorporate data-driven behavior into your applications and start building these capabilities with Apache Spark.
William Benton leads a data science team at Red Hat, where he has applied analytic techniques to problems ranging from forecasting cloud infrastructure costs to designing better cycling workouts. His current development focus is contributing to open-source distributed computing projects, but he has also conducted research and development in the areas of static program analysis, managed language runtimes, logic databases, cluster configuration management, and music technology. Benton holds a PhD in computer sciences from the University of Wisconsin.
For the latest information on OpenShift 3.1 and available briefings, please visit http://commons.openshift.org or subscribe to the OpenShift Blog (https://blog.openshift.com).
The OpenShift Commons exists to provide a platform for customers, partners, developers and other open source technology initiatives to collaborate, share and accelerate the pace of innovation and adoption of OpenShift globally.
The OpenShift Commons represents a new open collaborative community model designed to facilitate communication and sharing of best practices, feedback and development across the many open source initiatives that integrate with OpenShift. The best way to get involved is to join the conversation today at http://commons.openshift.org
- 3 participants
- 39 minutes
