Jeff Markham讲解Spark和YARN的融合以及YARN应用的开发

由哈里森 · 发布日期 2016年3月2日 · 已更新 2016年4月10日

Jeff Markham

Hortonworks亚太区技术总监

Hortonworks是一家开源Hadoop版本的提供商。在此之前，Jeff曾在VMware,Red Hat和IBM使用分布式数据来帮助企业构建分布式应用程序。他曾为Hadoop的用户组写过Java应用程序开发方面的文章并在会上公开演讲。Jeff对Apache Pig和ApacheHDFS做出过贡献。他也是《Apache Hadoop YARN: Moving Beyond MapReduce and BatchProcessing with Apache Hadoop 2》这本书的合著者。

本次会议Jeff为我们带来了两个议题，分别是Spark与YARN的融合、YARN应用的开发。具体内容请往下看：

Keynote Subject: Spark and YARN

Abstract: Since the introduction of Hadoop 2, YARN hasbeen the cornerstone of new applications built on Hadoop. Applicationscome in various forms: existing open source applications ported to YARN,proprietary software made to run on a Hadoop cluster, or even customapplications. In this session, we’ll talk about how the two most popularopen source projects of the past 5 years have come together. Spark andYARN come together to create a compelling technical and business solution. Find out the latest on what’s happening in this space.

Session Subject: YARN Application Development

Abstract: Building custom applications on top ofYARN is a daunting task. Some frameworks exist to address some of thecomplexity but none address all the issues you’ll face. In this session,we’ll have a code review of a simple application that we can test, debug, anddeploy to YARN. Find out how to incorporate newer YARN features such asthe Registry and Timeline Server into your own custom apps.