F嘉阳 - YARN
type: channel
link: https://blog.fjy8018.top/index.php/tag/YARN/
items: YARN单节点伪分布式搭建

YARN单节点伪分布式搭建
type: item
date: 2018-04-09T20:14:00+08:00
description: 配置YARN 首先启动之前配置好的HDFS服务，然后进入/www/hadoop/hadoop-2.6.0-cdh5.7.0/etc/hadoop目录，按照官方指导配置YARN单节点环境 使用cp mapred-site.xml.template mapred-site.xml命令把mapred-site.xml.template复制一份，加入官方的配置 mapreduce.framework.name yarn 同时对yarn-site.xml也进行配置 yarn.nodemanager.aux-services mapreduce_shuffle 启动YARN 进入/www/hadoop/hadoop-2.6.0-cdh5.7.0/sbin目录，使用./start-yarn.sh命令启动YARN服务 [hadoop@localhost sbin]$ ./start-yarn.sh starting yarn daemons starting resourcemanager, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/yarn-hadoop-resourcemanager-localhost.localdomain.out localhost: starting nodemanager, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/yarn-hadoop-nodemanager-localhost.localdomain.out 用jps命令查看是否启动成功，看到如下显示即为启动成功，主要是NodeManager和ResourceManager两个服务 [hadoop@localhost sbin]$ jps 7056 SecondaryNameNode 7475 NodeManager 6775 NameNode 6872 DataNode 7374 ResourceManager 7679 Jps 开放端口 YARN的浏览器客户端占用8088端口，故使用宝塔面板开放8088端口 访问YARN图形化界面 使用浏览器打开http://ip:8088/可见如下界面，则YARN启动成功 测试 YARN官方提供了示例java程序来测试资源调度，进入/www/hadoop/hadoop-2.6.0-cdh5.7.0/share/hadoop/mapreduce目录，可见 [hadoop@localhost mapreduce]$ ls hadoop-mapreduce-client-app-2.6.0-cdh5.7.0.jar hadoop-mapreduce-client-nativetask-2.6.0-cdh5.7.0.jar hadoop-mapreduce-client-common-2.6.0-cdh5.7.0.jar hadoop-mapreduce-client-shuffle-2.6.0-cdh5.7.0.jar hadoop-mapreduce-client-core-2.6.0-cdh5.7.0.jar hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar hadoop-mapreduce-client-hs-2.6.0-cdh5.7.0.jar lib hadoop-mapreduce-client-hs-plugins-2.6.0-cdh5.7.0.jar lib-examples hadoop-mapreduce-client-jobclient-2.6.0-cdh5.7.0.jar sources hadoop-mapreduce-client-jobclient-2.6.0-cdh5.7.0-tests.jar 其中hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar就是MapReduce的测试程序，通过hadoop jar的方式来运行 用命令查看可配置的参数hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar [hadoop@localhost mapreduce]$ hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar An example program must be given as the first argument. Valid program names are: aggregatewordcount: An Aggregate based map/reduce program that …