扫二维码与项目经理沟通
我们在微信上24小时期待你的声音
解答本文疑问/技术咨询/运营咨询/技术建议/互联网交流
jdk下载地址 http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html 添加hadoop用户 [root@centos67 ~]# useradd hadoop [root@centos67 ~]# passwd hadoop [root@centos67 ~]# ssh-keygen #生成密钥 [root@centos67 ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@127.0.0.1 将主机名写入hosts文件中 [root@centos67 ~]# echo "127.0.0.1 `hostname`" >> /etc/hosts 下载jdk-8u71-linux-x64.tar.gz [root@centos67 ~]# tar xf jdk-8u71-linux-x64.tar.gz -C /usr/local/ [root@centos67 ~]# cd /usr/local/ [root@centos67 local]# mv jdk1.8.0_71 java [root@centos67 ~]# vim /etc/profile.d/java.sh #!/bin/bash export JAVA_HOME=/usr/local/java #这里是你对应的jdk目录 export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib export PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/jre/bin [root@centos67 ~]# bash /etc/profile.d/java.sh 测试jdk是否安装成功 [root@centos67 ~]# java -version java version "1.8.0_71" Java(TM) SE Runtime Environment (build 1.8.0_71-b15) Java HotSpot(TM) 64-Bit Server VM (build 25.71-b15, mixed mode) hadoop下载地址 http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.6.3/hadoop-2.6.3.tar.gz 下载hadoop-2.6.3.tar.gz [root@centos67 ~]# tar xf hadoop-2.6.3.tar.gz -C /usr/local/ [root@centos67 ~]# cd /usr/local/ [root@centos67 local]# mv hadoop-2.6.3 hadoop [root@centos67 ~]# vim /etc/profile.d/hadoop.sh #!/bin/bash export HADOOP_HOME=/usr/local/hadoop export PATH=$PATH:$HADOOP_HOME/bin [root@centos67 ~]# bash /etc/profile.d/hadoop.sh 测试hadoop是否安装成功 [root@centos67 ~]# hadoop version Hadoop 2.6.3 Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r cc865b490b9a6260e9611a5b8633cab885b3d247 Compiled by jenkins on 2015-12-18T01:19Z Compiled with protoc 2.5.0 From source with checksum 722f77f825e326e13a86ff62b34ada This command was run using /usr/local/hadoop/share/hadoop/common/hadoop-common-2.6.3.jar 下面开始正式的配置hadoop 涉及到的文件有: /usr/local/hadoop/etc/hadoop/ hadoop-env.sh yarn-env.sh yarn-site.xml core-site.xml hdfs-site.xml mapred-site.xml 修改hadoop-env.sh配置文件 [root@centos67 ~]# cd /usr/local/hadoop/etc/hadoop [root@centos67 ~]# vim hadoop-env.sh #添加以下内容 export JAVA_HOME=/usr/local/java export HADOOP_PREFIX=/usr/local/hadoop 修改yarn-env.sh配置文件,将以下内容替换 [root@centos67 ~]# sed -i 's@^JAVA_HOME=$JAVA_HOME@JAVA_HOME=/usr/local/java@' yarn-env.sh 修改core-site.xml配置文件,配置hdfs的访问地址和端口 [root@centos67 ~]# vim core-site.xml修改hdfs-site.xml配置文件,备份方式默认为3,我的是单机模式,所以改为1了 [root@centos67 ~]# mkdir -p /home/hadoop/hdfs/{name,data} [root@centos67 ~]# chown -R hadoop:hadoop /home/hadoop/hdfs [root@centos67 ~]# vim hdfs-site.xml fs.default.name hdfs://centos67:9000 hdfs的访问地址 hadoop.tmp.dir /home/hadoop/tmp/ hdfs的临时目录 修改hadoop配置文件MapReduce的配置文件mapred-site.xml, [root@centos67 ~]# vim mapred-site.xml dfs.name.dir file:/home/hadoop/hdfs/name dfs.data.dir file:/home/hadoop/hdfs/data dfs.replication 1 修改hadoop配置文件yarn-site.xml [root@centos67 ~]# vim yarn-site.xml mapreduce.framework.name yarn 至此,hadoop可以格式化并启动了。 [root@centos67 ~]# hadoop namenode -format [root@centos67 ~]# /usr/local/hadoop/sbin/start-dfs.sh [root@centos67 ~]# jps 1472 DataNode 1660 SecondaryNameNode 2159 Jps 1375 NameNode 有这么几个进程,说明已经成功启动了 [root@centos67 ~]# /usr/local/hadoop/sbin/start-yarn.sh 1472 DataNode 2294 NodeManager 2392 Jps 1660 SecondaryNameNode 2205 ResourceManager 1375 NameNode 查看是否有9000端口 [root@centos67 ~]# netstat -lnpt | grep 9000 网页访问 http://localhost:50070 yarn.nodemanager.aux-services mapreduce_shuffle yarn.resourcemanager.webapp.address ${yarn.resourcemanager.hostname}:8099
我们在微信上24小时期待你的声音
解答本文疑问/技术咨询/运营咨询/技术建议/互联网交流