This website requires JavaScript.

Oozie 安装配置记录

编译Oozie

下载Oozie

wget http://apache.01link.hk/oozie/4.2.0/oozie-4.2.0.tar.gz
tar -xzf oozie-4.2.0.tar.gz

修改配置文件

<targetJavaVersion>1.7</targetJavaVersion>
<hadoop.version>2.6.3</hadoop.version>
codehaus 源已经废弃,因此注释掉
<!-- <repository>
     <id>Codehaus repository</id>
     <url>http://repository.codehaus.org/</url>
     <snapshots>
         <enabled>false</enabled>
     </snapshots>
</repository> -->

编译

cd bin
./mkdistro.sh –DskipTests

Oozie服务器部署准备

输出目录中 /usr/local/oozie-4.2.0/distro/target/oozie-4.2.0-distro/oozie-4.2.0  包含了所有Hadoop群集要用到的组件 把oozie部署到Slave1上

scp oozie-4.2.0-distro.tar.gz hadoop@Slave1:~/ tar -xzf oozie-4.2.0-distro.tar.gz
解压后在目录中创建库目录
mkdir libext cd libext
复制Hadoop 库
cp ${HADOOP_HOME}/share/hadoop//.jar ./ cp ${HADOOP_HOME}/share/hadoop//lib/.jar ./
去掉hadoop与tomcat冲突的jar包
mv jasper-compiler-5.5.23.jar jasper-compiler-5.5.23.jar.bak mv jasper-runtime-5.5.23.jar jasper-runtime-5.5.23.jar.bak mv jsp-api-2.1.jar jsp-api-2.1.jar.bak
同时放置ExtJS 2.2 进该目录

下载MySQL数据库的jar 包放进去

准备War文件

cd /home/hadoop/oozie-4.2.0
bin/oozie-setup.sh prepare-war
结果如下 [![image](http://www.bihell.com/wp-content/uploads/2016/02/image_thumb.png "image")](http://www.bihell.com/wp-content/uploads/2016/02/image.png)

上传这些jar包

bin/oozie-setup.sh  sharelib  create -fs hdfs://<namenode-hostname>:8020
接着编辑Hadoop的core-site.xml ,添加以下属性
<property>
    <name>hadoop.proxyuser.hadoop.hosts</name>
    <value>*</value>
</property>
<property>
    <name>hadoop.proxyuser.hadoop.groups</name>
    <value>*</value>
</property>
编辑位于oozie-4.2.0/conf 中的oozie-site.xml  添加以下内容,注意修改hadoop配置文件的位置以及用户
<property>
     <name>oozie.service.JPAService.create.db.schema</name>
     <value>true</value>
</property>
<property>
     <name>oozie.service.JPAService.jdbc.driver</name>
     <value>com.mysql.jdbc.Driver</value>
</property>
<property>
     <name>oozie.service.JPAService.jdbc.url</name>
     <value>jdbc:mysql://localhost:3306/oozie?createDatabaseIfNotExist=true</value>
</property>

<property>
     <name>oozie.service.JPAService.jdbc.username</name>
     <value>oozie</value>
</property>

<property>
     <name>oozie.service.JPAService.jdbc.password</name>
     <value>oozie</value>
</property>
<property>
     <name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
     <value>*=/opt/hadoop/etc/hadoop</value>
</property> 
<!-- 下面两个配置是hue用到的-->
<property>
     <name>oozie.service.ProxyUserService.proxyuser.hue.hosts</name>
     <value>*</value>
</property>

<property>
     <name>oozie.service.ProxyUserService.proxyuser.hue.groups</name>
     <value>*</value>
</property>

创建Oozie DB

安装MySQL

sudo yum install mysql-server sudo service mysqld restart sudo mysql_secure_installation
创建oozie用户
create database oozie; grant all privileges on oozie.* to oozie@'%' identified by 'oozie'; FLUSH PRIVILEGES;
初始化数据库

bin/ooziedb.sh create -sqlfile oozie.sql –run

启动Oozie

设置把环境变量

#set Oozie home export OOZIE_VERSION=4.2.0 export OOZIE_HOME=/usr/local/oozie-4.2.0 export PATH=$OOZIE_HOME/bin:$PATH
启动

oozied.sh start
日志文件
cat logs/oozie.log
查看状态,正常情况下应该显示System mode: NORMAL
bin/oozie admin -oozie http://localhost:11000/oozie –status

测试一个工作流

将Oozie样例文件放到Hadoop 文件目录中

tar -xzf oozie-examples.tar.gz hadoop fs -put examples examples bin/oozie job -oozie http://localhost:11000/oozie/ -config examples/apps/map-reduce/job.properties  -run
实际情况不一样,可能要修改一下job.properties的属性
nameNode=hdfs://Master:8020 jobTracker=Master:8032 queueName=default examplesRoot=examples

oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/apps/map-reduce/workflow.xml outputDir=map-reduce

参考文档

Building Oozie

Oozie Quick Start Beginner’s Guide For Oozie Installation oozie 安装过程总结 Oozie4.2.0配置安装实战 Hadoop 2.0中用户安全伪装/模仿机制实现原理 Oozie Examples Apache Oozie Installation on Hadoop 2.4.1

0条评论
avatar