This website requires JavaScript.

Hadoop:Cloudera Manager 数据库配置

安装MySQL服务器

yum install mysql-server
1.停止MySQL服务
service mysqld stop
2.将老的/var/lib/mysql/ib_logfile0 和 /var/lib/mysql/ib_logfile1 移到其他地方备份好 3.修改 /etc/my.cnf 对MySQL进行配置
  • 为了防止死锁,设置隔离级别为read committed (这样能防止死锁? 求解)
  • 将引擎配置为InnoDB ,默认情况下用的是MyISAM引擎,Cloudera Manager就没法运行.因此需要配置,可以使用该命令来查看当前所用的引擎mysql> show table status;
  • 默认情况下MySQL安装好以后使用了比较保守的内存和缓存配置.CM服务需要进行大量的读写.建议将innodb_flush_method 设置为 O_DIRECT
  • 根据你的群集大小设置max_connections 小群集(小于50台) -- 你可以在一台机器上配置多个数据库(比如Activity Monitor或者Service Monitor) .如果你要这样做需要进行以下处理: 每个数据库放在各自的存储卷中. 同事需要为每个数据库设置100个最大连接,然后附加50个连接.比如有两个数据库设置最大连接为250. 如果你在一个主机放了5个数据库(he databases for Cloudera Manager Server, Activity Monitor, Reports Manager, Cloudera Navigator, and Hive metastore),那就要将最大连接数设置为550.大型集群(超过50台) -- 不要把数据库装在一个机器上,虽然不要求用一个服务器专门来放数据库,但是必须把每个数据库放在不同的主机上.
  • Binary logging 对Cloudera Manager 来说不是必要的,not a requirement提供了复制,增量备份还原等好处.具体可以看The Binary Log 以下是Cloudera 推荐的配置:
[mysqld]
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
# symbolic-links = 0

key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M

#log_bin should be on a disk with enough free space. Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your system
#and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log

# For MySQL version 5.1.8 or later. Comment out binlog_format for older versions.
binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

sql_mode=STRICT_ALL_TABLES
我的配置:
mysqld]
default-storage-engine=INNODB
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
4.如果装了AppArmor 则需要针对MySQL配置,允许它写二进制. 5.设置MySQL启动运行
$ sudo /sbin/chkconfig mysqld on
$ sudo /sbin/chkconfig --list mysqld
mysqld          0:off   1:off   2:on    3:on    4:on    5:on    6:off
6.启动MySQL服务器
$ sudo service mysqld start
7\. 设置root密码
$ sudo /usr/bin/mysql_secure_installation
[...]
Enter current password for root (enter for none):
OK, successfully used password, moving on...
[...]
Set root password? [Y/n] y
New password:
Re-enter new password:
Remove anonymous users? [Y/n] Y
[...]
Disallow root login remotely? [Y/n] N
[...]
Remove test database and access to it [Y/n] Y
[...]
Reload privilege tables now? [Y/n] Y
All done!

安装MySQL JDBC驱动

下载地址: http://dev.mysql.com/downloads/connector/j/

wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.38.tar.gz tar zxvf mysql-connector-java-5.1.38.tar.gz
拷贝JDBC驱动,放置到相关主机

cp mysql-connector-java-5.1.38/mysql-connector-java-5.1.38-bin.jar /usr/share/java/mysql-connector-java.jar
如果目标目录不存在,请手动创建
mkdir -p /usr/share/java/
cp mysql-connector-java-5.1.38/mysql-connector-java-5.1.38-bin.jar /usr/share/java/mysql-connector-java.jar

创建并配置Activity Monitor, Reports Manager, Hive Metastore Server, Sentry Server, Cloudera Navigator Audit Server, and Cloudera Navigator Metadata Server

登陆后按照以下命令根据表格创建各个数据库

mysql> create database database DEFAULT CHARACTER SET utf8; Query OK, 1 row affected (0.00 sec)

mysql> grant all on database.* TO 'user'@'%' IDENTIFIED BY 'password'; Query OK, 0 rows affected (0.00 sec)

image

配置Oozie服务器

$ mysql -u root -p
Enter password: ******

mysql> create database oozie;
Query OK, 1 row affected (0.03 sec)

mysql>  grant all privileges on oozie.* to 'oozie'@'localhost' identified by 'oozie';
Query OK, 0 rows affected (0.03 sec)

mysql>  grant all privileges on oozie.* to 'oozie'@'%' identified by 'oozie';
Query OK, 0 rows affected (0.03 sec)

mysql> exit
Bye
拷贝JDBC驱动至Oozie目录 /var/lib/oozie/
0条评论
avatar