Winse Blog

走走停停, 熙熙攘攘, 忙忙碌碌, 不知何畏.

使用Puppet安装配置Ganglia

前面写过完全纯手工和用yum安装依赖来安装ganglia的文章,最近生产安装了puppet,既然已经手上已有牛刀,杀鸡就不用再取菜刀了。今天记录下前几天使用puppet安装ganglia的经历。

前提(自己操作过熟悉怎么用)

  • 配置过私有仓库 (createrepo)
  • 安装好puppet
  • 编译过自己的rpm (rpmbuild)

编译gmetad,gmond,gweb

点击链接下载SPEC:

然后编译打包:

先手动编译安装 ganglia ,把依赖的问题处理好。编译安装没问题,然后再使用 rpmbuild 编译生成 rpm 包!!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
# 1> 建立目录结构
mkdir ganglia-build
cd ganglia-build
mkdir BUILD RPMS SOURCES SPECS SRPMS

# 2> 修改配置
# ganglia-web-3.7.1.tar.gz的makefile、conf_default.php.in修改下,根据等下要配置gmetad的参数进行修改

less ganglia-web-3.7.1/Makefile 
  # Location where gweb should be installed to (excluding conf, dwoo dirs).
  GDESTDIR = /var/www/html/ganglia

  # Location where default apache configuration should be installed to.
  GCONFDIR = /usr/local/ganglia/etc/

  # Gweb statedir (where conf dir and Dwoo templates dir are stored)
  GWEB_STATEDIR = /var/www/html/ganglia

  # Gmetad rootdir (parent location of rrd folder)
  GMETAD_ROOTDIR = /data/ganglia

  APACHE_USER = apache

# 连外网太慢,下载放到本地
less ganglia-web-3.7.1/conf_default.php.in 
  #$conf['cubism_js_path'] = "js/cubism.v1.min.js";
  $conf['jquery_js_path'] = "js/jquery.min.js";
  $conf['jquerymobile_js_path'] = "js/jquery.mobile.min.js";
  $conf['jqueryui_js_path'] = "js/jquery-ui.min.js";
  $conf['rickshaw_js_path'] = "js/rickshaw.min.js";
  $conf['cubism_js_path'] = "js/cubism.v1.min.js";
  $conf['d3_js_path'] = "js/d3.min.js";
  $conf['protovis_js_path'] = "js/protovis.min.js";

# 3> 源文件
# 把文件放到SOURCES目录下,
ls SOURCES/
  ganglia-3.7.2.tar.gz  ganglia-web-3.7.1.tar.gz

# 4> 编译生成RPM
rpmbuild -v -ba SPECS/gmetad.spec 
rpmbuild -v -ba SPECS/gmond.spec 
rpmbuild -v -ba SPECS/gweb.spec 

# 5> 查看内容
rpm -qpl RPMS/x86_64/ganglia-3.7.2-1.el6.x86_64.rpm 

本地仓库

这里假设已经把系统光盘做成了本地仓库。

先安装httpd、php、createrepo,然后按照下面的步骤创建本地仓库:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# 系统带的可以从光盘拷贝,直接映射到httpd的目录下即可
[hadoop@hadoop-master1 rhel6.3]$ ls 
Packages  repodata
[hadoop@hadoop-master1 html]$ pwd
/var/www/html
[hadoop@hadoop-master1 html]$ ll
lrwxrwxrwx.  1 root root   20 2月  15 2014 rhel6.3 -> /opt/rhel6.3

[hadoop@hadoop-master1 ~]$ sudo mkdir -p /opt/dta/repo
[hadoop@hadoop-master1 ~]$ cd /opt/dta/repo
[hadoop@hadoop-master1 repo]$ ls *.rpm
gmetad-3.7.2-1.el6.x86_64.rpm  gmond-3.7.2-1.el6.x86_64.rpm  gweb-3.7.1-1.el6.x86_64.rpm  libconfuse-2.7-4.el6.x86_64.rpm

[hadoop@hadoop-master1 repo]$ sudo createrepo .
3/3 - libconfuse-2.7-4.el6.x86_64.rpm                                           
Saving Primary metadata
Saving file lists metadata
Saving other metadata

# 映射到httpd目录下
[hadoop@hadoop-master1 yum.repos.d]$ cd /var/www/html/
[hadoop@hadoop-master1 html]$ sudo ln -s /opt/dta/repo dta

# 加入本地仓库源
[hadoop@hadoop-master1 yum.repos.d]$ sudo cp puppet.repo dta.repo
[hadoop@hadoop-master1 yum.repos.d]$ sudo vi dta.repo 
[dta]
name=DTA Local
baseurl=http://hadoop-master1:801/dta
enabled=1
gpgcheck=0

注意: 在安装的时刻找不到gmond,可以先清理yum的缓冲: yum clean all

puppet模块

添加了三个模块,用于主机添加repo配置和sudo配置,以及安装配置gmond。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[root@hadoop-master1 modules]# tree $PWD
/etc/puppetlabs/code/environments/production/modules
├── dtarepo
│   ├── manifests
│   │   └── init.pp
│   └── templates
│       └── dta.repo
├── gmond
│   ├── manifests
│   │   └── init.pp
│   └── templates
│       └── gmond.conf
└── sudo
    ├── manifests
    │   └── init.pp
    └── templates
        └── sudo.erb

都比较简单,通过init.pp来进行配置,然后加载模板,写入到同步主机本地文件中。

  • dtarepo
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
./dtarepo/manifests/init.pp
class dtarepo {

file{'/etc/yum.repos.d/dta.repo':
  ensure => file,
  content => template('dtarepo/dta.repo'),
}

}

./dtarepo/templates/dta.repo
[dta]
name=DTA Local
baseurl=http://hadoop-master1:801/dta
enabled=1
gpgcheck=0
  • sudo:用于sudo切root,方便调试
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
./sudo/manifests/init.pp
class sudo {

if ( $::hostname =~ /(^cu-omc)/ ) {
  $user = 'omc'
} elsif ( $::hostname =~ /(^cu-uc)/ ) {
  $user = 'uc'
} elsif ( $::hostname =~ /(^cu-ud)/ ) {
  $user = 'ud'
} elsif ( $::hostname =~ /(^cu-db)/ ) {
  $user = 'mysql'
} else {
  $user = 'hadoop'
}


file { "/etc/sudoers.d/10_$user":
  ensure => file,
  mode => '0440', 
  content => template('sudo/sudo.erb'),
}


}

./sudo/templates/sudo.erb
<%= scope.lookupvar('sudo::user') %> ALL=(ALL) NOPASSWD: ALL
  • gmond

在默认的gmond.conf基础上修改一下两个配置: globals.deaf, cluster.name

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
./gmond/manifests/init.pp
class gmond {

$deaf = $::hostname ? {
  'hadoop-master1' => 'no',
  'cu-omc1' => 'no',
  default => 'yes',
}

if ( $::hostname =~ /(^cu-)/ ) {
  $cluster_name = 'CU'
} else {
  $cluster_name = 'HADOOP'
}

package { 'gmond':
  ensure => present,
  before => File['/usr/local/ganglia/etc/gmond.conf'],
}

file { '/usr/local/ganglia/etc/gmond.conf':
  ensure => file,
  content => template('gmond/gmond.conf'),
  notify => Service['gmond'],
}

service { 'gmond':
  ensure    => running,
  enable    => true,
}

}

./gmond/templates/gmond.conf
/* This configuration is as close to 2.5.x default behavior as possible
   The values closely match ./gmond/metric.h definitions in 2.5.x */
globals {
...
  mute = no
  deaf = <%= scope.lookupvar('gmond::deaf') %>
  allow_extra_data = yes
...
cluster {
  name = "<%= scope.lookupvar('gmond::cluster_name') %>"

参考下逻辑即可(也可以通过hiera配置)。

最后在 site.pp 引用加载编写的Module:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[root@hadoop-master1 modules]# cd ../manifests/
[root@hadoop-master1 manifests]# cat site.pp 
file{'/etc/puppetlabs/mcollective/facts.yaml':
  owner    => root,
  group    => root,
  mode     => '400',
  loglevel => debug, # reduce noise in Puppet reports
  content  => inline_template("<%= scope.to_hash.reject { |k,v| k.to_s =~ /(uptime_seconds|timestamp|free)/ }.to_yaml %>"), # exclude rapidly changing facts
}


include dtarepo
include gmond

# include sudo

一键安装

安装gmetad:

首先在主机上安装gmetad,由于只需要在一台机器安装,配置没有整成模板,这里直接手动弄。

1
2
3
4
5
6
7
8
9
10
[root@hadoop-master1 dtarepo]# mco rpc package install package=gmetad -I cu-omc1 (或者直接yum install -y gmetad)

# 注意:主机多网卡时可能需要添加route
[root@cu-omc1 ~]# route add -host 239.2.11.71 dev bond0

[root@cu-omc1 ~]# /etc/ganglia/gmetad.conf 注意!! 这里的rrd_rootdir配置与上面gweb/makefile是对应的!!
data_source "HADOOP" hadoop-master1
data_source "CU" cu-omc1
gridname "CQCU"
rrd_rootdir "/data/ganglia/rrds"

注意: data_source 需要配合 gmond/manifests/init.pp 中的 deaf 属性值。

php的时区调整:vi /etc/php.ini date.timezone = “Asia/Shanghai”

安装gmond:

在cu-omc2上安装gmond(正则表达式,想怎么匹配就怎么写):

1
[root@hadoop-master1 production]# mco shell -I /^cu-omc2/ run -- "/opt/puppetlabs/bin/puppet agent -t"

puppet同步好后,就安装好puppet,以及启动gmond服务。

同时看看web是否已经有图像。不要看一分钟负载,搞一个明显一点的,如磁盘容量内存容量可以明确判断数据有没有采集到的。 没有话可以试着重启gmond:

1
[root@hadoop-master1 production]# mco shell -I /cu-/ run -- service gmond restart

不要一次重启太多机器,时间比较长的话可以结合screen命令使用:

1
2
3
4
5
6
[root@hadoop-master1 ~]# screen
[root@hadoop-master1 ~]# mco shell -I hadoop-slaver2 -I hadoop-slaver3 -I hadoop-slaver4  -I hadoop-slaver5 -I hadoop-slaver6 -I hadoop-slaver7 -I hadoop-slaver8 -I hadoop-slaver9  run -- service gmond restart ; 
[root@hadoop-master1 ~]# for ((i=1;i<17;i++)) ; do mco shell -I /hadoop-slaver${i}.$/ run -- service gmond restart ; sleep 60 ; done  

[root@hadoop-master1 ~]# screen -ls 
[root@hadoop-master1 ~]# screen -r 22929

–END

Comments