如何百度搜到网站,俄语网站服务器,外贸网站推广方式,软件网站是怎么做的吗转载说明#xff1a;如果您喜欢这篇文章并打算转载它#xff0c;请私信作者取得授权。感谢您喜爱本文#xff0c;请文明转载#xff0c;谢谢。 问题描述#xff1a;
机房突然停电#xff0c;rabbitmq的主机异常断电#xff0c;集群服务全部需要重启。但是在执行service… 转载说明如果您喜欢这篇文章并打算转载它请私信作者取得授权。感谢您喜爱本文请文明转载谢谢。 问题描述
机房突然停电rabbitmq的主机异常断电集群服务全部需要重启。但是在执行service rabbitmq-server start 启动主节点服务的时候没有反应服务没有启动命令也执行卡住了。必须CtrlC结束进程
[rootmaster-2 rabbitmq]# service rabbitmq-server start
Starting rabbitmq-server (via systemctl): ^C
[rootmaster-2 rabbitmq]#查看/var/log/rabbitmq/startup_log 发现有如下报错信息
[rootmaster-2 rabbitmq]# tail -1000 startup_log
BOOT FAILED
Timeout contacting cluster nodes: [rabbits1-1,rabbitslave-2].BACKGROUND
This cluster node was shut down while other nodes were still running.
To avoid losing data, you should start the other nodes first, then
start this one. To force this node to start, first invoke
rabbitmqctl force_boot. If you do so, any changes made on other
cluster nodes after this one was shut down may be lost.DIAGNOSTICS
attempted to contact: [rabbits1-1,rabbitslave-2]rabbits1-1:* connected to epmd (port 4369) on s1-1* epmd reports: node rabbit not running at allno other nodes on s1-1* suggestion: start the node
rabbitslave-2:* unable to connect to epmd (port 4369) on slave-2: address (cannot connect to host/port)current node details:
- node name: rabbitmaster-2
- home dir: /var/lib/rabbitmq
- cookie hash: oqRyxdQQXO31mzM8U0ysNA{init terminating in do_boot,timeout_waiting_for_tables}解决方法1
根据/var/log/rabbitmq/startup_log日志最后的报错信息{“init terminating in do_boot”,timeout_waiting_for_tables}在网上查询到原因和linux下rabbitmq大致有关系的主要有这三种说法 1、5672端口被占用了导致服务起不来 2、/var/log/rabbitmq目录的权限不对需要重新赋权限 3、/var/lib/rabbitmq/mnesia这个数据目录异常删除原来的数据目录重新启动服务
方法一检查端口发现并没有5672的这个端口
[rootmaster-2 rabbitmq]# netstat -anp|grep 5672
tcp 0 0 193.168.0.90:3306 131.10.10.120:56727 ESTABLISHED 3666/mysqld
tcp6 0 0 193.168.0.90:56727 193.168.0.93:9092 ESTABLISHED 4891/java
[rootmaster-2 rabbitmq]# netstat -ano|grep 5672
tcp 0 0 193.168.0.90:3306 131.10.10.120:56727 ESTABLISHED keepalive (54.12/0/0)
tcp6 0 0 193.168.0.90:56727 193.168.0.93:9092 ESTABLISHED keepalive (50.53/0/0)方法二修改/var/log/rabbitmq权限进去/var/log/rabbitmq/目录发现该目录下面的文件确实存在权限不统一的问题于是修改权限重新启动服务还是失败
[rootmaster-2 rabbitmq]# cd /var/log/rabbitmq/
[rootmaster-2 rabbitmq]# ll
total 11740
-rw-r--r-- 1 rabbitmq rabbitmq 29075 May 14 11:14 rabbitmaster-2.log
-rw-r--r-- 1 rabbitmq rabbitmq 159053 Apr 29 03:19 rabbitmaster-2.log-20180429.gz
-rw-r--r-- 1 rabbitmq rabbitmq 1756006 May 7 03:11 rabbitmaster-2.log-20180507.gz
-rw-r--r-- 1 rabbitmq rabbitmq 9881632 May 13 03:17 rabbitmaster-2.log-20180513
-rw-r--r-- 1 rabbitmq rabbitmq 3108 May 14 11:14 rabbitmaster-2-sasl.log
-rw-r--r-- 1 rabbitmq rabbitmq 950 Apr 28 14:22 rabbitmaster-2-sasl.log-20180429.gz
-rw-r--r-- 1 rabbitmq rabbitmq 1677 May 4 15:25 rabbitmaster-2-sasl.log-20180507.gz
-rw-r--r-- 1 rabbitmq rabbitmq 159530 May 11 10:11 rabbitmaster-2-sasl.log-20180513
-rw-r--r-- 1 root root 0 May 7 15:14 shutdown_err
-rw-r--r-- 1 root root 44 May 7 15:14 shutdown_log
-rw-r--r--. 1 root root 103 May 14 11:15 startup_err
-rw-r--r--. 1 root root 1323 May 14 11:15 startup_log
[rootmaster-2 rabbitmq]# chown -R rabbitmq:rabbitmq /var/log/rabbitmq/
[rootmaster-2 rabbitmq]# ll
total 11740
-rw-r--r-- 1 rabbitmq rabbitmq 29075 May 14 11:14 rabbitmaster-2.log
-rw-r--r-- 1 rabbitmq rabbitmq 159053 Apr 29 03:19 rabbitmaster-2.log-20180429.gz
-rw-r--r-- 1 rabbitmq rabbitmq 1756006 May 7 03:11 rabbitmaster-2.log-20180507.gz
-rw-r--r-- 1 rabbitmq rabbitmq 9881632 May 13 03:17 rabbitmaster-2.log-20180513
-rw-r--r-- 1 rabbitmq rabbitmq 3108 May 14 11:14 rabbitmaster-2-sasl.log
-rw-r--r-- 1 rabbitmq rabbitmq 950 Apr 28 14:22 rabbitmaster-2-sasl.log-20180429.gz
-rw-r--r-- 1 rabbitmq rabbitmq 1677 May 4 15:25 rabbitmaster-2-sasl.log-20180507.gz
-rw-r--r-- 1 rabbitmq rabbitmq 159530 May 11 10:11 rabbitmaster-2-sasl.log-20180513
-rw-r--r-- 1 rabbitmq rabbitmq 0 May 7 15:14 shutdown_err
-rw-r--r-- 1 rabbitmq rabbitmq 44 May 7 15:14 shutdown_log
-rw-r--r--. 1 rabbitmq rabbitmq 103 May 14 11:15 startup_err
-rw-r--r--. 1 rabbitmq rabbitmq 1323 May 14 11:15 startup_log但是修改了权限之后服务还是起不来
[rootmaster-2 rabbitmq]# service rabbitmq-server start
Starting rabbitmq-server (via systemctl): ^C
[rootmaster-2 rabbitmq]# 方法三删除原有的数据目录然后重新启动服务
[rootmaster-2 rabbitmq]# cd /var/lib/rabbitmq/
[rootmaster-2 rabbitmq]# ll
total 4020
-rw-r----- 1 rabbitmq rabbitmq 4114398 May 14 11:15 erl_crash.dump
drwxr-x--- 4 rabbitmq rabbitmq 94 May 14 11:38 mnesia
[rootmaster-2 rabbitmq]# mv mnesia mnesia.bak
[rootmaster-2 rabbitmq]# ll
total 4020
-rw-r----- 1 rabbitmq rabbitmq 4114398 May 14 11:15 erl_crash.dump
drwxr-x--- 4 rabbitmq rabbitmq 94 May 14 11:38 mnesia.bak然后重新启动服务成功
[rootmaster-2 rabbitmq]# service rabbitmq-server start
Starting rabbitmq-server (via systemctl): [ OK ]
[rootmaster-2 rabbitmq]# ps -ef|grep rabbitmq
rabbitmq 3131 1 0 May13 ? 00:00:00 /usr/lib64/erlang/erts-5.10.4/bin/epmd -daemon
root 19908 1 0 11:41 ? 00:00:00 /bin/sh /etc/rc.d/init.d/rabbitmq-server start
root 19910 19908 0 11:41 ? 00:00:00 /bin/bash -c ulimit -S -c 0 /dev/null 21 ; /usr/sbin/rabbitmq-server
root 19914 19910 0 11:41 ? 00:00:00 /bin/sh /usr/sbin/rabbitmq-server
root 19932 19914 0 11:41 ? 00:00:00 su rabbitmq -s /bin/sh -c /usr/lib/rabbitmq/bin/rabbitmq-server
rabbitmq 19935 19932 0 11:41 ? 00:00:00 /bin/sh /usr/lib/rabbitmq/bin/rabbitmq-server
rabbitmq 20158 19935 17 11:41 ? 00:00:04 /usr/lib64/erlang/erts-5.10.4/bin/beam.smp -W w -A 64 -P 1048576 -t 5000000 -stbt db -zdbbl 128000 -K true -B i -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -pa /usr/lib/rabbitmq/lib/rabbitmq_server-3.6.12/ebin -noshell -noinput -s rabbit boot -sname rabbitmaster-2 -boot start_sasl -config /etc/rabbitmq/rabbitmq -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,/var/log/rabbitmq/rabbitmaster-2.log} -rabbit sasl_error_logger {file,/var/log/rabbitmq/rabbitmaster-2-sasl.log} -rabbit enabled_plugins_file /etc/rabbitmq/enabled_plugins -rabbit plugins_dir /usr/lib/rabbitmq/plugins:/usr/lib/rabbitmq/lib/rabbitmq_server-3.6.12/plugins -rabbit plugins_expand_dir /var/lib/rabbitmq/mnesia/rabbitmaster-2-plugins-expand -os_mon start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false -mnesia dir /var/lib/rabbitmq/mnesia/rabbitmaster-2 -kernel inet_dist_listen_min 25672 -kernel inet_dist_listen_max 25672
rabbitmq 20316 20158 0 11:41 ? 00:00:00 inet_gethost 4
rabbitmq 20317 20316 0 11:41 ? 00:00:00 inet_gethost 4
root 20406 16497 0 11:42 pts/5 00:00:00 grep --colorauto rabbitmq
[rootmaster-2 rabbitmq]# 注意这只是主节点的处理方法在两台从节点需要做如下操作 1、检查两台从节点的/var/lib/rabbitmq/.erlang.cookie文件内容是否和主节点是保持一致的 2、删除原有的数据/var/lib/rabbitmq/mnesia目录执行rabbitmq-server -detached重新启动服务 3、在两台从节点上执行下面的命令重新加入集群
#rabbitmqctl stop_app
#rabbitmqctl reset
#rabbitmqctl join_cluster rabbitmaster-2 # rabbitmaster-2里面的master-2是主节点的主机名注意修改
#rabbitmqctl start_app备节点执行完毕上面的步骤之后需要在主节点验证集群的正确性
[rootmaster-2 rabbitmq]# rabbitmqctl cluster_status
Cluster status of node rabbitmaster-2
[{nodes,[{disc,[rabbitmaster-2,rabbits1-1,rabbitslave-2]}]},{running_nodes,[rabbits1-1,rabbitslave-2,rabbitmaster-2]},{cluster_name,rabbitmaster-2},{partitions,[]},{alarms,[{rabbits1-1,[]},{rabbitslave-2,[nodedown]},{rabbitmaster-2,[]}]}]集群验证成功之后使用主节点IP端口登录界面发现输入之前的用户名和密码已经登录不进去了需要在主节点重新创建管理用户并且赋予密码和访问权限
[rootmaster-2 rabbitmq]# rabbitmqctl add_user admin password123 #创建用户和密码
Creating user admin
[rootmaster-2 rabbitmq]# rabbitmqctl set_user_tags admin administrator #给用户赋予管理员权限
Setting tags for user admin to [administrator]
[rootmaster-2 rabbitmq]# rabbitmqctl set_permissions -p / admin .* .* .* #给管理员赋予访问权限
Setting permissions for user admin in vhost /然后再使用主节点的IP端口重新登录输入用户名和密码服务恢复。
解决方法2推荐
问题解决之后发现还有一种说法 Are you running in a clustered configuration? If so, rabbit might be waiting for the other nodes to come up.
在后面的工作中发现确实存在个问题。当整个集群重启的时候如果关掉了整个集群所有的节点再启动服务。若先启动主节点而备节点全部没有启动就会出现上述的启动不了的问题。 如果先将从节点全部起起来再启动主节点就一切顺利数据也不会丢失这个应该是比上面更简便的方法。 从节点启动命令
rabbitmq-server -detached主节点启动命令
service rabbitmq-server start备注本文为迁移博客非近期遇到的故障
