Redis Sentinel集群方案--单机测试

方法一：（未验证）

1，建议下载redis-3.0.1以上版本，redis-3.0之前的版本都不支持集群，但是redis-3.0.0版本不太稳定。

2，将redis安装包解压，复制到/usr/local中（建议），进入/usr/local/redis文件夹，进行编译‘make’,进入/usr/local/redis/src中进行安装‘sudo make install’；

3，/usr/local/redis/redis.conf是redis服务启动的配置文件，/usr/local/redis/src下面有很多命令，包括启动命令redis-server；

4，启动redis服务，进入usr/local/redis/src中，用redis-server /usr/local/redis/redis.conf,就启动了，关闭，直接kill进程就行；

下面说一下在一台电脑上建集群

1，在/usr/local中创建cluster/6380、cluster/6381、cluster/6382、cluster/6383、cluster/6384、cluster/6385,然后将/usr/local/redis/redis.conf文件依次复制到新建的文件夹中，用vi编辑器修改一下参数：

port 6379

daemonize yes

cluster-enabled yes

cluster-config-file nodes-6379.conf

cluster-node-timeout 5000

appendonly yes

注意，port为对应的6380、6381、6382、6383、6384、6385，cluster-config-file也要对应，注释一定一定要去掉

2，分别启动这6个配置文件；

3，进入/usr/local/redis/src中，执行  ./redis-trib.rb create --replicas 1 127.0.0.1:6380 127.0.0.1:6381 127.0.0.1:6382 127.0.0.1:6383 127.0.0.1:6384 127.0.0.1:6385；

等会提示是否接受redis自动默认的配置，输入yes,

4,redis-cli -c -p 端口号，进入某一个redis服务

到此集群已搭建好了。

方法二：（已验证）

 Redis Sentinel集群方案--单机测试

1.测试环境
master：   127.0.0.1 6379
slave1:    127.0.0.1 6479
slave2:    127.0.0.1 6579
master-sentinel: 127.0.0.1 26379
slave1-sentinel: 127.0.0.1 26479
slave2-sentinel: 127.0.0.1 26579
2.下载安装redis-3.0.5

cd /usr/local/redis-3.0.5
yum  install  gcc
make MALLOC=libc PREFIX=/usr/local/redis install  (此处可用PREFIX参数将redis安装到其他目录)

3.配置测试环境
----创建目录：
cd /usr/local/redis
mkdir redis_cluster
mkdir redis_cluster/master_6379
mkdir redis_cluster/slave_6479
mkdir redis_cluster/slave_6579
----配置redis：
master:
cp ../redis-3.0.5/redis.conf ./redis_cluster/master_6379/
cp ../redis-3.0.5/sentinel.conf ./redis_cluster/master_6379/6379-sentinel.conf
vi ./redis_cluster/master_6379/redis.conf(将对应配置修改成如下)
-------------------------------------------------------------------------------------------
####master  redis.conf
daemonize no
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 0
loglevel notice
logfile ""
databases 16
save 900 1
save 300 10
save 60 10000
save ""
stop-writes-on-bgsave-error yes
rdbchecksum yes
dbfilename dump.rdb
dir ./
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes

-------------------------------------------------------------------------------------------
vi ./redis_cluster/master_6379/6379-sentinel.conf
-------------------------------------------------------------------------------------------
####master  sentinel.conf
port 26379
dir "/tmp"
sentinel monitor mymaster 127.0.0.1 6379 2
sentinel failover-timeout mymaster 900000
sentinel config-epoch mymaster 0
sentinel leader-epoch mymaster 0
-------------------------------------------------------------------------------------------
slave1:
cp -a -R -p ../redis/redis_cluster/master_6379/redis.conf ./redis_cluster/slave_6479/
cp -a -R -p ../redis/redis_cluster/master_6379/6379-sentinel.conf ./redis_cluster/slave_6479/6479-sentinel.conf
vi ./redis_cluster/slave_6479/redis.conf(将对应配置修改成如下)
-------------------------------------------------------------------------------------------
####slave1 redis.conf
port 6479
slaveof 127.0.0.1 6379
##-----------其他配置和master  redis.conf保持一致-----------##
-------------------------------------------------------------------------------------------
vi ./redis_cluster/slave_6479/6479-sentinel.conf
-------------------------------------------------------------------------------------------
####slave1 sentinel.conf
port 26479
##--------其他配置和master  sentinel.conf保持一致-------##
-------------------------------------------------------------------------------------------
slave2:
cp -a -R -p ../redis/redis_cluster/master_6379/redis.conf ./redis_cluster/slave_6579/
cp -a -R -p ../redis/redis_cluster/master_6379/6379-sentinel.conf ./redis_cluster/slave_6579/6579-sentinel.conf
vi ./redis_cluster/slave_6579/redis.conf(将对应配置修改成如下)
-------------------------------------------------------------------------------------------
####slave1 redis.conf
port 6579
slaveof 127.0.0.1 6379
##-----------其他配置和master redis.conf保持一致-----------##
-------------------------------------------------------------------------------------------
vi ./redis_cluster/slave_6579/6579-sentinel.conf
-------------------------------------------------------------------------------------------
####slave1 sentinel.conf
port 26579
##--------其他配置和master sentinel.conf保持一致-------##
-------------------------------------------------------------------------------------------

----启动查看：
注意：首次构建sentinel环境时，必须首先启动master。
启动master和master-sentinel：
redis-server --include /usr/local/redis/redis_cluster/master_6379/redis.conf &
redis-sentinel /usr/local/redis/redis_cluster/master_6379/6379-sentinel.conf &
克隆会话，启动slave1和slave1-sentinel：
redis-server --include /usr/local/redis/redis_cluster/slave_6479/redis.conf &
redis-sentinel /usr/local/redis/redis_cluster/slave_6479/6479-sentinel.conf &
克隆会话，启动slave2和slave2-sentinel：
redis-server --include /usr/local/redis/redis_cluster/slave_6579/redis.conf &
redis-sentinel /usr/local/redis/redis_cluster/slave_6579/6579-sentinel.conf &
查看master的状态：
[root@yqy103 redis]# redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> info Replication
# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.1,port=6479,state=online,offset=54227,lag=0
slave1:ip=127.0.0.1,port=6579,state=online,offset=54094,lag=1
master_repl_offset:54360
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:54359

查看slave的状态：
[root@yqy103 redis]# redis-cli -h 127.0.0.1 -p 6479info Replication
127.0.0.1:6479> info Replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:75514
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0


4.测试：
----场景1：slave宕机
关闭slave1：
redis-cli -h 127.0.0.1 -p 6479
[root@yqy103 redis]# redis-cli -h 127.0.0.1 -p 6479info Replication
127.0.0.1:6479> info Replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:75514
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6479> shutdown
12326:S 30 Aug 16:12:08.825 # User requested shutdown...
12326:S 30 Aug 16:12:08.827 * Calling fsync() on the AOF file.
12326:S 30 Aug 16:12:08.836 # Redis is now ready to exit, bye bye...
12199:M 30 Aug 16:12:08.838 # Connection with slave 127.0.0.1:6479 lost.
not connected> info Replication
Could not connect to Redis at 127.0.0.1:6479: Connection refused

查看sentinel状态：
not connected> 12359:X 30 Aug 16:12:38.889 # +sdown slave 127.0.0.1:6479 127.0.0.1 6479 @ mymaster 127.0.0.1 6379
查看master的Replication信息，此时只存在一个slave0
[root@yqy103 redis]# redis-cli -h 127.0.0.1 -p 6379info Replication
127.0.0.1:6379> info Replication
# Replication
role:master
connected_slaves:1
slave0:ip=127.0.0.1,port=6579,state=online,offset=201388,lag=1
master_repl_offset:201388
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:201387

----场景2：slave恢复
重新开启slave1:
redis-server --include /usr/local/redis/redis_cluster/slave_6479/redis.conf &
redis-server /usr/local/redis_cluster/slave-6479/redis.conf
查看sentinel状态：
sentinel能快速的发现slave加入到集群中：
12241:X 30 Aug 16:22:29.572 * +reboot slave 127.0.0.1:6479 127.0.0.1 6479 @ mymaster 127.0.0.1 6379
12428:X 30 Aug 16:22:29.615 * +reboot slave 127.0.0.1:6479 127.0.0.1 6479 @ mymaster 127.0.0.1 6379
12428:X 30 Aug 16:22:29.697 # -sdown slave 127.0.0.1:6479 127.0.0.1 6479 @ mymaster 127.0.0.1 6379
12241:X 30 Aug 16:22:29.694 # -sdown slave 127.0.0.1:6479 127.0.0.1 6479 @ mymaster 127.0.0.1 6379

查看master的Replication信息：
[root@yqy103 redis]# redis-cli -h 127.0.0.1 -p 6379 info Replication
# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.1,port=6579,state=online,offset=321643,lag=1
slave1:ip=127.0.0.1,port=6479,state=online,offset=321643,lag=1
master_repl_offset:321643
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:321642

----场景3：master宕机
master-sentinel作为master 1的leader，会选取一个master 1的slave作为新的master。slave的选取是根据一个判断DNS情况的优先级来得到，
优先级相同通过runid的排序得到，但目前优先级设定还没实现，所以直接获取runid排序得到slave 1。
然后发送命令slaveof no one来取消slave 1的slave状态来转换为master。当其他sentinel观察到该slave成为master后,就知道错误处理例程启动了。
sentinel A然后发送给其他slave slaveof new-slave-ip-port 命令，当所有slave都配置完后，sentinel A从监测的masters列表中删除故障master，然后通知其他sentinels。
关闭master:
[root@yqy103 redis]# redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> shutdown

查看sentinel状态：
6379-sentinel:
自动将slave2即6579切换成master，原来的master变成slave。
6579-sentinel:
显示了failover的过程：
12428:X 30 Aug 16:34:33.995 # -odown master mymaster 127.0.0.1 6379
12428:X 30 Aug 16:34:33.996 # +failover-end master mymaster 127.0.0.1 6379
12428:X 30 Aug 16:34:33.996 # +switch-master mymaster 127.0.0.1 6379 127.0.0.1 6579
12428:X 30 Aug 16:34:33.999 * +slave slave 127.0.0.1:6479 127.0.0.1 6479 @ mymaster 127.0.0.1 6579
12428:X 30 Aug 16:34:34.010 * +slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6579
12359:X 30 Aug 16:35:02.989 # +sdown slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6579
12241:X 30 Aug 16:35:03.014 # +sdown slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6579
12428:X 30 Aug 16:35:04.066 # +sdown slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6579

----场景4：master恢复
重新启动原来的master：
redis-server --include /usr/local/redis/redis_cluster/master_6379/redis.conf &
查看sentinel状态：
原来的master自动切换成slave，不会自动恢复成master：
13373:M 30 Aug 16:38:44.015 * The server is now ready to accept connections on port 6379
12428:X 30 Aug 16:38:44.089 # -sdown slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6579
12241:X 30 Aug 16:38:44.148 # -sdown slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6579
12359:X 30 Aug 16:38:44.158 # -sdown slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6579
12359:X 30 Aug 16:38:54.087 * +convert-to-slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6579

查看新master：（发现新master在6579,并没有发生变化）
[root@yqy103 redis]# redis-cli -h 127.0.0.1 -p 26379 info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
master0:name=mymaster,status=ok,address=127.0.0.1:6579,slaves=2,sentinels=3

测试完成。
注意：若在sentinel已选出新主但尚未完成其它实例的reconfigure之前，重启old master，则整个系统会出现无法选出new master的异常。

redis调用配置：

mymaster
redis: 127.0.0.1:26379,127.0.0.1:26479,127.0.0.1:26579