Redis使用及相关问题

Redis Redis 缓存
Created At :
  1. Redis基本使用
    1. Redis数据结构
    2. Docker搭建 主从 哨兵 集群
      1. Docker搭建单例Redis
      2. 主从模式
      3. 哨兵模式
      4. 集群模式
        1. 哈希槽hashslot
        2. 持久化
  2. Redis常见问题汇总及解决方案
    1. 缓存穿透
    2. 缓存雪崩
    3. 如何利用jvm内存和redis做二级缓存 并且保证缓存数据的一致性
    4. 发现大key
    5. DB和缓存的一致性问题
      1. 延迟双删
      2. 异步更新缓存

Redis基本使用

Redis数据结构

  • 字符串(String)

    set

  • 列表(List)

    list

  • 哈希(Hash)

    hash

  • 集合(Set)

    set

  • 有序集合(Zset)

    set

  • HyperLogLogs (基数统计)

    …

  • Bitmap (位存储)

    …

  • geospatial (地理位置)

    …

Docker搭建 主从 哨兵 集群

配置文件示例 https://github.com/redis/redis/blob/7.4.0/redis.conf https://github.com/redis/redis/blob/7.4.0/sentinel.conf

Docker搭建单例Redis


#单例运行
docker run -d --name redis -v /home/docker/redis/config/redis-single.conf:/usr/local/etc/redis/redis.conf -v /home/docker/redis/data/:/data/  -p 6379:6379 --rm redis:7.4.0 redis-server /usr/local/etc/redis/redis.conf

#操作
docker exec -it containerId redis-cli

#redis.conf
################################## NETWORK #####################################
#不要用127.0.0.1 绑定具体的ip 否则无法通过redis-cli -h ip -p port 进行外部访问
# bind 192.168.1.100 10.0.0.1     # listens on two specific IPv4 addresses
# bind 127.0.0.1 ::1              # listens on loopback IPv4 and IPv6
# bind * -::*                     # like the default, all available interfaces
bind * -::*
#关闭保护模式否则只能本机访问
protected-mode no
port 6379
################################# GENERAL #####################################
pidfile /var/run/redis_6379.pid
loglevel notice
#日志文件
logfile "/data/redis-master.log"
databases 16
################################ SNAPSHOTTING  ################################
save 3600 1 300 100 60 1
#存储文件
dbfilename redis_master_dump.rdb
# Note that you must specify a directory here, not a file name.
dir ./
################################# REPLICATION #################################
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync yes
repl-diskless-sync-delay 5
repl-diskless-sync-max-replicas 0
repl-diskless-load disabled
repl-disable-tcp-nodelay no
...

127.0.0.1:6379> info server
# Server
redis_version:7.4.0
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:a4a1a8bba24ad866
redis_mode:standalone
os:Linux 6.8.0-40-generic x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:12.2.0
process_id:1
process_supervised:no
run_id:076e63887ee1e69251e0c523f27e2ee28399f9ee
tcp_port:6379
server_time_usec:1726197123775604
uptime_in_seconds:1438
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:14921091
executable:/data/redis-server
config_file:/usr/local/etc/redis/redis.conf
io_threads_active:0
listener0:name=tcp,bind=127.0.0.1,bind=-::1,port=6379

主从模式

#使用自定义的网络
#master
docker run -d --name redis-master -v /home/docker/redis/config/redis-master.conf:/usr/local/etc/redis/redis.conf -v /home/docker/redis/data/:/data/ -v /home/docker/redis/logs:/usr/local/etc/redis/log/ --network inet -p 6379:6379 --rm  redis:7.4.0 redis-server /usr/local/etc/redis/redis.conf  &&
#slave1
docker run -d --name redis-slave1 -v /home/docker/redis/config/redis-slave1.conf:/usr/local/etc/redis/redis.conf -v /home/docker/redis/data/:/data/ -v /home/docker/redis/logs:/usr/local/etc/redis/log/ --network inet -p 6380:6379 --rm redis:7.4.0 redis-server /usr/local/etc/redis/redis.conf  &&
#slave2
docker run -d --name redis-slave2 -v /home/docker/redis/config/redis-slave2.conf:/usr/local/etc/redis/redis.conf -v /home/docker/redis/data/:/data/ -v /home/docker/redis/logs:/usr/local/etc/redis/log/ --network inet -p 6381:6379 --rm redis:7.4.0 redis-server /usr/local/etc/redis/redis.conf

#redis.conf
################################## NETWORK #####################################
#不要用127.0.0.1 绑定具体的ip 否则无法通过redis-cli -h ip -p port 进行外部访问
# bind 192.168.1.100 10.0.0.1     # listens on two specific IPv4 addresses
# bind 127.0.0.1 ::1              # listens on loopback IPv4 and IPv6
# bind * -::*                     # like the default, all available interfaces
bind * -::*
#关闭保护模式否则只能本机访问
protected-mode no
#master
port 6379
#salve1
#port 6380
#salve2
#port 6381
################################# GENERAL #####################################
pidfile /var/run/redis_6379.pid
#pidfile /var/run/redis_6380.pid
#pidfile /var/run/redis_6381.pid

loglevel notice
#日志文件
logfile "/data/redis-master.log"
#logfile "/data/redis-salve1.log"
#logfile "/data/redis-salve2.log"
databases 16
################################ SNAPSHOTTING  ################################
save 3600 1 300 100 60 1
#存储文件
dbfilename redis_master_dump.rdb
#dbfilename redis_salve1_dump.rdb
#dbfilename redis_salve2_dump.rdb

# Note that you must specify a directory here, not a file name.
dir ./
################################# REPLICATION #################################
#当在从节点配置时指定master
# replicaof <masterip> <masterport>
#replicaof redis-master 6379
...

 docker exec -it b822875a5056 redis-cli info REPLICATION
# Replication
role:master
connected_slaves:2
slave0:ip=172.18.0.3,port=6380,state=online,offset=14672,lag=0
slave1:ip=172.18.0.4,port=6381,state=online,offset=14672,lag=0
master_failover_state:no-failover
master_replid:bff2af97b86f24cffb57550003a93538fd3aed59
master_replid2:c85dd5eb038484fbd9faca0a429066a496722fec
master_repl_offset:14672
second_repl_offset:11859
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:11859
repl_backlog_histlen:2814

哨兵模式

# 在主从基础上加入哨兵
#Sentinel1
docker run -d --name redis-sentinel1 -v /home/docker/redis/config/redis-sentinel.conf:/usr/local/etc/redis/sentinel.conf  -v /home/docker/redis/logs/sentinel1.log:/tmp/sentinel.log  --network inet -p 26379:26379 --rm  redis:7.4.0 redis-sentinel /usr/local/etc/redis/sentinel.conf  &&
#Sentinel2
docker run -d --name redis-sentinel2 -v /home/docker/redis/config/redis-sentinel.conf:/usr/local/etc/redis/sentinel.conf  -v /home/docker/redis/logs/sentinel2.log:/tmp/sentinel.log  --network inet -p 26380:26379 --rm redis:7.4.0 redis-sentinel /usr/local/etc/redis/sentinel.conf  &&
#Sentinel3
docker run -d --name redis-sentinel3 -v /home/docker/redis/config/redis-sentinel.conf:/usr/local/etc/redis/sentinel.conf  -v /home/docker/redis/logs/sentinel3.log:/tmp/sentinel.log --network inet -p 26381:26379 --rm redis:7.4.0 redis-sentinel /usr/local/etc/redis/sentinel.conf 

#sentinel.conf
protected-mode no
daemonize no
pidfile /var/run/redis-sentinel.pid
loglevel notice
logfile "/data/sentinel.log"
# dir <working-directory>
dir /tmp
#  监控的master的ip 端口 以及 需要几个sentinel同意
#6.2 以上版本的 sentinel 才能解析主机名
# sentinel monitor <master-name> <ip> <redis-port> <quorum>
sentinel monitor mymaster redis-master 6379 2
# HOSTNAMES SUPPORT 主机名支持
SENTINEL resolve-hostnames yes
#
SENTINEL announce-hostnames yes
...

#master 正常时
docker exec redis-master redis-cli info Replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.18.0.3,port=6380,state=online,offset=103968,lag=0
slave1:ip=172.18.0.4,port=6379,state=online,offset=103968,lag=0
master_failover_state:no-failover
master_replid:26974803eec7dc51775e322c38428771928691ae
master_replid2:aa8b907c48b8501f8d8d5f895d4d359913c47897
master_repl_offset:103968
second_repl_offset:17260
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:17260
repl_backlog_histlen:86709

#master挂掉  redis-slave1 被选为master
docker stop redis-master
docker exec redis-slave1 redis-cli info Replication
# Replication
role:master
connected_slaves:1
slave0:ip=172.18.0.4,port=6379,state=online,offset=140844,lag=0
master_failover_state:no-failover
master_replid:b04d5e13f36a9d9dc8795cf15558b0b4b60f9260
master_replid2:26974803eec7dc51775e322c38428771928691ae
master_repl_offset:140844
second_repl_offset:130355
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:17260
repl_backlog_histlen:123585

#master回归正常后 master 变为redis-slave1的从节点
docker exec redis-slave1 redis-cli -p 6380 info Replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.18.0.4,port=6379,state=online,offset=170223,lag=0
slave1:ip=172.18.0.2,port=6379,state=online,offset=170223,lag=1
master_failover_state:no-failover
master_replid:b04d5e13f36a9d9dc8795cf15558b0b4b60f9260
master_replid2:26974803eec7dc51775e322c38428771928691ae
master_repl_offset:170358
second_repl_offset:130355
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:17260
repl_backlog_histlen:153099

集群模式

sudo sysctl vm.overcommit_memory=1

#docker stop cluster1 cluster2 cluster3 cluster4 cluster5 cluster6
docker run -d --name cluster1 -v /home/docker/redis/config/redis-cluster.conf:/usr/local/etc/redis/redis.conf -v /home/docker/redis/data/cluster1/:/data/  --network inet -p 6379:6379 -p ::16379  --rm  redis:7.4.0 redis-server /usr/local/etc/redis/redis.conf --cluster-announce-hostname cluster1 --cluster-announce-human-nodename cluster1\
&&\
docker run -d --name cluster2 -v /home/docker/redis/config/redis-cluster.conf:/usr/local/etc/redis/redis.conf -v /home/docker/redis/data/cluster2/:/data/  --network inet -p 6380:6379 -p ::16379  --rm redis:7.4.0 redis-server /usr/local/etc/redis/redis.conf  --cluster-announce-hostname cluster2 --cluster-announce-human-nodename cluster2\
&&\
docker run -d --name cluster3 -v /home/docker/redis/config/redis-cluster.conf:/usr/local/etc/redis/redis.conf -v /home/docker/redis/data/cluster3/:/data/  --network inet -p 6381:6379 -p ::16379  --rm redis:7.4.0 redis-server /usr/local/etc/redis/redis.conf  --cluster-announce-hostname cluster3 --cluster-announce-human-nodename cluster3\
&&\
docker run -d --name cluster4 -v /home/docker/redis/config/redis-cluster.conf:/usr/local/etc/redis/redis.conf -v /home/docker/redis/data/cluster4/:/data/  --network inet -p 6382:6379 -p ::16379  --rm redis:7.4.0 redis-server /usr/local/etc/redis/redis.conf  --cluster-announce-hostname cluster4 --cluster-announce-human-nodename cluster4\
&&\
docker run -d --name cluster5 -v /home/docker/redis/config/redis-cluster.conf:/usr/local/etc/redis/redis.conf -v /home/docker/redis/data/cluster5/:/data/  --network inet -p 6383:6379 -p ::16379  --rm redis:7.4.0 redis-server /usr/local/etc/redis/redis.conf  --cluster-announce-hostname cluster5 --cluster-announce-human-nodename cluster5\
&&\
docker run -d --name cluster6 -v /home/docker/redis/config/redis-cluster.conf:/usr/local/etc/redis/redis.conf -v /home/docker/redis/data/cluster6/:/data/  --network inet -p 6384:6379 -p ::16379  --rm redis:7.4.0 redis-server /usr/local/etc/redis/redis.conf  --cluster-announce-hostname cluster6 --cluster-announce-human-nodename cluster6
# 执行加入集群命令
# docker exec -it cluster1 redis-cli --cluster create  cluster1:6379 cluster2:6379 cluster3:6379 cluster4:6379 cluster5:6379 cluster6:6379   --cluster-replicas 1 
#输入yes


# 修改节点端口
port 6379
bind * -::*
logfile "/data/redis.log"
# 保护模式改为no，允许外部网络访问
protected-mode no
# 设置后台运行，改为yes
daemonize no
# pid文件路径配置
pidfile /var/run/redis_6379.pid

save 3600 1 300 100 60 1
dbfilename redis_dump.rdb
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes


# 开启集群模式
cluster-enabled yes
# 集群配置文件，根据端口不同修改
cluster-config-file nodes-6379.conf
#通过主机名找到集群
cluster-preferred-endpoint-type hostname
...

#查看集群信息
docker exec -it cluster1 redis-cli cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:178
cluster_stats_messages_pong_sent:172
cluster_stats_messages_sent:350
cluster_stats_messages_ping_received:172
cluster_stats_messages_pong_received:178
cluster_stats_messages_received:350
total_cluster_links_buffer_limit_exceeded:0
#
docker exec -it cluster1 redis-cli cluster nodes
49ae9c854a0122709c784a5653a7a7873d91db4e 172.18.0.7:6379@16379,cluster6 slave d450087aae42a2b9a724e92db72cd0c3402753e7 0 1726544317000 2 connected
704651b58d37c065c848faf2a65580e78fec219e 172.18.0.4:6379@16379,cluster3 master - 0 1726544318000 3 connected 10923-16383
2cdc0f8009cbc5fd39b12ef49a4010cb9dffbd35 172.18.0.5:6379@16379,cluster4 slave 704651b58d37c065c848faf2a65580e78fec219e 0 1726544316459 3 connected
36e60ceae53ead77730569651a1780c7b518e9df 172.18.0.2:6379@16379,cluster1 myself,master - 0 0 1 connected 0-5460
d450087aae42a2b9a724e92db72cd0c3402753e7 172.18.0.3:6379@16379,cluster2 master - 0 1726544318483 2 connected 5461-10922
0dccee28bc5b8baa01568f13382cac380f0e66d9 172.18.0.6:6379@16379,cluster5 slave 36e60ceae53ead77730569651a1780c7b518e9df 0 1726544316000 1 connected


# 直接连接会让你去cluster2上去操作
zhangzhitong@zhangzhitong-virtual-machine:/home/docker$ docker exec cluster1 redis-cli get key1
MOVED 9189 cluster2:6379
#集群方式连接redis-cli -c
zhangzhitong@zhangzhitong-virtual-machine:/home/docker$ docker exec cluster1 redis-cli -c get key1

zhangzhitong@zhangzhitong-virtual-machine:/home/docker$ docker exec cluster1 redis-cli -c set key1 zzt
OK
zhangzhitong@zhangzhitong-virtual-machine:/home/docker$ docker exec cluster1 redis-cli -c get key1
zzt
哈希槽hashslot

Redis 集群并没有使用一致性 hash，而是引入了哈希槽的概念。Redis 集群有 16384（2^14）个哈
希槽，每个 key 通过 CRC16 校验后对 16384 取模来决定放置哪个槽，集群的每个节点负责一部分
hash 槽。
集群使用公式slot=CRC16（key）/16384来计算key属于哪个槽，其中CRC16(key)语句用于计算key的CRC16 校验和

持久化

RDB：RDB是Redis默认的持久化方式。按照一定的时间周期策略把内存的数据以快照的形式保存到硬盘的二进制文件。即Snapshot快照存储，对应产生的数据文件为dump.rdb，通过配置文件中的save参数来定义快照的周期。（ 快照可以是其所表示的数据的一个副本，也可以是数据的一个复制品。）

AOF：Redis会将每一个收到的写命令都通过Write函数追加到文件最后，类似于MySQL的binlog。当Redis重启是会通过重新执行文件中保存的写命令来在内存中重建整个数据库的内容。 当两种方式同时开启时，数据恢复Redis会优先选择AOF恢复。

Redis常见问题汇总及解决方案

缓存穿透

数据从缓存中查不到之直接查库仍然未查到 可以通过布隆过滤器解决

缓存雪崩

大量数据从缓存中失效掉 导致数据库被大量访问

如何利用jvm内存和redis做二级缓存 并且保证缓存数据的一致性

发现大key

bigkey 是指键值占用内存空间非常大的 key。通过命令可以找出

#迭代数据库中的所有键，并为每种类型（字符串、列表、集合、有序集合和哈希）报告最大的键和其大小。
redis-cli --bigkeys

DB和缓存的一致性问题

1.先删除缓存，再更新数据库。如果数据库更新失败了，那么数据库中是旧数据，缓存中是空的，那数据就不会不一致。因为读的时候缓存没有，所以去读了数据库中的旧数据，然后更新到缓存中 2.在高并发时 会出现 一个请求删除了缓存 但在数据写操作未完成时 另一个请求将数据库中的旧值再次写入缓存 导致缓存和数据库不一致。 此时可以对操作进行加锁 保证一个请求的删除和写数据库完成时另一个在去读，或者对同一数据的操作路由分发到同一台机器 将写读操作写入队列 顺序完成 保证数据一致性 同时也会导致请求的倾斜.

延迟双删

先删除缓存 再写数据库 再延迟删除缓存 出现不一致情况较小

异步更新缓存

MySQL binlog 配合canal+kafka增量订阅消费->消息队列>增量数据更新到redis
–

©2016-2025 zzt

Built with Hexo and 3-hexo theme