Interviewer JD.COM: Redis, I have to ask these questions.

Cache benefits: high performance+high concurrency

The database query takes 8ms, and other users query the same data again, assuming that the data has not changed within 1 minutes, and 1 users have queried the same data within 1 minutes, within 1 minutes, each of the 1 users, Everyone feels very slow to query this data for 8ms

For example, the information of a certain commodity will not change within one day, but it takes 2s to query this commodity every time, and it will be browsed for 1W times in one day

mysql is only 2qps, and it is easier to cache several tens of thousands of qps on a single machine, and the concurrent load of a single machine is dozens of times that of mysql.

During the noon rush hour, 1W users access System A, and there are 4, requests to query the database every second. The database carries 4, requests per second, which will cause downtime. After adding the cache, 3, requests can go to the cache and 1, requests can go to the database.

cache is memory, which can naturally support 4w/s requests. Database (disk-based) generally recommends that concurrent requests should not exceed 2/s

redis single thread. Memcached multithreading

redis is a single-threaded nio asynchronous thread model

one thread+one queue

redis developed a network event handler based on reactor mode, which is called file event handler, and this file event handler is single-threaded. So redis is a single-threaded model, which uses io multiplexing mechanism to monitor multiple sockets at the same time, and selects the corresponding event handler to handle this event according to the events on the sockets.

the file event handler includes: multiple sockets, io multiplexing program, file event dispatcher, and event handler (command request handler, command recovery handler, connection response handler)

The file event handler is single-threaded and listens to multiple sockets through IO multiplexing mechanism. Achieve high performance and simplicity of thread model

When the monitored socket is ready to perform operations such as accept, read, write and close, it will generate corresponding file events. Before calling the associated time processor, it will handle

multiple socket concurrent operations and generate different file events. i/o multiplexing will monitor multiple sockets and put them in a queue. The event dispatcher takes the socket out of the queue and gives it to the corresponding event handler.

after a socket time is processed, the event dispatcher can get the next socket from the queue and give it to the corresponding event handler for processing.

file event:

the socket corresponding to AE _ readable becomes readable (the client performs a write operation on redis)

the socket corresponding to AE _ writeable becomes writable (the client performs a read operation on redis)

I/ O multiplexing can listen to AE _ reliable and AE_WRITABLE at the same time. If they are reached at the same time, the AE_REABLE time

file event handler:

Connect the response handler, the corresponding client wants to connect with redis

Command request handler, the corresponding client writes data to redis

Command reply handler, the corresponding client reads data from redis

Flow:

How many times can it be processed in one second? Ten thousand requests

ordinary set,get kv cache

type map structure, such as an object (without nested objects) cached in redis, and then when reading and writing the cache, you can directly manipulate the hash field (for example, change the age to 21, Other things remain the same)

key=15

value = {

}

Ordered list, elements can be repeated

List can be used to store some tabular data structures, similar to fan list and article comment list.

For example, fans of WeChat big V can put it in redis in the form of list to cache

key= a big V value = [zhangsan, lisi, wangwu]

For example, how many elements can lrange read from a certain element, and it can realize paging query function based on list, and realize high-performance paging based on redis, similar to Weibo's continuous paging.

you can set up a simple message queue, which is arranged in disorder from the head of the list (lpush) and the tail of the list (brpop)

. Automatic deduplication

requires fast global deduplication of some data. (Of course, it can also be based on HashSet, but it is stand-alone)

Set-based operations of difference set, union set and intersection. For example, the fan lists of two people intersect as a whole, and who are their * * * friends?

put the fans of the two big V's in two sets, and make a sinter for the two sets.

set that is sorted, but can be sorted, and give a score when you write it in. Automatically sort

leaderboards according to scores:

zadd board score username

For example:

Zaddboard 85 zhangsan

Zaddboard 72 wangwu

Zaddboard 96 lis

Zaddboa. Rd 62 zhaoliu

The automatic sorting is:

96 lisi

85 zhangsan

72 wangwu

62 zhaoliu

Get the top 3 users: zrevrange board3

96 lisi

85 zhangs. An

72 wangwu

Check zhaoliu's ranking: zrank board zhaoliu returns that 4

memory is precious and disks are cheap

After setting the expiration time for keys, redis deletes these keys regularly+lazily

Periodically:

redis randomly selects some keys with expiration time every 1ms by default to check whether they are valid or not.

Note: redis randomly selects some keys to check and delete every 1ms, instead of traversing all keys with expired time (otherwise, the CPU load will be high and it will be consumed in checking expired keys)

Inert deletion:

When obtaining a key, redis will check to see if the key has expired if it has expired, and delete it if it has expired.

if you delete a lot of expired keys regularly, and then you don't check them in time or take lazy deletion, if a large number of expired keys accumulate in the memory, which leads to the exhaustion of redis memory blocks, then take the memory elimination mechanism.

Memory elimination strategy:

LRU algorithm:

Cache architecture (multi-level cache architecture, hotspot cache)

The bottleneck of Redis' high concurrency lies in a single machine, and reading and writing are separated. Generally speaking, it supports high concurrency of reading, with few write requests, that is, one or two thousand per second, and a large number of requests for reading, 2, times per second.

One master has many slaves, and the master is responsible for writing, copying data synchronously to other slave nodes, and the slave nodes are responsible for reading, and all reading requests go from the slave nodes. Mainly to solve the high concurrency of reading.

master-slave architecture -> Separation of reading and writing -> Support 1W+ reading QPS architecture

master-> Slave replication is asynchronous

Core mechanism:

Significance of master persistence to master-slave architecture:

If master-slave architecture is enabled, persistence of master node must be enabled, otherwise the data of master downtime restart will be empty. Once replicated, Slave's data has also been lost

master-slave replication principle:

First start-up or disconnection reconnection situation:

Under normal circumstances:

Master sends a piece of data to Slave asynchronously

99.99% of the whole year, which is in an available state, so it can be called high availability

redis high availability architecture is called failover.

sentinel node

Role:

quorum =1 (representing the minimum number of sentinels who can try to fail over and elect the sentinels)

master goes down, and only S2 survives, because quorum = 1 can try to fail over. However, the standard of majority =2 (the minimum number of sentinels allowed to perform failover) is not met, and the failover cannot be performed

If M1 is down, S2 and S3 think that the master is down, and choose one to perform failover, because the Majority of the three sentinels is 2. Therefore, failover can be performed

data loss:

Solution:

sdown is down subjectively, and the sentry feels that a master is down (ping exceeds the number of is-master-down-after-milliseconds)

odown is down objectively. Sentries in quorum feel that master is down

Sentries perceive each other and send messages (their own host,ip,runid) to the same channel every 2 seconds through redis' pub/sub system, so other sentries can consume this message

and exchange the monitoring information of master synchronously.

the sentry ensures that other slave modify the master information to the newly elected master

when a master is considered o down &; & Marjority sentinels all agree, then a sentinel will switch between master and standby, and elect a slave as the master (consider 1. the time of disconnecting from the master 2. the priority of slave 3. copy offset 4. runid)

election algorithm:

quorum number Sentinels think odown-> Elect a sentry to switch -> Get the authorization of the majority sentry (quorum majority needs the authorization of the majority sentry, quorum >; = majority needs quorum sentry authorization)

The first elected sentry failed to switch. After waiting for failover-time, other sentries re-use confiuration epoch as a new version switch to ensure the latest configuration for configuration communication (via pu/sub message mechanism, Other sentries update the master configuration compared with the old version)

High concurrency: master-slave architecture

High capacity: Redis cluster, supporting hundreds of thousands of read-write concurrency per second

High availability: master-slave+sentries

The significance of persistence lies in fault recovery data backup (to other servers)+fault recovery (in case of disaster, the computer room is cut off, The cable is cut)

There is only one AOF, the data in redis is limited, and the memory size is fixed. AOF is used to store write commands. When it is large enough, AOF will rewrite the data in Redis memory at that time, and then reconstruct a smaller AOF file, and then delete the old expanded file. AOF files will always be limited to the same data as in Redis memory. The synchronization interval of AOF is shorter than that of RDB, and the data is more complete < P > Advantages: < P > Disadvantages: < P > When data is recovered, all the instruction logs stored in AOF need to be played back, and RDB is a data file, which is directly loaded into memory.

Advantages:

Disadvantages:

AOF ensures that data is not lost, RDB does cold standby at different times

N Redis master node are supported, and each master node mounts multiple slave node

Multiple master+read-write separation+high availability

The amount of data is very small, with high concurrency -> Replication+sentinal cluster

massive data+high concurrency+high availability -> Redis cluster

hash algorithm -> Consistent hash algorithm->; redis cluster-> Hash slot algorithm

redis cluster: automatic logarithm