A query analyzer that parses Redis' MONITOR command for counter/timing stats about query patterns
At its core, redis-faina uses the Redis MONITOR command, which echoes every single command (with arguments) sent to a Redis instance. It parses these entries, and aggregates stats on the most commonly-hit keys, the queries that took up the most amount of time, and the most common key prefixes as well.
Usage is simple:
# reading from stdin
redis-cli -p 6490 MONITOR | head -n <NUMBER OF LINES TO ANALYZE> | ./redis-faina.py
# reading a file
redis-cli -p 6490 MONITOR | head -n <...> > /tmp/outfile.txt
./redis-faina.py /tmp/outfile.txt
The output (anonymized below with 'zzz's) looks as follows:
Overall Stats ======================================== Lines Processed 117773 Commands/Sec 11483.44 Top Prefixes ======================================== friendlist 69945 followedbycounter 25419 followingcounter 10139 recentcomments 3276 queued 7 Top Keys ======================================== friendlist:zzz:1:2 534 followingcount:zzz 227 friendlist:zxz:1:2 167 friendlist:xzz:1:2 165 friendlist:yzz:1:2 160 friendlist:gzz:1:2 160 friendlist:zdz:1:2 160 friendlist:zpz:1:2 156 Top Commands ======================================== SISMEMBER 59545 HGET 27681 HINCRBY 9413 SMEMBERS 9254 MULTI 3520 EXEC 3520 LPUSH 1620 EXPIRE 1598 Command Time (microsecs) ======================================== Median 78.25 75% 105.0 90% 187.25 99% 411.0 Heaviest Commands (microsecs) ======================================== SISMEMBER 5331651.0 HGET 2618868.0 HINCRBY 961192.5 SMEMBERS 856817.5 MULTI 311339.5 SADD 54900.75 SREM 40771.25 EXEC 28678.5 Slowest Calls ======================================== 3490.75 "SMEMBERS" "friendlist:zzz:1:2" 2362.0 "SMEMBERS" "friendlist:xzz:1:3" 2061.0 "SMEMBERS" "friendlist:zpz:1:2" 1961.0 "SMEMBERS" "friendlist:yzz:1:2" 1947.5 "SMEMBERS" "friendlist:zpz:1:2" 1459.0 "SISMEMBER" "friendlist:hzz:1:2" "zzz" 1416.25 "SMEMBERS" "friendlist:zhz:1:2" 1389.75 "SISMEMBER" "friendlist:zzx:1:2" "zzz"
One caveat on timing: MONITOR only shows the time a command completed, not when it started. On a very busy Redis server (like most of ours), this is fine because there's always a request waiting to execute, but if you're at a lesser rate of requests, the time taken will not be accurate.
Have more stats / improvements you'd like to see to Redis-Faina? Please fork and send pull requests! And if analyzing hundreds of thousands of requests per second across many systems is interesting to you, drop us a note and tell us a bit about yourself--we're building out our dev & devops team