What is a Document Store?
- Type of nonrelational database
- Designed to store and query data as JSON-like documents
- Makes it easy for developers to store and query data in a database using the same document-model format used in app
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image14_0df3e0eeb8.png"> </div>
<br/>
Difference of Document Store from Relational Database
- Doesn’t require a schema before inserting data
- Stores data for a single entity in a single document rather than multiple tables
- More flexible in managing application changes over time
- Relatively simpler
<br/>
What is Redis?
- High-performance in-memory data structure store
- Advanced key-value model that supports different data structures
- Can be used as a database, cache, and message broker
- Open source
- Developed by Salvatore Sanfilippo and sponsored by Redis Labs
<br/>
Benefits of Using Redis as a Document Store
- High-Performance
- Natively written in C
- Extremely efficient machine code and requires little overhead
- Support for Arbitrary Data
- Binary Safe - Can store any data from human readable text to encoded binaries
- Supports a maximum string length of 512MB
- In-memory Store
- Allows the delivery of the fastest possible access times to the data for both read and write
- Key-Based Access
- Extremely optimized and efficient access times
- Atomicity of Operations and Transactions
- Has clients in all the popular programming languages
- Redis Benchmark Example:
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image44_f9625addb2.png"> </div>
<br/>
Trade-offs of Using Redis
- In-memory Store (Dataset always resides in RAM)
- Expensive
- Redis uses periodic memory dump to create a persistency snapshot on disk
- May need manual configuration and triggering
- May affect performance of live running Redis server
- Non-Relational Database
- Does not support structured query language (SQL) and useful complex relational queries
<br/>
Redis Commands and Data Structures Hands-on Exercise
- Prerequisite
- Install redis-cli via Homebrew
- Start Redis Server
<div> <img class="img-fluid img-width-50 mb-2" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image8_5a6a4ee7fc.png"> </div>
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image51_88b36cb462.png"> </div>
- Some Useful General Redis Client Commands
- PING - Checks if connection to server is alive (redis-cli defaults connection to server 127.0.0.1 port 6379)
- Connect to remote server via redis-cli: 'redis-cli -h <host> -p <port> -a <password>'
- FLUSHALL - Delete all keys from database
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image24_defc231df1.png"> </div>
- Keys and Commands
- Redis keys are generally in the form <string>[:<string>]...
- Useful commands
- KEYS <pattern> - Finds all keys matching the specified pattern
- DEL <key> - Deletes the key if it exists
- EXISTS <key> - Checks if keys exists
- RENAME <key> - Changes the key name
- EXPIRE <key> <seconds> - Sets expiry of key after specified time
- TYPE <key> - Returns the data type of the value stored in the key
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image7_17d51e6775.png"> </div>
- Strings
- Binary-safe data structure
- Can store any kind of data (string, integer, float, JPEG image, etc.)
- Example Commands
- SET <key> <value> - Sets value at the specified key
- GET <key> - Gets the value of a key
- MSET <key1> <value1> <key2> <value2> - Sets multiple values to multiple keys
- MGET <key1> <key2> - Gets the value of given keys
- GETSET <key> <value> - Sets the string value of a key and returns old value
- STRLEN <key> - Gets the length of the value stored in key
- INCR <key> - Increments the integer value of a key
- DECR <key> - Decrements the integer value of a key
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image45_06081d9e68.png"> </div>
- Lists
- Collection of string elements sorted according to order of insertion
- Duplicates are allowed
- Example Commands
- LPUSH <key> <value1> <value2> - Prepends one or more multiple values to a list
- RPUSH <key> <value1> <value2> - Appends one or more multiple values to a list
- LRANGE <key> <start> <stop> - Gets a range of elements from a list
- LPOP/RPOP <key> - Gets element from the list
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image1_c08aeb050b.png"> </div>
- Sets
- Unordered collection of unique string elements
- Example Commands
- SADD <key> <value1> <value2> - Adds one or more members to a set
- SMEMBERS <key> - Gets all the members in a set
- SPOP <key> - Removes and returns a random member from a set
- SISMEMBER <key> <value> - Determines if values is member of the set
- SDIFFSTORE <destination> <key1> <key2> - Subtracts multiple sets and stores to destination key
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image17_2d746c9138.png"> </div>
- Sorted Sets
- Collection of unique string elements sorted according to order of floating-point score
- Example Commands
- ZADD <key> <score1> <value1> <score2> <value2> - Adds one or more members to a sorted set, or updates its score if already exists
- ZRANGE <key> <start> <stop> [WITHSCORES] - Returns a range of members in a sorted set, by index
- ZRANGEBYSCORE <key> <min> <max> [WITHSCORES] [LIMIT] - Returns a range of members in a sorted set, by score
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image23_46209c9554.png"> </div>
- Hashes
- Stores a set of field-value pairs
- Best way to store objects with key <object-type>:<id>
- Example Commands
- HSET <key> <field> <value> - Sets the value of a hash field
- HGET <key> <field> - Gets the value of a hash field
- HGETALL <key> - Gets all the fields and values in a hash at the specified key
- HMSET <key> <field1> <value1> <field2> <value2> - Sets multiple hash fields to multiple values
- HMGET <key> <field1> <field2> - Gets values of multiple hash fields
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image11_3d494e46d4.png"> </div>
<br/>
Redis Node.JS Application Use Case: Key-Value Store
- One effective way to use Redis is to use it as a database
- Can be used alongside a relational database (e.g. MySQL)
- Separation of Tasks
- Relational Database: Data for Listing Page (Flat Table Implementation)
- Redis Key Value Store: Data for Detail Page
- App Example
<div> <img class="img-fluid img-width-50 mb-2" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image52_07ae73955e.png"> </div>
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image53_e4dad49634.png"> </div>
- Performance Benefits
- Relational database schema will be compact
- Data needed for detail page is quickly accessible because of Redis’ key-value approach and high performance
<br/>
Node.JS Redis Client: ioredis
- One of the most popular Redis clients for Node.js
- Initialization and Connection to Redis Server
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image25_420fd70edc.png"> </div>
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image50_1d65cbf408.png"> </div>
- Redis Commands
- Returns a promise
- General Form: redis.<command>(<arguments>)
- Supports callbacks
- Has Built-in Arguments and Replies Transformers
<div> <img class="img-fluid mb-2 img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image47_c511539f54.png"> </div>
<div> <img class="img-fluid mb-2 img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image18_13b9ade0e0.png"> </div>
<div> <img class="img-fluid mb-2 img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image34_cc111739df.png"> </div>
<div> <img class="img-fluid mb-2 img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image35_490432fe15.png"> </div>
- Redis Pipelining
- optimization approach in Redis to reduce network latency and improve round trip time between Redis client and server communication
- main idea is to send multiple commands at once
- ioredis pipeline exec returns a Promise
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image29_3feefb339a.png"> </div>
<br/>
Redis as Node.JS Key Value Store Hands-on Exercise
- Goal: Add Redis Key Value Store on sample Node.JS Application
- Post User
- Get User
Exercise Steps
1. Open provided Express app and explore
2. Go to express folder and run npm install
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image33_1c33a4ba68.png"> </div>
3. Install ioredis
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image32_c77b5354e2.png"> </div>
4. Go to parent directory and run docker-compose-up
5. Initialize and connect ioredis to redis docker server in repository
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image42_c866d9845a.png"> </div>
6. Populate User Redis Key Value Store on Post API Call
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image10_e4feeeccbb.png"> </div>
7. Test Post User Using Postman and redis-cli
<div> <img class="img-fluid mb-2 img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image28_9b1f5a8570.png"> </div>
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image39_9b78029566.png"> </div>
8. Get User Data from Redis on Get User API Call
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image19_27987df49e.png"> </div>
9.Test Get User via Postman
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image16_d9d97dfee4.png"> </div>
10. Optimize Redis Key Value Store by Implementing Pipelining
a. Optimize Post User
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image15_4baf26c85a.png"> </div>
b. Test Optimized Post User Using Postman and redis-cli
<div> <img class="img-fluid mb-2 img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image5_62510ccb11.png"> </div>
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image55_f5db535fbf.png"> </div>
c. Optimize Get User
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image30_7b8f0c0cad.png"> </div>
d. Test Optimized Get User via Postman
<div> <img class="img-fluid img-width-50" src="https://strapi-saperium.s3.ap-southeast-1.amazonaws.com/redis_image31_a8aca6d219.png"> </div>