Document Store Using Redis

What is a Document Store?

• Type of nonrelational database
• Designed to store and query data as JSON-like documents
• Makes it easy for developers to store and query data in a database using the same document-model format used in app

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis14_3a2de51f9d.png"> </div>

<br/>

Difference of Document Store from Relational Database

• Doesn’t require a schema before inserting data
• Stores data for a single entity in a single document rather than multiple tables
• More flexible in managing application changes over time
• Relatively simpler

<br/>

What is Redis?

• High-performance in-memory data structure store
• Advanced key-value model that supports different data structures
• Can be used as a database, cache, and message broker
• Open source
• Developed by Salvatore Sanfilippo and sponsored by Redis Labs

<br/>

Benefits of Using Redis as a Document Store

• High-Performance
• Natively written in C
• Extremely efficient machine code and requires little overhead
• Support for Arbitrary Data
• Binary Safe - Can store any data from human readable text to encoded binaries
• Supports a maximum string length of 512MB
• In-memory Store
• Allows the delivery of the fastest possible access times to the data for both read and write
• Key-Based Access
• Extremely optimized and efficient access times
• Atomicity of Operations and Transactions
• Has clients in all the popular programming languages
• Redis Benchmark Example:

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis44_078161946a.png"> </div>

<br/>

Trade-offs of Using Redis

• In-memory Store (Dataset always resides in RAM)
• Expensive
• Redis uses periodic memory dump to create a persistency snapshot on disk
• May need manual configuration and triggering
• May affect performance of live running Redis server
• Non-Relational Database
• Does not support structured query language (SQL) and useful complex relational queries

<br/>

Redis Commands and Data Structures Hands-on Exercise

• Prerequisite
  • Install redis-cli via Homebrew
  • Start Redis Server

<div> <img class="img-fluid img-width-50 mb-2" src="http://strapi.saperium.com/uploads/redis8_f1e55c5429.png"> </div>

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis51_41d8c7f8b2.png"> </div>

• Some Useful General Redis Client Commands
  • PING - Checks if connection to server is alive (redis-cli defaults connection to server 127.0.0.1 port 6379)
  • Connect to remote server via redis-cli: 'redis-cli -h &lt;host> -p &lt;port> -a &lt;password>'
  • FLUSHALL - Delete all keys from database

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis24_aa4959432a.png"> </div>

• Keys and Commands
  • Redis keys are generally in the form &lt;string>[:&lt;string>]...
  • Useful commands
    • KEYS &lt;pattern> - Finds all keys matching the specified pattern
    • DEL &lt;key> - Deletes the key if it exists
    • EXISTS &lt;key> - Checks if keys exists
    • RENAME &lt;key> - Changes the key name
    • EXPIRE &lt;key> &lt;seconds> - Sets expiry of key after specified time
    • TYPE &lt;key> - Returns the data type of the value stored in the key

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis7_f2b1f1fc6e.png"> </div>

• Strings
  • Binary-safe data structure
  • Can store any kind of data (string, integer, float, JPEG image, etc.)
  • Example Commands
    • SET &lt;key> &lt;value> - Sets value at the specified key
    • GET &lt;key> - Gets the value of a key
    • MSET &lt;key1> &lt;value1> &lt;key2> &lt;value2> - Sets multiple values to multiple keys
    • MGET &lt;key1> &lt;key2> - Gets the value of given keys
    • GETSET &lt;key> &lt;value> - Sets the string value of a key and returns old value
    • STRLEN &lt;key> - Gets the length of the value stored in key
    • INCR &lt;key> - Increments the integer value of a key
    • DECR &lt;key> - Decrements the integer value of a key

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis45_1f9b32a3e1.png"> </div>

• Lists
  • Collection of string elements sorted according to order of insertion
  • Duplicates are allowed
  • Example Commands
  • LPUSH &lt;key> &lt;value1> &lt;value2> - Prepends one or more multiple values to a list
  • RPUSH &lt;key> &lt;value1> &lt;value2> - Appends one or more multiple values to a list
  • LRANGE &lt;key> &lt;start> &lt;stop> - Gets a range of elements from a list
  • LPOP/RPOP &lt;key> - Gets element from the list

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis1_536040e557.png"> </div>

• Sets
  • Unordered collection of unique string elements
  • Example Commands
    • SADD &lt;key> &lt;value1> &lt;value2> - Adds one or more members to a set
    • SMEMBERS &lt;key> - Gets all the members in a set
    • SPOP &lt;key> - Removes and returns a random member from a set
    • SISMEMBER &lt;key> &lt;value> - Determines if values is member of the set
    • SDIFFSTORE &lt;destination> &lt;key1> &lt;key2> - Subtracts multiple sets and stores to destination key

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis17_1ff9b49c09.png"> </div>

• Sorted Sets
  • Collection of unique string elements sorted according to order of floating-point score
  • Example Commands
    • ZADD &lt;key> &lt;score1> &lt;value1> &lt;score2> &lt;value2> - Adds one or more members to a sorted set, or updates its score if already exists
    • ZRANGE &lt;key> &lt;start> &lt;stop> [WITHSCORES] - Returns a range of members in a sorted set, by index
    • ZRANGEBYSCORE &lt;key> &lt;min> &lt;max> [WITHSCORES] [LIMIT] - Returns a range of members in a sorted set, by score

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis23_e5f5d56392.png"> </div>

• Hashes
  • Stores a set of field-value pairs
  • Best way to store objects with key &lt;object-type>:&lt;id>
  • Example Commands
    • HSET &lt;key> &lt;field> &lt;value> - Sets the value of a hash field
    • HGET &lt;key> &lt;field> - Gets the value of a hash field
    • HGETALL &lt;key> - Gets all the fields and values in a hash at the specified key
    • HMSET &lt;key> &lt;field1> &lt;value1> &lt;field2> &lt;value2> - Sets multiple hash fields to multiple values
    • HMGET &lt;key> &lt;field1> &lt;field2> - Gets values of multiple hash fields

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis11_3880d8e9be.png"> </div>

<br/>

Redis Node.JS Application Use Case: Key-Value Store

• One effective way to use Redis is to use it as a database
  • Can be used alongside a relational database (e.g. MySQL)
  • Separation of Tasks
    • Relational Database: Data for Listing Page (Flat Table Implementation)
    • Redis Key Value Store: Data for Detail Page
  • App Example

<div> <img class="img-fluid img-width-50 mb-2" src="http://strapi.saperium.com/uploads/redis52_22d0967ab5.png"> </div>

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis53_8840901b5d.png"> </div>

• Performance Benefits
  • Relational database schema will be compact
  • Data needed for detail page is quickly accessible because of Redis’ key-value approach and high performance

<br/>

Node.JS Redis Client: ioredis

• One of the most popular Redis clients for Node.js
• Initialization and Connection to Redis Server

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis25_62d16f5588.png"> </div>

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis50_1fcf9134a0.png"> </div>

• Redis Commands
  • Returns a promise
  • General Form: redis.&lt;command>(&lt;arguments>)
  • Supports callbacks
  • Has Built-in Arguments and Replies Transformers

<div> <img class="img-fluid mb-2 img-width-50" src="http://strapi.saperium.com/uploads/redis47_7abcd3e3f0.png"> </div>

<div> <img class="img-fluid mb-2 img-width-50" src="http://strapi.saperium.com/uploads/redis18_3c9c49b660.png"> </div>

<div> <img class="img-fluid mb-2 img-width-50" src="http://strapi.saperium.com/uploads/redis34_c177ed942a.png"> </div>

<div> <img class="img-fluid mb-2 img-width-50" src="http://strapi.saperium.com/uploads/redis35_c33e636811.png"> </div>

• Redis Pipelining
  • optimization approach in Redis to reduce network latency and improve round trip time between Redis client and server communication
  • main idea is to send multiple commands at once
  • ioredis pipeline exec returns a Promise

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis29_85c3ca0bfb.png"> </div>

<br/>

Redis as Node.JS Key Value Store Hands-on Exercise

• Goal: Add Redis Key Value Store on sample Node.JS Application
  • Post User
  • Get User

Exercise Steps

1. Open provided Express app and explore

2. Go to express folder and run npm install

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis33_f473776f0c.png"> </div>

3. Install ioredis

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis32_142ec374b6.png"> </div>

4. Go to parent directory and run docker-compose-up

5. Initialize and connect ioredis to redis docker server in repository

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis42_8ee90c9f48.png"> </div>

6. Populate User Redis Key Value Store on Post API Call

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis10_718a7bed1d.png"> </div>

7. Test Post User Using Postman and redis-cli

<div> <img class="img-fluid mb-2 img-width-50" src="http://strapi.saperium.com/uploads/redis28_0976e1956a.png"> </div>

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis39_cca2b572d8.png"> </div>

8. Get User Data from Redis on Get User API Call

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis19_66d9db24ec.png"> </div>

9.Test Get User via Postman

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis16_8d50d5cb24.png"> </div>

10. Optimize Redis Key Value Store by Implementing Pipelining

a. Optimize Post User

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis15_d062f834e1.png"> </div>

b. Test Optimized Post User Using Postman and redis-cli

<div> <img class="img-fluid mb-2 img-width-50" src="http://strapi.saperium.com/uploads/redis5_f8e8cf102d.png"> </div>

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis55_db71a4635c.png"> </div>

c. Optimize Get User

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis30_8de3e343eb.png"> </div>

d. Test Optimized Get User via Postman

<div> <img class="img-fluid img-width-50" src="http://strapi.saperium.com/uploads/redis31_77f7e50330.png"> </div>