Recently at a monthly DevJam meeting Ted Naleid presented on the topic of Redis and how its use as a data store and cache. I was intrigued and wanted to delve into Redis more as a semi-persistent data store. I came across the Redis GORM plugin that allows you to persist POGOs directly to Redis via normal GORM operations. With the release of Grails 1.4.0 M1, there have been some changes to the default persistent data store from HSQL to an H2 database. I wondered just how much faster read/write operations could be with Redis (if at all) over the H2 database so I set out to write a little test app to see for myself.
I am not going to get into the specifics of the jQuery widget I wrote to do this, how to configure it or the semantics on how/why I chose to use the deferred object, but please grab the code and have a look at that stuff too.
The source code for this prototype is available at https://github.com/ctoestreich/redis-gorm.
Getting Redis
The first thing I needed to do was download the windows port of Redis from https://github.com/dmajkic/redis/downloads. I downloaded version 2.2.5 and edited the following lines in the 32bit (or 64bit) redis.conf file:
Uncomment/change the following line (probably a good idea to always have a PW on any persistent stores) requirepass a1s2d3f4qwert!
You can start redis now while we get the project running
Running Grails
For this particular task I decided to use Grails 1.4.0 M1. It was a bit different than 1.3.7 in getting the client setup working well with the newly integrated resources plugin. I assumed that the wiring would be a little more automatic, and it was, but I was still required to manually add the r:resources and the jqui tags. I also had a bit of an issue with IntelliJ 10.5 not recognizing the Grails 1.4.0 M1 as a valid SDK. I basically ended up coding all the stuff old school in textpad (so forgive any non-static analysis friendly code).
I installed the Grails Redis GORM plugin which allows you to add the following line and expose normal Gorm operations against Redis.
static mapWith = "redis"
My objects were pretty simple and consisted of the following nearly identical code.
class RedisObject {
static mapWith = "redis"
String name
List someList
static mapping = {
name(index:true)
}
}
class GormObject {
String name
List someList
}
I also added the following to the Config.groovy to get Grails talking to Redis
grails.redis.host="localhost"
grails.redis.port=6379
grails.redis.password="a1s2d3f4qwert!"
grails.redis.pooled=true
grails.redis.resources=15
grails.redis.timeout=5000
With your grails home and path set to recognize Grails 1.4.0 M1, start the server up.
grails run-app
After starting the server navigate to http://localhost:8080/redis- gorm/dataTest/index
Under The Covers
What is happening under the covers is multiple ajax requests are being performed for both create/save and retrieve against Redis and H2. I am using Highcharts to render the results for the operations realtime using the jQuery ajax and deferred libraries. Every time both (Redis/H2) requests complete it doubles (configurable) the number of operations it is trying to perform. The ajax calls return back to the client the time in ms it took to execute the request number of operation. The results will not be graphed until both Redis and H2 ajax operations have returned results.
Here is a very simplified flow of what is happening:
create widget –> performTest –> defer(getRedis, getGorm) –> both completed –> collect results data –> update graph –> increase record count –> start from performTest again
Here is a snippet of the code used on the controller to conduct the tests. I would encourage you to browse and download the code from github for more specifics on the client and server code.
def getGorm = {
def gorms = []
def duration = benchmark {
def count = GormObject.count()
(1..Integer.parseInt(params?.recordCount ?: "2")).each {
gorms << GormObject.findById((Math.random() *
count).asType(Integer.class))
}
}
println "Gorm found: ${gorms.size()}"
render text: duration, contentType: "text/plain"
}
def getRedis = {
def redises = []
def duration = benchmark {
def count = RedisObject.count()
(1..Integer.parseInt(params?.recordCount ?: "2")).each {
redises << RedisObject.findById((Math.random() *
count).asType(Integer.class))
}
println "Redis found: ${redises.size()}"
}
render text: duration, contentType: "text/plain"
}
def makeGorm = {
Integer total = Integer.parseInt(params?.recordCount ?: "100")
def duration = benchmark {
(1..total).each {
new GormObject(name: "GormObject" + it, someList:
makeListData(10)).save()
}
}
render text: duration, contentType: "text/plain"
}
def makeRedis = {
Integer total = Integer.parseInt(params?.recordCount ?: "100")
def duration = benchmark {
(1..total).each {
new RedisObject(name: "RedisObject" + it, someList:
makeListData(10)).save()
}
}
render text: duration, contentType: "text/plain"
}
private makeListData(count) {
def listData = []
(1..count).each {
listData << ["listData${it}"] } listData } private
benchmark = { closure –>
def start = System.currentTimeMillis()
closure.call()
def now = System.currentTimeMillis()
now - start
}
When invoking the read operation I wanted to grab a random set of results so I opted to use a random number generator and findByID(rand) to grab single records as opposed to a list operation. There is probably some inherent overhead in this type of operation as opposed to grabbing a whole list at a time. There is probably a better way to do this, but as the count of records to retrieve goes up, I was really hoping to see the difference in read operation time between the two by forcing more reads.
My Observations
First I should mention that I haven’t collected and analyzed with any detail specific numbers. I have mainly just used the graphical output and results generated from the controllers as my point of reference. I think depending on the machine the mileage of the tests and results will vary.
Redis is consistently faster on the read operations while the save seems to be pretty close and H2 keeps pace with Redis. The tests do seem to jump around a bit, but generally speaking Redis read/write operations are faster that those of H2 running in memory. I suppose I expected this result as Redis is best suited as a key/value store and lacks some of the more robust operations supported in a product more suited for relational data modeling, but I think as a simple and fast data store, Redis is more than a viable option.
Things To Note
Running the tests more times up to a number like 65k test items without stopping the server (just refreshing the browser) will help get items into the databases so there is more objects to randomize against the read operation.
I would like to see these same tests conducted using different criteria-based and list operations so the db has to do more of the work in joining and aggregating data on the read operations. Maybe next week.
Sample Output
The results from this graph show the time in ms it took to process a set number of records per operation type. The script will process them in blocks of 2,4,8,16,32 and so on until it hits some limit set via the widget. Then the results are graphed for the time to execute 2 saves, 4 saves, 8 saves, etc. The same is true of the read operations graph and test.
Conclusion
I have more research I would like to conduct on this topic by comparing a physical data store like MySQL against Redis for example. I don’t have an immediate need for a simple key/value store or cache at the moment, but I would love to incorporate Redis into our application stack in the the near future.
The source code for this prototype is available at https://github.com/ctoestreich/redis-gorm.
(Grails 1.4.0 M1)
content/uploads/2011/06/start_redis.png (start_redis)
content/uploads/2011/06/start_redis.png
content/uploads/2011/06/graph1.png (Results Graph)
content/uploads/2011/06/graph1.png