When you’re sending a lot of messages into RabbitMQ message broker and have started to experience RabbitMQ slowdowns, maybe it’s time to think about message compression. Good news: if you’re using Sneakers, you can do it without any ado!
The main idea is to reduce the amount of data RabbitMQ operates by compressing and decompressing messages on the client’s side.
The great thing about Sneakers is that it allows us to define our own content type and implement serializer and deserializer causing the work to be as easy as a pie.
In the code snippet below we define a custom content type
and two lambdas, one to convert data string into its compressed representation
and another one to turn compressed data back into the string it used to be.
Here I use
Zlib::GzipWriter for compression and
decompression but implementation is totally up to you.
Sneakers::ContentType.register( content_type: 'application/json+gzip', serializer: lambda do |payload| io = StringIO.new('w') w = Zlib::GzipWriter.new(io) w.write(payload) w.close w.string end, deserializer: lambda do |payload| io = StringIO.new(payload, 'rb') Zlib::GzipReader.new(io).read end )
Make sure you’ve implemented custom content type support in the message publisher and the message consumer both!
Place this code into
initializers/sneakers.rb if you are using Rails, or
another suitable place if you don’t.
A few months later, I discovered Google’s Brotli compression might be up to several times more effective for compressing json than the Gzip compression. Thanks to an awesome brotli gem, making use of it is a piece of cake:
Sneakers::ContentType.register( content_type: 'application/json+br', serializer: lambda do |payload| Brotli.deflate(payload) end, deserializer: lambda do |payload| Brotli.inflate(payload) end )
CommentHas something on your mind? Tweet it!