Can YOU Still Receive Connections At REDIS_URL

It’s Friday morning and the Training Tracker production error handler started squawking.

Redis::CannotConnectError: SSL_connect returned=1 errno=0 peeraddr=XX.XXX.XXX.XXX:XXXXX state=error: certificate verify failed (self signed certificate in certificate chain)

Training Tracker is deployed on Heroku, and uses a mini Heroku Key-Value Store to manage a Redis server. Redis, on Heroku, largely “just works”™️ and once configured isn’t something you have to worry about. On Friday morning, the first entry in the TT activity log was as follows:

So something changed, our application was no longer connecting to Redis, errors were being thrown left and right. If you find yourself in a similar situtation, read on. Hopefully this blog can help get you back up and running.

How Fix 🔨

Your application isn’t broken. Heroku isn’t a villain. You will get things working again. Heroku regurally rotates your Redis URL, and for the most part that is a good thing.

What is going on here is that the Environment Variable REDIS_URL, which is managed by the Heroku Key-Value Store, when on the mini plan will henceforth use a secure TLS connection. In order for your application to communicate with Redis using that URL you will need to take that TLS connection into consideration.

For a Rails app that means adjusting the configurations of ANY service that is connecting to Redis. If your Rails server is connecting to Redis via the redis gem, you are going to want to create an initializer like the one below.¹

# config/initilizers/redis.rb
$redis = Redis.new(
  url: ENV["REDIS_URL"],
  ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE }
)

If ActionCable needs to connect to Redis you will need to make adjustment to your cable.yml config. The below should do the trick but your milage may vary.

# config/cable.yml
adapter: redis
  url: <%= ENV.fetch('REDIS_URL') %>
  password: <%= ENV.fetch('REDIS_PASSWORD', nil).presence %>
  ssl_params:
    verify_mode: 0

If Sidekiq is connecting you will need an initializer that looks something like the below.

# config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
  config.redis = {
    url: ENV["REDIS_URL"],
    ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE }
  }
end

Sidekiq.configure_client do |config|
  config.redis = {
      url: ENV["REDIS_URL"],
      ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE }
  }
end

Here is an article that Heroku published detailing changes that is making to the REDIS_URL environment variable. And hopfully that gets you sorted.

Footnotes

If you’re looking for a team to help you discover the right thing to build and help you build it, get in touch.