Since the release of Firefox 47, automated tests using Selenium are pretty much broken: browser opens and instantly closes. This seems to be related to the Extension Signing addon and indeed temporarily downgrading to FF 46 solves the problem. However this addon will be added in FF 48 and starting from this version Selenium won't work with Firefox at all (at least, judging by the comments of the gem's maintainer).

Therefore it is really suggested to use Marionette, a Firefox driver, instead. Getting started with it is really simple: navigate to github.com/mozilla/geckodriver/releases, download and extract version that works for you. Then place the executable file to a directory that's present in your PATH. The guide also mentions that's it is advisable to rename the file to wires because some older versions of Selenium may still expect this name.

Once you do that, simply register a new driver in your test helper. For example, here is my setup for RSpec, rails_helper.rb file:

Capybara.register_driver :selenium_marionette do |app|
  Capybara::Selenium::Driver.new(app, browser: :firefox, marionette: true)
end

Capybara.javascript_driver = :selenium_marionette

And now my tests marked with js: true work perfectly! Note however that currently Capybara does not work with Marionette, but hopefully this will change soon.