There is a production system that has been running for many years, first as a PHP application, then as a hybrid with Rails, and now completely in Rails. It's unclear how long it has been around. The oldest git commit is from 5 years ago.
The goal is to keep the system running at all costs. It doesn't matter what code we use as long as nothing breaks. Currently it's at Rails version 3.2.33.
If we don't upgrade any gems we run the chance of becoming obsolete and undeployable. If we upgrade we will need to make changes to the code causing potential bugs to creep in. Not only do we face code rot, but also downtime due to AWS outages.
What would be the first step to make sure nothing breaks? I've spent months writing cucumber (integration) tests but it's hard to cover every edge case. The app has been running so long that most bugs have been fixed and there are few new exceptions. Testing was not a priority from the beginning so most of the code is undocumented.
Honestly, I find that Ruby on Rails is not ideal for this sort of application. Both Ruby and Rails have a very aggressive release schedule, and Rails especially isn't afraid to ditch backwards compatibility. Rails is great for agile development where things are always changing, but at the cost of long-term stability.
I'm assuming your application is big enough that you don't want to switch to anything else. Sinatra, for instance, doesn't change much and would be a much more stable option.
If you're sticking with Rails, I'd recommend getting rid of as many gem dependencies as possible. There is always a danger that they will no longer be developed or that they could introduce bugs or vulnerabilities.
Also, as much as possible, it's a good idea to favor POROs (plain old Ruby objects) over Rails dependent code. It generally takes more work but you wind up with more stable and reusable code.
I realize that may be more work than you want to put into an application like this, but that's my best advice.