Empirically, software with higher complexity metrics, such as cyclomatic complexity, are harder to maintain. There is research supporting this dating back to the 1970s. There is also work that suggests that complexity density, which is cyclomatic complexity over size of the system also relates to maintenance time. Unfortunately, an IEEE subscription is necessary to access both of those papers, but you can try to look them up on other sources if you're interested.
From a quality perspective, it's often worth it to spend some time refactoring, assuming you have a test framework in place to prevent the introduction of new defects. This will allow you to more easily implement new features to your system, add additional tests, and train new developers to work.
Ultimately, however, there is the pressure to deliver new functionality and added value. You need to balance refactoring with the implementation of new functionality and repair of defects.