Industry reports suggest Ross has walked away from the hit franchise following a much-publicised pay dispute.
One of the film's stars, Josh Hutcherson, previously claimed Ross was unlikely to leave as he was "the only one that could ever direct the second one".
However, the moviemaker has announced he will bow out of the series, insisting he doesn't have the time to ensure the sequel is completed to his satisfaction.
In a statement, Ross says, "Despite recent speculation in the media, and after difficult but sincere consideration, I have decided not to direct Catching Fire. As a writer and a director, I simply don't have the time I need to write and prep the movie I would have wanted to make because of the fixed and tight production schedule."
A representative for film studio Lionsgate adds, "We're very sorry that Gary Ross has chosen not to direct Catching Fire. We were really looking forward to making the movie with him. He did an incredible job on the first film and we are grateful for his work. This will not be the end of our relationship, as we consider Ross to be part of the Lionsgate family and look forward to working with him in the future."