Ledger was shooting scenes for the film before his death from a prescription drug overdose in January 2008, and Johnny Depp, Jude Law and Colin Farrell were drafted in by director Terry Gilliam to carry on the project.
All three stars play different versions of Ledger's character Tim - a move Troyer is pleased about, because it fits in so well with the weird fantasy film.
But the pint-sized star is adamant he would rather Ledger had played the role in its entirety.
He says, "I don't really want to say this but it's almost like it was meant to be because it flows so well. I'd rather have Heath for the whole thing but for some reason it worked.
"It was tough. I'm sure everybody else felt the same way I did. I just couldn't believe it. It was hard to go back to work because you were used to seeing Heath and he just wasn't there."