James-Collier, who plays conniving footman Thomas Barrow in the hit period drama, eventually plucked up the courage to talk to Smith and was surprised by her down-to-earth attitude and "wicked" sense of humour.
However, the actor admits the veteran actress can still be very intimidating with her "death-ray" stare, which he personally experienced when he accidentally trod on her toe while filming a dance scene.
He tells Britain's Lancashire Life magazine, "A lot of us were a bit intimidated by her. There are great actors in the cast, then there is Maggie. Nobody does it better.
"She's the master of the withering put-down and has a death-ray stare. I experienced it when I trod on her toe when we had to dance... It took me two years to pluck up the courage to have a proper conversation (with her). Then I found she is incredibly down-to-earth. No one calls her Dame Maggie and she has a wicked sense of humour."