Gaynes' feature film work began with a small role in "P.T. 109" (1962), and he played small parts in Sidney Lumet's "The Group" (1966), Sydney Pollack's "The Way We Were" (1973) and Peter Bogdanovich's "Nickelodeon" (1976). His first substantial screen role was in Pollack's 1982 comedy "Tootsie". In this Dustin Hoffman vehicle, Gaynes was John Van Horn, the silly old soap opera actor whose big kick is sticking his tongue into the actresses' mouths during love scenes. It turned out to be the magic his film career needed. Mel Brooks cast Gaynes as one of the troupe of Polish actors in "To Be or Not to Be" (1983), and in 1984 he played Commandant Lassard, the titular leader, in the first of seven "Police Academy" movies. At the same time, Gaynes' TV career was also on the rise. Having begun in TV in the 1950s and marking time in the daytime dramas "Search for Tomorrow" (1971) and "General Hospital" (1980), he went on to be featured in several miniseries and TV-movies. His chance for primetime stardom came with the NBC series "Punky Brewster", in which he was the curmudgeonly Henry Warnimont who takes in and raises an orphaned little girl. Gaynes also provided Henry's voice for the animated series version of the show. After the demise of "Punky Brewster", he played a recurring role on "The Days and Nights of Molly Dodd" (on which his wife, Allyn Ann McLerie, was a regular), and was the air-headed Senator Smithers on "Hearts Afire" during that show's 1992-93 season. He has also directed several episodes of the syndicated sitcom, "WKRP in Cincinnati".
In 1994, Gaynes was a member of the cast of Louis Malle's acclaimed independent feature, "Vanya on 42nd Street", as Serybryalzov, the impoverished estate owner married to the much younger Yelena (Julianne Moore). He was also featured in the film adaptation of Arthur Miller's "The Crucible" (1996).