Dunn went freelance in 1936, and his career began a downslide. He did small films for medium-sized studios (Columbia, Universal, RKO) and even smaller films on Poverty Row (Monogram, Republic, Grand National). He returned to Broadway in 1940 with a role in "Panama Hattie", which at least kept his name before the public.
Elia Kazan came to Dunn's rescue with the period "A Tree Grows in Brooklyn" (1945). His performance as the ne'er-do-well Irish father garnered him a Best Supporting Actor Oscar; but disappointingly, his film roles didn't much improve. He did several stage productions, including "Harvey" on Broadway in 1948. Dunn made another eight unremarkable films, ending with "The Oscar" (1966).
If his film and theater appearances didn't rekindle his fame, Dunn did get a lot of TV work in the 1950s. He appeared on nearly every "Golden Age" anthology, including "Studio One" (CBS), "Pulitzer Prize Playhouse" (ABC), "Curtain Call Theater" (NBC) and "First Person Singular" (Dumont). He also had recurring roles in the series "It's a Great Life" (NBC, 1954-56), "Mr. Broadway" (NBC, 1957), and "Ben Casey" (ABC, 1961-66).