Ryan Jarvis, starting alongside his brother Rossi for the first time, hit two second half goals to help overcome a 2-1 deficit after Lee Williamson scored a penalty early in the second half.
The Jarvis brothers were joined by fellow academy graduates Robert Eagle and Ian Henderson in a youthful line-up that also benefitted from the experience of fringe first-teamers Craig Fleming and Peter Thorne.
Thorne took only six minutes to get off the mark, but Millers midfielder Michael Keane levelled things up and the sides went in at the break all square.
Williamson's penalty after a foul by Jason Shackell ensured a nightmare start to the second half for Nigel Worthington after having brought Paul McVeigh and Dickson Etuhu into the action, but two goals in a minute, from Ryan Jarvis and Craig Fleming, put the visitors back in front.
Jarvis added his second in the dying moments to ensure Norwich's place in the Third Round draw.