His team produced a vastly improved performance to impress their new boss and inflict a rare defeat on the Tigers.
Goals from Damian Spencer, Mark Yates and Paul Brayson sent the home side into the second round of the Cup for the fourth season running.
There was an all-action start to the game with Hull striker Ben Burgess heading just wide from a corner kick and Cheltenham's Mark Yates brushing the post with a shot from the edge of the area.
Home goalkeeper Shane Higgs touched a shot from Stuart Green over the bar.
All of those incidents came within the first five minutes and the opening goal, which had looked likely, arrived with seven minutes on the clock.
Right-back David Bird lifted a ball over the defence for Spencer, who took it down and shook off a defender before squeezing a shot in off the far post.
Hull should have levelled straight away but Jason Price, playing as a makeshift striker in place of the injured Danny Allsopp, blazed over the crossbar from only eight yards out with just Higgs to beat.
Spencer then missed a chance to double his tally when a corner from Ben Cleverley sailed over the defence but he was unable to force in a shot from close range.
Hull drew level only seconds after the re-start when Price raced onto a through ball from Damien Delaney and scored past a hesitant goalkeeper, but only two further minutes had passed before the Robins got their noses back in front.
Midfielder Yates scored with a simple header from a Cleverley corner and from that point on the Robins worked hard to stay ahead.
John Finnigan forced a good save from Paul Musselwhite and Michael Duff had a shot cleared off the line by Dawson before the tie was put beyond doubt a minute from time.
Paul Brayson latched onto a long clearance from the back and produced a neat turn of skill to shake off two defenders before shooting low past Musselwhite.
"I asked the players to enjoy themselves and put on a performance and that was what they did," said Ward.
"They did everything that was asked of them and they can enjoy this win but the real work starts on Monday."
Hull boss Peter Taylor added: "I was disappointed with some of the performances out there but more disappointed by the assistant referee. He allowed their first goal to stand despite our players saying Spencer was offside. And he then disallowed a perfectly good goal by Jamie Forrester."