City pushed for an equaliser but made it difficult for themselves when having a man dismissed 10 minutes before half time. There was confusion as to who was shown the red card after Duane Darby was brought down just outside the box. It looked as though Delaney would walk, but it was in fact captain Justin Whittle who was dismissed.
Minutes before half time Jamie Forrester hit the post with a great chance to get an equaliser.
However, City's fortune seemed to change in the second half with two goals in four minutes. Debutant Jon Otsemobor levelled the scores when he headed home Dean Keates' corner just before the hour mark, then Jon Walters got on the end of a Martin Reeves corner minutes later to make it 2-1.
But Paul Hall restored parity in the 68th minute, and the home side took control of the game. Onandi Lowe had two shots on target while David Bell hit the crossbar, and a winner looked imminent.
It came in the 87th minute when a failed clearance fell to Lowe who made it 3-2.
The game was finished two minutes into stoppage time when Hall pulled it back from the left to Stuart Wardley who made no mistake in making it 4-2 and securing the three points for the home side.
City line-up (4-4-2): Fettis; Otsemobor, Whittle, Anderson, Delaney; Reeves (Melton 75), Appleby (Smith 45), Keates, Elliott; Walters, Forrester.
Un-used subs: Musselwhite, Regan, Jevons.