Trinity will be wondering how they didn't get something out of the game having led twice and equalised late on.
The home side took the lead through Ashley Worsfold but Town levelled courtesy of Louis Jones' own-goal, although Bradley Wells regained Gainsborough's lead just before half-time.
Harrogate manager Simon Weaver got the reaction he wanted after the break as Dominic Knowles equalised before Joe Leesley struck twice in six minutes to give Town a 4-2 lead.
Gainsborough weren't prepared to give up and like the visitors had done to them, goals from Craig King and Shane Clarke in the space of three minutes tied things up at 4-4.
There was to be more drama in the 83rd minute though when substitute Terry Kennedy headed in what proved to be the winner for Harrogate.
"We got ourselves in a bad position first-half, losing at half-time, and we haven't been used to that," Weaver explained to the club's website. "We had a few words to say. We were well below our expectations of how we wanted to perform and were deservedly losing at that time.
"We got a good reaction and were an attacking threat for 20 minutes. Then we conceded two goals from nowhere. Fortunately we got the winner and the three points.
"We were weak as a team defensively. Our overall game was poor. We weren't energetic enough and that's not our identity. But you have to find ways of winning on days like this."