It seemed like the Stones were cruising to their fourth consecutive win until two goals in the last 15 minutes for the depleted visitors turned things around - watch match highlights here.
Zavon Hines scored from the penalty spot after Theo Vassell brought down the tricky forward and was sent off.
Alex Wynter looked to have put the result beyond doubt in the second half but Richard Peniket pulled one back and then Danny Johnson equalised ten minutes from time.
"Defensively, the two goals were a shambles, that's the only way I can put it," said Saunders. "But I don't want to take anything away from Gateshead, I thought they were outstanding when they went down to ten men.
"In an ideal world I'd have rather they stayed with 11 on the pitch. We looked good for the first 15 minutes and then the sending off changed the game. They put two banks of four behind the ball which is fair enough and they worked their socks off. I thought they deserved something out of it, sometimes you've got to hold your hands up.
"At 2-0 up we've got to do better for their two goals. If we don't switch off for those two moments, we win the game."
For more coverage, reaction and some of the best moments from games going throughout the season, head to our social media channels on Twitter, Facebook and now Instagram.