The brave 1-0 win was only their second success since early September - and the first for the new manager.
It wasn’t enough to lift them off the bottom of the table but the three points closed the gap between them and Orient, who sit just above the bottom four.
Liam Daly’s headed just after half time did the job - and Yates says this needs to be just the start.
The new manager told the club’s website: “We’re still bottom of the table so it’s just a building block on top of another. I think the fans have just enjoyed the performance.
“I said to my players, give me that every week, give me that effort and do you know what, in four games they’ve given me everything.
“I asked for a little bit of quality, I thought we showed little bits of quality today and we’re going to keep asking for more.”
Yates feels there is more than enough time left to pull off the Great Escape.
“There are plenty of games to go and we have to believe,” he said. “There’s a lot to do, one game doesn’t all of a sudden mean that we’re sorted – we’ve got some serious work to do.
“The board, the staff, the players – they’re up for it. They’ve let me bring in a few players and we’re going to keep looking. We need to keep improving as a club, as a team and we’ll do everything we can to get us further up the table. We’ll work hard and we’ll give it a right good go.”