The Bucks were looking to move out of the drop zone – and even a point could have been enough to go above fourth-bottom Tamworth.
And it was Ron Edwards` side who broke the deadlock 2 minutes before half-time when Marcus Dinanga was brought down just inside the box, and the striker dusted himself off before slipping home the resulting penalty.
Avenue grabbed an equaliser just 9 minutes after the restart with a header from sub Frank Mulhern.
It looked though as if the Bucks were heading for a point, until Mulhern popped up with a carbon-copy header in the 87th minute to win it for Mark Bower`s side.
The win leaves Avenue in ninth, four points behind the top seven, but if they are to make it into the play-offs they may need favours from teams above them as they have played more games than most around them.