The National League side tumbled out to Southern League opposition as the Bulls stuck a killer blow.
Speaking to the Barking & Dagenham Post, Still was unhappy as his team became the latest victim of a Trophy upset on Sunday.
“We made some attacking changes and Hereford scored from a needless penalty,” said the manager.
“The ball’s going out of play and Scott (Doe) has given away a penalty — I don’t know why. We had a couple of chances after that, but there wasn’t much in the game in the second half.
“I think first half, we were still on the bus and I don’t think we did very well,” he added.
“Hereford were deservedly in front, so I don’t have any complaints with that.
“Second half, we got ourselves back in it and I think both teams wanted to win it and not have a replay.”