The Saints have been in superb league form this season and sit sixth in the table.
But their FA Cup drubbing by Taunton Town last week has left Allison demanding a response, but also questioning some of his players.
“We haven’t done our jobs properly,” he told the Hertfordshire Advertiser. “It could have been six or seven, but we were chasing the game second-half.
“We were second best all over the park. We were poor.
Saturday sees Clarence Park host Non-League Day and the club will be expecting a larger crowd than normal leaving Allison determined not to let down the paying fans again.
“I know who hasn’t applied themselves and they will have to improve if they want to be in the side on Saturday.
“I’m disappointed, I’m disappointed for the fans that made the journey and as a dressing room we haven’t been good enough and we have to respond on Saturday.”