The Bluebirds went down 2-0 at the Evo-Stik Southern League side in the FA Cup second qualifying round.
Goals in each half from Charlie Davis and Mark Cooper gave the home side the win despite playing with ten men for 30 minutes of the second half after Calvin Brooks was sent off.
"To say I'm disappointed is an understatement," said Collier. "It was a poor, poor performance by some of our players. We have spoken to them about it and I have some big decisions to make on certain players.
"We've played ok but ok is not good enough for Chippenham Town. Too many people played within themselves and didn't have enough determination. We've gone out of the Cup with a whimper.
"Credit to Weymouth, they've played well and had a good shape about them. They defended properly when they had to.
"Too many of our players were way off the mark. Perhaps they're finding it tough pressure wise to be a Chippenham player and reach the levels we're trying to achieve, that's something for me to think about."