The Ryman League Division 1 North side could even afford a penalty miss on their way to securing an Essex derby against Braintree Town in the 4th Qualifying Round.
The Hawks took the lead 4 minutes before the interval when Alex Read racing onto a through ball to shoot past Steve Phillips.
City were largely restricted to long-range efforts and with 8 minutes to go it looked as though the Romans were virtually defeated when Harlow won a penalty after David Pratt brought down Fabion Simms, but Phillips saved Read's effort to keep City's hopes alive.
However, a couple of minutes later Harlow broke away and substitute Lewis Perkins did make it 2-0.
Bath managed a consolation 2 minutes from time when Pratt headed home but they were out and victims of another FA Cup `shock`.
Gosport Borough finished with only nine-men as they bowed out of the competition after a controversial 2-1 defeat by National League South rivals Whitehawk at Privett Park.
Boro took a 5th minute lead through George Barker, but had both Dan Wooden and striker Justin Bennett sent-off before Jake Robinson scored Whitehawk’s late winner.
The referee also red-carded Whitehawk’s Nick Arnold during the first half of an incident filled tie.
The all-National League North replay saw veteran striker Lee Hughes score an injury-time winner to put knock high-flying Solihull Moors out.
Hughes' added time winner put City through to face National League side Gateshead in two weeks` time, ending Solihull`s unbeaten start to the season.
Worcester`s league rivals Brackley Town survived a tricky journey to Butlin Road to face NPL First Division South outfit Rugby Town, but came through 2-0 and now take on Bamber Bridge in the next round.
Richard Batchelor gave the Saints the boost of an early 4th minute lead which was eventually doubled to secure their passage 10 minutes from time through Steve Diggin.
Another National League North side through against NPL First Division South opposition were Stalybridge Celtic, who won 2-0 at Bower Fold against famed FA Cup fighters Chasetown.
Jack Higgins gave the home side the boost of a goal just before half-time.
But it wasn`t until 3 minutes from time that Celtic fans could relax when Adam Farrell added number two.