The former central defender, appointed in January 2020, has departed by what the National League North club call mutual consent.
They posted a statement on their website announcing the news to their fans.
Hereford sit 15th in the table and fell to a 1-0 defeat at struggling Farsley Celtic on Saturday.
It was their third defeat in a row, leaving them eight points off the play-offs.
Assistant Manager Yan Klukowski will take the team on an interim basis, with the help of experienced Jared Hodgkiss.
A club statement read: “We are looking to appoint a new manager as soon as possible, further details will be provided about the recruitment process in due course.”
The Edgar Street club’s next match is at home to Southport on Tuesday night.
Bulls chairman Jon Hale added: “We’d like to thank Josh for his hard work and dedication during his time as manager of Hereford FC and wish him well for the future.”