Williams, on loan from Southend having only signed for the Clarets on Thursday, headed in a cross from Leon Redwood for the simplest of finishes, but the lead was short lived as Lloyd Macklin equalised five minutes later.
City were controversially denied the lead again early in the second half as Michael Cheek’s goal was ruled out after the referee had already blown for a City free kick.
But the Clarets went back in front after Shaun McAuley brought Cheek down in the area as he was running away from goal, with Lee Sawyer stepping up to win all three points for the away side.
The victory was another one to add to an impressive away record so far this season for Mark Hawkes' side as the Clarets boss made two changes to the team that lost to Barnet on Tuesday night.
Mark Haines was replaced by Marvin Ekpiteta through injury with Hawkes giving a debut to on-loan Southend United striker Jason Williams in place of Christian Smith.
The Clarets faced a tough test at the Soccer AM Stadium as Basingstoke sat top of the league and were in the midst of an unbeaten run that dated back to September 20th.
Both sides created early openings on goal with Basingstoke’s top scorer, Chris Flood, testing the palms of Tony Thompson on two occasions and Joe Ward curling an effort just over from outside the area.
But it was City who surprised the league leaders by taking the lead early on after a good overlap from Leon Redwood allowed the left back to cross into the area. His cross found Williams unmarked in the six yard box leaving the striker to head the away side in front on his debut.
Chelmsford had so far been unbeaten in games this season where they had scored the first goal, but their lead was short lived as five minutes after Williams put the Clarets in front Basingstoke were back level.
A long ball over the top had Ekipteta and Lloyd Macklin battling it out for the ball. The Basingstoke striker won the battle, outmuscling the defender to be in on goal, drilling the ball into the bottom corner to level it up.
Having equalised so quickly, City could have expected Basingstoke to push them back but the rest of half played out evenly as the two sides headed into the break level with Yado Mambo coming closest to a third goal of the afternoon, getting underneath a Ward corner to head wide.
The second half started controversially as City should have been back in front after Cheek got on the end of a weak strike from Rohdell Gordon to round the goalkeeper and put the ball in the back of the net.
Referee, Carl Brook, though failed to allow City any advantage blowing for a Clarets free kick to the dismay of City boss Mark Hawkes. But their anger was not to last as minutes later City were back in the lead as they were awarded yet another penalty.
Cheek was brought down in the box by Shaun McAuley whilst running away from goal, giving Brook little choice but to point to the spot.
Having missed his last two penalties, Cheek handed the responsibility to Lee Sawyer who placed the ball into the bottom left corner to put City back in front.
The Clarets were once again proving the form guide wrong as they lead away from home against the league leaders, and they almost doubled their advantage after Cheek found Gordon whose drilled shot was cleared desperately by the legs of Moore.
Moore again was called into action as Williams this time muscled his way into the area as City looked for a potentially decisive third but at an acute angle the Southend loan-man couldn’t find a way past the Basingstoke ‘keeper.
As the game wore on Williams was replaced by Christian Smith as City went to shut up shop by packing the midfield, leaving Cheek up front on his own and any last ditch onslaught from the Dragons failed to materialise.
Basingstoke did have the ball in the net momentarily when Flood had failed to realise the ball had gone out of play for a goal kick. As Thompson set up to restart play Flood run up and kicked the ball into the net with the linesman awarding the goal only for the referee to point out it was in fact a goal kick to Chelmsford.
And Flood was inches away from levelling deep into stoppage time as he headed just wide from a corner leaving City to come away with all three points.
"From the first minute to the last the lads were superb" said boss Mark Hawkes at full time. "The work rate, their attitude, their application was great.
"We changed it up a little bit today. I just felt coming to Basingstoke - top of the league - with the couple they had missing that it was time to change formations and get at them and I think that's what we done in the first 20 minutes.
"We got in front, a little sloppy with their goal and we went in level but I think in the second half there was only one winner. The boys in the claret shirts were absolutely great."
1. Stuart Moore
2. Rob Rice
3. Tom Bird
4. Louie Soares
5. Jay Gasson (C)
6. David Ray
7. Simon Dunn
8. Shaun McAuley
9. Lloyd Macklin
10. Chris Flood
11. Jamie Brown
12. Nathan Smart
14. Andrew Jenkinson (On for Shaun McAuley 81’)
15. James Harper (On for Simon Dunn 75’)
16. Julian Owusu (On for Lloyd Macklin 89’)
17. George Hallahan
1. Tony Thompson
2. Rob Girdlestone
3. Leon Redwood
4. Mark Hughes (C)
5. Marvin Ekpiteta
6. Yado Mambo
7. Lee Sawyer
8. Joe Ward
9. Rohdell Gordon
10. Michael Cheek
11. Jason Williams
12. Christian Smith (On for Jason Williams 75’)
14. Luke Callander
15. Evans Kouassi
16. Nicky Nicolau
17. James Love (On for Joe Ward 90’)
Referee: Carl Brook
Assistant Referees: David Spain & Stephen Brown
Basingstoke: Jamie Brown (64’ – Descent)
Chelmsford City: Marvin Ekpiteta (24 – Foul), Michael Cheek (35 – Foul), Jason Williams (72’ – Failing to retreat), Lee Sawyer (74’ – Foul)
Basingstoke: Lloyd Macklin 21’
Chelmsford City: Jason Williams 16’, Lee Sawyer (p)53’
City Man of the Match: Lee Sawyer