Despite fielding an almost full-strength side the Clarets failed to get on top of their hosts who had initially taken the lead through Matty Bakare, only for Michael Cheek to equalise a minute later.
The Millers went into the break in front though after Kyle Asante's turn and shot found the bottom corner, but Cheek once more wiped out the home side's lead as he looped the ball over James Marrable early in the second half.
Two superb stops from Niklas Freund prevented Aveley from going ahead once more, but he was unable to prevent Tyler Devlin scoring from close range after City failed to clear a corner in a night to forget for Mark Hawkes and his team.
Having been dumped out of the FA Trophy by local rivals Bishop's Stortford on Saturday, Chelmsford now find themselves out of all this season's cup competitions with just the league left to compete for.
Of the two cup exits this one will arguably hurt Hawkes and his staff the most, especially when you consider the side that the Clarets boss put out against a team from two divisions below them in the pyramid.
Bradley Watkins, Marvin Ekpiteta, Lee Sawyer and Christian Smith were replaced by Niklas Freund, Yado Mambo, Harry Morgan and Leon Redwood as City looked to ensure they weren't on the end of a cup upset.
But it was the home side that faired the better in the opening exchanges with Theo Ola calling Freund into action early after the nippy midfielder had gotten the wrong side of the City back line.
Chelmsford momentarily thought they'd taken the lead when Mark Hughes headed in a Nicky Nicolau free kick only for it to be ruled out by the offside flag, but just after the half hour the home side opened the scoring through Matty Bakare.
Neat build-up play between Ola and Tyler Devlin saw the ball fall to the feet of Bakare who opened up his body to caress the ball past Freund and into the bottom corner.
The Millers' joy though was short lived as within a minute the Clarets were back level after Michael Cheek smashed home from close range after failing to connect initially with Rohdell Gordon's cross from the right.
Having gotten back on level terms Hawkes would have hoped his side would kick on but ended up behind again just three minutes after levelling the scores, this time through Kyle Asante.
Sammy Knott was the provider as he whipped in a low cross from the left. Asante's first touch was perfect allowing him space to get his shot away on the turn as the Millers went in front at the break.
With an upset looming City made the ideal start to the second period by restoring parity within 10 minutes of the restart with Cheek getting his fourth goal in his last three games, and his second of the night.
Again Gordon was the provider as he came out the stronger in a challenge with Arian Retkoceri before chipping a delightful pass into the path of Cheek who looped the ball passed James Marrable and into the net.
The Clarets were now in the ascendency but they failed to continue their forward momentum and as penalties began to loom Freund was called upon on two occasions to keep City's cup hopes alive.
The young German first stopped Millers captain Junior Baker from scoring with a far post header from point blank range, before then preventing Devlin with an even better stop with his legs when the forward was one-on-one.
Devlin though was not to be denied what many would argue was a deserved winner for the home side as a corner from the right was swung into a dangerous area by Knott, breaking for Devlin at the far post to score off the underside of the bar from close range.
With little time to find another equaliser City did manage to create one clear opportunity to take the game to penalties as Cheek managed to find himself bearing down on goal.
The Clarets striker though, on a hat-trick, decided to try and round Marrable rather than go for goal and following a heavy touch he could only manage to get a cross into the area with Sam Gregory's header being blocked on the line as the Millers went through to the last eight.
1. James Marrable
2. Paul Preston
3. Sammy Knott
4. Junior Baker (C)
5. Billy Sendall
6. Brian Okwera
7. Theo Ola
8. Arian Retkoceri
9. Tyler Devlin
10. Kyle Asante
11. Matty Bakare
12. Alex Stavrinou
14. George Essuman
15. Marcus Elliott (On for Kyle Asante 72’)
16. Hassan Jollah (On for Theo Ola 80’)
17. Lewis Rowland
1. Niklas Freund
2. Rob Girdlestone
3. Nicky Nicolau
4. Mark Hughes (C)
5. Mark Haines
6. Yado Mambo
7. Harry Morgan
8. Joe Ward
9. Rohdell Gordon
10. Michael Cheek
11. Leon Redwood
12. Marvin Ekpiteta
14: Jake Gordon
15. Sam Gregory (On for Rohdell Gordon 82')
16. Eamonn Payne
Referee: C Pullen
Assistant Referees: J Pickford & D Harrison
Chelmsford City: Michael Cheek (Foul – 64’)
Aveley: Matty Bakare 33', Kyle Asante 38', Tyler Devlin 87'
Chelmsford City: Michael Cheek 35', 54'
HT Score: 2-1
City Man of the Match: Niklas Freund