Jack Magill put Kent ahead on 37 minutes only for the visitors to level through an own goal seconds late.
But the hosts triumphed in the third minute of stoppage time thanks to Simon Richards’ winner at the death.
AFC Dunstable Res and Kent Athletic Res drew 2-2 in Reserve Division One, while in Reserve Division Two Totternhoe beat Winslow United 1-0.
Meanwhile, the Division Two game between The 61 FC (Luton) and New Bradwell St Peter and the Bedfordshire Senior Trophy semi-final between Wootton Blue Cross and Crawley Green were both postponed.
Former Aston Villa and Crystal Palace youngster Williams elated to make his full Hatters debut in Carabao Cup
Jones could loan out Muskwe and Mendes Gomes as he admits certain players would have left regardless of cup exit
Hatters were aware of McAtee's shoulder injury before agreeing to sign Grimsby striker
Jones still keen on pushing for defensive signing ahead of deadline day
Potts determined to nail down centre half role at Luton following Naismith's exit to Bristol City
Tonight, AFC Dunstable Res go to Kings Langley Res in Reserve Division One and on Wednesday, Crawley Green entertain Hertford Town in the second round of the Challenge Trophy.
This Saturday Crawley head to Buckingham Athletic in Division One, in Division Two Kent Athletic are home to New Bradwell St Peter and Totternhoe entertain Mursley United, while in Reserve Division One
AFC Dunstable Res visit Baldock Town Res and in Reserve Division Two Totternhoe Res go to Mursley United Res.
Meanwhile, on Tuesday week, Kent Athletic Res travel to Holmer Green Res in Reserve Division One.