Harvey joined the OD's from Dunstable Town in 2020 and played 51 times, scoring six goals.
With competition high in the central midfield areas at the present time he has decided to pursue an opportunity at Bell Close.
A statement from AFC said: "Jack is a great lad and liked by everyone at the club.
Former Aston Villa and Crystal Palace youngster Williams elated to make his full Hatters debut in Carabao Cup
Jones could loan out Muskwe and Mendes Gomes as he admits certain players would have left regardless of cup exit
Jones still keen on pushing for defensive signing ahead of deadline day
Luton boss will make a late decision in the transfer window on whether to loan out Pereira
Hatters were aware of McAtee's shoulder injury before agreeing to sign Grimsby striker
"There is no animosity at all and we fully understand why Jack wants to make the move.
"He did everything the right way in leaving us and goes with our best wishes and good luck for the coming season.
"He always gave his all on the pitch and was a very popular character off the pitch.
"He is welcome back anytime to Creasey Park.”
AFC have also added two more partners to the club for the coming season with Go Bowling Dunstable joining up along with MunchBoxHQ.