The man and his child had to escape to the roof of the four-storey building in Ketton Close when it was engulfed in smoke.
A community nurse, who was visiting, knocked on doors in the flats so other occupants could escape, Luton crown court heard.
George Smith, 32, and his sister Jane, 45, were convicted by a jury of arson being reckless as to whether life is endangered.
The fire had been started on 3 August last year because the siblings wanted to move from the one-bed flat. George Smith was living there with his sister and her partner.
When the blaze broke out three fire engines and the police were called to the scene. Their private landlord put the damage caused at £6,000.
Defending George Smith, Tim Nutley said he had suffered a traumatic brain injury in 2004. “At the time he had moved in with his sister Jane. Their mother had passed away two years previously. They had visited her grave that day. “
He said his client was vulnerable and could not assist the court as to what happened that day.
“It was selfish behaviour, but was not born out of malice or a wish to harm anyone. It may have been due to a wish to move flats.”
He said the blaze was out by the time the fire brigade arrived.
Paul Webb, for Jane Smith said she is horrified by what happened and is unlikely to come before the court again. He said she suffers chronic back pain and depression.
Judge Michael Kay QC jailed George Smith for 4 years and Jane Smith for 30 months.
He said: “The motivation for this arson was to get out of the flat. There had been previous inquiries about a move.”
He went on to say the arson was extremely dangerous. “ Fortunately the fire did not catch and was confined to hallway. There was considerable smoke causing the father to take the baby onto the roof.”