When the Persians defeated the Babylonians in 539 BCE things began to change.  The Persians maintained a benevolent dictatorship.  They were happy to allow deported populations to return to their homes and rebuild their places of worship.  King Cyrus wanted to sustain tranquility in his empire by respecting local populations and encouraging them to worship as they saw fit.

Those who did return were from the tribes of Judah and Benjamin.  They were primarily property owning families who were in control of the first temple.  Those who were returning felt that they were in charge of a divine task. 

Not everyone decided to return home though.  Some decided to offer their monetary support instead.  After all, this had been their home for the last 60 years, many had been born there.  They had intermarried with the local population.  They had roots there and decided not to make the long journey home.  Instead they maintained their Jewish culture and beliefs and stayed in Babylon.