An estimated 15,000 people marched through the city streets on Sunday, demanding the Abbott government close the trouble-plagued immigration centres on Nauru and Manus Island.
The demonstrators marched from Parliament House to Queen Victoria Gardens on St Kilda Road, chanting condemnations of the government's asylum seeker policy.
Representations of several churches led the march, along with former refugees. Refugee Action Collective spokesman Chris Breen said the government had recently stepped up its efforts to deport people found "unfairly" not to be refugees.
Marches were held across 12 other Australian cities, and 19 cities overseas, demanding refugees be released from detention.
Former refugee Mohammad Ali Baqiri who arrived in Australia as a child from Afghanistan without his parents, said the government was acting without compassion or dignity towards people fleeing desperate situations.
"This same government continues to damage our national image and hurt our reputation," he said. "We desperately need leaders with a human approach who offer care and protection to those who need peace and security."
The rally was protesting against the government's compulsory detention of asylum seekers and for allegedly failing to provide basic care to thousands of people locked in centres in Nauru.
Two recent reports, the Moss report and the Human Rights Commission, have been scathing of the centres, with the latter finding incidents of sexual abuse, rape and intimidation at Nauru.
The government has been attacked for its strong criticism of both reports and for allegedly trying to induce Commission president Gillian Triggs to resign.
The United Nations has also found Australia's treatment of asylum seekers to breach international anti-torture laws.
Daniel Webb from the the Human Rights Law Centre told the crowd the government's policies failed to do justice to who "Australians are".
"We are not nasty and we are not cruel but our asylum seeker policies are. We are the only country in the world that subjects children to automatic and indefinite detention," he said.
The Palm Sunday rally drew 5000 more people compared to numbers from last year's event, police reported.