The body of the 50th victim was discovered at the al Noor mosque, where most victims were killed, when officials were removing the victims' bodies on Saturday, Bush told reporters on Sunday.
The names of the victims were not made public. While a preliminary list of the victims has been shared with families, Bush said their bodies have not yet been released.
The number of injured also increased to 50. Of those injured, 36 remain in the hospital and two are in critical condition , he said.
Two days after the shooting gripped the country, Brenton Harris Tarrant, 28, appears to be the only person in custody that has been linked to the attack.
Three other people who were initially detained were not involved in the shootings, Bush said, and authorities are not ruling out the possibility of other suspects.
"I will not be saying anything conclusive until we are absolutely convinced as to how many people were involved, but we hope to be able to give that advice over the next few days, "the police commissioner said.
Suspect's manifesto was emailed before attack
The suspect sent an 87-page manifesto to New Zealand Prime Minister Jacinda Ardern minutes before the attack began.
Ardern's chief press secretary, Andrew Campbell, told CNN that the email was sent to a "generic" email account maintained by staff, and was not seen by the Prime Minister.
Tarrant, who is facing one murder charge, made a hand gesture associated with white supremacists when he appeared in court on Saturday.
He was remanded in custody and will reappear in court April 5.
Some victims had found refuge in New Zealand
A Syrian refugee, a Pakistani academic and their sons were among the 50 people killed in the terror attacks, family members and nonprofit organizations confirmed.
Syrian refugees Khaled Mustafa and his family had moved to New Zealand in 2018 because they saw it as a safe heaven, Syrian Solidarity New Zealand said on its Facebook page.
He was at the mosque with his two sons for Friday prayers when the shooter opened fire. His older son, Hamza Mustafa, 14, was killed and his younger son was injured.
Victims hailed from around the world. Naeem Rashid, 50, and his son Talha Rashid, 21, were among six Pakistanis who were killed in the mosques, according to Mohammad Faisal, spokesman for Pakistan's Ministry of Foreign Affairs.
"He used to teach at a university," Dr. Khurshid Alam said of his brother, the elder Rashid. "My nephew (Talha) was a student."
Makeshift memorials have sprung up in the area around the mosques, with flowers and notes bearing messages of hope and love.
"They may take our innocence but we will show the world the meaning of love and compassion," said one note taped to flowers left on a road divider.
Survivors tried to warn others
 Before two lightly armed community police officers ended the attack by running the gunman's car to the side of the road, some attempted to elude the suspect, according to survivors.
Survivor Ahmed Khan, after dodging a bullet fired by the shooter, ran to a mosque to warn others.
Inside he found a friend bleeding – he had just been shot in his right arm. "I said to him 'Calm down the police is here now,'" Khan recalled. "And then the gunman came through the window and shot him – when I was holding him – in the head. And he was dead."
In the video posted online by the shooter, he is greeted as he arrives at the first location, the Al Noor mosque, by a man who says "hello brother." Less than a second later, the attacker raises his semi-automatic shotgun and fires his first round of bullets.
Suspect traveled to Turkey and Pakistan
Tarrant is an Australian citizen who had been living in the southern city of Dunedin, about 225 miles from Christchurch, Ardern said. He had traveled around the world and was in New Zealand sporadically, she added.
Officials said that he had no criminal history in New Zealand or Australia and had not drawn the attention of the intelligence community for extremist views.
Tarrant visited Pakistan last October, the owner of a hotel in Nagar said. ] "He was a regular tourist," Syed Israr Ahmed told CNN. "All I remember was that he was a fan of local food. He would leave the hotel in the morning and come back in the evenings."
A senior Turkish official told CNN Tarrant had traveled to Turkey a number of times and spent "an extended period of time" there. Turkey is "currently investigating the suspect's movements and contacts within the country," the official told CNN. The suspect may also have traveled to other countries in Europe, Asia and Africa, the official added.
In a chapter of Tarrant's hate-filled manifesto he called for the assassination of Turkish President Recep Tayyip Erdogan, declaring "he must bleed his last. " State broadcaster TRT, citing Turkish authorities, said officials were investigating whether Tarrant was in the country "to carry out a terror attack and / or an assassination."
Erdogan condemned Friday's attack in a post on Twitter, calling it "the latest example of rising racism and Islamophobia. "
CNN's Mohammed Tawfeedq, Susanna Capelouto and Sophia Saifi contributed to this report.