The 38-year-old arrived in the country on Friday and first appeared in court on Saturday charged with burglary and assault with intent to commit sexual violation.
He had been remanded in custody until today.
The charges related to events that allegedly happened at the home of Tania Billingsley on May 9.
Judge Arthur Tomkins granted bail to a temporary, undisclosed address, but imposed a number of conditions.
They were that he was to surrender his passport, have an overnight curfew, have no contact with the complainant and not to travel to Newtown, Berhampore or Cuba Street.
Rizalman was due to reappear next week to finalise bail conditions.
Judge Tomkins granted applications for media to take photos and footage of Rizalman during today's hearing.
Rizalman, wearing a grey striped shirt, stood quietly in the dock while arguments were being heard.
About half a dozen people, believed to be from the Malaysian High Commission, were in the gallery for the hearing.
Outside court, Rizalman was escorted to a waiting taxi without making any comment to media.
His lawyer, Dr Donald Stevens QC and the police prosecutor Grant Burston also declined to make a comment.
Following Rizalman's arrest on May 10, New Zealand asked Malaysia to waive diplomatic immunity but on May 22 he left the country after a mix-up between officials.
The Malaysian Foreign Affairs Ministry said Rizalman was accompanied to the country last week by two New Zealand police officers and two Malaysian armed forces personnel.
Malaysian newspaper The Star reported Rizalman was escorted onto an aircraft in Kuala Lumpur without handcuffs and had reportedly expressed his wish to voluntarily return to New Zealand.
Rizalman was a former defence staff assistant at the High Commission of Malaysia.
At the end of June the case was made public and an inquiry began over how the diplomat was allowed to leave.
Malaysia confirmed on July 2 that Rizalman would be returned to New Zealand after psychiatric assessments.
The Star reported he was discharged from the Tuanku Mizan Military Hospital at the end of July.
- NZME.
