The Justice Minister's comments to the New Zealand Herald come after New Zealand's suppression laws were labelled "nonsense" this morning by National MP Nick Smith following Google's mass email naming the accused.
The generic no-reply email, viewed by the Herald, names the accused in its subject heading.
More than 100,000 searches of the 26-year-old's name were also made, according to the email.
The Silicon Valley company's publicly available analytics also showed the accused's name was the second-most searched item in New Zealand on Monday with over 50,000 searches.
A Google spokesperson told the Herald it would comply with any court order it was made aware of; however, they said initial investigations by the company showed it did not know about the suppression order.
Little, who has already rebuked UK media for naming the murder accused, said any publication of the man's details in New Zealand constitutes a breach of the court order.
The man applied for interim name suppression when he appeared in the Auckland District Court for his first appearance on Monday.
Despite Judge Evangelos Thomas refusing to grant the application, the man's lawyer, Ian Brookie, indicated he would appeal the decision to the High Court - invoking an automatic 20 working day suppression order.
As of yesterday, no formal appeal has been received by the High Court.
"Who you chase after to call to account is the difficulty," Little said of the numerous breaches.
"There are plenty of people who have said, 'well this is the internet' ... but we still have to defend the basic principles of our justice system," he added, listing the presumption of innocence and the right to a fair trial."
Little continued: "The very least we can do is call out those publishers, whether they are here or overseas.
He said the British press were able to name the accused after his court appearance because "someone in New Zealand provided those details".
"I'm not quite sure how much time the police want to expend trying to track who that might be," he said.
"Certainly, technically, there is somebody in New Zealand who is responsible for breaching the suppression orders who could be called to account if there is a willingness to do so."
As for Google, Little continued, if the email breach was traced to any of its New Zealand infrastructure then the Silicon Valley company could be prosecuted.
"Google has staff in New Zealand, I know because I've got Christmas cards from them," Little said.
The risk, he said, was one day a suppression order will simply be "flouted and the consequence will be catastrophic" and potentially invite "clumsy" legislation as a fix which could also breach people's right to freedom of expression.
On Tuesday, Prime Minister Jacinda Ardern acknowledged the online environment had evolved since Parliament last looked at suppression laws, but she had no plans to look at changing them.
But this morning at Parliament's justice select committee, National MP Nick Smith grilled the heads of the Justice Ministry about name suppression laws.
Smith said the current system was "a nonsense" given the Google alerts, and asked questions about the 20-working day interim suppression which is granted on appeal when name suppression is refused.
"The 20-day suppression order appeal mechanism is not just. Are you reconsidering [changing it] or is the Justice Ministry pretty sure there isn't a problem?" Smith asked.
He said defence lawyers have told him it was much better for their clients to simply lodge an appeal to delay the name becoming public for effectively a month, even if there are no substantial grounds for name suppression.
"It's much better for the defendant to have their name disclosed a significant period after an awful tragedy, such as in the Millane case," Smith said.
Ministry Deputy Secretary Rajesh Chhana said name suppression was looked at a few years ago, but there was no active review at it at the moment.
Millane was murdered on the weekend of her 22nd birthday, just a day after she arrived in Auckland as part of a one-year solo OE.
She had landed in New Zealand 10 days earlier and had been travelling around the North Island before settling in Auckland for a visit.
Her body has been returned to her family and they are in the process of taking her home to Essex in England during the next few days.
Google is not the only internet behemoth disseminating and allowing her murder accused's name to be published.
Herald searches on social media platforms Twitter and Facebook show hundreds of posts, comments hashtags and photos of the accused.
Twitter's search function automatically completes the accused's full name when simply typing the first letter of his name.
A Herald investigation earlier this year found some high-profile New Zealand court cases were having suppressed details published on Google.
Auckland University law professor Dr Bill Hodge told Newstalk ZB's Larry Williams this week the breaches by overseas websites was just one of the issues of modern media.
Detective Inspector Scott Beard, the police officer in charge of the Grace Millane investigation, also warned Kiwi social media users this week.
