The Dunedin protest was one of 11 around the country today.
Among the speakers expressing concerns about the Government Communications Security Bureau and Related Legislation Bill were Dunedin Mayor Dave Cull and Dunedin South Labour MP Clare Curran.
The bill is expected to be passed in Parliament with a single vote majority.
The bureau's empowering legislation prohibits it from spying on New Zealanders but it has done so 88 times since 2003, mainly in helping other domestic agencies.
The amendment bill will explicitly allow it to do so now, and it will also allow it to intercept the communications of New Zealanders in its role as the national cyber security agency.
In Auckland, traffic was at a standstill as protesters, many wearing masks and waving signs, spilled across the streets.
Police were diverting traffic away from the area.
Labour Party leader David Shearer said it would be an "uphill battle" to stop the passage of the GCSB bill.
"It'll be difficult to do but we'll do everything we can to oppose it.
"There's now a lack of confidence by New Zealanders in our intelligence agencies," he said.
Kim Dotcom told the crowd spies had invaded his privacy and "if they can do that to me they can do that to any of you".
Another speaker at the Auckland march was Dr Rodney Harrison, QC, who called the planned legislation the "Dunne Deal" - referring to the United Future MP.
In Wellington, more than 500 protesters held up buses and cars as they marched in a crowd more than 200m long from Cuba St to the steps of Parliament.
Many wore masks or held signs aloft as they chanted "we don't want your spying state" and "surveillance is for war" as they marched along Willis St and Lambton Quay.
They were met at the steps of Parliament by speakers including Green Party co-leader Russel Norman, Labour MP Chris Hipkins and Council of Trade Unions president Helen Kelly.
Vikram Kumar, the chief executive of Kim Dotcom's new Mega website and the former head of Internet NZ, was met with a resounding "no" when he asked the crowd whether they should be spied on for marching today.
"This was a country that stood up and said no to nuclear power, it said no to apartheid. And what do we say today? No!
"The land of the long white cloud becoming the land of the NSA spy cloud? No!"
Dr Norman called on everyone at the protest to file an Official Information Act (OIA) request with the GCSB asking how many people were at the rallies across the country today.
"Maybe if they're so tied up dealing with 10s of thousands of OIA requests, it might give them less time to go around spying on us with their special powers."
Civil liberties campaigner and Tech Liberty co-founder Thomas Beagle said the GCSB bill was about "mass surveillance".
"It's about spying on everyone, no matter what they've done, no matter what they're gong to do," he said.
"This sort of mass surveillance changes the balance of power in our society away from the people and towards the state."
Mr Beagle said it was wrong to suggest people who had done no wrong had nothing to fear from surveillance.
"I believe in the right to privacy, I believe in the right to sit in my house and call my friends on the phone without the Government listening.
"I believe in freedom of expression and freedom of association, for people not being scared into silence because they are being watched by Government spies."
Prime Minister John Key told reporters on his visit to South Korea that it was important to understand what was actually in the GCSB legislation.
He said the bill would give greater oversight and was "a strengthening of the previous legislation".
"I accept there are some that will always feel a bit nervous about privacy and their own rights, but I can give you the best assurance I can that we're very careful and cautious about what we do as a state. But in the end we do have to protect the interests in New Zealanders."
Asked about Kim Dotcom 's involvement in the protests, Mr Key said "he loves the limelight".
"But you've got to remember he's got a long-term agenda here, and that is to try and convince people that he should be allowed to stay [in New Zealand] forever."
However, Mr Key said Dotcom needed to make that case to the United States, which was seeking Dotcom's extradition.
- additional reporting APNZ