Seafood company Sanford announced a change to its processing sites earlier this week.
The workers' union, Etu, said the proposal would cut "almost half" of Sanford's Bluff fish-processing plant workforce and move white fish processing to Timaru.
A Sanford worker, who asked not to be named, said it had been "pretty hard" to go back to work after hearing the news.
She said employees were unhappy with the potential outcome.
"It has been very emotional. There is no feeling of hope there. People are very upset. It's been very hard."
She said she felt betrayed by the company because the news "came out of the blue".
"They never said anything. Just called us on Tuesday and said we have some news ... This is not going to affect just the workers, it will affect the whole community."
Etu organiser Anna Huffstutler said Sanford recently gained resource consent to expand its salmon farms at Bluff, arguing this would bring jobs and benefits to the community.
"The community thought things were solid in Bluff because Sanford made a commitment that there would be more jobs if they got the consent. Now what they're saying is that's not going to happen for three years. People are furious, and they feel very misled."
Bluff Community Board chairman Raymond Fife said if the proposal was to go ahead, the results would be devastating for the community.
"For a small community it is a big loss. The company said in the medium to a long term, it will have more jobs, but people need jobs right now. Three years is too far away."
He said Bluff relied on those big industries and local business would suffer from the cut.
Sanford chief executive Volker Kuntzsch said the move was about creating "three centres of excellence" across the South Island to improve production.
"As we concentrate the white fish part of our South Island operations in Timaru, sadly we will see a loss of some jobs in Bluff. We are working through exactly what this will mean, as we consult with the team there on how many roles we can create on the salmon side of the business."
He said the company's target was to "only cut 20" of its 30 employees.
Mr Kuntzsch said the company had been transparent with the Bluff community during the resource consent process, but information about the "optimisation" was only announced this week.
"The problem with this kind of information, you can't share in advance ... It would create some uncertainty and a very unhappy workforce. We have an unhappy workforce now, but at least we know what we have and are trying to find solutions."
Mr Kuntzsch said the company would not know how many jobs would be cut until the consultation process finished in 10 days.
The company planned to end white fish processing at the Bluff plant at the end of July.
