The concern stems from the work of Broad Bay Waterfront Inc, a group formed in 2007 to develop the idea of a shared boat club and community centre building.
Some residents spoken to by the Otago Daily Times - worried the building will dominate the bay and be paid for by selling the town's existing hall - have warned the project is "not a done deal".
Keep Our Hall spokeswoman Sarah McDougall, whose group comprised 15 members, said the issue was threatening to divide the community.
She feared losing the old hall would rob Broad Bay of its connection to past school plays and other community events, and mean the loss of a building "built by locals for locals".
The new building would also discourage some residents from swimming in the bay, she believed.
"It's really basically a clash of values about what people appreciate.
"We appreciate having the old quintessential village hall ... compared to those ... who want to have a flash boat club," she said.
However, Broad Bay Waterfront Inc secretary Allan Potter hit back, rejecting the concerns and saying objectors were "from a culture that's used to complaining".
"They are probably living in the past," Mr Potter said yesterday.
"They don't want to see progress in the bay."
The old community-owned hall was run by a committee reliant on a $15 annual membership fee, and fundraising events, to pay for its upkeep.
However, the hall's committee approached the boating club in 2005 to suggest a shared building, as it struggled to maintain the hall amid dwindling membership, he said.
Broad Bay Waterfront Inc was formed as a result in 2007, with representatives from the existing hall's committee and the Broad Bay Boating Club.
Mr Potter - a boating club member - said the group's plan included the sale of the old hall, which would raise about $600,000 towards the new building.
A survey of both groups found "unanimous" support for the new building, although the wider public sentiment had not yet been tested.
Detailed designs were being prepared, although a final decision would not be made until next year when other funding sources were also confirmed, he said.
Construction was expected to take "at least 12 months", costing between $1.1 million and $1.4 million.
The building would cater for both groups, for community events and private functions, and possibly a cafe/bar development later, he said.
The building would be larger than the existing club, extending into the bay with a deck and plaza, and would require both Dunedin City Council and Otago Regional Council consents.
The community had been kept updated, but a feasibility report prepared by the group warned the consents process could prompt opposition and appeals to the Environment Court, adding "scary" costs of "tens of thousands of dollars".
That meant the committee would not proceed without "wholehearted" community support, and "outright objection could be fatal", the report said.
Mr Potter did not think isolated opposition would stop the project, although the cost of proceeding, only to end up in the Environment Court, would need to be weighed up.
"That could stop it dead ... we might find there's half a dozen people out of 400 [residents] that have the power to do that."
