The building, constructed for the 1925 South Seas Exhibition, was the Dunedin public art gallery for almost 60 years.
Demolition is expected to be completed by the end of the month.
It will allow the expansion of the University Oval cricket ground, which needs to be larger to host tests against major cricketing nations.
The work is costing about $230,000, while expanding the cricket venue is expected to cost about $5 million.