A Christchurch-based bomb squad was called in to the after a Waikawa and Districts Museum in the Catlins on Wednesday following information received about the substance.
Museum vice president Janette Buckingham said a visiting Southland District Council roving museum officer yesterday spotted the small bottle or jar in a medicine kit.
It had been at the museum since 1982.
"We've had it on display, quite harmless looking."
She said they did not know how hazardous it was, but it was better to err on the side of caution.
"Until we have a conclusion, it could be just a great big storm in a tea-cup. We don't know that it is hazardous, it is potentially hazardous."
Her understanding of the substance, known as picric acid, was that it could crystallize over time and become potentially explosive.
"After she alerted me to it we didn't dare touch it.
"It hasn't been handled, it's just sitting there doing nothing. But it's the fact that it's there...knowing what we know now, you would be very uneasy about it so we had to evacuate the museum."
The New Zealand Defence Force Explosive Ordnance Disposal unit based in Christchurch was notified and they have sent a team down to assess the situation and dispose of the substance safely.
Police said the disposal would happen today "in a safe environment away from the museum", which has been closed to the public.
Ms Buckingham said everyone was just doing their job.
"The fire crew is just doing their job, as was the museum officer yesterday who was working with me."
They would not be allowed back in to the museum until the substance was taken care of.
