And then there were nine

Census 2018 was a mess. A year on, big chunks are still missing from the data needed to plan the country’s future. Was it Statistics New Zealand’s clumsiness, or was it pushed? Bruce Munro investigates.

In Bathgate St, South Dunedin, the horizon is your neighbour's roof line. A rusty corrugated iron affair punctured by a century-old brick chimney. Or, perhaps, newer, green tile atop a small, orange brick ownership unit.

In this flat-as-a-pancake, renter-plentiful neighbourhood it is hard to see further than the next pay check, the next electricity bill.

Most of the residents of Bathgate St, however, are conscientious about their civic duties. Lawns are cut, rubbish bins wheeled to the kerb. The importance of filling out the five-yearly national census is not lost here.

Andrew Sporle, a senior research fellow in the department of statistics at the University of...
Andrew Sporle, a senior research fellow in the department of statistics at the University of Auckland. Photo: Supplied
Lorraine Smith stands at the door of a small unit, whose ranch slider spans more than a third of the building's length. She remembers getting a letter in the mail before last year's March census. She used the code it contained to fill out her form online.

She doesn't know many of her neighbours.

"People move addresses here a lot," she says.

A little further down the street, a woman in a dressing gown struggles to remember. She has no internet but thinks she probably did fill out the form.

Nearby, a boy answers the door and calls his father, a big man with an Eastern European accent. He is intrigued to know what the census is. He wonders why the Government would want to know so much about him.

"We just build our lives, one brick at a time," he says.

Others remember, or cannot remember, receiving a letter, doing the online census or ordering a paper copy.

The tally, when 10 houses have been canvassed on the subject of last year's legally compulsory, national census, is six "yes", two "not sure" and two "didn't get the letter".

Census 2018 has been described as many things: a mess, a shamozzle, a debacle, a policy disaster, a failure. It is a sorry state for an event that is vital to managing the country's resources, planning its prosperity and ensuring the wellbeing of its citizens.

For the past 167 years, New Zealand has held a regular nationwide count-up; a census of every person and dwelling in the country. The census gathers information on a wide range of topics, from the number of people living in each dwelling, types of heating used and relationship status to means of transport, hours of paid work and health issues.

The data is used for everything from deciding where schools should be built and how much funding hospitals should receive to determining electoral boundaries and figuring out which industries will need more workers. The fair distribution of tens of billions of dollars depends on quality census data. It is also used by non-government organisations and businesses in their research and planning.

At first, the census was held every three years, but it has been five yearly since 1881.

Census day is always a Tuesday in March - statistically the day and month when Kiwis are least likely to be travelling.

New Zealand has had one of the longest running, best censuses in the world, with, until recently, a 97% response rate. It has only been cancelled twice - in 1931, during The Great Depression, and in 1941, during World War 2 - and postponed once, after the 2011 Christchurch earthquake. That one was deferred to 2013.

Five years later, on cue, on Tuesday, March 6, Census 2018 took place. Only, something went wrong.

It was supposed to be the first primarily online census. A brave step into the digital age. But one in 10 people did not make the leap. More than 400,000 people failed to fill out a census form.

Ninety percent might sound like a pretty good response rate. But clear insight and effective planning is not likely when the equivalent of the entire population of Wellington, Tauranga and Dunedin, combined, is absent without leave.

On top of that, the fear is that those most likely to be missing are Maori, the elderly, the poor and those in rural areas. Their non-response rates are likely to be quite a bit higher than 10%.

Census field worker Em Oysten delivers forms for the 2006 Census. Photo: ODT files
Census field worker Em Oysten delivers forms for the 2006 Census. Photo: ODT files

One year on, Statistics New Zealand (Stats NZ), the guardians of the census, are still trying to pick up the pieces. An independent review is still investigating what went wrong. A panel of statistics gurus is still trying to patch serious holes in the data.

What exactly happened is hotly debated, but not loudly. Most of those close to the action will not talk on the record.

At an operational level, when it came to rolling out New Zealand's first largely online census, several wheels fell off, they say.

There wasn't enough publicity. Statistics New Zealand relied on the diminished postal system to get initial information to people. There were not enough forms. Fewer people were employed to follow up on those who had not filled out their form.

"There were a lot of cock-ups," one well-informed person says.

For previous censuses, squads of collectors were hired by local managers. Those census collectors were allocated neighbourhoods that they then visited, repeatedly. First they went to familiarise themselves, next to hand out census forms and gather information on the number of dwellings and occupants, then to follow up (as many times as necessary) to get people to complete their forms.

One person, who was involved in three censuses, says a trial of the approach used in 2018 was carried out in 2013, in one area of Oamaru. When that letter-only approach got a good response rate, albeit lower than the census collector approach, the green light was given for 2018. But when it came to it, what worked in urban Oamaru in 2013 did not translate to the whole of New Zealand five years later.

"I would have hated to have had anything to do with this last census," the person says.

The implications are "potentially disastrous", Andrew Sporle says.

Sporle is an Auckland-based statistician and a founding member of Te Mana Rauranga - the Maori data sovereignty network.

"If you under-count the number of kids in a region by say 10% or 15%, that could be 2000 kids. That's five schools," Sporle says.

"So, if you're trying to do some planning about how much to spend and where to spend it, it means you're going to be, not necessarily driving blind, but you're going to be seeing through some fairly foggy glasses."

Prof Peter Crampton puts it this way.

Physical education student Jade Humphrey (20) completes the census on census day afternoon last...
Physical education student Jade Humphrey (20) completes the census on census day afternoon last year. Photo: ODT files
The groups most likely to have been missed are the marginalised, disenfranchised, high-needs people, whanau, and communities - "those for whom good social policy is of the highest priority", the professor of public health at the University of Otago says.

"If these groups become invisible in the census then policy-making becomes doubly difficult and some of the least advantaged communities are at risk of becoming further marginalised."

The missing data affects the quality of information about what is going on right now. It also makes it difficult to see how things are changing over time, Sporle says.

"It is going to be harder to generate things like the Consumer Price Index and the unemployment rate from now on because it is going to be hard to do the surveys that inform those key statistics."

Statistics New Zealand has delayed the release of census data. It is working desperately to find other sources of data and new statistical methods that will enable it to fill in the missing pieces.

Sporle describes the group pulled together to work this magic as "fantastic people".

"If anybody can figure it out, the people on that panel can".

But he predicts the final result will inevitably be of lower precision than previously, with wider margins of error and less timely.

How it got to this place - the "cock-ups" - are easy enough to pinpoint. But why is a bit murkier.

Suggestions range from mindless cost-cutting to the inevitable outworking of a long-term contempt for expertise. Most fingers point at the previous government.

Could the disaster have been precipitated by an ideological commitment to a new form of census that is barely a census at all?

There is an ongoing argument over whether Stats NZ got the funding needed to do Census 2018 properly.

Nick Smith, the National Party's spokesman for State Services, says it was generously given 20% more than the budget for the previous census.

Kathy Connolly, Stats NZ's general manager census, says the organisation got $120million for Census 2018, compared with $74million for Census 2013. Connolly adds that the 2013 figure does not count the cost of developing processes and systems for the postponed Census 2011 that were able to be used in 2013. The 2011 non-census cost $64million. Adding the two figures together would not give an accurate figure but she is unable to provide one. Dead end.

Three facts, however, are clear.

Firstly, as early as 2014, the government had an independent report, commissioned by Stats NZ, that showed net benefit of the census to New Zealand was well in excess of $1billion. Or as the report put it, "every dollar invested in the census generates a net benefit of five dollars in the economy."

Secondly, the same year, despite that knowledge, the government told Stats NZ to cut the cost of the census by 5% over the next two census cycles.

And thirdly, when, in the run up to Census 2018, Stats NZ warned the government it needed a $3million contingency fund so it could respond quickly if something went wrong with the roll-out of the new online census, it was turned down.

Sporle calls it "extraordinary", "dumb" and "unwise".

"You don't put a $1 billion investment at risk," he says.

"That penny-pinching has come back and bitten us in the arse."

Why would they do that?

Curiously, the way Stats NZ is having to respond to the crisis is by fast-tracking methods that were being developed for a possible new type of census. One that the previous government favoured. One that some say is not really a census at all.

Former Prime Minister Sir Bill English championed the use of digital data for policy development. He drove what is called the Integrated Data Infrastructure (IDI) - a sharing of people's information held by 10 government agencies, including health, education, social development, justice and Inland Revenue.

In 2015 when Sir Bill was finance minister, the Cabinet agreed Stats NZ should work towards a future census, a census transformation programme based primarily on government administrative data supported by surveys.

The IDI would be the mainstay of that administrative data.

Sir Bill was prime minister by the time Stats NZ was turned down for the $3million contingency fund. Stats NZ did not have the resources to respond quickly when it became clear that the Census 2018 response rate was plummeting to a historic low.

Its only hope of redeeming the situation was to resort to the half-ready methods that it was hoped would make an IDI-based census a reality.

Smith rejects any suggestion the previous government undermined the census. He asserts National thinks the transformed, IDI-based census would not be robust enough. If Stats NZ had needed any extra money for Census 2018, the government would have given it, he says.

Has that always been National's position on the IDI-based census, or since Sir Bill retired? Always, Smith says.

What about the 2015 Cabinet paper saying the government agreed with the census transformation programme?

He cannot recall it.

"Cabinet did about 50 Cabinet papers a week. I was in Cabinet for about nine years."

It is hard to argue with the facts provided by Stats NZ's Connolly.

"In July 2017, Stats NZ requested a further $5million from the Finance and Expenditure Committee," she says.

Peter Crampton, professor of public health at the University of Otago, says it it essential the...
Peter Crampton, professor of public health at the University of Otago, says it it essential the census process is properly resourced. Photo: ODT files
"Two million was requested to cover additional costs that had arisen as a result of the 2016 Kaikoura earthquake and the impact on the organisation's IT infrastructure.

"Three million dollars was requested as contingency should the organisation need to move quickly to more manual processes to mitigate anything unplanned.

"The programme was granted $2million for additional costs."

Prof Crampton is not sure about the suggestion the failure of Census 2018 was an orchestrated attempt to force Stats NZ down the IDI-based census road. Only time will tell, he says.

But he is certain that across the State Services there was years of political disdain for civil servants' expertise and a bias against what was perceived to be "wasteful bureaucracy".

Asked whether that had been Stats NZ's experience, Connolly replied, "No comment."

Was the messaging "You guys cost us way too much money. This is the age of computers. Go do a cheap census"? Again, the reply, "No further comment."

Chris Hipkins, who is the minister of State Services, says the previous government cut funding throughout the public service.

"Every government should run a careful ruler over its spending to ensure value for money, but you can only cut so far before you hit bone," Hipkins says. "National was caught out and the census process has suffered."

Sporle says an IDI-based census, supported by surveys, still looks likely at some stage. He is strongly opposed to the idea.

"I don't believe it would be a good census. There is key information that only the census gathers.

"And surveys need a base line population to compare against. At the moment, the census is our base line."

Prof Crampton also has major concerns.

"It is essential that the census process is properly resourced and that the resulting data are as complete as possible and high quality," he says.

What is most at risk, is the belief that the census is important - worthy of the money spent on it, worthwhile giving time to fill out, worth keeping.

The danger is the legacy of Census 2018 will be that it undermines public confidence in the census - an instrument that relies heavily on people seeing value in it in order to fulfil its promise of being valuable.

Kaye Irving Arthur, lives on a semi-rural property less than 2km from the centre of Mosgiel.

She did not get a census letter ahead of Census 2018. Neither did her brother, who lives at the same address.

What is "even more random", she says, is that her husband did receive a census letter.

She remembers later reading about the debacle and thinking "Oh, yeah, that'd be right. Why fix something that's not broken?".

"If they want to do something right then for goodness sake just do it properly," she says.

"It's the way our country is supposedly run, with the statistics from census. How are they going to use the information if it's clearly not correct?"

 

Comments

"Those census collectors were allocated neighbourhoods that they then visited, repeatedly. First they went to familiarise themselves, next to hand out census forms and gather information on the number of dwellings and occupants, then to follow up (as many times as necessary) to get people to complete their forms".
Does this include collectors that attempt to force their way into your home to 'fill out' the forms. This has actually happened.

"Distrust of expertise" is the New Zealand distrust of professionalism and attention to detail, sadly fallen under the rubric of 'PC'.