Sadness, not just at his absence, but also that he did not live to see the outcome of the latest moves to clear his name. Not long before he died, the Supreme Court granted him leave to appeal his conviction.
Ellis, the man at the centre of what became known as the Civic Creche case, has always maintained his innocence. He received a 10-year jail sentence for abusing seven children at the Christchurch Civic Childcare Centre in 1991.
Many of the claims made against him could only be described as bizarre. Some of his fellow creche workers - all women - were also charged, but charges against them were dropped.
After a High Court trial in 1993, Ellis was convicted of 16 counts of sexual offending. He was released in 2000 after serving seven years. Three charges were overturned in 1994 after one of the children said she lied, but a second challenge to the Court of Appeal was dismissed. A ministerial inquiry in 2001 by Sir Thomas Eichelbaum found there was no risk of a miscarriage of justice.
There have been decades of disquiet about the case, particularly with regard to the reliability of the children's evidence. Children were repeatedly interviewed using techniques which would no longer be considered acceptable. Increasingly, the case has been seen as part of the world-wide moral panic and hysteria in the 1980s and early 1990s where bands of childcare workers were accused of far-fetched ritual abuse and torture of children.
As Ellis put it himself, shortly before he died, it was a phenomenon that could have hit any creche in New Zealand.
However, other countries had dealt with the resulting wrongful convictions while New Zealand had yet to "sort it out", he said.
Concerns have also been raised at the off-putting impact the case had on men who might have wanted to work in early childhood education or other teaching jobs, and the fear of false accusation it provoked in some men already in such roles.
Otago people have been at the forefront of the latest moves to have the case re-examined.
A University of Otago team, including vice-chancellor Harlene Hayne, who has done considerable analysis of the children's interviews, and former law school dean Mark Henaghan, have built on the research of Dunedin author Dr Lynley Hood, and work done by Ellis' former lawyer, Judith Ablett-Kerr QC.
The importance of Dr Hood's courageous and meticulously researched book A City Possessed: The Christchurch Civic Creche Case deserves special mention for influencing thinking on the case. Published in 2001, it laid out the many layers of the case and its context .
Former National Party leader Don Brash, an ardent supporter of Ellis, said he had not met anyone who had read Dr Hood's book who thought Ellis was guilty. (Dr Brash, with former Dunedin MP Katherine Rich, launched a petition in 2003 campaigning for a commission of inquiry into the case. Although it attracted high-profile support, it was unsuccessful.)
It remains to be seen if the appeal will be heard by the Supreme Court in November. It is the court's decision on whether it should go ahead now Ellis has died.
Let's hope it does. Given the twists and turns in this unfortunate story, not proceeding would be the ultimate sadness.
