Mr Baker's sister, Sarah Baker, received the "devastating" news on Friday.
The NZ Transport Agency said it was working with the family to "find a resolution to this unfortunate situation".
Mr Baker died after his motorcycle was hit by a vehicle driven by a tourist on State Highway 1 near the Shag Point turnoff between Palmerston and Moeraki on August 13 last year.
The family subsequently sought permission to erect a memorial bench and plaque, which contained Mr Baker's ashes, at a site near where the crash happened. Miss Baker said it was away from the road in bushes and could not be seen from the road.
Miss Baker said she received an email advising her of the situation from Downer, the contractor responsible for maintaining the site on behalf of NZTA.
"I was shocked," she said.
Miss Baker said she had sought permission for the project from NZTA. She was referred to Downer and a representative agreed to the plans.
"I just don't understand why they would do this, she told the Otago Daily Times on Monday.
"If I had gone and done it without permission I could understand, but the fact that I did it all correctly - I just don't understand."
The plaque had remained in place for nine months since being unveiled on January 13 to mark Mr Baker's birthday.
The email from Downer said the memorial was reviewed following complaints from the Waitaki District Council and members of the public.
After reviewing the project with NZTA, it was decided the plaque would have to be removed as NZTA did not want to set a precedent which it would "find hard to control in the future".
The email said as the Downer representative accepted some of the responsibility, a small bronze plaque would be installed in its place.
"I want to come to some sort of resolution to keep it there," Miss Baker said.
NZTA transport system management senior manager Steve Mutton said it did not give permission for the memorial as it now stands.
"The Transport Agency gave permission for the family to install a seat to look out to the ocean with a plaque on the seat as a memorial to Riley," he said.
"Transport Agency contractors were present when this seat was installed. The Transport Agency did not give permission for the subsequent installation of a grave site/headstone containing ashes behind and under the seat on public road reserve land.
"However, we acknowledge that this was not clearly communicated to Riley’s family at the time of installation.
"We sincerely regret the distress that this has now caused Riley’s family, and we are committed to working with them to find a resolution to this unfortunate situation.
"We understand the desire of families to commemorate the passing of those killed in road crashes by erecting roadside memorials, with the hope of preventing similar tragedies happening to other families, as long as these are deemed to be safe and not considered a driver distraction.
"Complaints regarding this headstone were referred to the Transport Agency via the Waitaki District Council but the complaints did not originate from the council or its staff."
Miss Baker said she kept Downer fully informed of her plans and the decision to remove what was agreed to, nine months on, was devastating.
