Carter has been replaced by Colin Slade in the starting XV for the match at Waikato Stadium, with halfback Jimmy Cowan coming on to the bench, and Piri Weepu now to provide first-five cover.
Carter suffered a stiff back in the All Blacks World Cup-opening match against Tonga.
All Blacks Doctor Deb Robinson said: "The key thing is that Daniel had been recovering well from a stiff lower back. However, he pulled up sore at an indoor training session this morning so we have made the decision not to risk him for the Japan match and we will aim to have him back fully training next week in preparation for the France Test.''
Carter was also seen rubbing his left wrist in the Tonga match at Eden Park.
He said after the match: "I have a bit of a dodgy wrist. It's why it's strapped. It's been affecting me for a while now. I got a whack to that but it's nothing too serious.
"My back tightened up about 20 minutes into the game so I was a bit stiff running around for most of the game but it didn't stop me being out there and it's something I need to manage over the next couple of days.''
All Blacks team to play Japan in their Rugby World Cup match in Hamilton on Friday (kickoff 8pm).
All Blacks (with number of caps):
Mils Muliaina (98), Cory Jane (27), Conrad Smith (49), Ma'a Nonu (61), Richard Kahui (13), Colin Slade (6), Andy Ellis (21), Adam Thomson (22), Richie McCaw (c _ 99), Jerome Kaino (42), Sam Whitelock (19), Brad Thorn (53), Owen Franks (25), Keven Mealamu (86), Tony Woodcock (77). Reserves: Andrew Hore (56), John Afoa (34), Ali Williams (67), Victor Vito (9), Jimmy Cowan (48), Piri Weepu (50), Sonny Bill Williams (8).