John Kirwan's side dominated with its physical play in the second half, shutting down the US to secure the victory.
Playing before a partisan crowd at Chichibunomiya Stadium, Japan scored first on a try by Taihei Ueda six minutes into the contest, which began in a driving rain. Ueda took a pass from Alisi Tupuailai, who set up the score with a strong open-field run, near the goal line.
Kevin Swiryn responded for the US seconds later with a try to level the scores at 5-5. The US moved ahead 7-5 on the conversion by Nese Malifa but Japan retook the lead with a drop goal from Ryan Nicholas in the 12th minute.
Paul Emerick then scored a try on a long uncontested run to give back the advantage to the Americans just before halftime. Swiryn added the conversion to make it 14-8 at the break.
In the second half, Kikutani barreled through the American defense from just outside the goal line for a try in the 53rd minute to move the hosts within a point. Shaun Webb put Japan back on top 15-14 with the conversion.
Usuzuki extended Japan's winning margin to 20-14 with a try in the 66th minute.
Japan, which lost to Italy 31-24 in Cesena last week, made 12 changes for the match.
US coach Eddie O'Sullivan made eight changes to his side after losing 27-7 to Canada in Glendale, Colorado, last weekend.
Japan will face France in its opening Pool A World Cup match on September 10.
The US begins its campaign in Pool C against Ireland on September 11.