It will be the first time North Korea has missed a summer Olympics since it boycotted the Seoul games in 1988, during the Cold War.
South Korean President Moon Jae-in had hoped the two countries, still technically at war after their 1950-53 conflict ended in a truce, not a peace treaty, could field a combined team in Tokyo and rebuild momentum for an improvement in relations.
The withdrawal is also a setback for Moon and North Korean leader Kim Jong Un's plan, agreed in 2018, to pursue a joint Korean bid to host the 2032 Games.
When South Korea hosted the Pyeongchang Winter Olympics in 2018, Kim sent his sister to head the national delegation, and both teams marched under a unified flag at the opening ceremony and fielded a combined women's ice hockey team.
Tensions on the Korean Peninsula rose last month when the North resumed missile tests, although both sides said they wanted to continue dialogue.
South Korea's Unification Ministry said Seoul had hoped the Tokyo Olympics would be a chance to "foster peace and reconciliation" and expressed regret that it could not happen.
US State Department spokesman Ned Price said the North Korean move "would appear consistent ... with the DPRK's stringent response to COVID-19," referring to North Korea by the acronym of its official name, the Democratic People's Republic of Korea.
The administration of US President Joe Biden, which took office in January, has sought to engage North Korea in dialogue but has been rebuffed.
The Tokyo Organising Committee said it would "continue to prepare the best possible stage to welcome athletes from all countries and regions".
The International Olympic Committee said it had asked the North Korean organising committee several times for a telephone conference to discuss the coronavirus situation in North Korea, but North Korea had been unable to comply.
"The IOC has not received any official application from the NOC of DPRK to be released from their obligation to take part in the Olympic Games according to the Olympic Charter," an IOC spokesperson said.
North Korea's sports ministry said on its website that the decision to pull out of Tokyo had been made at a meeting of its Olympic committee with Sports Minister Kim Il Guk on March 25.
"The committee decided not to join the 32nd Olympics Games to protect athletes from the global health crisis caused by the coronavirus," it said.
North Korea says it has not had any coronavirus cases.
The March 25 meeting also discussed ways to develop professional sports technologies, earn more medals at international competitions and expand public sports activities over the next five years, the ministry said.
Kim, known to be a fan of US basketball, has expressed a desire to promote professional sports in North Korea.
