Holder met with his European counterparts in Paris on Sunday (local time) to discuss ways to prevent violent extremism, after Islamist militants killed 17 people in several attacks in the French capital.
Holder said on CBS's "Face the Nation" programme that the "decimation" of core al Qaeda has reduced or eliminated that group's ability to carry out a 9/11-style attack, but he said affiliates like al Qaeda in the Arabian Peninsula (AQAP) have stepped in to try to inspire people to carry out smaller attacks.
"I think the possibility of such attacks exists in the United States," Holder said.
"It is something that, frankly, keeps me up at night worrying about the lone wolf or a very small group of people who decide to get arms on their own and do what we saw in France this week," he said.
Holder said authorities have not determined if AQAP or a group like Islamic State, which has declared a caliphate and seized large areas of Iraq and Syria, is responsible for the Paris attacks.
The two brothers who carried out the attack in Paris on a satirical weekly newspaper traveled in 2011 to Yemen, where AQAP is based, for weapons training, two senior Yemeni sources said on Sunday. The gunman who staged an attack on a Paris deli appeared in a video declaring his allegiance to the Islamic State group.
Holder, who appeared on several US television shows on Sunday morning, said the United States and its allies need to do a better job sharing information about possible threats.
"One nation cannot by itself hope to forestall the possibility of terrorism even within its own borders," he told ABC's "This Week" programme.
President Barack Obama will hold a global security meeting in Washington in February to discuss domestic and international efforts to counteract violent extremism.
Holder said Americans should feel secure with U.S. officials' efforts to prevent attacks by Islamist militants.
He told CNN's "State of the Union" programme that about 150 Americans had gone or attempted to travel to Syria or Iraq to fight with militants there, and about 12 were there now.
