The assault on the US Consulate in Benghazi September 11 that killed Ambassador Christopher Stevens "was a terrorist attack," Panetta told a news conference at the Pentagon.
"A group of terrorists obviously conducted that attack on the consulate and against our individuals. What terrorists were involved, I think, still remains to be determined by the investigation," he added.
Panetta's remarks came a day after Secretary of State Hillary Clinton seemed to link the Benghazi attack to militants with ties to al Qaeda in the Islamic Maghreb.
Stevens died of smoke inhalation when he was trapped alone inside the burning consulate in Benghazi after it was attacked by militants. Another diplomat, Sean Smith, and two US security men were also killed.
Libyan Prime Minister-elect Mustafa Abushagur told a news conference in Tripoli on Thursday that the "investigation is under way, it is progressing," but added there was no "complete definite investigation to say who did this yet."
He said the Libyan and US governments were closely cooperating on the investigation.
There was initial confusion about whether the attack had been planned in advance or was opportunistic, taking advantage of mob violence over an anti-Islam film made in the United States.
Panetta and Army General Martin Dempsey, the chairman of the Joint Chiefs of Staff, told the Pentagon news conference it took time before officials determined that extremists were behind the attack.
"As we determined the details of what took place there, and how that attack took place, ... it became clear that there were terrorists who had planned that attack," Panetta said. "That's when I came to that conclusion."
Dempsey said he wasn't aware of any specific threat to the consulate before the attack.
He said intelligence reporting from eastern Libya indicated some militant groups were trying to work together, but "there wasn't anything specific and certainly not a specific threat to the consulate."
