"The diplomat was near the embassy building when men came by in a car and took him by force. He is responsible for security in the embassy," a security source said, adding that it was not known where the men had taken the diplomat.
Earlier, tribesmen briefly held interior ministry employees hostage. They freed the ministry personnel a few hours later but continued to occupy the building, a ministry official said.
The incidents highlighted the continuing turmoil in Yemen despite a peace deal under which Saleh stood down after months of protests against his 33-year rule and was replaced in February by his deputy, Abd-Rabbu Mansour Hadi.
The ministry storming was a direct challenge to Hadi's authority. He is trying to restructure the armed forces and stabilise the impoverished Arab nation, where Saleh's legacy still looms large.
The Interior Ministry official said the tribesmen were Saleh loyalists, who were promised they would be enrolled in the police force in return for helping tackle last year's uprising. The promise has not been fulfilled.
"At midday, the armed tribesmen... stormed the ministry building, took control of it and climbed onto the roof with their guns," the official said. "They refuse to leave until their demands are met."
Tribesmen have fought alongside government troops in a U.S.-backed offensive against al Qaeda-linked militants that drove insurgents out of several towns in the south of the country last month. Many tribal fighters also sided with Saleh who was toppled by a popular uprising.
Disgruntled tribesmen often kidnap foreigners and bomb oil and gas pipelines as a way to press demands on authorities.
In April, officers and tribesmen loyal to Saleh forced Yemen's main airport to close for a day in protest at the sacking of the air force commander, a half-brother of Saleh.
