More than 100 fighters carried out the assault in Wazei Zadran in a mountainous region of Paktia province near neighbouring Pakistan's lawless tribal areas, said Galaxy Sky construction company owner Noorullah Bidar.
He described the assailants as Taliban fighters, though there was no immediate claim of responsibility for the attack.
The region is thought to be controlled largely by the Haqqani network, a fiercely independent branch of the Afghan Taliban and a major enemy of US and NATO forces. The al-Qaeda-affiliated network, led by ailing Jalaluddin Haqqani and his son Sirajuddin, is based in neighbouring Pakistan's tribal belt.
US officials say Pakistan has not done enough to go after insurgent safe havens in the restive area, especially in North Waziristan where the Haqqanis are thought to be based. The issue is a sore point between the two countries and the area has been the target of repeated US drone strikes.
About 85 people were at the construction site at the time of the attack, including 65 guards recruited from surrounding villages, Bidar said. He said the firefight lasted about two hours and the insurgents shot down at the camp from a nearby hill. The fighting also destroyed a number of construction vehicles.
"This was a brutal attack," Bidar said.
Rohullah Samon, a spokesman for Gov. Juma Khan Hamard, said 35 workers were killed and 20 were wounded. He said eight insurgents also died in the attack, which ended around midnight when the fighters retreated after stealing some trucks.
"It is a very remote area and there is no police presence there and we are investigating," said provincial police chief Gen. Abdul Ghafar Safi.
Galaxy Sky had a contract to build a 27km stretch of road through the mountainous area that was being funded by a NATO provincial reconstruction team, said Abdullah Durani, head of the public works department of Paktia province.
Afghan President Hamid Karzai condemned the attack. "The terrorists once again make clear their hostility against the development and progress of Afghanistan," he said in a statement on Thursday.
The new top US commander in eastern Afghanistan, Maj. Gen. Daniel B. Allyn, said securing the nation's border with Pakistan remains a priority.
"We will work in close cooperation with our Pakistan military partners across the borders," Allyn told reporters after formally assuming command. "They continue to conduct complementary operations and we remain in daily dialogue ... to ensure that we can provide stability and security ... against the common enemies of both Pakistan and Afghanistan."
Insurgents declared the start of a spring offensive against NATO and the Afghan government last month. NATO has been expecting the Taliban to stage a series of spectacular and complex attacks, and the group has already carried out a number of them recently
On Wednesday, a suicide bomber crashed a car into a police bus in eastern Afghanistan, killing 14 people and wounding 16, the Interior Ministry said. Most of the casualties were police officers. The bus was travelling to a police academy in the city of Jalalabad.
There was no claim of responsibility for that attack, but it matched the pattern of Taliban attacks against government workers and security forces.
In southern Kandahar city, a roadside bomb killed a child and wounded eight others, including several children, the governor's office said on Thursday, adding that the target of the explosive was unclear.
In the northern city of Taloqan, demonstrators took to the streets for a second day on Thursday to protest a nighttime NATO raid that left four people dead. NATO later said the dead were insurgents, while Afghan officials said they were civilians.
Provincial Gov. Abdul Jabbar Taqwa said that about 100 demonstrators gathered outside his office to protest the deaths and set fire to two shipping containers being used to sell books.
Meanwhile, provincial police chief Gen. Shah Jahan Noori said about 400 people chanting slogans against NATO had gathered outside police headquarters on Thursday, setting small fires and hurling stones. He said 15 people were arrested.
Residents often charge that international forces go after the wrong people or mistreat civilians as they search compounds
The effectiveness of the Taliban's long-awaited spring campaign, code-named Badr after one of the Prophet Muhammad's decisive military victories, could affect the size of President Barack Obama's planned drawdown of US troops in July. Gen. David Petraeus, the top US and NATO commander in Afghanistan, has said the size of the withdrawal will depend on conditions on the ground.
The alliance has committed itself to handing over control of security in the country to Afghans by 2014.
