Lina Hurtig struck the winning spot-kick at Melbourne Rectangular Stadium to cap a madcap shootout in the round of 16 clash, though US goalkeeper Alyssa Naeher parried the ball on the line and the result needed to be confirmed by VAR.
Sweden head into a quarter-final against Japan in Auckland on Friday but it was the first exit before the semi-finals in nine World Cups for the Americans, who were bidding for an unprecedented third consecutive title.
Gloomy US coach Vlatko Andonovski was not convinced Hurtig's spot-kick had crossed the line
"It’s a tough moment, a moment where it's hard to go through," he told a press conference.
"You have hopes that the ball didn’t cross the line.
"That just shows how cruel this game can be."
Sweden's Magda Eriksson said her team had shown heart if not their best game, having been dominated for much of the match.
"We're so happy right now," she said in a radio interview.
"I don't know how we managed to stay in this game.
"We didn't play our best game, but in some way we showed we're hard-headed, we showed heart, we never give up. We took it to extra time, we took it to penalties and we managed to win."
Young US forward Sophia Smith was left shattered, having had a chance to win the match for the four-times champions when she stepped up in the penalty shootout only to blast the ball over the bar.
She was not the first player in her team to miss.
NINE MONTHS
Andonovski said the US had been practising penalties for the last nine months.
"I know we were criticised for the way we played ... I think we came out today and were all about the grit, the resilience," he said.
Goalkeeper Zecira Musovic was Sweden's hero, having made 11 saves before the shootout to keep them in the match.
Often starved of possession, the Swedes did not have a shot on goal until the 85th minute when substitute Sofia Jakobsson danced into the area, only to shoot straight at Naeher.
Naeher saved Rebecka Blomqvist's fourth shot for the Swedes in the shootout but Smith's miss kept the Scandinavians in it.
After Naeher and Magdalena Eriksson converted their spot-kicks, Kelley Ohara sent her effort pinging off the right post, allowing Hurtig to step up and send the champions out of the tournament.
The US will wonder how the match had to go to a shootout, having dominated the play and unleashed 21 shots, including 11 on target.
Trinity Rodman proved a menace down the right wing early, testing the Swedish keeper twice in the first half-hour.
Lindsey Horan thumped a header against the bar and Musovic denied her again with a low dive when the US skipper fired a deflected cross at the far post.
Musovic produced another wonderful save to stop an Alex Morgan header and see the Swedes into extra time, then foiled a Lynn Williams shot in the first period.
Rapinoe came on to replace Morgan in extra time but could not conjure the magic that helped the US to back-to-back World Cup triumphs in Canada in 2015 and France four years ago.
