Lisa Dahlkvist converted Sweden's final penalty after Alex Morgan and Christen Press missed for the holders in the shootout. Linda Sembrant also missed for Sweden.
The United States, who dominated large parts of the match, have won four of the last five Olympic golds and had never not reached the Games final.
They will be joined in the semi-finals by Germany, who beat China 1-0, and Canada, who overcame France by the same margin.
Melanie Behringer got the winner for Germany after 76 minutes and Wang Shuang missed a penalty for China with eight minutes to go.
Germany will face Canada in Belo Horizonte on Tuesday, while Sweden will face the winner of Friday's other quarter-final between hosts Brazil and Australia.
Canada beat France thanks to a 56th minute goal from Sophie Schmidt.
Sweden scored against the United States in 61 minutes when Stina Blackstenius ran on to a defence-splitting pass and calmly slotted the ball past U.S. keeper Hope Solo.
The US, however, kept plugging away and scored a deserved equaliser after 77 minutes as the ball ricocheted off the face of Jessica Samuelsson and fell perfectly for Morgan to fire home from eight metres.
