Sweden, whose previous best showing at an Olympics was a fourth place finish in 2004, will now face the winner of the other semifinal between Germany and Canada.
Lisa Dahlkvist scored the decisive penalty on Sweden's fifth attempt after Andressa, the youngest player on the Brazil squad, saw her penalty saved by Hedvig Lindahl.
Brazil's Cristiane and Sweden's Kosovare Asllani also had their kicks saved.
The result was a huge letdown for the host nation, who were vastly superior and had the raucous support of a capacity crowd at the Maracana stadium.
Brazil had most of the play throughout as Sweden tried to the same smash and grab tactics that took them past the United States in the quarterfinals. Brazil had 33 shots at goal, compared to Sweden's six.
