Ivanovic, the 2008 French Open champion and a semi-finalist at Roland Garros this year, fell victim to an unlucky draw as the 50th-ranked Cibulkova of Slovakia was ranked in the top 10 last year before slipping after an Achilles-related surgery.
"It was a very tough matchup, I thought, especially for the first round," said the 27-year-old Serb.
"It's definitely very disappointing, because I felt like I did a lot of hard work over last few months. Over the last week, I had great preparation, lots of good training and everything."
Cibulkova seized the advantage with a service break in the fourth game of the third set for a 3-1 lead and consolidated it with a backhand drop shot to make it 4-1.
A backhand long from Ivanovic, who had her blistered left foot taped midway through the final set, ended the nearly two-hour contest.
"Today I know that my game was there on the court," said 26-year-old Cibulkova, who was coming off a quarter-finals berth last week in New Haven.
"I just had to stay tough and know I had to do it, she doesn't give away shots."
Cibulkova, a 2014 Australian Open finalist, will next face American qualifier Jessica Pegula, a 7-5 6-3 winner over Belgian Alison Van Uytvanck.
For Ivanovic, the result continued an up-and-down season.
She reached the semi-finals at Roland Garros this year and was coming off back-to-back quarter-finals at US Open tune-up events in Toronto and Cincinnati.
She also has lost three times to players outside the top-100 this year, including twice at grand slams.
"I really felt like I had been improving. Last week I saw a huge improvement in my game overall. My serve was really good today," said Ivanovic.
"It was very tough first match I think for both of us. First match of the tournament, as well."
