After restricting Sri Lanka to 233 for eight, Virat Kohli (77) and Sachin Tendulkar (48) put India in a strong position before a middle order collapse nearly derailed their innings.
It was left to India's last recognised batting pair of Ravindra Jadeja (24 not out) and Ravichandran Ashwin (30 not out) to steer them home with 20 balls to spare.
In a rematch of the 2011 World Cup final which India won in Mumbai, Sri Lanka struggled for momentum with the bat.
Dinesh Chandimal (64) and Tillakaratne Dilshan (48) contributed nearly half of the Sri Lankan total which hardly vindicated skipper Mahela Jayawardene's decision to bat first.
India lost Virender Sehwag (10) early but Kohli and Tendulkar added 75 runs to put the chase back on track.
However, the milestone every Indian fan has been waiting for once again did not materialise.
Chasing his 100th international century, Tendulkar looked comfortable against the Sri Lankan bowlers but two runs short of his 96th half-century he dragged an Angelo Mathews delivery onto his stumps.
Tendulkar remains stuck on 99th centuries since hitting his last against South Africa in Nagpur during the 50-over World Cup last March.
Tendulkar's dismissal triggered a mini-collapse and India slumped from 89 for one to 167 for five.
Rohit Sharma (10) could not do justice to his reputation, Suresh Raina (24) wasted the start he got and captain Mahendra Singh Dhoni (four) continued his poor form with the bat in Australia.
Kohli then ran himself out after going for a non-existent single and had to be helped off the field having injured himself.
Ashwin, adjudged man of the match, showed great determination in Jadeja's company and scored the winning run when his miscued shot landed in the mid-off region with three Sri Lankan fielders running for the catch before all pulling out at the last minute.
India lost their opening match when they lost to Australia by 65 runs in Melbourne on Sunday.
