Michu netted his 16th goal of the season to put Swansea ahead against the run of play in the 39th minute and Graham added a second in the 90th.
Chelsea signing Demba Ba came on for his home debut as a second-half substitute and had a penalty appeal turned down and a goal disallowed for offside late on.
The second leg is at Swansea on January 23.