Belgian Dieter Van Tornhout headed the winner on his 27th birthday six minutes from time following a superb whipped cross from Lee Johnson, his first goal for the club.
Celtic, 14-times League Cup winners and seeking a domestic treble, underperformed against a team six places and 41 points behind them in the Scottish Premier League.
Kilmarnock goalkeeper Cammy Bell saved well from Anthony Stokes and Victor Wanyama while Stokes had a strong penalty claim turned down in the dying minutes but Kilmarnock hung to win their first trophy since the 1997 Scottish Cup.