Islamists, some allied with al Qaeda, seized control of vast parts of northern Mali last year, raising concerns about human rights abuses and fears that the West African country could become a base for militant attacks in the region and beyond.
"Since the beginning of the armed conflict in January 2012, the people of northern Mali have been living in profound turmoil," ICC Prosecutor Fatou Bensouda said in a statement.
"At each stage during the conflict, different armed groups have caused havoc and human suffering through a range of alleged acts of extreme violence. I have determined that some of these deeds of brutality and destruction may constitute war crimes."
Bensouda said she would investigate crimes including murder, mutilation, torture and rape, as well as executions conducted without a judgment from an established court.
Human rights groups have long demanded an ICC investigation.
"The ICC's decision to open an investigation in Mali sends an important message to all forces including separatist rebels, Islamist fighters, government soldiers, and foreign troops that serious human rights abuses will not go unpunished," said Corinne Dufka of Human Rights Watch.
She said the ICC team would need strong backing from its member countries, given the difficult environment. ICC judges have in the past said the difficulties of operating in remote, dangerous regions have limited the scope of investigations.
Bensouda said the investigation would also cover "intentionally directing attacks against protected objects", a reference to the destruction by Islamists of traditional Sufi shrines in the ancient caravan town of Timbuktu - acts reminiscent of the Taliban in Afghanistan.
The investigation will focus on crimes committed since January 2012 in the three northern regions of Mali, Bensouda said.
"There is still turmoil in North Mali and populations there continue to be at risk of yet more violence and suffering," she said.