A police spokesman had said officers killed a Palestinian they suspected was carrying a weapon in Jerusalem's Old City on Saturday (local time), but according to Israeli media, the man was later found to have been unarmed.
Palestinian officials said the 32-year-old man, Iyad Khayri, suffered mental health issues and decried his killing.
In comments at Israel's Sunday cabinet meeting, Defense Minister Benny Gantz said: "We are really sorry about the incident in which Iyad Khayri was shot to death and we share in the family's sorrow - but I am certain this matter will be investigated swiftly and conclusions will be drawn."
He said Israel will make "every effort" to limit casualties while continuing to "maintain the proper level of security."
Israeli Prime Minister Benjamin Netanyahu did not mention the incident in his remarks.
Tension has risen in recent weeks with Israel saying it hopes to move ahead with a plan to extend sovereignty to Jewish settlements and the Jordan Valley in the occupied West Bank - land the Palestinians seek for a state.
The Palestinians, Arab states, the United Nations and European states have warned against the move and the Palestinians have declared an end to security cooperation with Israel and its ally, the United States, in protest.
