The offbeat tale of political resistance traded wins with the vampire story Sinners, setting up a fight to the end at the Dolby Theatre in Los Angeles on Sunday (local time).
"Let's have a martini! This is pretty amazing," director Paul Thomas Anderson said on stage after his One Battle was announced the recipient of the top award.
The Warner Bros movie stars Leonardo DiCaprio as a one-time revolutionary who grows up to become the weed-smoking single father of a teenager.
Before this year Anderson had 11 career Oscar nominations and no wins. In addition to best picture, Anderson won best director and best adapted screenplay.
One Battle star Sean Penn was named best supporting actor for his role as an obsessed military officer. It was the third Oscar for Penn, who frequently skips movie industry awards shows and was not in the Dolby Theatre audience.
"Sean Penn couldn't be here, or didn't want to, so I'll accept the award on his behalf," said presenter Kieran Culkin, last year's supporting actor winner.
"I stand here because of the people that came before me," Jordan said before naming previous black Oscar winners including Sidney Poitier, Denzel Washington and Halle Berry. "I'm going to keep stepping up and I'm going to keep being the best version of myself."
Irish actor Jessie Buckley landed the best actress accolade for playing William Shakespeare's wife, Agnes Hathaway, in Hamnet. The movie explores how the couple navigates the death of their 11-year-old son "Hamnet."
"I would like to dedicate this to the beautiful chaos of a mother's heart," Buckley said. "We all come from a lineage of women who continue to create against all odds."
In her remarks, the 75-year-old thanked director Zach Cregger.
"He just wrote a dream part and he just let me grab it by the throat," Madigan said.
KPop Demon Hunters, a Netflix movie that became a global phenomenon, was named best animated feature. Its catchy song, Golden, won the award for best original song.
New Zealand designer Kate Hawley won the Best Costume Oscar for her work on Guillermo Del Toro's Frankenstein.
Meanwhile, New Zealand's Wētā FX crew, who worked on James Cameron's film Avatar: Fire and Ash, won the Oscar for best visual effects.
More than 1200 Wētā FX artists contributed to the project, delivering more than 90% of the film's visual effects. Only about 11 seconds did not contain special effects.
Redford and Reiner remembered
Amid the celebration, the Academy Awards took on a serious tone to honour two major losses in the film world - Robert Redford and Rob Reiner - who both acted and became directors.
Billy Crystal, star of When Harry Met Sally, said Reiner's films including A Few Good Men and This Is Spinal Tap would "last for lifetimes." He was joined on stage by Demi Moore, Meg Ryan and other cast members from Reiner classics.
Barbra Streisand, who played opposite Redford in The Way We Were, called Redford a "brilliant, subtle actor" and an "intellectual cowboy." She finished her remarks by singing a few lines from the movie's well-known title song.
Host Conan O'Brien opened the festivities by joking that he was honoured to be "the last human host" of the awards at a time when Hollywood is worried about artificial intelligence taking over jobs.
The glitzy celebration, Hollywood's most over-the-top gala of the year, took place as the US wages war on Iran.
Security was tight in and around the ceremony. Organisers said they were working closely with the FBI and Los Angeles police after a federal warning of a possible Iranian threat against California, though authorities have cited no specific or credible danger to the Academy Awards.
Attendees had to cross through several traffic checkpoints and go through metal detectors to make their way into the event.
The festivities masked the unease in the film business over where movies are being made as studios chase tax incentives and lower costs elsewhere in the United States and overseas, weakening Hollywood’s grip on production.
Warner Bros. the studio behind One Battle and Sinners, is in the process of being sold to Paramount Skydance in a deal that will narrow the ranks of major film distributors.
A media watchdog group, Free Press, circulated a roving billboard around Hollywood over the weekend airing its opposition to the merger.
Winners of the gold Oscar statuettes are chosen by the roughly 10,000 actors, producers, directors and film craftspeople who make up the Academy of Motion Picture Arts and Sciences.
The Academy took steps this year to try to ensure voters have actually watched the movies they are voting on. The online balloting system for the first time tracks whether a voter has streamed each movie. Voters, however, can check a box to say they watched the movie elsewhere outside the Academy website.
