Both sides had won their opening two games to set the early pace and it looked like they had opted to keep their powder dry for the battles ahead until right back Szoboszlai stepped up in the 83rd minute to seal it.
After Arsenal's Martin Zubimendi brought down the speeding Curtis Jones around 30m from goal, it looked a long way out but Hungarian Szoboszlai struck his right-foot effort with power and precision to leave David Raya clawing at thin air.
"I had to take a risk and shoot a little bit harder because I knew Raya likes to jump behind the ball. He's an unbelievable goalkeeper. If it goes a bit more inside he saves it," said Szoboszlai, who was outstanding all afternoon.
It was a rare moment of quality in a game that promised a great deal but delivered little in terms of excitement.
But champions Liverpool, who looked rock-solid again after having conceded four goals in their opening two games, will not care about that as they made a big statement of intent in their quest to win back-to-back titles.
Arsenal have not won a league game at Liverpool since 2012 and that sequence now extends to 16 matches.
"It was going to be decided in two possible actions, an individual error or a magic moment and it was decided in a magic moment," Arsenal manager Mikel Arteta said.
Arteta's Arsenal, having spent £260 million in the summer window in a bid to dethrone Arne Slot's Liverpool and claim their first title since 2004, might have been expected to show more adventure against the hosts.
They largely controlled the game and prevented Liverpool from registering a touch in the visitors' area for 33 minutes and did not concede a corner until the 70th.
But apart from the pace of Noni Madueke they looked tame and were punished for their conservatism as the hosts floored them with a late sucker punch.
"In the end you need to find a way to win these big matches," said Spaniard Arteta, whose side have finished runners-up for three successive seasons.
Fans of tactical chess matches would have found some intrigue in the opening half at a sunny Anfield but the cautious approach of both sides hardly made for a great spectacle.
Arsenal were not helped by the loss of central defender William Saliba after five minutes to an ankle injury, the Frenchman being replaced by new signing Cristhian Mosquera.
Madueke's pace troubled Liverpool fullback Milos Kerkez and it was the former Chelsea player who finally forced a goalkeeper to make a save with a first-time shot from a half-cleared corner that Alisson kept out with a regulation stop.
Liverpool did have the ball in the net on the hour after Hugo Ekitike bundled home when Raya failed to hold a shot by the otherwise quiet Florian Wirtz but the offside flag was up.
With 20 minutes remaining Arsenal's Eberechi Eze came off the bench for his first appearance since his big-money move from Crystal Palace earlier this month, but this was not a day for their attacking players, including striker Viktor Gyokeres, to remember with any degree of affection.
Liverpool did exert more control as the minutes ticked down but a point apiece looked a foregone conclusion until Szoboszlai sent the Kop into raptures.
Early in the season it may be, but Liverpool will head into the international break right where they want to stand at the top of the table, two points clear of second-placed Chelsea and three ahead of a chasing pack led by Arsenal.
"If you win against a team like Arsenal, who last season were such a good team and have strengthened so much, to win against them is very positive," Liverpool boss Arne Slot said.
"We will know at the end of the season what it's worth."
City fall late
Brighton & Hove Albion's Brajan Gruda snatched the winner a minute from time as the hosts beat Manchester City 2-1 in the Premier League, after James Milner had cancelled out Erling Haaland’s opener, in a stunning turnaround.
Haaland put City in front with a well-worked goal after 34 minutes as he wriggled between two defenders to score from close range, his third goal in as many games this season.
Brighton levelled after the ball struck Matheus Nunes' arm as he tried to protect his face from a cross and Milner drilled the spot kick home in the 67th to become the oldest scorer of a penalty in the Premier League at 39 years 239 days.
The momentum had swung Brighton's way by then and it was the home side who claimed the points when Gruda beat the offside trap and rounded James Trafford in the City goal to score and condemn City to back-to-back league defeats.
Palace thrash Villa
Aston Villa's dismal start to the season continued as they crashed 3-0 to Crystal Palace in the Premier League, their first home defeat in any competition for 10 months.
Palace thoroughly deserved the three points, dominating after Jean-Philippe Mateta's penalty gave them an early lead.
England defender Marc Guehi, linked with a move to Liverpool, made it 2-0 with a superb finish after 68 minutes before Ismaila Sarr wrapped up the win for the visitors.
Palace's first victory of the season after two draws lifted them to eighth, while second-bottom Villa have one point from their opening three games and are yet to score.
"Every game he brings something new, and he is a fantastic player. I hope he stays with us," Mateta said of Guehi.
Villa had been unbeaten in their last 19 home league matches, a streak lasting more than a year.
Palace were the last team to beat them at Villa Park, a 2-1 victory in the League Cup last October.
Meanwhile West Ham got their season back on track with a 3-0 win over Chris Wood's Nottingham Forest.
