Climate Change Minister Simon Watts will take Lee's spot in Cabinet.
Simmonds was a minister outside of Cabinet.
Prime Minister Christopher Luxon announced the changes in a statement today.
He said Justice Minister Paul Goldsmith - currently overseas - would take over the Media and Broadcasting role, while Social Development Minister Louise Upston would pick up Disability Issues.
Lee was under pressure after Warner Brothers Discovery announced it would stop producing local news through Newshub, and shutter the majority of its operations in New Zealand.
Simmonds had also been in headlines over the handling of changes to disability-related funding.
Luxon said it had "become clear in recent months that there are significant challenges in the media sector. Similarly, we have discovered major financial issues with programmes run by the Ministry of Disabled People".
"I have come to the view it is important to have senior Cabinet Ministers considering these issues."
“There are significant synergies between the Arts, Culture and Heritage portfolio and Media and Communications – as such, I’ve asked Paul Goldsmith to take the lead on media issues for the government.
“The challenges facing our media are being seen all over the world. While there are limited levers for government here, I look forward to Paul progressing work to ensure regulatory settings are appropriate to enable the media to modernise and adapt to a changing media environment.
“The Ministry of Disabled People is a departmental agency within the Ministry of Social Development, so I have asked Louise Upston to take responsibility for the Disabilities Issues portfolio. This will free Penny Simmonds up to focus on the Environment portfolio and the major changes she is progressing to improve tertiary education.
The Prime Minister has advised the Governor-General to give effect to the changes today.
Luxon said that as issues changed in prominence, he planned to switch ministerial responsibilities to make sure the government was best placed to deliver for New Zealanders.
