Larry "Major Nelson" Hyrb from Microsoft's Xbox division tweeted today (April 7) that from now on, Xbox Series X|S users can use the console controller to directly switch the TV's input signal source back to the console.

Hyrb says the feature is part of consumer electronic device control (CEC). Hosts that originally had this feature can send commands to other devices through the handle. CEC has been around on Xbox consoles for a while, but starting today, users can officially use the feature. The same functionality may have been around on the PS5 for some time.
As long as the TV connected to the Xbox Series console supports CEC, players can use their Xbox controller to switch inputs. All they have to do is find the TV and A/V power options menu in the console's settings. From there, users can open HDMI-CEC and choose which commands their console can issue to their TV.
In addition to being able to switch TV inputs using their handles, users can also have their console change the volume of the device or turn other devices on or off.