Option for keep sending game controller input to VM even when not in focus

I have a Windows 11 VM on a Mac Book Pro with an external projector attached. My son is playing games on the projector screen using an Xbox Controller and it works fine, until I move focus onto the laptop screen to try and do some other work while he's playing.
I understand that the VM screen is losing focus when I activate another window, but it would be amazingly nice for the VM to keep receiving just the Xbox controller input while I can use the laptop trackpad and keyboard on the laptop screen. In the VM USB/Bluetooth configuration I have the Xbox controllers permanently mapped to the VM, but when loosing focus, the VM apparently doesn't receive the input anymore, or it doesn't want to keep processing it.
It would be amazing to be able to use the laptop and play games at the same time on different screens!
Thank you!