If you already have an old mini PC, SBC, or cheap laptop at home, then it’s the best option and costs nothing! Set it up with Debian, install some emulators, and you’re good to go.
If you don’t have any old hardware to make use of, then you can pop over to the used market to find good deals on those sorts of things for a lot cheaper than a brand new device!
As for controller recommendations, there are lots of really good cheap ones that use hall effect sticks. The 8BitDo Ultimate 2C are very good + come in lots of fun colours, and the GameSir Nova 2 Lite is also good (both are wireless). The GameSir Tegenaria Lite is a stellar deal if you want wired controllers! I have the Ultimate 2 (non-C) and the Gamesir Super Nova, and both of them are great! You might also want to look at the Gulikit controllers, but I don’t think they have many options in the price bracket of the Ultimate 2C and Nova 2 Lite



It has worked very well for me, and there hasn’t been any weird shenanigans as of right now. It was a bit messy but it’s all good! If you are very paranoid, I think you can use Termux with the Linux version of Syncthing (or hey, go all the way and experiment with a Linux phone if you’re daring!)