Remote play is a bit like video recording... both are plugins that needs to make video and audio encoding/compression and this takes memory and computing power
Some games are going to work, the bad ones, and the most optimized, that doesnt uses much resources, so the PS3 has room to run additional process
But usually triple AAA games are not going to work at all, in some cases could even crash if you force it to do this kind of tasks "unnofficially" with custom code
This is the real problem the rec_plugin released by
@mysis had, but is not a problem of the plugin the code is correct, is the PS3 that says "no way i cant handle this" and says good bye
So always test with a low requirement game (or a game well optimized, but is hard to know wich ones falls in this category)
And use a video/audio format that doesnt requires much memory and processing power, neither compression (technically is better to encode in the format most closer to "raw" posible)