As the other guy said run power shell as admin, also yes you can shift delete it and you will be safe from it getting detected as a shift deleted exe if you do it correct. Cause if you don't delete it you can find it in executed programs list.
You don't even need to use a icon its not needed, you replace it as a legit .exe in a directory like steam and its fine you cant find it in LastActivity view etc.
All you need to do is to restart your pc so they cant see what you cleared. But yeah if you used recycle bin a while ago it doesn't matter - and just shift delete from now on.