Because there are humans out there that say leave chimps alone. Thus I am inferring that there would be aliens that say leave humans alone.

EDIT: Not to mention they would have lifeforms on their own planet to mess with. What says they wouldn't have their own "chimps"?