vout_macosx: check whether the receiving window can respond to non-standard calls prior to calling them (close #6034)