macosx: set a sane minimum size for the advanced prefs window, so the options remain useable (close #5994)
(cherry picked from commit 219dd45d31f36be3f93c9331c2ffac4114b42566)