Also set the content area to expand vertically.

Opened privacy dialog, changed setting around and saw that the list of specific contacts came up depending on the option chosen. The add/remove buttons also became enabled as expected.