Set with non-empty reference attribute

Hi,

Is it possible to have a set of users where a reference attribute is not empty?

Basically, I have created a new reference attribute and attached it to the Person object, but I would like to make a set so that only a user who has that reference attribute set to something becomes a member of it. I know this is relatively simple for string type attributes, but just wondering if it can be done for reference attributes?

e.g. A set of all users who have a manager assigned?

Thanks