You're right. The solution is the Joomla categories field plugin (select the tree or the categories, apply or not Joomla ACL..). It's not public. Will be a commercial plugin. Please contact us with the form http://www.seblod.com/contact
Or you can restrict category access by Joomla ACL and add multible Category select lists to your form. The User A sees only the category select list A he is allowed to see. The User B has list B. The User C has more ACL rights and has two lists A and B.