I've looked into the Multivalue control and I understand that the control has a list of hardcoded values in the suggested values fields:

<FIELD name="Triage" refname="Microsoft.VSTS.Common.Triage" type="String" reportable="dimension">
<SUGGESTEDVALUES expanditems="false">
<LISTITEM value="Approved" />
<LISTITEM value="Investigate" />
<LISTITEM value="Rejected" />
<LISTITEM value="Submit" />
</SUGGESTEDVALUES>
</FIELD>

And while I could hard code in a list of values, what I'd like to do is use the multivalue control to be able to assign multiple team members to the work item.

In this case I've looked at the assigned to field, and it uses the <ValidUserRule /> for looking up users.

How might I setup the multivalue control to query the <ValidUserRule /> and list each user in the fields?

There are some issues with using such control for assigned to. The way the control works, is that it enclose each value with square brackets []. so you have to supply suggested values but not restrict the field values to those suggestion. I don't belive the AssignedTo Field can work with the control. this will make querying for that field very tricky. Not to mention that it will break the "My Work Items" query, and the assigned to @me clause.