Azure - Load balancing/HA - two PIPs

I am trying to set up high availability for two servers in Azure. The problem is - both servers have to use instance-level public IP addresses as described here - http://msdn.microsoft.com/en-us/library/azure/dn690118.aspx. They are used to bypass the limitation on the number of endpoints per vm (we are using around 50000 of those as opposed to the allowed 150 without PIP).

The problem is that neither traffic manager (routes to endpoint) nor vm load-balancing (limits endpoints) seem to be able to achieve that.

Does anyone have experience with this scenario and/or can advise a method that does not rely on DNS failover?

Thanks!