Azure load balancer rule not working

I'm new to Azure Load balancing.. so I'm trying to add 3 VM's to a load balancer and configured this as follows:

1. Added 3 VM's with an availability set.
2. Created Load Balancer and used the Availability set.
3. Added inbound firewall rule for the TCP port on all 3 VM's
4. Configured inbound NAT rules so that the client can hit for a specific VM at a particular port internally -This worked fine!

Now I need to test a scenario when I need clients to send data to a specific TCP port which should be distributed to the 3 VM's and I did the following steps:

1. Removed the previously created NAT Rules.
2. Added the 3 VM's under a single back-end pool.
3. Added Probes to non http TCP port.
4. Created Load Balancer rule where I have defined the incoming port and the outgoing TCP port which is different.

This does not seem to be working, any idea why ?