Hi All,
I am setting up a HDP cluster on Azure. In azure, all the machines of the cluster have the same PUBLIC VIRTUAL IP (VIP) ADDRESS and unique internal IP address.
On a vm of the cluster if I want to open some port I am adding end points for the virutal machine through Azure Management portal. I added port 8080 for a VM in the end point and configured it to be accessible. But for some services which are running on all the nodes of the cluster (for example data node in the case of Hadoop), I could not configure it as they are behind the same public Virtual IP. Learnt that we need to create Network Security groups. I used the azure blog for reference.
I created network security group using the following commands in the powershell to open port 50070.
Creating Security Group:
New-AzureNetworkSecurityGroup -Name "sectest1-sg" -Location "West Europe"
Adding a rule to the security group to open port 50070:
Get-AzureNetworkSecurityGroup -Name "sectest1-sg" | Set-AzureNetworkSecurityRule -Name NNUI -Type Inbound -Priority 100 -Action Allow -SourceAddressPrefix '*' -SourcePortRange '*' -DestinationAddressPrefix '*' -DestinationPortRange "50070" -Protocol TCP
Associating NSG with VM:
Get-AzureVM -ServiceName "clusterServiceName" -Name "VMOnWhichPortNeedsToBeOpened" | Set-AzureNetworkSecurityGroupConfig -NetworkSecurityGroupName "sectest1-sg" | Update-AzureVM
In the powershell I see the success messages like:
VERBOSE: 7:04:38 PM - Completed Operation: Get Deployment
VERBOSE: 7:04:40 PM - Completed Operation: Get Deployment
VERBOSE: 7:04:40 PM - Begin Operation: Update-AzureVM
OperationDescription OperationId OperationStatus
-------------------- ----------- ---------------
Update-AzureVM 8587690e-f8ea-0614-ad79-a44615f7cas2 Succeeded
VERBOSE: 7:05:43 PM - Completed Operation: Update-AzureVM
However when I try to access the port 50070 through web portal I see"This webpage is not available" error message.
I am not sure if I missed here something or something additional needs to be done.
I would be greatful if you could give some helpful suggestions/pointers to solve this issue.
Many thanks.
-A
I am setting up a HDP cluster on Azure. In azure, all the machines of the cluster have the same PUBLIC VIRTUAL IP (VIP) ADDRESS and unique internal IP address.
On a vm of the cluster if I want to open some port I am adding end points for the virutal machine through Azure Management portal. I added port 8080 for a VM in the end point and configured it to be accessible. But for some services which are running on all the nodes of the cluster (for example data node in the case of Hadoop), I could not configure it as they are behind the same public Virtual IP. Learnt that we need to create Network Security groups. I used the azure blog for reference.
I created network security group using the following commands in the powershell to open port 50070.
Creating Security Group:
New-AzureNetworkSecurityGroup -Name "sectest1-sg" -Location "West Europe"
Adding a rule to the security group to open port 50070:
Get-AzureNetworkSecurityGroup -Name "sectest1-sg" | Set-AzureNetworkSecurityRule -Name NNUI -Type Inbound -Priority 100 -Action Allow -SourceAddressPrefix '*' -SourcePortRange '*' -DestinationAddressPrefix '*' -DestinationPortRange "50070" -Protocol TCP
Associating NSG with VM:
Get-AzureVM -ServiceName "clusterServiceName" -Name "VMOnWhichPortNeedsToBeOpened" | Set-AzureNetworkSecurityGroupConfig -NetworkSecurityGroupName "sectest1-sg" | Update-AzureVM
In the powershell I see the success messages like:
VERBOSE: 7:04:38 PM - Completed Operation: Get Deployment
VERBOSE: 7:04:40 PM - Completed Operation: Get Deployment
VERBOSE: 7:04:40 PM - Begin Operation: Update-AzureVM
OperationDescription OperationId OperationStatus
-------------------- ----------- ---------------
Update-AzureVM 8587690e-f8ea-0614-ad79-a44615f7cas2 Succeeded
VERBOSE: 7:05:43 PM - Completed Operation: Update-AzureVM
However when I try to access the port 50070 through web portal I see"This webpage is not available" error message.
I am not sure if I missed here something or something additional needs to be done.
I would be greatful if you could give some helpful suggestions/pointers to solve this issue.
Many thanks.
-A