Unable to delete Azure Virtual Network Gateway using Resource Manager

I created an Azure Virtual Network using PowerShell:

        $gwpip=New-AzureRmPublicIpAddress-NameTestVNet-ResourceGroupNameTestRg-Location'West Europe'-AllocationMethodDynamic-DomainNameLabelTestVNet
        $vnet =Get-AzureRmVirtualNetwork-NameTestVNet-ResourceGroupNameTestRg
        $subnet =Get-AzureRmVirtualNetworkSubnetConfig-Name'GatewaySubnet'-VirtualNetwork $vnet
        $gwipconfig =New-AzureRmVirtualNetworkGatewayIpConfig-NameTestVNet-SubnetId $subnet.Id-PublicIpAddressId $gwpip.IdNew-AzureRmVirtualNetworkGateway-NameTestVNet-ResourceGroupNameTestRg-Location'West Europe'-IpConfigurations $gwipconfig -GatewayTypeVpn-VpnTypeRouteBased

This worked perfectly. I now want to delete this Virtual Network Gateway, but that doesn't seem to work. When I use the delete button in the Portal I get a message: "Successfully saved configuration changes to virtual network TestVNet"

When I use PowerShell it doesn't return anything, unless I use the -debug and -verbose switch.

Remove-AzureRmVirtualNetworkGateway-NameTestVNet-ResourceGroupNameTestRg-Force-Verbose-Debug

The final HTTP response says:

Body:{"status":"Failed","error":{"code":"InternalServerError","message":"An error occured.","details":[]}

When I check the audit logs in the Portal I see two Microsoft.Network/virtualNetworkGateways/delete events, both Informational. The first has status Started, the second Accepted. After that, nothing happens anymore. Using the Portal or PowerShell doesn't make a difference in the audit log.

Any suggestions on how I can remove the gateway?

Thanks in advance.

Marcel