Sometimes hostname not updated by waagent about walinuxagent

[TuxInvader]

More digging has revealed that this happens to me when the hostname of the instance is the same as the hostname used in building the image. It's because the the waagent process doesn't reset the set-hostname file when deprovisioning. This results in setting the hostname being skipped in cc_set_hostname.py during cloud-init:

https://github.com/canonical/cloud-init/blob/695c4f8f46585dd7feac2fdc0729f410c539d3bc/cloudinit/config/cc_set_hostname.py#L80

I'm going to open a bug to request that waagent removes or updates /var/lib/cloud/data/set-hostname on deprovision. As a workaround I manually rm the file as the last step in my packer script.

 {
    "execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh '{{ .Path }}'",
    "inline": [
      "/usr/sbin/waagent -force -deprovision+user && export HISTSIZE=0",
      "rm /var/lib/cloud/data/set-hostname || echo 'no set-hostname file'",
      "sync; sync; sync"
    ],
    "inline_shebang": "/bin/sh -x",
    "type": "shell"
  }

[yuezh]

What is the cloud-init version the image is using? There was a bug with cloud-init that it will remove the file "SharedConfig.xml" which is essential for waagent.

[mhmxs]

We are using cloud-init.x86_64 0:0.7.5-10.el7.centos.1. Sorry maybe i was misunderstood, i pasted the full log of the waagent. Our problem is not around the SharedConfig.xml, but in the Ovf XML processing. On the rigth node i see:

2015/10/28 13:01:16 Provisioning image using OVF settings in the DVD.
...
2015/10/28 13:01:20 EnvMonitor: Detected host name change: localhost.localdomain -> vmslave12
2015/10/28 13:01:20 Setting host name: vmslave12
2015/10/28 13:01:21 Ovf XML process finished

but on the failed:

2015/10/28 13:00:53 Provisioning image using OVF settings in the DVD.
2015/10/28 13:00:53 Wrote /var/lib/waagent/CustomData
2015/10/28 13:00:53 Disabled SSH password-based authentication methods.
2015/10/28 13:00:55 Created user account: asd
2015/10/28 13:00:55 Deploy public key:750A...
2015/10/28 13:00:55 Ovf XML process finished

So there are missing Detected host name changeand Setting host name lines.

Update: Is it true, that waagent is just reacting to the hostname change which caused by cloud-init and publishing new hostname to Azure services?

[yuezh]

Sorry. I may mislead you in the preview post. Your problem is that Network Manager is conflict with waagent. I assume you are using a customized image. You need to uninstall the Network Manager first.

[mhmxs]

Thank you, we would try it, but really hard to catch the problem, because for example yesterday i created almost 40 instances without any problem.

[mhmxs]

We are running on Centos7, and i found the text below in the official documentation:

Preparing a CentOS 7 virtual machine for Azure is very similar to CentOS 6, however there are several important differences worth noting:

The NetworkManager package no longer conflicts with the Azure Linux agent. This package is installed by default and we recommend that it is not removed.

[szarkos]

Right, NetworkManager is required in CentOS 7. The issue here is probably with cloud-init. Canonical wrote the cloud-init data provider for Azure, but it's really only been tested on Ubuntu. If you just install cloud-init it will work partially and probably try to provision the VM at the same time as the Azure Linux agent, which will probably be very racy and lead to odd issues like this. So in this case I would recommend removing cloud-init.

[mhmxs]

Thank you.

[TuxInvader]

I think this is still an issue, at least for custom images. I'm using an Ubuntu 18.04 image created and provisioned by packer. After installing my software waagent is called with -deprovision+user.

The first machine deployed from this image in a new VNET will register with DHCP using localhost. The default config for provision.MonitorHostName in ubuntu is no so even if the hostname is changed in cloud-init or via some other mechanism the DHCP wont get updated (not unless the lease is renewed).

Subsequent hosts will get the hostname they were given through the ARM template, presumably because DHCP rejects the attempt to register localhost, and gives them the name configured for them in the Azure.

I'm guessing the issue is caused by the set_dhcp_hostname function being called during the packer build

WALinuxAgent/azurelinuxagent/common/osutil/default.py

Line 1135 in 8087dbf

def set_dhcp_hostname(self, hostname):

There should be an unset_dhcp_hostname() called during -deprovision

[TuxInvader]

Actually send host-name gethostname(); is in the original dhclient.conf, so waagent isn't adding it, it would have hardcoded the hostname if it had anyway.

For me, removing the send_hostname line from dhclient in my packer script stops localhost getting added to DNS, and modifying setting Provisioning.MonitorHostName=y in waagent.conf adds the hostname I set in cloud-init.