Find File differences using for loop with bash or python

Question

I am trying to add Linux machines to ansible host file with automation script. Automation process is:

1. Ansible machine gets Linux vm list which named LinuxVms.txt from one of Vmware server.

2. I developed a shell script as shown below. It adds servers into the [all_linux_host] tag, from LinuxVms.txt file. The script works when used in a single operation.

After this process, what I want to do is:

• The VM team will automatically send the LinuxVm.txt list to the ansible server and if the LinuxVm.txt file has a new IP address I need to add this IP address to the ansible hosts file, under the [all_linux_host] tag.

I am thinking that a for loop should work for this. The for loop has to control the new arrived LinuxVm.txt file and only [all_linux_host] tag, not all tags in ansible host file. If there is a differences between file and tag it has to find that differences and add to the [all_linux_host] tag.

For example

• LinuxVms.txt

  1.1.1.1       
  2.2.2.2           
  3.3.3.3   
  12.12.12.12
  

• current, ansible host file /etc/ansible/hosts

  [test]  
  8.8.8.8         
  12.12.12.12   
  13.13.13.13
  
  [all_linux_hosts]
  1.1.1.1     
  2.2.2.2
  

  Note that [all_linux_hosts] is the last section in the ansible host file.
• After for loop, the ansible host file has to be like this

  [test]  
  8.8.8.8         
  12.12.12.12   
  13.13.13.13
  
  [all_linux_hosts]
  1.1.1.1       
  2.2.2.2           
  3.3.3.3   
  12.12.12.12
  

  The order of the IP addresses is not important.

Can you help me to develop the for loop?

---

One time operation you don't need to consider this part.

#!/bin/bash
sudo cp /home/vmteam/LinuxVMs.txt /home/xxx
sudo chown xxx: /home/vmteam/LinuxVMs.txt
sudo dos2unix /home/xxx/LinuxVMs.txt
awk '{print $1}' /home/xxx/LinuxVMs.txt >> ansible_host_file  ##file correction
awk '{print $2}' /home/xxx/LinuxVMs.txt >> ansible_host_file  ##file correction
sed -i 's/PublicIp//g'    /home/xxx/ansible_host_file
sed -i 's/-//g' /home/xxx/ansible_host_file
sed -i '/^\s*$/d' /home/xxx/ansible_host_file
sed -i 's/IpAddress1/ /g' /home/xxx/ansible_host_file
sed -i '/^\s*$/d' /home/xxx/ansible_host_file``` 

Answer 1

Populate an associative arraya[".."] keyed on the linux hosts mentioned in the vms file. Then tick off hosts seen in ansible file and print what remains in the end.

awk '
  NR==FNR {a[$1];next}

  $1=="[all_linux_hosts]",0 {
    if ( $1 in a ) delete a[$1]
  };1

  END {
    for (host in a) print host
  }
' LinuxVms.txt /etc/ansible/hosts
[test]  
8.8.8.8         
12.12.12.12   
13.13.13.13

[all_linux_hosts]
1.1.1.1     
2.2.2.2
12.12.12.12
3.3.3.3