win_copy: Unable to copy large folder about ansible.windows

[Udayendu]

I did the ansible upgrade and tried but still the same behavior. Seems some timeout is happening somewhere.

# pip3 show ansible
Name: ansible
Version: 2.9.13
Summary: Radically simple IT automation
Home-page: https://ansible.com/
Author: Ansible, Inc.
Author-email: [email protected]
License: GPLv3+
Location: /usr/local/lib/python3.8/dist-packages
Requires: cryptography, jinja2, PyYAML
Required-by:

Also I observed that its able to make the complete tarball of the folder under /root/.ansible/tmp location but after that its not copying the data to the target location. But if the size is small then its working perfectly.

[Udayendu]

I tried to copy a single file of 6.7GiB and that worked. But when keeping that in a folder its failing with the same error.

[Udayendu]

After switching from ntlm to credssp based method, its able to do the copy from ansible provision server to windows guest system. I tried multiple size like 3GB, 5GB, 7GB and upto 8.5GB and all worked successfully.

So not sure if the issue with the module or the auth method.

[jborean93]

Hmm I can't seem to replicate this problem, I created a 4.5GB file with fallocate -l 4.5G large.file then ran the following playbook

- hosts: '2019'
  gather_facts: no
  tasks:
  - win_copy:
      src: large.file
      dest: C:\temp\large.file

  - win_file:
      path: C:\temp\large.file
      state: absent

I made sure to use NTLM auth through WinRM based on your last comment but it succeeded in the end. Are you finding this is failing when the task starts or after some time and some data has been transferred already. Is it happening with just 1 Windows host or can you replicate it across other. When the copying is happening have you seen any large memory spikes on the Windows host that may contribute to the failure?

[Udayendu]

Are you finding this is failing when the task starts or after some time and some data has been transferred already.

Yes, after creating the zip file its failing. Its not able to start the copy activity.

Is it happening with just 1 Windows host or can you replicate it across other.

 I tried with two systems in parallel and for both its failing at the same point with same error. And even for a single windows system also the same error at same stage. So I thought its the issue with NTLM. Because wen I am using CredSSP and WinRM, its working perfectly.

When the copying is happening have you seen any large memory spikes on the Windows host that may contribute to the failure?

No notable change in CPU & Memory utilization both at source and destination.

[jborean93]

Yes, after creating the zip file its failing. Its not able to start the copy activity.

How long does it take before it actually fails. Is it typically once it tries to start copying the zip and it's near the end of the copy (i.e. 2+GB has been copied)?

Can you update your ntlm-auth and cryptography Python libraries and try again. Can you run that playbook (with the fallocate command beforehand) to see if that works for you. Can you slowly decrease the size until you find out when it starts to fail.

From my perspective I cannot replicate this problem on any of my hosts so it's going to be hard to figure out what's going wrong here. Until then there's not much I can do to try and fix the problem.

[Udayendu]

Yes, after creating the zip file its failing. Its not able to start the copy activity.

How long does it take before it actually fails. Is it typically once it tries to start copying the zip and it's near the end of the copy (i.e. 2+GB has been copied)?

Usually it do create the dip properly of whatever size the actual content is. I have see it upto 8.5GiB. creating zip is never an issue. Its the copy the zip file that never worked.

Can you update your ntlm-auth and cryptography Python libraries and try again. Can you run that playbook (with the fallocate command beforehand) to see if that works for you. Can you slowly decrease the size until you find out when it starts to fail.

In my system both the packages are latest:

$ pip3 show ntlm-auth
Name: ntlm-auth
Version: 1.5.0
Summary: Creates NTLM authentication structures
Home-page: https://github.com/jborean93/ntlm-auth
Author: Jordan Borean
Author-email: [email protected]
License: MIT
Location: /usr/local/lib/python3.8/dist-packages
Requires:

$ pip3 show cryptography
Name: cryptography
Version: 3.1
Summary: cryptography is a package which provides cryptographic recipes and primitives to Python developers.
Home-page: https://github.com/pyca/cryptography
Author: The cryptography developers
Author-email: [email protected]
License: BSD or Apache License, Version 2.0
Location: /usr/local/lib/python3.8/dist-packages
Requires: cffi, six
Required-by: requests-ntlm, requests-credssp, pyspnego, paramiko, ansible

From my perspective I cannot replicate this problem on any of my hosts so it's going to be hard to figure out what's going wrong here. Until then there's not much I can do to try and fix the problem.

Yes, thats true. So may be you can close this as I am no longer going to use 5985 in my platform. I am using CredSSP with 5986 and thats working perfectly.

[jborean93]

I would also be curious if the problem with NTLM over port 5985 is also an issue when you use the psrp connection plugin. It also runs over WinRM/WSMan but in a protocol inside that with different deps.

I'm going to close the ticket for now as you've got a workaround. If you do come across any other key information please share it as we might be able to figure out what's going on from there.