Addon manager causes temporary ban from github (ratelimit) about freecad

[kadet1090]

I can reproduce, banned two times today 😅

[maxwxyz]

I cannot reproduce on Windows

[diggit]

With git enabled (Disable git not ticked)
stdout log

FreeCAD 0.22.0, Libs: 0.22.0devR37923 (Git)
© Juergen Riegel, Werner Mayer, Yorik van Havre and others 2001-2024
FreeCAD is free and open-source software licensed under the terms of LGPL2+ license.
FreeCAD wouldn't be possible without FreeCAD community.
  #####                 ####  ###   ####  
  #                    #      # #   #   # 
  #     ##  #### ####  #     #   #  #   # 
  ####  # # #  # #  #  #     #####  #   # 
  #     #   #### ####  #    #     # #   # 
  #     #   #    #     #    #     # #   #  ##  ##  ##
  #     #   #### ####   ### #     # ####   ##  ##  ##

Sheet Metal workbench loaded
The `.FCStd#` backup format is deprecated as of v0.21 and may be removed in future versions.
To update, check the 'Preferences->General->Document->Use date and FCBak extension' option.
The cached device pixel ratio value was stale on window update.  Please file a QTBUG which explains how to reproduce.
Previous cache process was interrupted, restarting...
The cached device pixel ratio value was stale on window update.  Please file a QTBUG which explains how to reproduce.

fc_addon_manager_github_git_enabled.pcapng.gz

[diggit]

Ok now I tried it via mobile data and at least GH does not ban me, but Addon manager gets stuck on same percentage (41%, Downloaded icon for OSH Automated DOcumentation)

Edit:
even though the progress bar got stuck at 41% it was possible to stop the current operation. Then I could browse the catalog and install addons.

[diggit]

My wired connection ISP is Vodafone CZ and they are using CGNAT with DSlite (default). So my public IPv4 is shared with others and that could explain rate limit issue (even though I did not notice it anywhere else than with FreeCAD).

@kadet1090 could you share some details about your connectivity?

[chennes]

@diggit are you self-compiling? If so, the latest main should have fixed the "getting stuck at OSH" issue on Qt6 -- it may also fix the temp-ban problem, I'm not 100% sure which rate-limiter is being tripped. None of this has anything at all to do with the "use zip download" setting -- we're not using git (the app) to access this data, we're using GitHub (the website) to pull down one specific file.

[kadet1090]

@kadet1090 could you share some details about your connectivity?

Hard to say if I'm behind ISP NAT - my ISP is Orange PL. However I'm also using arch and get basically same results - after opening addon manager I can't access github anymore. I doub that this is API rate limiting - more like some firewall on some part of the network triggers and cuts of all traffic from my IP as I also cant access github via SSH.

So either GitHub changed their firewalls and they are blocking traffic from IPs making rapid requests or there is another firewall that we have in common that is doing exactly that.

Nontheless - tried again and cannot reproduce now, maybe github were under DDoS attack and had enabled more strict firewall rules.

Edit:

Managed to get banned, it appears that I get cut out at this node as it is first missing from my traceroute after ban:

9  github-ic-350972.ip.twelve99-cust.net (62.115.182.171)  42.335 ms  43.810 ms  42.261 ms

[Carpetner]

I also got flooded with the killTimer msg today, playing with FEM and maybe accessing examples. Everything seemed to work fine though, except for me.

OS: Linux Mint 21.3 (X-Cinnamon/cinnamon)
Word size of FreeCAD: 64-bit
Version: 0.22.0dev.37928 (Git) AppImage
Build type: Release
Branch: main
Hash: e7699b531eba4bee262f7214699f07b96f66ba7b
Python 3.11.9, Qt 5.15.13, Coin 4.0.2, Vtk 9.2.6, OCC 7.7.2
Locale: English/United States (en_US)
Installed mods: 
  * PieMenu 1.7.0
  * Assembly4 0.50.13
  * sheetmetal 0.4.18
  * fasteners 0.5.24
  * Render 2024.6.5
  * OpenTheme 2024.5.16
  * EM 2.1.1

[obelisk79]

This issue is still present, using an AUR build. If I download the latest appimage, everything works fine.

OS: EndeavourOS (KDE/plasmax11)
Word size of FreeCAD: 64-bit
Version: 0.22.0dev.38025 (Git)
Build type: Release
Branch: main
Hash: 04cc890fd0685bc4143de1a8000e92dce0580d28
Python 3.12.4, Qt 6.7.2, Coin 4.0.2, Vtk 9.3.0, OCC 7.7.2
Locale: English/United States (en_US)

[chennes]

Is that a clean rebuild? In particular, I want to ensure that the Addon Manager's Python files got copied correctly. Current MD5sum of NetworkManager.py should be 6898c9fe2ad4ea03135d375591435565.

[obelisk79]

hmmm, looks like the checksum here is : 2a165bb8303af497941997d4c5c8ca3d I'll try and force it on my install.

[obelisk79]

@chennes the MD5 I posted in above comment matches the latest file after downloading it directly from the repo using my browser.

[chennes]

OK, thanks. Then the rate limit issue was not related to the redirect issue (which is the one I fixed last week). It's odd to me that the rate limiter would affect different platforms differently, but perhaps we are just skating on the edge of it, and some platforms are slightly faster than others. I'll try to code up something to detect it.

[chennes]

This is really probably a deeper issue: it's not a great idea for every copy of FreeCAD to do this scrape every time. I was planning on doing the cache server-side post-1.0, but it appears that I am going to have to do it now.