Feature request -- Specify country/country list for mirror auto-selection about flexo

[cmsigler]

Hi,

One more observation now that I'm playing with this a bit more. For some reason the auto-selection of mirrors always seems to select the same mirrors. However, none of these mirrors is a "fast" mirror according to rankmirrors, and rankmirrors has served me very well.

The auto-selected list just looks "strange" based on my manually managing my mirrorlist for a very long time in order to saturate my DSL line. These seem to be slow compared to my manually managed list.

Obviously the mirrors chosen come from the latest archlinux.org mirrors status JSON. I assume when I run rankmirrors it does the same thing. The only difference is I restrict rankmirrors to those from the US.

Again, I really appreciate flexo because I really need a pacman pkg proxy cache since I now find myself with four Arch boxes at home (I'm a masochist ;). I hope my comments will help you. Thank you again very much!

Clemmitt

[nroi]

Thanks for reporting this. I would like to prioritize the auto-selection mechanism before implementing workarounds to circumvent the less-than-ideal auto-selection mechanism.

Could you paste your /var/cache/flexo/state/latency_test_results.json? This might help me troubleshoot this issue further.

[cmsigler]

Hi,

Well I wish I could send you a copy of the old latency_test_results.json, but when I restarted flexo after finding mirrors from Sweden and Australia it was overwritten :(

I haven't read your program. Is it supposed to have "adaptive logic" that learns over time? (I'm guessing it isn't....)

I can't explain it, but now flexo is choosing much closer mirrors all in the US. Why it had that glitch when I was first trying it out is beyond me.

Just FYI, here is the current latency_test_results.json:

---

{
"timestamp": "2020-12-07T18:03:31.023427394Z",
"download_providers": [
{
"uri": "https://arch.mirror.constant.com/",
"mirror_results": {
"total_time": {
"secs": 0,
"nanos": 222952000
},
"namelookup_duration": {
"secs": 0,
"nanos": 3235000
},
"connect_duration": {
"secs": 0,
"nanos": 44685000
},
"pretransfer_time": {
"secs": 0,
"nanos": 151003000
},
"starttransfer_time": {
"secs": 0,
"nanos": 222889000
}
},
"country": "United States"
},
{
"uri": "https://mirror.dal10.us.leaseweb.net/archlinux/",
"mirror_results": {
"total_time": {
"secs": 0,
"nanos": 294726000
},
"namelookup_duration": {
"secs": 0,
"nanos": 2232000
},
"connect_duration": {
"secs": 0,
"nanos": 64915000
},
"pretransfer_time": {
"secs": 0,
"nanos": 228982000
},
"starttransfer_time": {
"secs": 0,
"nanos": 294661000
}
},
"country": "United States"
},
{
"uri": "https://arlm.tyzoid.com/",
"mirror_results": {
"total_time": {
"secs": 0,
"nanos": 331767000
},
"namelookup_duration": {
"secs": 0,
"nanos": 2473000
},
"connect_duration": {
"secs": 0,
"nanos": 86599000
},
"pretransfer_time": {
"secs": 0,
"nanos": 232167000
},
"starttransfer_time": {
"secs": 0,
"nanos": 331717000
}
},
"country": "United States"
},
{
"uri": "https://mirror.wdc1.us.leaseweb.net/archlinux/",
"mirror_results": {
"total_time": {
"secs": 0,
"nanos": 286177000
},
"namelookup_duration": {
"secs": 0,
"nanos": 3678000
},
"connect_duration": {
"secs": 0,
"nanos": 87520000
},
"pretransfer_time": {
"secs": 0,
"nanos": 249499000
},
"starttransfer_time": {
"secs": 0,
"nanos": 286130000
}
},
"country": "United States"
},
{
"uri": "https://arch.hu.fo/archlinux/",
"mirror_results": {
"total_time": {
"secs": 0,
"nanos": 309015000
},
"namelookup_duration": {
"secs": 0,
"nanos": 2321000
},
"connect_duration": {
"secs": 0,
"nanos": 93721000
},
"pretransfer_time": {
"secs": 0,
"nanos": 208679000
},
"starttransfer_time": {
"secs": 0,
"nanos": 308965000
}
},
"country": "United States"
}
]
}

---

It would probably help to know I am (physically and) network hop-wise located close to the Washington, DC/metro area, so the mirror.wdc1.us.leaseweb.net mirror is a good one and is also retrieved when I use rankmirrors.

One other peculiarity of my network connection -- My ISP is a dinosaur and refuses to support IPv6. So I use a Hurricane Electric 6-in-4 tunnel with the other endpoint in the DC area. This means my IPv4 and IPv6 routing are asymmetric. FWIW....

Please let me know how else I can help :) Thank you again!

Clemmitt

Edit: Correct stupid wiki editing mode mistake; mention IPv6 tunnel config

[nroi]

Thanks for the detailed info, I'm going to collect everything related to the auto-selection issue in #29. I'll keep this issue open for the feature request of including a list of mirrors.

[nroi]

This has been implemented on version 1.0.9. To make use of this setting, proceed as follows:

• Install the most recent version of Flexo.
• Edit your /etc/flexo/flexo.toml: Add something like allowed_countries = ["US"] (or whatever country makes sense for you) at the bottom of the mirrors_auto section.
• Restart Flexo.
• Wait for 10 seconds or so.

The file /var/cache/flexo/state/latency_test_results.json should then include only mirrors from the US.

[cmsigler]

Hi,

Thank you for your hard work to implement this, and for flexo :)

Clemmitt