iTesco_cz/sk vrací 0 items about hlidac-shopu

[JJetmar]

@rarous With @MartinaGelnerova we changed the proxies to RESIDENTIAL, and we wanted to wait like a week to see how much would the price per run change.

Currently we are waiting for confirmation based on the new price, right? @MartinaGelnerova

[MartinaGelnerova]

@JJetmar Oh, I didn't see "Build" option but that was because I was checking the task, not actor itself. Good to know... Thank you.

But it's not working still (different reasons for CZ and SK):
https://console.apify.com/admin/users/iMWJjifpQdTwbkKYn/actors/tasks/L48jIjbkN9oQlUQY9/runs/eLnYhfxC9nHVTs4IA#log
https://console.apify.com/admin/users/iMWJjifpQdTwbkKYn/actors/tasks/wASaIqALSVN65yR2o/runs/LzmXHpa4Vp92FykUv#log

[JJetmar]

Checked this one, it seems that the proxy groups we used were blocked. I did some testing and changed the groups. No implementation needed. We will evaluate the results tomorrow.

[rarous]

@JJetmar is this solved?

[MartinaGelnerova]

@JJetmar @rarous we decided with Kuba B. to keep current setup with RESIDENTIAL, so I am closing the issue as solved

[MartinaGelnerova]

@JJetmar @rarous sorry, re-opening the issue again: today both CZ and SK runs took over 5 hours with only few returned results. I aborted both runs but there should be some time-limit in order not to let run last (and consume residential proxies) forever...
Can you please have a look at it?
https://console.apify.com/admin/users/iMWJjifpQdTwbkKYn/actors/n8WwDXAF6HXzNBPYm/runs

[JJetmar]

Hmm... When the Actor works, the run takes about 40min, I will setup (temporary) timeout to 60min. So we don't have to abort the actor manually and check if there are any other options.

[JJetmar]

Looks like there is some new additional protection.
I did few changes, currently having test run under my account.
Currently just checking if it make it through. Once it finishes I will try to do some optimizations for the resources/cost.

[JJetmar]

PR #1490

[JJetmar]

So far it looks like that after update no Residential proxies are longer needed and we are obtaining more results (there is no blocking in log from what I see), the cost is lower than with Residential proxies (currently waiting for optimized run to finish).

Highly suggest to increase the memory for the actor to 4096MB, since it is using Puppeteer now. Or ping me, when merged - I can do that.

[JJetmar]

Highly suggest to increase the memory for the actor to 4096MB, since it is using Puppeteer now. Or ping me, when merged - I can do that.

@MartinaGelnerova I set up the Memory on the Tasks for this Actor, so far everything should be good to go (I noticed that running the Actor was unscheduled recently). Additionally I set the timeout for run to 2hrs which should be more than fine for the results, and also it will save the cost in case that something unpredictable happens.

[MartinaGelnerova]

@JJetmar thank you, I have re-set schedule for both CZ and SK task.

[MartinaGelnerova]

@JJetmar Am I missing something? I tried to manually run iTesco_cz and iTesco_sk tasks, but both ended with 0 results. Logs:
https://console.apify.com/admin/users/iMWJjifpQdTwbkKYn/actors/tasks/L48jIjbkN9oQlUQY9/runs/Fa3A8XNPokTq8DpvV#log
https://console.apify.com/admin/users/iMWJjifpQdTwbkKYn/actors/tasks/wASaIqALSVN65yR2o/runs

[salax]

yeah, the actor wasn't rebuilt (it's a manual process). It should work now.

[JJetmar]

Ah, sorry about that, I remember pressing the build button on platform but based on the builds it was not really rebuilt at all 🤔

[JJetmar]

@MartinaGelnerova Weird,
I just started run under my account and everything works fine. I am investigating what is happening 🤔

[MartinaGelnerova]

@JJetmar fyi:
CZ finished successfully today, but returns low amount of results (used to be 7t items, today only 1832 items)
cost is low ($0.482)
log: https://console.apify.com/admin/users/iMWJjifpQdTwbkKYn/actors/tasks/L48jIjbkN9oQlUQY9/runs/oEHTYdRtfmmdG1MUj#log

SK: failed, ERROR Selected proxy groups have no usable proxies from country 'SK'.
log: https://console.apify.com/admin/users/iMWJjifpQdTwbkKYn/actors/tasks/wASaIqALSVN65yR2o/runs/C1asnkHq52fe9Wbd6#log

[JJetmar]

I did another two runs today under my account both with more than 7k results.
I noticed, that only difference is that I am running it with the development mode which somehow affects the proxyConfiguration, so I changed that in the code.
I also fixed the issue with the SK version.

@salax PR #1504

[JJetmar]

@MartinaGelnerova I see both of the runs are fine for today.

[MartinaGelnerova]

@JJetmar You are right - number of results is correct and the cost is reasonable ($1). Thank you for fixing it. Closing the issue.

[MartinaGelnerova]

@JJetmar sorry to bother you again:

SK 0 items during last 2 days, logs:
https://console.apify.com/admin/users/iMWJjifpQdTwbkKYn/actors/tasks/wASaIqALSVN65yR2o/runs/ojbC9KdExBoJfVaAT#log
https://console.apify.com/admin/users/iMWJjifpQdTwbkKYn/actors/tasks/wASaIqALSVN65yR2o/runs/nPJcLn6ir8Oh2s1eB#log

CZ 0 items on Sat 26.8., but succeeded with items on Sun 27.8.. Log with 0 items:
https://console.apify.com/admin/users/iMWJjifpQdTwbkKYn/actors/tasks/L48jIjbkN9oQlUQY9/runs/fXNvPkAIn6sLG8NcM#log

[JJetmar]

It gave all the results today without any changes - so it is happening occasionally.

Hopefully I will be able to reproduce this behavior and fix it by better turning of sessions when request is blocked.

I will check today in the evening.

[JJetmar]

It works pretty well - the only issue is it get blocked sometimes - I did some minor improvement for the session management so it tries more than just 3 retries before it dies. For some reason I can not really reproduce this behavior under my account. Let's merge the changes and see.

@salax PR #1530

[JJetmar]

@MartinaGelnerova I see it was merged + built yesterday, today we are still obtaining the results regulary.

[JJetmar]

@MartinaGelnerova I see the drops are still comming back, I am currently testing new implementations under my account.

[rarous]

Mostly successful. Good enough