Comments (10)
Option 2 seems nice.
from puppeteer-cluster.
I removed the await
s from all cluster.queue
examples and added this hint to the docs:
"Be aware that this function only returns a Promise for backward compatibility reasons. This function does not run asynchronously and will immediately return."
I guess that should clear any confusion in the future.
from puppeteer-cluster.
The await
there is currently not really needed (it's also not needed for cluster.queue
by the way). It's there for two reasons:
- Consistency: All functions on
cluster.*
return Promises. That way you don't need to worry whether to useawait
or not when dealing with the cluster functions. Just use it everywhere and you are fine. - Forward compatibility: Right now the functions
task
andqueue
have only synchronous calls inside. That means right now we don't need Promises. I can imagine that in the future this might change. There are several use cases for this. Imaging we want to log data asynchronously whenqueue
ortask
is called. Or maybe we want to store the queued data into a database instead of an array.
Hope that makes sense.
from puppeteer-cluster.
Right now the functions task and queue have only synchronous calls inside.
You mean cluster.queue('http://www.wikipedia.org/')
will be called after cluster.queue('http://www.google.com/')
finish?
from puppeteer-cluster.
No, I meant there are no await (asynchronous) calls inside of the task and queue functions. The functions are very simple and don't need to be async
right now.
from puppeteer-cluster.
I got it. The doc indicate that all functions on cluster.*
return Promises. The example may cause confusion that cluster.queue('http://www.wikipedia.org/')
will be called after cluster.queue('http://www.google.com/')
finish, take axios
as example:
await axios.get('http://api.org/get-name');
// will be called after the above function is finished
await axios.get('http://api.org/get-age');
// both will be called asynchronously
axios.get('http://api.org/get-name');
axios.get('http://api.org/get-age');
Hope it only causes confusion for me lol.
from puppeteer-cluster.
Thank you. I had not thought about it that way. Maybe the await
should be removed then?
I have to think about this. Feedback is welcome :)
from puppeteer-cluster.
await
really seems to confuse people: https://stackoverflow.com/questions/46597304/how-to-enable-parallel-tests-with-puppeteer/51408815#comment92960172_51408815
Currently I'm in favor of removing await
from the documentation, but keeping the returning of a Promise to not break compatibility with previous versions.
from puppeteer-cluster.
I think people would use queue
to define a task and add it to end of the queue. Is it really necessary to return a Promise in this case?
from puppeteer-cluster.
I implemented the execute
function which hopefully makes things clearer. Right now I think there are three options on how to proceed:
- Remove Promise from
queue
function and returnvoid
instead ofPromise<void>
- Still return a Promise, but remove all
await cluster.queue(..)
from documentation - Continue as is.
I prefer option 2 as this keeps backward compatibility and will hopefully make it more clear through documentation.
from puppeteer-cluster.
Related Issues (20)
- I think a timeout of `0` should disable timeouts HOT 2
- Clear up Concurrency wording incorrect usage HOT 5
- Feature: Lifetimes
- How To Stop Worker To Become Idle automatically
- Expose stats via prometheus HOT 2
- Screen shot getting stuck forever
- Use same URL but diffetent logic on each browser HOT 1
- Concurrency launch: CONCURRENCY_BROWSER definition slightly misleading HOT 2
- Error detection super slow with new Puppeteer versions HOT 1
- Support to new puppetter versions HOT 1
- share the dockerfile I'm using
- Suggestion: Allow pool of already instantiated browser workers
- how to open the progress view and monitoring statistics? HOT 1
- cluster concurrent seems not work HOT 1
- Regarding resource usage HOT 1
- Worker Error getting browser page HOT 1
- How to set args like .launch({ args: [] }) ? HOT 3
- Has anyone managed to use separate data for each browser?
- browser crushing due to "open /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq: No such file or directory (2)"
- puppeteer-v22.0.0 breaking changes HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from puppeteer-cluster.