Comments (3)
metalwarrior665
commented on May 18, 2024
I would like to add this, it seems straightforward. It is in the client.
The thing is if we want to somehow control the memory to be 2 to the power of something. Right now in API it is not controlled at all, you can run the act with memory 1234.
from crawlee.
mtrunkat
commented on May 18, 2024
We let use enter any value in SDK/client. If the memory won't be a power of 2 then server throws an error which appears in actor log. I think that this is better behavior than having some validation in SDK, client and server.
from crawlee.
mtrunkat
commented on May 18, 2024
Fixed in new version that will be shortly released as BETA
from crawlee.
Related Issues (20)
- Huge sitemap takes forever to load
- Make RequestQueueV2 default
- type error `puppeteerUtils.gotoExtended` ?
- Issue with decoding quotation mark HOT 2
- Incorrect Request Timeout in Error Message
- The request queue scans all 450k (99.999% of which are done towards the end) requests for each iteration HOT 2
- Missing `create*Router` helper for AdaptivePlaywrightCrawler
- Support for crawling from secondary IP address HOT 1
- Statistics does not use crawler log HOT 1
- Race conditions in CI/CD HOT 4
- Malformed Sitemap content when url contains searchParams HOT 7
- Mysterious timeout hard-kills `CheerioCrawler` script HOT 7
- The default value of `availableMemoryRatio` is too low HOT 10
- Some "run on Apify" examples do not work HOT 1
- Control proxy-chain's 'host' parameter from Apify's launchContext HOT 2
- Handle `Crawl-delay` directive in robots.txt
- RobotsFile.isAllowed returns false for allowed routes HOT 3
- Refactor `retireOnBlockedStatusCodes` to `isBlockedStatusCode` and move the retiring out of the `Session` class
- `AdaptivePlaywrightCrawler`: programmatically deciding when to render JS HOT 3
- Cheerio crawler going out of memory unexpectedly with lot of concatenated strings
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from crawlee.