Comments (6)
My use case is a bit different, I rarely need to run the entire archive at a time. Instead I usually only run the last few months worth, so I use smaller instances and only run the service on a subset of the snapshots.
When I need the entire chain I usually use i3en.12xlarge
or i3en.24xlarge
as spot instances, since I am not worried if they go down unexpectedly (due to my use case).
I would not suggest using EBS volumes, they are extremely slow and it is often cheaper to use instances with ephemeral storage than smaller instances with EBS.
If cost is really a concern and you don't need it to be super fast, the version of geth
I provided in the install script has modified the timeout of calls. You can just use git diff
to find out what the changes are. If you increase it even more, I have had success using the d3en
class of instances with much smaller EBS (configured as io2
) and then use ZFS and configure it to be an L2ARC. You can configure those HDD's to be RIAD0'ed together. I haven't tested this in a long time, but I was able to keep up with the chain when I tested back in May.
from bsc-archive-snapshot.
Is it possible that the AWS user you are using is not allowed to be charged for S3 actions from the account owner it's under?
I have reports of other users being able to use it without issue just yesterday.
from bsc-archive-snapshot.
I suggest trying in us-west-2
(as it's pretty much free to download) and if it works there then do your testing in your preferred data center.
Can you run:
aws sts get-caller-identity
And make sure it succeeds (you don't need to post results here). If it does not succeed, it means that you don't have an AWS role setup on that instance.
I suspect the issue is that the instance is either missing a role or the role it has doesn't have permissions to be changed for requester pays configuration. The easiest way to test this is to put in your personal AWS credentials on the instance using aws configure
then try, if it works, it's a role issue.
I have was helping out another company at one point and had ssh access into the instance and was able to download it a few months ago and have changed no permission stuff, so I am pretty sure it should be fine on my end.
Let me know,
-Allada
from bsc-archive-snapshot.
from bsc-archive-snapshot.
Hi, not sure if you got this working @jsvisa - but I also get the 403 error. Even from us-west-2
# aws configure
AWS Access Key ID [None]:ACCESSKEYID
AWS Secret Access Key [None]: SECRETKEY
Default region name [None]: us-west-2
Default output format [None]:
# aws sts get-caller-identity
{
"UserId": "XXXXX",
"Account": "7011111111111",
"Arn": "arn:aws:iam::7011111111111:user/user"
}
# aws s3 ls --request-payer=requester s3://public-blockchain-snapshots/bsc/
An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
# aws s3 cp --request-payer=requester s3://public-blockchain-snapshots/bsc/erigon-latest.tar.zstd .
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
Any help would be appreciated @allada
from bsc-archive-snapshot.
Owner
** allada ** commented 1 hour ago •
Ah you're absolutely correct - thank you very much
from bsc-archive-snapshot.
Related Issues (20)
- how login account HOT 1
- AWS fatal error: Unable to locate credentials HOT 10
- Problem when calling the ListObjectsV2 operation: Access Denied HOT 3
- The whole snapshot only 500GB? HOT 3
- ETH vs BSC Archive Nodes` HOT 1
- fatal error: An error occurred (404) when calling the HeadObject operation: Key "bsc/erigon-latest.tar.zstd" does not exist HOT 1
- Download failed HOT 42
- Download snapshots failed HOT 7
- How to store 30TB in AWS EBS? HOT 4
- Sync slow HOT 8
- fail to start bsc archive with new snapshot HOT 5
- Empty file after downloading erigon-bsc-archive HOT 2
- Current version failure to download/run... HOT 2
- The script stopped working HOT 3
- Script finishes with no errors at 568GB. HOT 9
- getting block not found for older blocks HOT 1
- Always about 8 minutes behind the latest block HOT 5
- S3 folder structure possibly wrong HOT 4
- On (probable) discontinuation of erigon support for BSC HOT 4
- Slow synchronization HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bsc-archive-snapshot.