storj / docs

Source for Storj docs

Storj V3 Network

Storj is building a distributed cloud storage network. Check out our white paper for more info!

---

Storj is an S3-compatible platform and suite of distributed applications that allows you to store data in a secure and distributed manner. Your files are encrypted, broken into little pieces and stored in a global distributed network of computers. Luckily, we also support allowing you (and only you) to retrieve those files!

Table of Contents

• Contributing
• Start using Storj
• License
• Support

Contributing to Storj

All of our code for Storj v3 is open source. If anything feels off, or if you feel that some functionality is missing, please check out the contributing page. There you will find instructions for sharing your feedback, building the tool locally, and submitting pull requests to the project.

A Note about Versioning

While we are practicing semantic versioning for our client libraries such as uplink, we are not practicing semantic versioning in this repo, as we do not intend for it to be used via Go modules. We may have backwards-incompatible changes between minor and patch releases in this repo.

Start using Storj

Our wiki has documentation and tutorials. Check out these three tutorials:

• Using the Storj Test Network
• Using the Uplink CLI
• Using the S3 Gateway

License

This repository is currently licensed with the AGPLv3 license.

For code released under the AGPLv3, we request that contributors sign our updated Contributor License Agreement (CLA) v2 so that we can relicense the code under Apache v2, or other licenses in the future.

Support

If you have any questions or suggestions please reach out to us on our community forum or file a ticket at https://support.storj.io/.

docs's Issues

can't open dashboard for the node

filezilla - update the way how to download the latest version

This ticket was escalated from Zendesk by Aleksey Leonov:

Ticket: #16026
Type: Ticket
Priority: -
Requester: Tomas
Organization: -
Assignee: Aleksey Leonov

---

I have some troubles with setting my account with filezilla... I dont see any option wit storj, I only  have default FTP/SFTP in Protocol window... do I need Filezilla Pro? What am I doing wrong?

I am using Fedora 35, filezilla installed using flatpak and rpm packege... none of it has that option... filezilla version 3.58.0

Attachments:
Screenshot from 2022-03-30 09-41-59.png
Screenshot from 2022-03-30 09-50-08.png gz#16026

How-To- to post - it's how to hot rod your performance

https://docs.google.com/document/d/1lNPsmTqpf2yIzNtAys4baHC2OveJ8oYYxOYvSmC1H2w/edit

docker exec -it storagenode /app/dashboard.sh

Hello

This command works find : docker exec -it storagenode /app/dashboard.sh

The problem is I can't access to the web page : Internal 127.0.0.1:7778
I suppose I can find my reputation on this page but like I can't acces to it I don't know if it's work well or no.

A last thing, when the docker process to a upgrade like today from v0.10 to v0.11, is the reputation is reset to 0 like the uptime ? I hope no but it to be sure.

Thanks, all the best.

dashboard reporting error

Anyone receiving wrong usage report on dashboard?
Everytime system reboots, the dashboard reports the newly updates not total.

Broken links?

Assuming this repo is https://docs.storj.io,
then e.g. https://docs.storj.io/dcs/api-reference/storj-client-libraries/ has 'Broken Reference':

Container error on raspberry with older database

2019-06-02T15:53:20.567388697Z 2019-06-02T15:53:20.566Z	INFO	Configuration loaded from: /app/config/config.yaml
2019-06-02T15:53:20.572806761Z 2019-06-02T15:53:20.572Z	INFO	Operator email: [email protected]
2019-06-02T15:53:20.573283734Z 2019-06-02T15:53:20.572Z	INFO	Operator wallet: 0x8472A2725Ef5bC8932c2733Bb5EE4401022bbb08
2019-06-02T15:53:20.575076110Z Fatal error: Error starting master database on storagenode: boltdb error: open config/kademlia: operation not supported
2019-06-02T15:53:20.575203817Z 	storj.io/storj/storage/boltdb.NewShared:71
2019-06-02T15:53:20.575240379Z 	storj.io/storj/storagenode/storagenodedb.New:58
2019-06-02T15:53:20.575270535Z 	main.cmdRun:134
2019-06-02T15:53:20.575299753Z 	storj.io/storj/pkg/process.cleanup.func1.2:270
2019-06-02T15:53:20.575329128Z 	storj.io/storj/pkg/process.cleanup.func1:284
2019-06-02T15:53:20.575359961Z 	github.com/spf13/cobra.(*Command).execute:762
2019-06-02T15:53:20.575389544Z 	github.com/spf13/cobra.(*Command).ExecuteC:852
2019-06-02T15:53:20.575418971Z 	github.com/spf13/cobra.(*Command).Execute:800
2019-06-02T15:53:20.575447981Z 	storj.io/storj/pkg/process.Exec:59
2019-06-02T15:53:20.575476835Z 	main.main:288
2019-06-02T15:53:20.575506730Z 	runtime.main:200

I want to run the node on my raspberry pi. I already have an older database that I want to use.
The older database I moved to an shared HDD. No special permision for HDD

Replace uplinkng to uplink in Download page

https://docs.storj.io/dcs/downloads/download-uplink-cli/

Update instruction for Rclone

This ticket was escalated from Zendesk by Aleksey Leonov:

Ticket: #16035
Type: Ticket
Priority: -
Requester: Visual
Organization: -
Assignee: Aleksey Leonov

---

I was trying to setup Rclone from your docs, however the latest Rclone no longer has storj on its list. In example you had N 31 which is now Palouse. Do you know which version worked so I can download that?

Now the integration is called Storj gz#16035

Windows 10 permission denied

running:
docker exec -it storagenode /app/dashboard.sh
or
docker logs -t storagenode
yield

Fatal error: open /app/config/config.yaml: permission denied

starting storagenode seems to work but running it shows this problem.

I'll will try to switch on/off to Windows Container mode bc it may be related to : https://docs.docker.com/v17.09/docker-for-windows/troubleshoot/#permissions-errors-on-data-directories-for-shared-volumes
edit> storagenode needs Linux container so switching was not useful

Move the section Get Started to the top of the Nav bar

As a user, I want to see first what I need to Get Started with Storj DCS before learning about concepts and company.

While we don't update the full documentation, I moved the section to the top of the navbar in gitbook, so it's more visible to users. I need this change to be merged.

#188 Moving "Getting Started" section to the top in Navbar
https://app.gitbook.com/o/-Lij5OMtVUHTiJ2J0i52/s/-LmlDzFHYn9g_f2QxmPe-887967055/~/changes/m7ojvkjYfVdPL83WhD9m/

Invalid CA for update checks

Seeing the following error on the latest storagenode update:

Failed to do periodic version check: %!(EXTRA *url.Error=Get https://version.alpha.storj.io: x509: certificate signed by unknown authority)

I should note that my browser resolves the above URL without a problem, and displays a valid cert.

Nixe Not found

Is this an issue with my node? Had this before the incompatibility update too...

2019-04-02T17:52:58.870975259Z 2019-04-02T17:52:58.870Z WARN agreements could not find satellite {"satellite id": "xxx", "error": "node not found", "errorVerbose": "node not found\n\tstorj.io/storj/pkg/kademlia.(*Kademlia).lookup:299\n\tstorj.io/storj/pkg/kademlia.(*Kademlia).FindNode:256\n\tstorj.io/storj/pkg/piecestore/psserver/agreementsender.(*AgreementSender).SettleAgreements:88\n\tstorj.io/storj/pkg/piecestore/psserver/agreementsender.(*AgreementSender).Run.func1:65\n\tstorj.io/storj/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1:57"}

Trying to get help

This is so hard to get going. No GUI? I have been hard drive mining with Burst and Sia and have not had this much issue getting setup. I am attempting to follow the instructions step by step. I have downloaded the identity_windows_amd64.exe. When I open it, It says try running from command prompt. I try that then powershell just flashes but nothing happens. I can not get it to work. I have made all exceptions with windows 10 pro and firewall. There is no reason for it to be blocked. I tried to get on the rocket chat for support. It says I have already signed up with my email as I have in the past. Yet, it says my password is invalid. Not sure how because I use 1password encryption and I can not loose passwords. I try to reset the password. It sends me an email to reset. When I try to it says error something went wrong. I've tried 3 times now with different pass combos and it still just says error. Therefore, I am locked out of rocket chat and I am staring at a brick wall. I really wanted to get on board with this node but you guys have got to make this easier to setup. What do I do from here? I can not install identity_windows_amd64.exe, I can not log into rocket chat, I can not progress forward. I have currently downloaded Docker desktop and that went fine and is running. I can not get identity_windows_amd64.exe to run.

Update Watchtower details

As I understand it, as of v1.52.2 Watchtower is not used for node updates, but is required for the base image, I want to update the documentation to reflect this change.

Windows Workabout...

After trials and errors...then I found my frigging mistakes...

so this is what I did ..

docker run -d --restart unless-stopped -p 28967:28967 -e WALLET="0x162C28E126A97D0009158Fbda0cA93E6e9CC13d0" -e EMAIL="[email protected]" -e ADDRESS="sg-01.azrin.info:4000" -e BANDWIDTH="2TB" -e STORAGE="50GB" -v "C:\Storagenode\":/app/identity -v "C:\new-storj\":/app/config --name storagenode storjlabs/storagenode:alpha

Copied IDENTITY to a new directory and found out if I have static IP..then I should rename the -p 28967:28967 to something else on the firewall map...

Windows is a bugger..so ensure c:\ (DOUBLE BACKSLASH) is needed for paths.. best use a single directory.

-V is your target directory....for stashing and if you have a few HDD lying about..then U may consider resilient drive in Windows 10...

I did an article in this before... short one...

So this is how U combine Drives...

The URL in the blog "fun and profit" is coming to 404

https://storj.io/blog/2019/01/sharing-storage-space-for-fun-and-profit/
The link check out the blog post on how node churn impacts network performance and file durability doesn't work, it shows 404
The right link is https://storj.io/blog/2019/01/why-proof-of-replication-is-bad-for-decentralized-storage-part-2-churn-and-burn/

Change project

How to tell uplink which project it should use?

I have created new project with new bucket and new access grant but

uplink ls

stil shows only buckets from another project

Why it take so long to create key with a minimum a difficulty of 30

My laptop take about 6-8 hours to generate the key, please help me understand why?
Thank you.

Cannot initialise database: invalid argument

Hi folks,

I am starting the storagenode for the fisrt time, but the init fails with the following message:

Fatal error: Error starting master database on storagenode: boltdb error: invalid argument

Docker says that the image is in sync with latest version, so I am raising this issue to follow the case.

Getting error at the last step

I am getting the following error when i am trying to start the node using that big last command. I am running windows 10

At line:9 char:3

• --name storagenode storjlabs/storagenode:alpha
• ~
  Missing expression after unary operator '--'.
  At line:9 char:3
• --name storagenode storjlabs/storagenode:alpha

Unexpected token 'name' in expression or statement.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : MissingExpressionAfterOperator

Easy way to see storage node reputation

Is there an easy way to see what my storage node's reputation is?

Clarify that ISP with NAT cannot do port forwarding

Besides issuing dynamic IP address, more and more ISPs are also using NAT, which prevents a trivial port forwarding by configurating the home gateway.

I suggest to integrate the PIA article (https://support.storj.io/hc/en-us/articles/360026892971-Running-a-V3-Storage-Node-with-PIA-VPN-) to the networking section of the prerequisites (https://documentation.storj.io/before-you-begin/prerequisites#internet-connection) and port forwarding page (https://documentation.storj.io/dependencies/port-forwarding).

Node Not found message in log

Is this an issue with my node? Had this before the incompatibility update too...

2019-04-02T17:52:58.870975259Z 2019-04-02T17:52:58.870Z WARN agreements could not find satellite {"satellite id": "xxx", "error": "node not found", "errorVerbose": "node not found\n\tstorj.io/storj/pkg/kademlia.(*Kademlia).lookup:299\n\tstorj.io/storj/pkg/kademlia.(*Kademlia).FindNode:256\n\tstorj.io/storj/pkg/piecestore/psserver/agreementsender.(*AgreementSender).SettleAgreements:88\n\tstorj.io/storj/pkg/piecestore/psserver/agreementsender.(*AgreementSender).Run.func1:65\n\tstorj.io/storj/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1:57"}

Node invitation

This ticket was escalated from Zendesk by Helene Unland:

Ticket: #16054
Type: Ticket
Priority: -
Requester: Raymholt
Organization: -
Assignee: Helene Unland

---

SNO docs section https://docs.storj.io/node/dependencies/identity/#authorize-the-identity needs to be updated.

We no longer send emails when an auth token is requested from https://www.storj.io/host-a-node - The SNO needs to copy the auth token directly from the screen instead.

gz#16054

Windows binary availability

Could you please update this ticket once there's a Windows binary available.

ERROR untrusted: unable to get signee: context canceled

This error message showed again and again
I've set up the node for 5 hours now(using identity generated on other machine), and getting this for ~ twenty times.

storj.io/storj/storagenode/piecestore.(*Endpoint).VerifyOrderLimitSignature:127
storj.io/storj/storagenode/piecestore.(*Endpoint).VerifyOrderLimit:68
storj.io/storj/storagenode/piecestore.(*Endpoint).Upload:138
storj.io/storj/pkg/pb._Piecestore_Upload_Handler:602
storj.io/storj/pkg/server.streamInterceptor:19
google.golang.org/grpc.(*Server).processStreamingRPC:1182
google.golang.org/grpc.(*Server).handleStream:1254
google.golang.org/grpc.(*Server).serveStreams.func1.1:690
2019-04-06T13:12:09.650Z ERROR untrusted: unable to get signee: context canceled

Bring back original title for FileZilla Pro 'How to' Doc

Hi there -

For some reason (?) the headline of this doc got changed when posted. Please update the title to be the title in the doc:

How to Integrate Filezilla Pro with Storj DCS to Easily Find, Transfer and Download All Of Your Files

doc here for reference - https://docs.google.com/document/d/1Dqf1y3zk3P3KRcxZ4OJfANTQtvdwkKQ0-QC-BI62dQ8/edit

Thank you!

Find an alternative for GitBook

GitBook was a perfect solution until they pushed a new version in October 2021 and broke everything - from editor and history to how unstable it works now, especially search.
They working hard to broke remaining.

The "Storage Sharing Terms and Conditions" has a wrong URL in the blog "fun and profit"

https://storj.io/blog/2019/01/sharing-storage-space-for-fun-and-profit/
The link Storage Sharing Terms and Conditions is coming to Terms of Service for the Storj network.
The "Storage Sharing Terms and Conditions" should have a different URL: https://storj.io/storj-share-terms/

How to Integrate FileZilla Pro with Storj DCS

We have a new 'how to' for our docs: How to Integrate FileZilla Pro with Storj DCS to Easily Find, Transfer and Download All of Your Files. You can use this shorter version for a title 'How to Integrate FileZilla Pro with Storj DCS GatewayMT

Please post this new doc right after the 'FileZilla Native Integration'. Can you change the current one (title) to (so they are consistent) How to Integrate FileZilla Standard with Storj DCS Native CLI

Here's the doc:

https://docs.google.com/document/d/1Dqf1y3zk3P3KRcxZ4OJfANTQtvdwkKQ0-QC-BI62dQ8/edit

Here's the folder with all the screen shots:

https://drive.google.com/drive/folders/19FBy6ylkbwBUjQ40loF8OR0W9_r86fDG

Thanks all!

Error message about Piecestore protocol

These two error message about Piecestore protocol error showed every several seconds
And the disk usage accumulated to 90gb in the past day.
Not sure is it's a issue or not.
I tried to join the rocket chat but still not getting approved by the admin after 5 days of registration.

Add --stop-timeout to docker run command

Instead of putting in the FAQ that we should use -t 300 with docker stop to take down the node for maintenance you should set --stop-timeout in the docker run command so docker takes care of it for you. This also allows tools like watchtower to follow this same procedure.

Replace copy paste of s3-compatibility with direct link to GitHub

https://docs.storj.io/dcs/api-reference/s3-compatible-gateway/supported-s3-commands/ have a limited content from
https://github.com/storj/gateway-st/blob/main/docs/s3-compatibility.md and it's outdated.

We need to link directly to the documentation in the repo.

Should I aws-sdk go documentation?

New! How to simplify file management with S3 Browser & Storj DCS

Hi all! Here's our third in a 3-part series about the best file management tools to use with Storj DCS. This one is for using S3 + our browser to organize, transfer and manage files in a super easy way.

The Google Doc for this is here: https://docs.google.com/document/d/1xlCKnGqB7Bfeh7N8yQAu86dbDR9tNxRr0WPKl-OuLKw/edit

The screen shots that are referenced in the doc are all included in this folder: https://drive.google.com/drive/folders/1YAXh85Bu6a80QdzGYA5oeL0fjduRjmhM

Let me know if there are any questions and/or concerns. We'll be able to post a couple of cool new blogs (top 3 file management tools; top 3 backup tools) once this is posted.

As always, I so appreciate y'all for reviewing these and getting them posted.

Typo in commands to download and install uplink

The filenames for downloading and unzipping do not match and this means it cannot be just copypasted. I highlighted it in red rectangles on the screenshots below. Can we edit it?
Link to the docs: https://docs.storj.io/dcs/downloads/download-uplink-cli/

Storage node dashboard impossible to contact , says offline

`Storage Node Dashboard ( Node Version: v0.12.2 )

======================

ID 12J3SUuX2bffprM3e5vRZ9Huktv9QLXW1YeV7bpikY2CfpgZjLx
Last Contact OFFLINE
Uptime 1m15s

               Available       Used     Egress     Ingress
 Bandwidth        5.0 TB     9.8 GB     1.0 GB      8.8 GB (since Jun 1)
      Disk        1.8 TB     2.3 GB

Bootstrap bootstrap.storj.io:8888
Internal 127.0.0.1:7778
External this.is.my.ip:28967

Neighborhood Size 139`
Here is what the storage node dashboard says
Running on Raspberry pi

unsupported protocol scheme

I'm getting the following errors when trying to create and authorize my v3 identity:

$ ./identity_linux_arm create storagenode 2019-05-03T23:37:48.094-0500 ERROR Failed to do periodic version check: %!(EXTRA *url.Error=Get : unsupported protocol scheme "")

It's non-fatal and I'm able to get an identity, from what I can tell.

When I go to authorize I get the same error as well as a no such host though I can do a DNS lookup just fine.

$ ./identity_linux_arm authorize storagenode TOKEN REMOVED 2019-05-03T23:53:03.265-0500 ERROR Failed to do periodic version check: %!(EXTRA *url.Error=Get : unsupported protocol scheme "") Fatal error: error occurred while signing certificate: transport error: connection error: desc = "transport: error while dialing: dial tcp: lookup certs.alpha.storj.io: no such host" (identity files were still generated and saved, if you try again existing files will be loaded)

This is on an odroid XU4 running an up-to-date armbian stretch.

The instruction is redirected to the wrong address

https://storj-labs.gitbook.io/docs/ is redirected to the https://docs.storj.io/docs where is only documentation for the v2

Ticket #15548. Missing mandatory `--not-after=` option in `uplink share`

page at https://docs.storj.io/dcs/getting-started/quickstart-uplink-cli/uploading-your-first-object/view-distribution-of-an-object/ needs to be updated. The share command requires an additional command line parameter --not-after=

Script links in payout dispute form go to main page

At the payout dispute form there are 2 scripts linked for checking stats locally, however both links redirect back to https://documentation.storj.io/

The right links:

• Script for checking contracts and GBh
• Check node(s) v2 upload and downloads

HOT! New 'How To' for new Storj IPFS Pinning Service Beta

@heunland @AlexeyALeonov hey there! Here is that very 'hot' (off the press!) How To document to support our IPFS Pinning launch tomorrow. It's been approved by all on the product team - now for your eyes and for publishing.

So sorry for the late notice but this was put together super fast to help stop lots of questions/tickets/etc with our launch tomorrow.

https://docs.google.com/document/d/1gh5EbPst_OQlGVd4nGAY4pin8R4dh8VA8TnPK9dvan0/edit#

As soon as it's posted, we'll add the link to the blog and some other comms. The Press Release for this launch goes live tomorrow at 8 am ET. THANK YOU x 1,000,000.!

Cyberduck integration

Identity Tool is unable to authorize identity due to not accepted certificate

Resulting issue from resolved previous issue #51

./identity_linux_amd64 authorize storagenode [email protected]:XXXX --log.level debug
2019-07-24T19:39:58.140Z DEBUG debug server listening on 127.0.0.1:42697
2019-07-24T19:39:58.615Z DEBUG allowed minimum version from control server is: v0.15.1
2019-07-24T19:39:58.615Z INFO running on version v0.15.4
2019-07-24T19:39:58.866Z FATAL error occurred while signing certificate: transport error: connection error: desc = "transport: authentication handshake failed: tls peer certificate verification error: not signed by any CA in the whitelist: CA cert"
(identity files were still generated and saved, if you try again existing files will be loaded)

Workaround

Setting --signer.tls.use-peer-ca-whitelist=false solves the issue.

Improve "Host a Node Signup" label in SNO docs

Sometimes users who want to become SNOs accidentally sign up for a Storj DCS account which we then have to delete. I believe this is because the very first link on the SNO documentation just says "Sign up" and sends them to https://storj.io/node where they see a button at top right called "Start for free" which is not the button they should push. Instead they should be clicking on "Host a Node" So I am proposing to change the label on the link in the SNO docs to be less confusing.

https://app.gitbook.com/o/-Lij5OMtVUHTiJ2J0i52/s/-LiiXe2K6q6NwWAdegg5/~/changes/jRUWu1MJQUTqWPsaWeEC/

Note v0.22.1 shows offline

It was connectet but no data was transferred. Today if checked again and now it says last contact offlinel. If you need more infos pls let me know.

Limit watchtower to monitor Storj container only

When running watchtower as suggested in docs, it monitors (and tries to update) all docker containers running on the host.
I suggest that this be limited to the storagenode container only.
According to https://hub.docker.com/r/containrrr/watchtower, this can be done by passing an argument to the watchtower container with the name(s)of the container to monitor.

So basically all we need is to update the docs accordingly.
i.e.
docker run -d --restart=always --name watchtower -v /var/run/docker.sock:/var/run/docker.sock storjlabs/watchtower storagenode <-- added the name of the container to monitor.

ERROR running on not allowed/outdated version v0.10.1

After I start the container and display the dashboard

filip@filip-pc:~$ docker exec -it storagenode /app/dashboard.sh
2019-05-09T20:04:00.802Z INFO Configuration loaded from: /app/config/config.yaml
2019-05-09T20:04:00.811Z INFO Node ID: 129V9YdhUSoKn3PRMDYWxmfS3kXm6jEtRpLT3io9Rpgo2t8fhHY

filip@filip-pc:~$ docker logs -t storagenode
2019-05-09T19:53:28.484181218Z 2019-05-09T19:53:28.484Z	INFO	Configuration loaded from: /app/config/config.yaml
2019-05-09T19:53:28.493262235Z 2019-05-09T19:53:28.493Z	INFO	Operator email: <my-mail>
2019-05-09T19:53:28.493317149Z 2019-05-09T19:53:28.493Z	INFO	Operator wallet: <my-wallet>
2019-05-09T19:53:44.045493174Z 2019-05-09T19:53:44.045Z	ERROR	running on not allowed/outdated version v0.10.1

Remove Gateway MT from mention on hosting a static website

https://docs.storj.io/dcs/how-tos/host-a-static-website/

We recommend using the Uplink CLI with Gateway MT but you may also use [...]

uplink share --url doesn't really do anything useful for the Gateway MT despite creating an access key. That access key is marked public so the intention is for it to work with linksharing.