The linksharing method presentWithProject is all of the logic around requesting data from Satellites/Storage nodes. https://github.com/storj/linksharing/blob/main/sharing/present.go#L50
Here is how it works:
-
If the path requested is empty or ends in a slash, it starts a goroutine to check if index.html exists. This is a round trip to the satellite (StatObject).
-
In parallel, it checks to see if the object requested exists (another round trip to the satellite (StatObject)). If it exists, it calls showObject (discussed below). If it doesn’t exist, it determines if the path requested is a prefix, in which case it redirects to have a slash. This does another round of round trips. We can ignore the isPrefix redirection for the purposes of this ticket.
-
If index.html exists, it calls showObject.
-
If it ends with a slash, it otherwise lists objects.
So, so far, we have done most likely one, potentially two round trips to the Satellite in parallel (StatObject), and in the case of not finding anything without a trailing slash, we may have done a bunch of sequential round trips to determine if the path is a prefix.
showObject then does yet another set of round trips! It calls httpranger.ServeContent with an objectranger. httpranger.ServeContent will parse the byte range requests and so on. This is a really complicated method, to the point that JT has been trying to break it apart into a more modular codebase here: https://github.com/jtolio/servecontent. Once it has parsed byte ranges, then it will go and make another round trip to the Satellite to get order limits.
Let’s talk about a specific example for the purpose of this ticket.
Let’s say the request is to load https://link.us1.storjshare.io/raw//roms/android/my-android.rom
This path does not end in a slash, so we’re going to do one StatObject request to find out if the object exists. Let’s say it does. Then we’re going to do one DownloadObject request (inside the objectranger) to get order limits. So this request is a best case scenario and still has an unnecessary round trip. We could have just done DownloadObject.
To fix this, we will need to explode the ServeContent method so we can do the byte range processing up front. This is a really big refactor, so I propose using https://github.com/jtolio/servecontent as a starting point.
We probably need some test cases to preserve the existing semantic behavior.
Specifically:
-
if there is an object that ends with a slash in its name, that currently is served and is not assumed to be a prefix.
-
if a path is requested that ends with a slash or is the empty key (special case), but no object exists with that exact name (ending with a slash), we first check to see if an index.html exists. if it does we serve that. if it doesn’t, we try to do a prefix listing if we have the listing permission.
-
if a path is requested that does not end in a slash but that object doesn’t exist, we assume we don’t have the listing permission and check to see if an object with a key addition of “/index.html” exists. if +“/index.html” doesn’t exist, we try and do a prefix listing. either way, if we confirm that the path is a prefix, we do a redirect with a trailing slash. if we can’t confirm the path is a prefix, we either return an access denied (no listing permission) or a 404
-
otherwise we serve correctly named objects
in all object serving cases, we do our best to serve with