GithubHelp home page GithubHelp logo

topperdel / uplink.net Goto Github PK

View Code? Open in Web Editor NEW
25.0 3.0 4.0 660.52 MB

A .Net-wrapper to connect to the storj v3 network.

License: MIT License

C# 97.32% Go 0.96% Batchfile 1.00% SWIG 0.72%
storj uplink tardigrade net

uplink.net's Introduction

uplink.NET

Codacy Badge Storj.io

A .Net/C#-wrapper for Storj (v3)

This library enables you to connect to the Storj network to upload and retrieve files to the distributed and secure cloud-storage. It is based on uplink-c, the C-bindings provided from Storj-labs to connect to the Storj network.

If you want to help out, check the open issues, create some or open a PR.

Use

Install uplink.NET from NuGet.

Also install the applicable platform-specific library:

See the wiki for details how to use the library.

You may also try the included sample-app for UWP and Android. There you should find additional details on how to use the library.

Build (Windows and Android)

Let's build the Windows- and Android-binaries.

Prerequisits

Building everything by yourself is possible on Windows. You'll need the following tools:

  • Git (to get the uplinkc-repository)
  • SWIG (to generate the C#-proxy)
  • Go (to build uplink-c)
  • MSYS2 (used by Go to compile a Windows-DLL) => see here for installation instructions
  • Visual Studio 2017 or higher to compile the uplink.NET-library
  • Android NDK (to build the Android-Release) => install Xamarin with your Visual Studio, start VS, go to Extensions => Android => SDA Manager, Choose Tools and install NDK. Check that your environment variables include "$ANDROID_HOME" and that the path contains no spaces. If it got installed in "C:\Program files (x86)" (default) change the path in the variable to "C:\PROGRA~2...". Otherwise you'll get an "exec: could not find file"-error during build.

Noob-Disclaimer

The process described here is the way I was successfull so far. It might not be the simplest way or you might be successfull using different compilers, versions, systems and whatever. I'm open to any adjustments, enhancements or proposials. Just let me know!

Building-steps

First you need to pull the latest version of this repository. Open a command prompt, choose a suitable working directory and enter:

git clone https://github.com/topperdel/uplink.net.git

If you want you can adjust the uplinkc-Version to use (the github-tag) by setting the STORJ_VERSION-Parameter within build.bat to the one to use.

Then do

cd \uplink.net
build

This will start the build-process. During the build it will clone the uplinkc-repository into the above chosen working directory.

Once finished it will open the folder "Build-results" within the explorer and it should contain a storj_uplink.dll (Windows x64), a storj_uplink-x86.dll, a "cs-Files"-Folder and an "Android"-folder (containing the so-files for android with the correct ABI-lib-path).

The files already got copied to the correct locations for the Visual Studio solution.

Build the solution.

Feel good.

Build (Linux) with WSL

Let's build the Linux-binaries.

Prerequisits

Building the linux .so-file on Windows is possible with Windows Subsystem for Linux (WSL). Currently you need four files from the build-process above for Windows and Android. They are available in my fork of the uplink-c-repo.

First of all make sure that git and go are installed on WSL. Starting with a fresh Debian, this would basically be the list of commands (according to these instructions):

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install wget
wget https://dl.google.com/go/go1.14.2.linux-amd64.tar.gz (you might find a more current version, but the golang-go-package is NOT working)
sudo tar -xvf go1.14.2.linux-amd64.tar.gz (adjust the version to the one downloaded in the step before)
sudo mv go /usr/local
sudo nano ~/.bashrc

Scroll down and add these to your .bashrc profile:

export GOROOT=/usr/local/go
export GOPATH=$HOME/go
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH

Save and close the file with "Ctrl + o" and "Ctrl + x". Then update the current session:

source ~/.bashrc

Then install git:

sudo apt-get install git

Building-steps

Clone and build the forked uplink-c-repo:

git clone --branch v1.0.2 https://github.com/topperdel/uplink-c.git
cd uplink-c

Now build the linux .so-file like this:

sh build-linux.sh

The following files are included in the fork of uplink-c. If you needed to change those files you need to update them before calling build-linux.sh. Just for your convenience calling explorer on the current WSL-folder is as easy as this:

explorer.exe .

These files are necessary for building linux (and MacOS/iOS; see above):

  • storj_uplink_second_wrap.c
  • storj_uplink.h
  • custom_metadata_helper.go
  • restrict_scope_helper.go

Copy the generated storj_uplink.so to the runtimes/linux-x64/native-folder under uplink.Net.

Build (MacOS)

For this task you need a Mac (oh Apple, you drive me nuts). The procedure is basically the same as for linux above:

  • install go
  • install git
  • clone the forked uplink-c-repo
  • run the build-command:
sh build-macos-ios.sh

Place the resulting storj_uplink.dylib from the build/macos-folder to the runtimes/osx-x64/native-folder under uplink.Net and also to libs/mac/ renaming it to "libstorj.dylib". Place the resulting libstorj_uplink.dylib from the build-root-folder to the runtimes/ios/native- AND the libs/ios-folder under uplink.Net.

Testing

To run the test within the VS-solution you have to set the VALID_API_KEY within TestConstants.cs of the test-project to a valid API-key. If you use a different satellite, change that address, too. Storj provides a local test-net you can spin up very quickly that would be possible to use, too.

uplink.net's People

Contributors

codacy-badger avatar erikvv avatar topperdel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

uplink.net's Issues

Win-Arm64 not packaged correctly in Nuspec

Describe the bug
The nupkg from Nuget installs the x64 binary on arm64, using latest 2.12.3363.

To Reproduce
Steps to reproduce the behavior:

  1. dotnet new console
  2. dotnet add package uplink.NET.Win
  3. dotnet add package uplink.NET.Linux
  4. dotnet build -r win-arm64
  5. dotnet build -r linux-x64
  6. In the publish folder, the Windows x64 storj_uplink.dll is installed for both win-arm64 and linux-x64.

Expected behavior
The package should only install the relevant files.
The same problem occurs when including the Windows+Linux package and targeting Linux.

Additional context
Looking inside the nupkg, it has the correct files, but the build process appears to be picking up the wrong version for some reason.

Console session showing the included files:

> shasum uplink.net.win.2.12.3363.nupkg/runtimes/win-x64/native/storj_uplink.dll
8c9f069cc657f16787347d1f916a67f756aedbd9  uplink.net.win.2.12.3363.nupkg/runtimes/win-x64/native/storj_uplink.dll

> shasum uplink.net.win.2.12.3363.nupkg/runtimes/win-x86/native/storj_uplink.dll
ba10f3492ea1e5c838a08a129fc951e3d5060716  uplink.net.win.2.12.3363.nupkg/runtimes/win-x86/native/storj_uplink.dll

> shasum uplink.net.win.2.12.3363.nupkg/runtimes/win-arm64/native/storj_uplink.dll
6ff71d75099f12ebf98c8ff5383f1af5fd0e5e13  uplink.net.win.2.12.3363.nupkg/runtimes/win-arm64/native/storj_uplink.dll

> shasum test-project/publish/linux-arm64-gui/storj_uplink.dll
8c9f069cc657f16787347d1f916a67f756aedbd9  test-project/linux-arm64-gui/storj_uplink.dll

> shasum test-project/publish/linux-x64-gui/storj_uplink.dll
8c9f069cc657f16787347d1f916a67f756aedbd9  test-project/linux-x64-gui/storj_uplink.dll

> shasum test-project/publish/linux-x64-gui/storj_uplink.so
3998df8c86149d31d249728d85f3a26b4d118fac  test-project/linux-x64-gui/storj_uplink.so

> shasum test-project/publish/win-arm64-gui/storj_uplink.dll
8c9f069cc657f16787347d1f916a67f756aedbd9  test-project/win-arm64-gui/storj_uplink.dll

> file test-project/publish/win-arm64-gui/storj_uplink.dll
test-project/win-arm64-gui/storj_uplink.dll: PE32+ executable (DLL) (console) x86-64 (stripped to external PDB), for MS Windows

> file test-project/publish/win-x64-gui/storj_uplink.dll
test-project/win-arm64-gui/storj_uplink.dll: PE32+ executable (DLL) (console) x86-64 (stripped to external PDB), for MS Windows

> file uplink.net.win.2.12.3363.nupkg/runtimes/win-arm64/native/storj_uplink.dll
uplink.net.win.2.12.3363.nupkg/runtimes/win-arm64/native/storj_uplink.dll: PE32+ executable (DLL) (GUI) Aarch64, for MS Windows

> ls test-project/linux-x64-gui/storj_uplink.*
test-project/linux-x64-gui/storj_uplink.dll	
test-project/linux-x64-gui/storj_uplink.so

> file test-project/publish/win-x64-gui/storj_uplink.dll
test-project/win-arm64-gui/storj_uplink.dll: PE32+ executable (DLL) (console) x86-64 (stripped to external PDB), for MS Windows

> file test-project/publish/win-x64-gui/storj_uplink.so
test-project/win-arm64-gui/storj_uplink.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=551f4f30f0fe83233ff6da071cfa448ef246ae6e, with debug_info, not stripped

My project file:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="uplink.NET.Linux" Version="2.12.3365" />
    <PackageReference Include="uplink.NET.Win" Version="2.12.3363" />
  </ItemGroup>

</Project>

I see a warning during build:

/Users/kenneth/Downloads/test-storj/test-storj.csproj : warning NU1701: Package 'uplink.NET.Win 2.12.3363' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8, .NETFramework,Version=v4.8.1' instead of the project target framework 'net8.0'. This package may not be fully compatible with your project.

I am guessing that it somehow triggers the target file which overrides the normal target selection.

Handle BucketList/ObjectList correctly with SWIG

Expected Behavior

The SWIG-Proxy should handle the items-pointer correctly as Array.

Actual Behavior

There are inline-function-declarations in storj_uplink_second.i called get_bucketinfo_at and get_objectinfo_at to retrieve Elements from C by index.

Help wanted

I did not get SWIG to handle this correctly, but I'm pretty sure that it is possible. Any help is greatly appreciated! This would help a little with performance fetching bucket-Contents.

Samples-app currently not up-to-date

Describe the bug
The samples-app included in this repo is out of date and Needs to be updated for the current Uplink.NET-Version. Should be an easy one, as it's mainly a renaming-thing.

Error WebApi C# Project - HRESULT: 0x8007000B

Can't use Uplink lib on WebApi C# Project Net 4.7

System.TypeInitializationException: 'L'inizializzatore di tipo di 'uplink.SWIG.storj_uplinkPINVOKE' ha generato un'eccezione.'
BadImageFormatException: Tentativo di caricare un programma con un formato non corretto. (Eccezione da HRESULT: 0x8007000B)
at uplink.SWIG.storj_uplinkPINVOKE.new_UplinkConfig()
at uplink.NET.Models.Uplink..ctor(UplinkConfig uplinkConfig, String tempDir)
at uplink.NET.Models.Scope.Init(UplinkConfig uplinkConfig)
at uplink.NET.Models.Scope..ctor(String apiKey, String satelliteAddress, String secret, UplinkConfig uplinkConfig)
at dev.loady.storj.Program.d__5.MoveNext()

The project is running on x64, and can't be modified. I think this is the problem.

Screenshot_10

Error when using API Key

I am trying to sync to storj via duplicati. When it is set to API key i get the following error when connecting.

System.ArgumentException: uplink: node id is required in satelliteNodeURL
storj.io/uplink.parseNodeURL:212
storj.io/uplink.Config.requestAccessWithPassphraseAndConcurrency:161
storj.io/uplink.Config.RequestAccessWithPassphrase:148
main.uplink_config_request_access_with_passphrase:38
main._cgoexpwrap_9685c5e68192_uplink_config_request_access_with_passphrase:765
runtime.cgocallbackg1:332
runtime.cgocallbackg:207
runtime.cgocallback_gofunc:794
at uplink.NET.Models.Access..ctor (System.String satelliteAddress, System.String apiKey, System.String secret, uplink.NET.Models.Config config) [0x000b5] in <782b73e966c3415bb5c6a264ad83577c>:0
at Duplicati.Library.Backend.Storj.Storj..ctor (System.String url, System.Collections.Generic.Dictionary2[TKey,TValue] options) [0x0012b] in <44365ce81c464c899da132942d0018f5>:0 at (wrapper managed-to-native) System.Reflection.RuntimeConstructorInfo.InternalInvoke(System.Reflection.RuntimeConstructorInfo,object,object[],System.Exception&) at System.Reflection.RuntimeConstructorInfo.InternalInvoke (System.Object obj, System.Object[] parameters, System.Boolean wrapExceptions) [0x00008] in <12b418a7818c4ca0893feeaaf67f1e7f>:0 --- End of stack trace from previous location where exception was thrown --- at Duplicati.Library.DynamicLoader.BackendLoader+BackendLoaderSub.GetBackend (System.String url, System.Collections.Generic.Dictionary2[TKey,TValue] options) [0x0017e] in <0f2eba409abd4b7ba27aae5e0bea8106>:0
at Duplicati.Library.DynamicLoader.BackendLoader.GetBackend (System.String url, System.Collections.Generic.Dictionary`2[TKey,TValue] options) [0x00000] in <0f2eba409abd4b7ba27aae5e0bea8106>:0
at Duplicati.Server.WebServer.RESTMethods.RemoteOperation.TestConnection (System.String url, Duplicati.Server.WebServer.RESTMethods.RequestInfo info) [0x000b0] in <59054a017605435993aba9f724246795>:0
at Duplicati.Server.WebServer.RESTMethods.RemoteOperation.POST (System.String key, Duplicati.Server.WebServer.RESTMethods.RequestInfo info) [0x00091] in <59054a017605435993aba9f724246795>:0
at Duplicati.Server.WebServer.RESTHandler.DoProcess (Duplicati.Server.WebServer.RESTMethods.RequestInfo info, System.String method, System.String module, System.String key) [0x00289] in <59054a017605435993aba9f724246795>:0

Handle project_salted_key_from_passphrase correctly with SWIG

Expected Behavior

The call to project_salted_key_from_passphrase returns a byte[] (correctly) and new_encryption_access_with_default_key expects a byte[].

Actual Behavior

The call to project_salted_key_from_passphrase returns a SWIGTYPE_p_uint8_t (wrong) Which can't be passed to new_encryption_access_with_default_key as it expects a byte[]. Therefore I've included a new_encryption_access_with_default_key2-inline-function to storj_uplink_second.i. It internally calls new_encryption_access_with_default_key but expects a uint8_t as import-parameter.

Help wanted

If someone knows how to let SWIG return a byte[] on new_encryption_access_with_default_key the inline-function in the interface-file could be removed.

Build iOS-version

It would be nice to have an iOs-binary of Uplink.NET available, too. If it is available then the sample-app should be enhanced, too.

Build 32bit-versions

Currently there is no x86/32bit version available. Would be nice to support that Architecture, too.

The nuget does not correctly deploy the binaries in all cases

Describe the bug
The current nuget seems to not always deploy the binaries correctly. There are currently differen storj_uplink-binaries fpr Windows, Android and Linux. They also need to be deployed differently with NetFramework and NetCore.

Expected behavior
The nuget needs to be setup correctly so that the nuget is only available in projects, where we have binaries for (e.g. iOs does not yet have binaries) and that the binaries are deployed so they can be found during runtime.

Add .NET 6.0 as target for the lib

Is your feature request related to a problem? Please describe.
Avoiding the warning about "using this is for .NET Framwork" message.

Describe the solution you'd like
Add .net-6.0-linux and net6.0-win to each of the os lib

Describe alternatives you've considered
Ignoring the warning when on a .NET 6.0 project

Additional context
I'm wondering what is the reason for using .NET Framework for the lib

Library not loaded: storj_uplink-simx64.dylib

Describe the bug
When debugging the iOS app, it crashes at startup and emits the following error: Library not loaded: storj_uplink-simx64.dylib

To Reproduce
Steps to reproduce the behavior:

  1. Create a Xamarin.iOS app
  2. Install the nuget packages
  3. Add the uplink.NET.Models.Access.Init_iOs(Foundation.NSBundle.MainBundle.BundlePath); at app's startup
  4. Debug the app
  5. See error

Expected behavior
The app should be able to locate the library and run.

Screenshots
If applicable, add screenshots to help explain your problem.

Smartphone (please complete the following information):

  • Device: Simulator
  • OS: iOS

Additional context
Add any other context about the problem here.

Listing Objects does not return Items

Describe the bug
Listing objects inside a bucket does not return Items

To Reproduce

  1. Get a bucket by calling var bucket = await bucketService.GetBucketAsync("demo-bucket").ConfigureAwait(false);
  2. List items by calling var items = await objectService.ListObjectsAsync(bucket, new ListObjectsOptions());
  3. items.Items is empty

Expected behavior
Listing items should return the items inside the bucket

List nested files without initial recursion

First of all thank you for your work, it is a great help to me to develop my graphical interface.

I can't figure out how to retrieve the list of files in a "sub-folder" (object whose IsPrefix property is set to True) other than by retrieving the list of objects in a bucket recursively, which is not suitable for use on a large amount of files.

Add pagination to ListObjects()

The current implementation loads all objects. It fetches 1000 per page but does not stop at one page. This might not always be the best as it makes ListObjects slower and produces higher traffic.

It would be better to add an overload that takes a "Cursor"-value and a max entries parameter.

Handle restrict_scope correctly

Expected Behavior

The SWIG-Proxy should handle the restrict_scope-function correctly.

Actual Behavior

I could not wrap EncryptionRestrictions correctly with SWIG, so I had to add the file "restrict_scope_helper.go" to overcome this issue.

Help wanted

I did not get SWIG to handle this correctly, but I'm pretty sure that it is possible. Any help is greatly appreciated!

Having issue with uploading large file.

Facing difficulty uploading large file. I have tried the UploadObjectChunkedAsync and the UploadObjectAsync method. I still can't make it work.

I have checked the wiki documentation, no documentation for such.

Is there a reference to any documentation on how this works?

Language: ASP.NET CORE.

Make SWIG-Models only internally visible

Expected Behavior

All SWIG-Models and -classes are not visible from outside. They are not necessary to use the library.

Actual Behavior

Alls SWIG-Models and -classes are visible.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.