topac / rmega Goto Github PK

Im a total noob with ruby, can you tell me how to do the DSL thing? I just want to store my account. Thanks

I noticed rmega doesn't have a retry attempt feature. Any network interruption causes download to stop.
I'm on one of the worst networks on earth, and any network-related interruptions causes the the error: ERROR: Unexpected data length to be thrown.

A way to specify retry attempts will be priceless, and I'll be the first to set mine at 2M retries!

This function is very slow and uses excessive amounts of CPU:

This is a pragmatic issue for me. I've been downloading a ~50G directory over several invocations, and each time I start rmega-dl, it takes a long time to verify the already-downloaded portion and makes my fan turn on. As the function is currently implemented, it only verifies about 30M per second.

The issue appears to be the fact that it processes the data 16 bytes at a time. This probably makes sense in a language like C, but in Ruby, a tight loop like this has a ton of overhead. I tried modifying this function to pass the data in all at once and then just extract the last 16 bytes, and not only did it approximately triple the speed of verification, it also didn't make my fan turn on. I don't think the specific modification I made is necessarily the best solution, but it does prove that the current one is suboptimal.

After v0.1.6 it is possible to read name of files that shared from others to you.

But it is still not possible to decrypt the name of nodes that shared by you to other guys. (for public and private shares)

storage = Rmega.login('[email protected]', 'xxx')

C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rmega-0.1.2/lib/rmega/session.rb:61:in 'request': Error -9: Object (typically, node or user) not found (Rmega::RequestError)         from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rmega-0.1.2/lib/rmega/session.rb:35:in 'login'         from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rmega-0.1.2/lib/rmega/session.rb:20:in 'initialize'         from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rmega-0.1.2/lib/rmega/session.rb:8:in 'new'         from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rmega-0.1.2/lib/rmega/session.rb:8:in 'login'         from C:/Users/xxx/Desktop/xxx.rb:2:in '<main>'
ruby 2.0.0p247 (x64-mingw32)
latest rmega

Hello, I'm trying to upload file. And I have issue with "exists?"

nikita@nikita-desktop:~/LineageOS> rmega-up.ruby3.2 out/target/product/j4lte/lineage-17.1-20230514-UNOFFICIAL-j4lte.zip -u [email protected]
/usr/lib64/ruby/gems/3.2.0/gems/rmega-0.3.1/lib/rmega/cli.rb:30:in `read_configuration_file': undefined method `exists?' for File:Class (NoMethodError)

        return {} unless File.exists?(configuration_filepath)
                             ^^^^^^^^
Did you mean?  exist?
	from /usr/lib64/ruby/gems/3.2.0/gems/rmega-0.3.1/lib/rmega/cli.rb:9:in `cli_options'
	from /usr/lib64/ruby/gems/3.2.0/gems/rmega-0.3.1/bin/rmega-up:11:in `<top (required)>'
	from /usr/bin/rmega-up.ruby3.2:25:in `load'
	from /usr/bin/rmega-up.ruby3.2:25:in `<main>'

How to solve it?
Thank you.

hello,
attempted some examples in the readme:

storage = Rmega.login('[email protected]', 'mypass')

storage.root.files.each { |file| puts file.name }
NoMethodError: undefined method `root' for #<Rmega::Storage:0x007fc3da597310>

storage.nodes
NoMethodError: undefined method `first' for -3:Fixnum
    from /Users/kain/.rvm/gems/ruby-2.1.1/gems/rmega-0.0.3/lib/rmega/session.rb:68:in `request'
    from /Users/kain/.rvm/gems/ruby-2.1.1/gems/rmega-0.0.3/lib/rmega/storage.rb:33:in `nodes'

Currently, for every call to http_get_content() a new connection is created. this is slow. very slow. 10Mbps on a 200Mbps network slow.

This also eliminates HTTP/2's throughput benefits for small requests (such as API requests interleaved with data)

A solution would be to modify Rmega::Net to a more object-oriented approach (or at least offer it) such that at least each file operates over a single connection. Possibly even going as far as one connection per host per thread.

A feature request. (I will look into it if I learn ruby quickly)

When having a list with links it would be nice that rmega tags completed links so it doesn't use them again, same with those that are dead. (Use again happens when a error occur and you don't know what link it was working with)
This could also be used when scraping websites, store in a file, tag those completed or faulty.

I'm testing the library in a threaded environment (sidekiq), I found an exception, will report more as I find them.

I'm not sure what caused this one, probably it's caused my multiple sessions being instanced between processes/threads.

WARN: undefined method `[]' for nil:NilClass
WARN: /Users/kain/.rvm/gems/ruby-2.1.2/bundler/gems/rmega-0839f38050a0/lib/rmega/crypto/crypto.rb:34:in `block in decrypt_rsa_privk'
/Users/kain/.rvm/gems/ruby-2.1.2/bundler/gems/rmega-0839f38050a0/lib/rmega/crypto/crypto.rb:33:in `times'
/Users/kain/.rvm/gems/ruby-2.1.2/bundler/gems/rmega-0839f38050a0/lib/rmega/crypto/crypto.rb:33:in `decrypt_rsa_privk'
/Users/kain/.rvm/gems/ruby-2.1.2/bundler/gems/rmega-0839f38050a0/lib/rmega/session.rb:44:in `login'
/Users/kain/.rvm/gems/ruby-2.1.2/bundler/gems/rmega-0839f38050a0/lib/rmega/session.rb:21:in `initialize'
/Users/kain/.rvm/gems/ruby-2.1.2/bundler/gems/rmega-0839f38050a0/lib/rmega/session.rb:8:in `new'
/Users/kain/.rvm/gems/ruby-2.1.2/bundler/gems/rmega-0839f38050a0/lib/rmega/session.rb:8:in `login'

I tried to use the "rmega-up" command to upload a test file called "a.pdf."
However, rmega keeps throwing up the error, "ERROR: end of file reached"
Here is the exact code I used:
"USER$ rmega-up a.pdf -u EMAIL_ID@gmail.com --pass"
(I also tried "a.pdf" with the quotes, didn't make a difference)
It logs into the account, and shows an initial upload phase where it says, "[Upload] 0.0 bytes of 737.46 kb, 0.0% @ 0.0 bytes/s, 4 threads", but then just throws up the error, "ERROR: end of file reached"

Kindly help me

Hello I would like to get some help
I never had contact Ruby, however I managed to install it in the 2.2.2 version on a VPS Centos 6.6 64bit.

The rmega-dl command worked perfectly.

However the rmega-up command returns me the following error:
Error: User email is required

Investigating a little more I noticed that one existed rmega_account.yml.example file.

I put my account details MEGA and rename it to rmega_account.yml.

Yet still get the error mentioned when I run rmega-up command

the I do to use only these two commands: download and upload files to the mega using ssh terminal?

Error when running rmega-dl even as root to download to external drive

[Allocate] 0.0 bytes of 8.71 GB, 0.0% @ 0.0 bytes/s, 4 threadsTraceback (most recent call last):
9: from /usr/local/bin/rmega-dl:23:in <main>' 8: from /usr/local/bin/rmega-dl:23:in load'
7: from /var/lib/gems/2.5.0/gems/rmega-0.2.4/bin/rmega-dl:32:in <top (required)>' 6: from /var/lib/gems/2.5.0/gems/rmega-0.2.4/lib/rmega/cli.rb:113:in cli_rescue'
5: from /var/lib/gems/2.5.0/gems/rmega-0.2.4/bin/rmega-dl:65:in block in <top (required)>' 4: from /var/lib/gems/2.5.0/gems/rmega-0.2.4/bin/rmega-dl:65:in each_with_index'
3: from /var/lib/gems/2.5.0/gems/rmega-0.2.4/bin/rmega-dl:65:in each' 2: from /var/lib/gems/2.5.0/gems/rmega-0.2.4/bin/rmega-dl:68:in block (2 levels) in <top (required)>'
1: from /var/lib/gems/2.5.0/gems/rmega-0.2.4/lib/rmega/nodes/downloadable.rb:76:in download' /var/lib/gems/2.5.0/gems/rmega-0.2.4/lib/rmega/nodes/downloadable.rb:12:in allocate': Unable to allocate space for file /media/ubuntu/hdd1/InstallESD.iso (RuntimeError)

I get this error when there is a space and or a : in the directory name.
Using rmega-dl.

Some companies will only use gems with a certain license.
The canonical and easy way to check is via the gemspec
via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

There is even a License Finder to help companies ensure all gems they use
meet their licensing needs. This tool depends on license information being available in the gemspec.
Including a license in your gemspec is a good practice, in any case.

How did I find you?

I'm using a script to collect stats on gems, originally looking for download data, but decided to collect licenses too,
and make issues for missing ones as a public service :)
https://gist.github.com/bf4/5952053#file-license_issue-rb-L13 So far it's going pretty well

Hi can you add code for download using PHP with resume support. There are codes already on github for mega download but none of them support resume.

Get this error when there is a @ in the filename when downloading. (In my case, in the beginning)

I'm not into maga.nz's api. Can't find method to rename file/node.
Any idea?

storage = Rmega.login('[email protected]', 'xxx')
JSON::ParserError: A JSON text must at least contain two octets!

ruby 2.0.0p247
Latest Rmega

I manage to download files with "rmega-dl" but can't sign in and download them.

I get an error: ERROR: undefined method folders' for nil:NilClass`

I use the code:
rmega-dl 'https://mega.nz/#F!LS***' -o /downloads/Mega/ -u ***@163.com --pass ***

I get the code working without the signing in part.
Can't figure out whats wrong.

Thanks for the contribution! 🙏

Are you planning to add a folder upload functionality? 🤔

using rmega-dl.

I encountered "ERROR: Unexpected Length". I figured it eventually out (Free Quote Exceeded), but the error message does not really indicated anything useful to the user.

Moreover, I have a pretty sizeable collection of links which is being downloaded - but as soons as one of the links does not work (after the 10 retries) (with the above mentioned error), rmega-dl stops. But links later in the list work - so, imo it would be much more useful to message the user that the download has failed, but then continue working on the rest of the list.

hello, is there any possibility you can add a web ui so we can
use it from mobiles.

Noticed that sometimes rmega-dl gives a checksum error on a file, if I run the tool again it does not give a error. Using 4 threads. (Noticed that using more threads can cause this more often)
Know this is difficult to troubleshoot but just wanted to put it out here.

Got this message when downloading from a link with a couple of files and folders.
The root directory has a space in it's name, maybe that can be a reason? (The root name is from mega so I can't change it)

You can add creation and removal of a link to file

here you can see the folder
https://mega.nz/#F!eds3WDpb!imq91_iprHOmEvc7shWZ_Q

but here link is encrypted and can only be download with JDownloader
https://mega.nz/#N!CA8SCYzI!vf_VIlNK6mAfsvt-wCcn2SXCN-zeJEZ23z5ysGkqNuI=###n=eds3WDpb

you can easily solve to download it from the terminal?

Download method not working as expected. I have called download method on storage object as follows:
@storage.download "https://mega.co.nz/#!op9jXaIQ!CgIXWBu35eFRmympa9Hk2GWqqa16H9v0ZxJ0hhwhJrc","/home/username/Downloads"

It is showing following output and hangs forever:
Download in progress 0.0bytes of 821.0bytes | 0.00% | ETA: ??:??:??

Hello,
is there a way to request a single file public url given a handle (or filename for instance) without listing nodes, folders, etc.?

I guess there is not, but consider a streaming endpoint, when a file gets requested in order to have a public_url I need to go through all nodes first in order to bring up a valid Node instance.

Probably there's no way around and I suspect that even a cache might not be a viable option due to all the session/crypto going around.

I'm caching the final public_url for at least 30 seconds to be resilient again multiple partial content requests, but what bothers me is that I need to query everything again for each new file:

    Rails.cache.fetch(media_file.cache_key, race_condition_ttl: 10, expires_in: 30.seconds) do
      service_response = trap_remote_call(:stream) do
        storage.nodes.lazy.select do |node|
          node.type == :file && node.name == media_file.remote_path # filename (not handle)
        end.to_a.first
      end
      service_response.public_url # this fires off a request(a: 'l', n: handle)

If I just do

require 'rmega'
storage = Rmega.login('valid_username', 'valid_password')

I get:

Traceback (most recent call last):
	5: from rmega.rb:2:in `<main>'
	4: from /home/sourav/.gem/ruby/2.7.0/gems/rmega-0.2.6/lib/rmega.rb:34:in `login'
	3: from /home/sourav/.gem/ruby/2.7.0/gems/rmega-0.2.6/lib/rmega/session.rb:94:in `login'
	2: from /home/sourav/.gem/ruby/2.7.0/gems/rmega-0.2.6/lib/rmega/session.rb:144:in `request'
	1: from /home/sourav/.gem/ruby/2.7.0/gems/rmega-0.2.6/lib/rmega/net.rb:7:in `survive'
/home/sourav/.gem/ruby/2.7.0/gems/rmega-0.2.6/lib/rmega/session.rb:150:in `block in request': The upload target URL you are trying to access has expired. Please request a fresh one. (Rmega::ServerError)

Error:
No such file or directory @ rb_file_s_size - C:/xampp/htdocs/descargas/file.rar

My code:
public_url = 'https://mega.nz/file/TctykRzL#p5xkGEjIz9VcegfbBh2maCD3JGKmsIdaOXahf_ZL-ZI' Rmega.download(public_url, Rails.root)

On some links people have uploaded file that are empty, for example some . files.
Is it possible to skip these empty files so the download can continue?

hello,
the mega documentation states that you need an api key to call the API, but this gem doesn't seem to provide a way to include it in requests; it's working though, so I don't understand if it's mandatory or not, do you have any more info about that?

about sessions: I saw other libraries (php, etc.) providing a way to represent and persist the session to be able to reconstruct it when needed. Do you have more information about mega sessions to share, how they roughly work, if they expire, etc.?

thanks

require "rmega"
storage = Rmega.login("[email protected]", "your_password")

result

Rmega::ServerError: Object (typically, node or user) not found
    from /cache/gems/rmega-0.2.2/lib/rmega/session.rb:150:in `block in request'
    from /cache/gems/rmega-0.2.2/lib/rmega/net.rb:7:in `survive'
    from /cache/gems/rmega-0.2.2/lib/rmega/session.rb:144:in `request'
    from /cache/gems/rmega-0.2.2/lib/rmega/session.rb:94:in `login'
    from /cache/gems/rmega-0.2.2/lib/rmega.rb:33:in `login'
    from (irb):6
    from /cache/gems/railties-4.2.7/lib/rails/commands/console.rb:110:in `start'
    from /cache/gems/railties-4.2.7/lib/rails/commands/console.rb:9:in `start'
    from /cache/gems/railties-4.2.7/lib/rails/commands/commands_tasks.rb:68:in `console'
    from /cache/gems/railties-4.2.7/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
    from /cache/gems/railties-4.2.7/lib/rails/commands.rb:17:in `<top (required)>'
    from bin/rails:4:in `require'
    from bin/rails:4:in `<main>'