aldefouw / redcap-api-transfer Goto Github PK
View Code? Open in Web Editor NEWThis project contains Ruby libraries that aim to make transferring a REDCap project from one server to another as simple as running a single script from command line.
This project contains Ruby libraries that aim to make transferring a REDCap project from one server to another as simple as running a single script from command line.
We are migrating the Redcap projects using this utility.
Issue: While importing the files other than Text files, file are uploaded successfully, however they are getting corrupted.
Cause: While importing, the file is being read using File.read(file) which corrupts the file if it's not a text file.
Importing a DOC/PDF/EXCEL file from API playground or Postman works fine. The code which is used is the same code which is being provided in the Redcap samples https://redcaptest.cc.nih.gov/api/help/?content=examples
Below is the code which is causing the issue, any help would be appreciated.
def body(boundary, id, field, event_name, file)
<<-EOF
--#{boundary}
Content-Disposition: form-data; name="file"; filename="#{File.basename(file)}"
Content-Type: application/octet-stream
#{File.read(file)}
--#{boundary}
#{import_file_fields(id, field, event_name).collect{|k,v|"Content-Disposition: form-data; name=\"#{k.to_s}\"\n\n#{v}\n--#{boundary}\n"}.join}
EOF
end
Not sure if you're still actively maintaining this project, but I'm getting the below error when trying to transfer a project. Note that I previously posted a bug report as "senorrojo" a few years ago (which you solved - thank you) and I've just started using this package again because we're migrating from one REDCap server to another. Thank you so much for creating this project - it truly saved me the last time I needed it and I had many, many successful transfers of large projects.
I get the same error for transferring a single record or all records:
[matt@localhost redcap-api-transfer]# sh transfer_single_record.sh Test 25
/app/library/export_data.rb:50:in throw': uncaught throw "Unable to find a destination data template file for the project /app/export_data/dest_template-1708116087.tmp." (UncaughtThrowError) from /app/library/export_data.rb:50:in
fetch_dest_data_template'
from /app/library/export_data.rb:43:in dest_data_cols' from /app/library/export_data.rb:15:in
initialize'
from /app/library/transfer_records.rb:50:in new' from /app/library/transfer_records.rb:50:in
initialize'
from transfer_single_record.rb:7:in new' from transfer_single_record.rb:7:in
<main>'
Loading Configuration ...
Downloading SOURCE Project Template ...
Downloading DESTINATION Project Template ...
Downloading Data Dictionary ...
Loading Export Data ...
[matt@localhost redcap-api-transfer]#
[matt@localhost redcap-api-transfer]# ls app/export_data/
data.csv
data.csv has a CSV export of the project data from the SOURCE. I also tried putting the data dictionary in that export_data directory to no avail. Below is my config file. I'm happy to provide verbose logging or any other information that might be helpful.
settings:
processes: 8 # Default number of processes the script will fallback to if none is specified in specific projects below
verbose: false # When true, outputs additional information to logs about the HTTP requests being made
projects:
Test:
source:
url: https://redcap.healthsciencessc.org/api/
token: XXXX
destination:
url: https://redcap.research.sc.edu/api/
token: YYYY
transfer_new_records_only: true # Must have API Export Rights on Destination REDCap Server!
processes: 8 # Takes precedence over the processes listed in settings
verbose: false # Takes precedence over the verbose flag listed in settings
Again, thank you!
When trying to transfer records (single or all records) with an uploaded file attached, I get the below error:
$ ruby -v
ruby 2.3.5p376 (2017-09-14 revision 59905) [x86_64-linux]
$ ruby single_record.rb
Loading Configuration ...
Loading Export Data ...
Loading Reporting Options ...
Completed request for 2a206 on source.
Successfully fetched 2a206 from source.
Completed request for 2a206 on destination.
Successfully created 2a206 on destination.
Completed request for 2a206 on source file.
Successfully fetched source file called 2a206 form.pdf from 2a206 / source.
/home/matt/.rvm/gems/ruby-2.3.5/gems/curb-0.9.4/lib/curl/easy.rb:68: warning: uncaught exception from callback
...and the file is not uploaded on the destination. I get the same thing if I try ruby-2.6.3. The only corresponding log entry is in info.log (nothing in error.log):
Successfully fetched source file called 2a206 form.pdf from 2a206 / source.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.