GithubHelp home page GithubHelp logo

maayanlab / biojupies Goto Github PK

View Code? Open in Web Editor NEW
105.0 9.0 29.0 69.32 MB

Automated generation of tailored bioinformatics Jupyter Notebooks via a user interface.

Home Page: http://biojupies.cloud

License: Other

Python 24.32% CSS 5.29% HTML 63.00% JavaScript 5.67% Shell 1.27% Dockerfile 0.39% R 0.06%
bioinformatics systems-biology jupyter rna-seq pipeline notebook cloud dcppc

biojupies's Introduction

BioJupies

Notebooks Generated Datasets Available Plugins

Automated generation of Jupyter Notebooks for RNA-seq data analysis via user interface (http://biojupies.cloud).

Source code for the RNA-seq analysis plugins available at https://github.com/MaayanLab/biojupies-plugins.

Overview

  1. What is BioJupies?
  2. How can I generate notebooks using BioJupies?
  3. How can I upload my own RNA-seq data to BioJupies?
  4. What analyses can BioJupies perform?
  5. Can I contribute my RNA-seq analysis tools?
  6. How can I download and rerun my notebooks?
  7. References

What is BioJupies?

BioJupies is a web server which allows users to automatically generate Jupyter Notebooks from RNA-seq datasets through an intuitive interface, with no knowledge of coding required. It can be accessed for free from http://biojupies.cloud. Screenshot of the BioJupies website landing page.

How can I generate a notebook using BioJupies?

Generating a notebook using BioJupies requires three steps:

Screenshot of the BioJupies website landing page.

  1. First, select an RNA-seq dataset you with to analyze. You can upload FASTQ files, gene expression tables, or use a search engine to browse over 6,000 public datasets published in the Gene Expression Omnibus and processed by ARCHS4.
  2. Second, add one or more computational tools to analyze the data. BioJupies currently supports 14 plugins to perform exploratory data analysis, differential gene expression, enrichment analysis, and small molecule queries.
  3. Third, generate the notebook with the desired settings. The notebook will be served to you through a URL, and can be easily downloaded and rerun on your local computer.

How can I upload my RNA-seq data to BioJupies?

Users can upload their RNA-seq datasets for analysis at https://amp.pharm.mssm.edu/biojupies/upload.

Screenshot of the BioJupies upload page.

BioJupies currently supports uploading RNA-seq datasets in the FASTQ format or tables of gene-level counts.

Note: While the user may successfully generate a notebook from normalized gene expression counts or microarray data tables, the tools are NOT yet optimized to handle such data. Such results should be interpreted with caution.

The Enrichment Analysis and Small Molecule Query plugins currently support datasets uploaded with gene symbols as row identifiers. When uploading datasets with different identifiers (e.g. ENSEMBL IDs, Entrez IDs), please note that these plugins may return empty results.


What analyses can BioJupies perform?

BioJupies currently provides 14 RNA-seq data analysis plugins, divided into four categories. For more information about the plugins, visit https://github.com/MaayanLab/biojupies-plugins.

Screenshot of the BioJupies upload page.

Can I contribute my RNA-seq analysis tools?

Developers are welcome to contribute their own RNA-seq data analysis plugins for integration in BioJupies. Once integrated, the plugin will be made available to all users for notebook generation and published on the BioJupies plugins GitHub repository.

To contribute your plug-in, please visit https://amp.pharm.mssm.edu/biojupies/contribute.


How can I download and rerun my notebooks?

After generating a notebook, users can download and execute it locally using the maayanlab/biojupies-notebooks Docker container.

To do this, the following steps are required:

  1. Pull the latest version of the Docker container from the console:
docker pull maayanlab/biojupies-notebooks
  1. Identify the notebook's unique ID (a 9-character alphanumeric string) from its URL - e.g. 5DAHiejHN.

Screenshot of the a BioJupies notebook UID.

  1. Launch the Docker container.
docker run -it -p 8888:8888 -e NOTEBOOK_UID=5DAHiejHN maayanlab/biojupies-notebooks

Note: replace 5DAHiejHN with the UID of the notebook you wish to rerun.

  1. Access the local Jupyter Notebook server from your browser at port 8888.

Screenshot of the a local Jupyter Server.

  1. Execute and customize your notebook from the Jupyter server.

Screenshot of the a local Jupyter Server.

For more information about reusing notebooks, see https://amp.pharm.mssm.edu/biojupies/docker.


References

BioJupies is available as a pre-print on BioRxiv: BioJupies: Automated Generation of Interactive Notebooks for RNA-seq Data Analysis in the Cloud Torre, D., Lachmann, A., and Ma’ayan, A. (Cell Systems, 2018)

License

This project is licensed under the Apache-2.0 License - see the LICENSE.md file for details

biojupies's People

Contributors

avimaayan avatar denis-torre avatar dt2539 avatar u8sand avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

biojupies's Issues

GTEx analysis seems to be broken

Selected 20-29 old and not exposed skin in group 1 and 20-29 exposed skin in group 2, click continue... ended back in the starting page.

Files not uploading

Tried to upload 6 files... No progress... See screenshot. Using Windows and Chrome.
biojupies-screenshot1

Only two classes in many GSEs - suggestion of enhancement

Hello,

I have been using biojupies satisfactorily for a while now, and I notice that for many studies, there are only two groups: one control and one perturbation.

Automatic labeling of classes is hard, and the underlying code does a nice job of identifying controls.

For binary cases (only one perturbation), it would be practical to have a "Tag all non-assigned samples to Perturbation" button.

It is minor, but might save many clicks for many users.

Thanks,

Clustergrammer and L1000FWD Query not loading

Tried to upload notebook from BioJupies to JShelf... Clustergrammer and L1000FWD did not load in the notebook.

Dump from Inspect is pasted below:

Failed to load resource: the server responded with a status of 404 (NOT FOUND)
login.gif:1 Failed to load resource: the server responded with a status of 404 (NOT FOUND)
(index):1 Refused to apply style from 'https://cdn.jsdelivr.net/npm/[email protected]/dist/pretty-checkbox.min2.css' because its MIME type ('text/plain') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
upload.gif:1 Failed to load resource: the server responded with a status of 404 (NOT FOUND)
launch.gif:1 Failed to load resource: the server responded with a status of 404 (NOT FOUND)
(index):1 Refused to apply style from 'https://cdn.jsdelivr.net/npm/[email protected]/dist/pretty-checkbox.min2.css' because its MIME type ('text/plain') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
css.css:1 Failed to load resource: the server responded with a status of 404 (NOT FOUND)
(index):1 This page includes a password or credit card input in a non-secure context. A warning has been added to the URL bar. For more information, see https://goo.gl/zmWq3m.
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
(index):881 Object
BioJupies.ipynb:740 Mixed Content: The page at 'https://nbviewer.jupyter.org/urls/s3.amazonaws.com/biodos/e62583354f9bb743b10359c2913aa82d/BioJupies.ipynb' was loaded over HTTPS, but requested an insecure resource 'http://amp.pharm.mssm.edu/clustergrammer/viz/5b4395231576c511ba09e19c/tmpqvbmaeb_.txt'. This request has been blocked; the content must be served over HTTPS.
BioJupies.ipynb:40453 Mixed Content: The page at 'https://nbviewer.jupyter.org/urls/s3.amazonaws.com/biodos/e62583354f9bb743b10359c2913aa82d/BioJupies.ipynb' was loaded over HTTPS, but requested an insecure resource 'http://amp.pharm.mssm.edu/l1000fwd/vanilla/result/5b43958f5c8056001639e5fe'. This request has been blocked; the content must be served over HTTPS.
(index):1 Refused to apply style from 'https://cdn.jsdelivr.net/npm/[email protected]/dist/pretty-checkbox.min2.css' because its MIME type ('text/plain') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
css.css:1 Failed to load resource: the server responded with a status of 404 (NOT FOUND)

Add progress bar for the alignment process

User is left waiting for a long time during the alignment process. It would be great to have a progress bar that keeps increasing, perhaps based on an estimate of the time it should take based on the file size and number of samples.

Allow users to specify private datasets

If toggled, the dataset will be made private on Google Cloud. This happens directly from the server. The notebook generator server will automatically set the dataset to public prior to notebook generation.

  • Add a private column to the dataset table.

  • Upon setting a dataset to private, toggle privacy on Gcloud.

  • Whenever a notebook is submitted through the website, toggle privacy on and then off.

Low priority.

Unable to upload an Expression Table File.

Hi!

I am trying to upload an Expression Table File (.xlsx), but I get the following error:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <title>500 Internal Server Error</title> <h1>Internal Server Error</h1> <p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
I tried several times with Firefox 57 and also with Chromium 67 using Linux.

Thank you!

Alert when user uploads FASTQ file over 5GB

function upload_reads() {
// Get credentials
username = 'biojupies';
passwd = 'sequencing';
// Get files
var files = $('#fileinput').prop('files');
// Loop through files
$.each(files, function(index, file) {
// Add progress bar
progress_bar(file['name']);
// Get Sign Policy
$.getJSON("https://amp.pharm.mssm.edu/charon/signpolicy?username=" + username + "&password=" + passwd, function (data) {
// Create Form
var FD = new FormData();
FD.append('key', data['uid'] + "/{{upload_uid}}-${filename}");
FD.append('AWSAccessKeyId', data['cid']);
FD.append('acl', 'private');
FD.append('success_action_redirect', 'success.html');
FD.append('policy', data['policy']);
FD.append('signature', data['signature']);
FD.append('Content-Type', 'application/octet-stream');
FD.append('file', file);
// Get filename
var filename = file['name'];
// Create Request
var xhr = new XMLHttpRequest();
xhr.open("POST", "https://" + data["bucket"] + ".s3.amazonaws.com/", true);
xhr.onloadstart = function (e) {
console.log("start")
}
xhr.onloadend = function (e) {
// Set complete status on progress bar
$('[data-filename="' + filename + '"] .progress-bar').attr('data-status', 'complete');
$('[data-filename="' + filename + '"] .progress-bar-text span:first-child').html('Successfully uploaded ');
$('[data-filename="' + filename + '"] .progress-bar-text span:last-child').html('');
// Activate button if all progress bars are complete
if ($('.progress-bar').length === $('.progress-bar[data-status="complete"]').length) {
$('[form="read-upload-form"]').prop('disabled', false).toggleClass('black white bg-white bg-blue');
}
console.log("end")
// Call FASTQ Upload Status API
}
xhr.upload.addEventListener('progress', function (evt) {
if (evt.lengthComputable) {
var progress = Math.ceil((evt.loaded / evt.total) * 100)+'%';
$('[data-filename="' + filename + '"] .progress-bar').html(progress);
$('[data-filename="' + filename + '"] .progress-bar').css('width', progress);
}
}, false);
xhr.send(FD);
});
})
}

Text edits to this page http://amp.pharm.mssm.edu/biojupies-dev/upload/reads

Use the form below to upload raw RNA-seq data in FASTQ format. The files will be aligned to the reference genome to generate the quantified reads expression table at the gene level. The table will be subsequently made available for you to download and to generate the interactive notebook. Once you have uploaded the desired files, click Continue to proceed.

Upload Read Files
To begin, select the files you wish to upload using the form below, then click on Upload Files to proceed.

Add progress bar to notebook generation stage

If you select a lot of tools, the notebook generation takes a long time. It would be great to see a progress bar that keeps increasing as time passes so users don't leave the site.

Landing page text edits

Biojupies Automatically Generates
RNA-seq Data Analysis Notebooks
With BioJupies you can produce in seconds a customized, reusable, and interactive report from your own raw or processed RNA-seq data through a simple user interface

log10P in the volcano plot instead of -log10P

Hello,

I was using biojupies and noticed that the volcano plot y-axis has 'log10P' while it should be - log10P, so all the values are positive.

It doesn't interefere with the analysis, but might be confusing for some people.

(great tool, nevertheless)

Prevent selection of same GTEx sample in two groups

$(document).ready(function () {
// Load GTEx metadata
$.ajax({
url: "{{ url_for('gtex_api') }}",
dataType: "json",
method: "POST",
success: function(json) {
// Loop through tables
$('.gtex-table').each(function(index, elem) {
// Create table
var table = $(elem).DataTable({
data: json,
aaSorting: [],
pagingType: 'simple',
columns: [
{data: 'checkbox', orderable: false, className: 'select-checkbox'},
{data: 'Age', orderable: false},
{data: 'Gender', orderable: false},
{data: 'Tissue', orderable: false}
],
select: {
style: 'multi'
},
fnCreatedRow: function (nRow, aData, iDataIndex) {
$(nRow).attr('data-sampleid', aData['id']); // or whatever you choose to set as the id
},
initComplete: function () {
this.api().columns().every(function () {
var column = this;
if (column.index() > 0)
{
var select = $('<select class="gtex-sample-filter"><option value=""></option></select>')
.appendTo($(column.header()).empty())
.on('change', function () {
var val = $.fn.dataTable.util.escapeRegex(
$(this).val()
);
column
.search(val ? '^' + val + '$' : '', true, false)
.draw();
});
column.data().unique().sort().each(function (d, j) {
select.append('<option value="' + d + '">' + d + '</option>')
});
}
});
}
})
// Add listener for checkbox
table.on('click', '.select-all', function(evt) {
// Get button
var $button = $(this);
// Deselect and select
if (table.rows({ selected: true }).count() > 0) {
table.rows().deselect();
$button.removeClass('active');
} else if (table.rows({ filter: 'applied' }).count() == table.rows().count()) {
alert('Please select at least one filter in the table such as age, gender or tissue.')
} else {
table.rows({ filter: 'applied' }).select();
$button.addClass('active');
}
})
// Add update number of samples
table.on('select deselect', function(evt) {
var $col = $(evt.target).parents('.gtex-table-col'),
selected_rows = table.rows({ selected: true }),
ids = Array.from(selected_rows.data().pluck('id'));
// Update number
$col.find('.gtex-table-header .nr-samples').html(selected_rows.count());
// Add samples to HTML
// $col.find('.selected-samples').data('id', ids);
$col.find('.selected-samples').val(ids);
})
// Change group names
table.on('change', '.gtex-sample-filter', function (evt) {
var $table = $(this).parents('table'),
group_name = $table.find('.gtex-sample-filter').map(function () { if ($(this).val().length) {return $(this).val()} }).get().reverse().join(', ');
$table.parents('.gtex-table-col').find('.gtex-group-name').first().val(group_name);
console.log(group_name);
})
})
// Show table
$('.gtex-table').removeClass('d-none');
$('.gtex-table-header').removeClass('d-none');
$('.gtex-loader').addClass('d-none');
}
})
// Check sample size
$('button[form="select-samples-form"]').click(function(evt) {
// Prevent submission
evt.preventDefault();
// Get group size
group_1_size = $('#DataTables_Table_0_wrapper tr.selected').length;
group_2_size = $('#DataTables_Table_1_wrapper tr.selected').length;
// Check sizes
if ((group_1_size < 2) || (group_2_size < 2)) {
// Alert
alert('Please select at least two samples for each group using the provided checkboxes.')
} else {
// Submit
$('#select-samples-form').submit();
}
})
})

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.