Tutorials not working, input/parameter files? about basenji

[JeevG]

Hi Dave,

Any advance on uploading a params_file we can start to play with? The paper doesn't mention how many filters etc were used

@vagarwal87 - if you use the tensor2tensor release they have data relating to this paper (although their implementation is currently broken)

Thanks!

Jeev

[davek44]

Hey all,

I've made some decent progress on these. And I will release models soon. The biggest hold up at this point is that we're working to integrate these architectures better with the TensorFlow input data format TFRecords and the new Dataset API. You can expect that stuff soon.

Best,
Dave

[davek44]

Hi all, the tutorials and both errors Hirak encountered should be fixed and working now.

[davek44]

Hi Vikram,

Thanks for your interest, and sorry for the incomplete status here. The tutorials, data, and pre-trained models are a work in progress that I'll be fleshing out throughout the summer. I'll leave comments here as I make progress.

Best,
Dave

[dylanmmarshall]

Hi Dave,

Adding my voice to the chorus here for people wanting an updated tutorial. Also, I'm wondering if it wouldn't be too much trouble to provide data used to make figures / conclusions in biorxiv paper in easily accessible form such as Google Drive link? Or perhaps whatever form a quickstart to basenji may be. Looking forward to further developments - awesome work!

Best,
Dylan

[ankitvgupta]

Hey Dave, could you post the umap_macro.bed file that you refer to in https://github.com/calico/basenji/blob/master/tutorials/preprocess.ipynb?
Or post a link to where it could be downloaded from?

[davek44]

Sorry, that was a typo. It's referring to this file: https://github.com/calico/basenji/blob/master/tutorials/data/unmap_macro.bed

[ankitvgupta]

Ah got it - I should have looking more closely for that. Thanks!

[goldmich]

Hi Dave,

A quick question related to the discussion above. I'm trying to work through the gene expression tutorial (and do some further work with pre-trained models) and am having trouble loading the pre-trained models from the .tf files in basenji_test_genes.py -o data/gencode_chr9_test --rc -s --table models/params_small.txt models/gm12878_best.tf data/gencode_chr9_l262k_w128.h5. Even after replacing _best with _d10 and downloading the .tf.data-00000-of-00001, .tf.meta, and .tf.index associated with the gm12878_d10 model, I receive the following error: tensorflow.python.framework.errors_impl.NotFoundError: Key cnn5/batch_normalization/renorm_mean_weight not found in checkpoint. I assume that this is because I am missing a checkpoint file, which I cannot manage to find on this page. Any pointers? Thanks in advance.

Best,
Michael

[davek44]

HI Michael,

Sorry about that--I have a bit of an incompatibility right now between the master branch, and the pre-trained models. I'll clean that up next week. In the meantime, the safest bet is to work off the release branch here: https://github.com/calico/basenji/releases/tag/0.2

Best,
Dave

[goldmich]

Hi Dave,

Thanks for working on this and pointing me to the release branch. I've setup the 0.2 release, but I am unfortunately still running into some issues with missing information from the checkpoint. Similar to the above error, the gene expression prediction from the tutorial fails with NotFoundError (see above for traceback): Key cnn0/batch_normalization/beta/Adam not found in checkpoint. Any tips? Thanks in advance, and please let me know if you need any more traceback.

Best,
Michael

[davek44]

Hi Michael,

You're right, the parameters file doesn't match the pre-trained model. I will clean it up this week. In the meantime, that tutorial can still demonstrate the steps involved. And if you'd like to see the output, you can train a model for a few epochs yourself with those parameters and ignore models/gm12878_d10.tf.

Best,
Dave

[hiraksarkar]

Hi Dave,
Awesome package, I am trying to create the files according to tutorial, but failed while generating data/heart_l262k.h5
It seems due to some update basenji currently does not have genome module

Traceback (most recent call last):
  File "/home/hirak/Projects/basenji/bin/basenji_hdf5_single.py", line 918, in <module>
    main()
  File "/home/hirak/Projects/basenji/bin/basenji_hdf5_single.py", line 205, in main
    chrom_segments = basenji.genome.load_chromosomes(fasta_file)
AttributeError: module 'basenji' has no attribute 'genome'

[hiraksarkar]

Hi Dave,
I guess it was an import issue, figured it out. The tutorial should have been run within the root directory of basenji.

[hiraksarkar]

Hi Dave,

The package dependencies are resolved, but I am stuck with another problem in the basenji_hdf5_single.py script

Traceback (most recent call last):
  File "/home/hirak/Projects/basenji/bin/basenji_hdf5_single.py", line 924, in <module>
    main()
  File "/home/hirak/Projects/basenji/bin/basenji_hdf5_single.py", line 477, in main
    data=seqs_na[train_indexes],
UnboundLocalError: local variable 'seqs_na' referenced before assignment