schornacklabslcu / amfinder Goto Github PK

Hello,

I recently solved an issue during installation on a M1 MacBook Pro with the ./install.sh command being unable to satisfy tensorflow-2.9.1 requirements, noting this version of mac lists it as tensorflow-macos-2.9.1. A quick edit to the requirements file fixed the issue. However, I am now faced with another issue related to tensorflow-io-gcs-filesystem-0.26.0.

I downloaded the correct version of the package via the "git clone -b v0.26.0 https://github.com/tensorflow/io.git"command, which then allowed me to install the ...-gcs-filesystem package through specifying the wheel that was set up. I can find it in my python3.9 frameworks folder, alongside the tensorflow-macos-2.9.1 & other required packages. (see screenshot) Despite this, it is still not recognized during installation. I do not believe this issue is due to file naming, as I edited the requirements doc briefly but did not resolve the issue. Is there a way to fix this error?

Note: I also made a minor edit to the ./install.sh file by changing PYTHON="${PYTHON:-$(which python3.7)}" > PYTHON="${PYTHON:-$(which python3)}" because the export PYTHON= function was not working for me for some reason.

Hello,

I have installed amf & amfbrowser on mac. When choosing files to upload for annotation, I am unable to load TIFF files correctly into the browser. After converting the file into jpeg it uploads correctly. I have attached an image of what an uploaded TIFF file looks like (when experiencing the error) & this link will take you to a drive folder with the jpeg & tiff versions of the image used.

Thank you for your time,

Hi,
Thanks for this amazing software. I got AMFinder to successfully generate a prediction for a test image, however when I attempt to open the prediction on the browser, I cannot select "ok" (or even "cancel"). The program freezes.

On WSL I get this notice when I open the browser:
(amfbrowser.exe:79): Gtk-CRITICAL **: 16:30:43.574: gtk_window_realize_icon: assertion 'info->icon_pixmap == NULL' failed

Would appreciate advice, thanks!

Hello,

first I like to say it's a great pipeline for AMF colonisation analysis.

I am reporting two changes that I had to do when installing (MacOS 10.14.6) - don't know if this is due to OS or updated libraries or something else:

In amfinder_model.py I was getting an error until I changed the line 53 from:
from keras.optimizers import Adam
to:
from keras.optimizers import adam_v2

Got the fix from: https://stackoverflow.com/questions/62707558/importerror-cannot-import-name-adam-from-keras-optimizers

In amfinder_superresolution.py I had to change the line 36 from:
from keras.engine.network import Network
to:
from tf_agents.networks import network
after instaling tensorflow agents.
Got the fix from: https://www.tensorflow.org/agents/tutorials/8_networks_tutorial

Now everything works. Maybe this will help someone in the same situation.

Hi,

I am trying to train a CNN2 model using some images I annotated and have been running into an error related to the sample weights.

The command I'm running is: amf train -net trained_networks/CNN2v1.h5 -s -o ~/Downloads/CNN2_model -CNN2 /Users/ethanbass/Pictures/mcrz_training_cropped_1500px/selected/CNN2_training/*.jpg.

The routine goes through the "tile extraction" OK, but then returns the following error:

[17:34:10] Class weights
    - ConvNet A: 969 active (weight: 0.93), 832 inactive (weight: 1.08).
    - ConvNet V: 965 active (weight: 0.93), 836 inactive (weight: 1.08).
    - ConvNet H: 690 active (weight: 1.31), 1111 inactive (weight: 0.81).
    - ConvNet I: 750 active (weight: 1.20), 1051 inactive (weight: 0.86).
Traceback (most recent call last):
  File "/opt/local/bin/amf", line 77, in <module>
    main()
  File "/opt/local/bin/amf", line 55, in main
    AmfTrain.run(input_files)
  File "/Users/ethanbass/software/amfinder-apple-silicon/amf/amfinder_train.py", line 597, in run
    his = model.fit(t_gen.flow(xt, yt, batch_size=bs),
  File "/Users/ethanbass/miniforge3/envs/amf6/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/Users/ethanbass/miniforge3/envs/amf6/lib/python3.9/site-packages/keras/engine/data_adapter.py", line 1424, in _make_class_weight_map_fn
    raise ValueError(error_msg)
ValueError: Expected `class_weight` to be a dict with keys from 0 to one less than the number of classes, found {'A': {0: 1.0823317307692308, 1: 0.9293085655314758}, 'V': {0: 1.0771531100478469, 1: 0.933160621761658}, 'H': {0: 0.8105310531053105, 1: 1.305072463768116}, 'I': {0: 0.8568030447193149, 1: 1.2006666666666668}}

I thought at first this might be related to the configuration of my python environment since I had to change the versions of some of the dependencies to get AMfinder installed on my M1 mac, but I am also getting the same error on a Windows 11 computer in the lab running a standard installation of amfinder on the Windows Subsystem for Linux.

I think the error may be related to this issue on the tensorflow github page (tensorflow/tensorflow#41448) and perhaps also to the issue mentioned in the documentation of the class_weights function:

    Tensorflow 2.1 (and Keras 2.3.1). A bug in TF makes it
    impossible to use class_weights to models with multiple
    outputs. This bug is active on January 2021.
    Reference: https://github.com/tensorflow/tensorflow/issues/40457

But if this is still an issue with tensorflow above 2.1, I'm not sure why the requirements of amfinder no longer reflect this? Probably there's something I'm missing. I would be grateful for any suggestions!

Ethan

Update: I was able to get this running on the WSL by reverting to the 2021 release of AMfinder (https://github.com/SchornacklabSLCU/amfinder/releases/tag/v2.0). So I guess it is indeed an issue with the tensorflow bug I mentioned. I don't think this will help me get it running on my mac unfortunately, since tensorflow 2.1 is not available (as far as I can tell) for the m1 architecture. If it is in fact the case that tensorflow 2.1 is required for CNN2 training, I think it would be helpful if this were reflected in the installation instructions. I'm also wondering if there might be some kind of workaround to get this working with newer versions of tensorflow? It seems like they are not going to fix this any time soon, given that people have been complaining about it for 3 years with very little response from the developers.

Having this error when I try to run the application.

I have followed the instructions for setting up on windows.
I downloaded WSL and python3.9, then ran ./install.sh and got the confirmation message at the end of the script.

I have the virtual environment set up and running when I run amf.

Did I miss something during the installation?

Hi,
Sorry to bother you, I was wondering whether the program is able to train an entire folder of annotated images at once or would I have to train it image by image? Somehow when I do "./amf train folder_name" it says that my images in the folder are not annotated for some reason, but when I train it image by image by drag and drop, it does not throw that error. (All images along with the zip files are in the same folder that I gave it). I also tried giving it multiple images by simple drag and drop, but the program only seems to take the first image as input and ignores the rest.

I also get the following error for some images that I input, which I'm not exactly sure why, since I annotated them all in the same manner.
ValueError: class_weight must contain all classes in the data. The classes {2} exist in the data but not in class_weight.

Any insight would be much appreciated. Thank you so much for all your help!

i have followed the instructions and obtained the amfbrowser.exe file on a ubuntu system, but i cannot run it with the "amfbrowser my_name.jpg", i cannot open it with double clicking it . So ,how can i run the exe file?

I am wondering what would be involved in adding support for additional classes of tiles. Specifically, I would be interested in being able to train a CNN to distinguish colonization by AMF from colonization by other root endophytes. A lot of our roots from the field are colonized by "dark septate endophytes" which appear quite different from AMF. For example, I believe the root on the left in the attached picture contains vesicles from AMF, but the root on the right contains microsclerotia from DSE. It would be great to be able to train a CNN to differentiate these different types of fungi

!

Hi sorry to bother, I was just wondering how one would go about getting the amfbrowser.exe to run on a newer version of MacOS? The one that is in the folder is Catalina64.exe but I have Big Sur. Is there a way I would be able to change the exe file to get it to run on my laptop?
Thank you so much for your help in advance.

Hello,

I've been trying to install amfbrowser on the linux subsystem for windows per your instructions & continue to run into an error effectively installing or compiling lablgtk and cairo2-gtk.I, which prevents me from completing the ./install.sh command. I've had a more coding-savy colleague attempt to resolve the issue to no avail so far. This Stack Overflow entry from another user has more details on the exact problem I am experiencing. Additionally I am not able to use the pre-built linux.exe provided to bypass this issue. Do y'all know of effective work arounds or solutions to this issue?
Thank you in advance, I'm excited to work with this program.

Best,
Lorenzo

Hello,

I am currently working to train a fluorescently stained image set. I have a total of 48 images, which I was able to input into the training pipeline. However, I only ended up with 1500 M+ tiles, 1300 M- tiles, and 1400 X tiles. This resulted in repetitive pre-mature termination at epoch 12-20, which I believe is due to pre-termination built into the code to prevent overfitting if the loss doesn't decrease for 12 training cycles in a row.

From the publication, these numbers seem very low even though I'm using more images in the training, it seems that there were 175105 total tiles from 32 images. I don't quite understand why how 48 images resulted in dramatically less tiles than 32, but maybe I am missing something big here.

So my questions are:

1. Is there a way to increase the tile # from an image? Currently, the max number of tiles I am getting from 1 image is 125 tiles (i.e. my images are only ever divided into 125 grids by the amfbrowser whenever I feed an image to it.)
2. Is the early pre-termination resulting from a small sample size or too few tiles?

Thank you in advance and please let me know if I can clarify on any of this,

Karl

I am trying to run the amfbrowser on my computer (running Mac OS 10.14.6). I am able to load and annotate images, but I can't figure out how to import predictions. The buttons under predictions are all grayed out on my computer and cannot be accessed. I'm not sure if there's something I'm misunderstanding here or if there is something wrong with my installation. My understanding from reading the documentation is that I should be able to use these buttons to import predictions. I'd greatly appreciate any thoughts you might have on how to resolve these issues (or if I am just completely misunderstanding something, please let me know that as well). Thank you!

I have the AMFinder downloaded to a HPC and am trying to run a prediction on 18 photos (which I have converted to JPEG images). When I run my command, the software recognizes the photos but comes back with an error saying they are not valid archives.

Hello,

I've just finished installing amf and amfbrowser and was able to generate predictions for an image using amf.

But when I run amfbrowser and try to select the prediction file, I cannot click on the OK button. I've tried clicking, selecting with Enter, and just about every other key on my keyboard.
My lab mate thinks it is because I am trying to run this on Windows 11. Do you know if there is any way to run this without downgrading to Windows 10? Alternatively I could try installing on a system with Linux.
What is your recommendation?

Thanks,
Robbie
University of Ottawa