automl / autofolio Goto Github PK

The facade API is really useful. However, when called to make a prediction it picks the first item in a schedule, i.e. the item with the shortest allocated budget. Is this intentional?

E.g. my problem sets the runtime limit to 3600 and in the AF logging output I get
Selected Schedule [(algorithm, budget)]: [('algo_tree', 1), ('algo_mdd',3601)]
But the output from AFCsvFacade.load_and_predict() just returns algo_tree. This is because the method (src) just returns pred[0][0].

Is this correct / intended? If only 1 sec of budget is set to algo_tree, but 3601 to algo_mdd, but I can only use 1 choice, which should it be? Perhaps I'm misunderstanding.

The yaml file to modify the pcs is fine for now,
but it would be more convenient to pass a pcs file which updates the current pcs.

Hello,
I'm trying to use Autofolio for a project on an ASlib scenario. I ran the example ASlib scenario, and information about what appears to be different models gets printed to the console. However, I cannot find a resource that explains how to use this information or what the recommended model should be. Could I get some information about this?
Thank you,
Austin

Hi, I want use AutoFolio to train a algorithm selector from a collection of algorithms. However, I encounter an issue with using option '-t'. For example, I have two csv files for storing feature data (each instance has 54 features, i.e., the size of each instance's feature vector is 54) and performance data (using runtime as performance metric), respectively.

I first run the command to construct the portfolio selector:
python3.5 <AutoFolio_Path>/AutoFolio-master/scripts/autofolio --performance_csv <Performance_Data_Path>/performance_data.csv --feature_csv <Feature_Data_Path>/ [csv_files.zip](https://github.com/mlindauer/AutoFolio/files/1205204/csv_files.zip) feature_data.csv --objective runtime --runtime_cutoff 10 --tune --save autofolio_test

Then I run the command to run the portfolio selector:
python3.5 Components/AutoFolio-master/scripts/autofolio --load autofolio_test --feature_vec [54 float numbers separated by space (character ' ')]

The terminal outputs:

/usr/local/lib/python3.5/dist-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
"This module will be removed in 0.20.", DeprecationWarning)
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/pandas/core/internals.py", line 4294, in create_block_manager_from_blocks
placement=slice(0, len(axes[0])))]
File "/usr/local/lib/python3.5/dist-packages/pandas/core/internals.py", line 2719, in make_block
return klass(values, ndim=ndim, fastpath=fastpath, placement=placement)
File "/usr/local/lib/python3.5/dist-packages/pandas/core/internals.py", line 115, in init
len(self.mgr_locs)))
ValueError: Wrong number of items passed 54, placement implies 3

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "Components/AutoFolio-master/scripts/autofolio", line 19, in
af.run_cli()
File "/home/lightning/luo/projects/Sparkle/Sparkle_source_codes/Components/AutoFolio-master/autofolio/autofolio.py", line 89, in run_cli
model_fn=args_.load, feature_vec=list(map(float, args_.feature_vec)))
File "/home/lightning/luo/projects/Sparkle/Sparkle_source_codes/Components/AutoFolio-master/autofolio/autofolio.py", line 293, in read_model_and_predict
feature_vec, index=["pseudo_instance"], columns=scenario.feature_names)
File "/usr/local/lib/python3.5/dist-packages/pandas/core/frame.py", line 306, in init
copy=copy)
File "/usr/local/lib/python3.5/dist-packages/pandas/core/frame.py", line 483, in _init_ndarray
return create_block_manager_from_blocks([values], [columns, index])
File "/usr/local/lib/python3.5/dist-packages/pandas/core/internals.py", line 4303, in create_block_manager_from_blocks
construction_error(tot_items, blocks[0].shape[1:], axes, e)
File "/usr/local/lib/python3.5/dist-packages/pandas/core/internals.py", line 4280, in construction_error
passed, implied))
ValueError: Shape of passed values is (54, 1), indices imply (3, 1)

It seems that AutoFolio recognizes the number of feature size is 3, and the program terminates. However, if I do not use the option '--tune', the program will outputs a schedule list with algorithm and budget.

Could you help me to solve this issue?
Please find my csv files in the attachment.

Thanks very much for your help in advance!

csv_files.zip

When try running the toyexample, I had encountered the TypeError for train_scenario

If I were to run the toyexample with the -t parameter, encounter another TypeError : sequence item 0: expected str instance, bool found

Could the reason because I am running on python 3.6.5?

Hi Marius,

I had encountered a problem with calculating the Normalized Score.

AutoFolio-master/autofolio/validation/validate.py:86: RuntimeWarning: invalid value encountered in double_scalars
self.logger.info("Normalized Score: %.4f" %( ( par10 - oracle) / (sbs - oracle)))
INFO:Stats:Normalized Score: nan

Initially I thought that my input data contains negative values or extremely small values causing the error. However, I noticed that the same issue also occurs when running the toy_example data.

Hi Marius,

when using the CSV-facade, one can specify the CV-folds with an extra CSV file:

It would be convenient, if one could, instead of having to create a full CVS specifying the splits, just pass a custom number of CV folds to use AutoFolio more easily with different dataset sizes.

Let me know if you'd like me to provide a pull request.

Best,
Jan

Hello

any updates on Python version 3.7?

@mlindauer Would it be possible for you to look at issue 13?

Hi Marius ​,
I encountered TypeError when run AutoFolio with option '--config',

The contents of the yaml file are as follows:

And I have a question about presolver, can I specify a solver as the presolver? could you please give me more details about presolver?
Thank you very much!