7ossam81 / evolopy

EvoloPy toolbox provides classical and recent nature-inspired metaheuristic for the global optimization.

EvoloPy: An open source nature-inspired optimization toolbox for global optimization in Python

The EvoloPy toolbox provides classical and recent nature-inspired metaheuristic for the global optimization. The list of optimizers that have been implemented includes Particle Swarm Optimization (PSO), Multi-Verse Optimizer (MVO), Grey Wolf Optimizer (GWO), and Moth Flame Optimization (MFO). The full list of implemented optimizers is available here https://github.com/7ossam81/EvoloPy/wiki/List-of-optimizers

If you like our framework then we would really appreciate a Star ⭐!

Features

• Six nature-inspired metaheuristic optimizers were implemented.
• The implimentation uses the fast array manipulation using NumPy.
• Matrix support using SciPy's package.
• More optimizers is comming soon.

Installation

• Python 3.xx is required.

Run

pip3 install -r requirements.txt

(possibly with sudo)

That command above will install sklearn, NumPy, and SciPy for you.

• If you are installing EvoloPy Toolbox onto Windows, please Install Anaconda from here https://www.continuum.io/downloads, which is the leading open data science platform powered by Python.

• If you are installing onto Ubuntu or Debian and using Python 3 then this will pull in all the dependencies from the repositories:

  sudo apt-get install python3-numpy python3-scipy liblapack-dev libatlas-base-dev libgsl0-dev fftw-dev libglpk-dev libdsdp-dev
  

Get the source

Clone the Git repository from GitHub

git clone https://github.com/7ossam81/EvoloPy.git

Quick User Guide

EvoloPy toolbox contains twenty three benchamrks (F1-F23). The main file is the optimizer.py, which considered the interface of the toolbox. In the optimizer.py you can setup your experiment by selecting the optmizers, the benchmarks, number of runs, number of iterations, and population size. The following is a sample example to use the EvoloPy toolbox.
Select optimizers from the list of available ones: "SSA","PSO","GA","BAT","FFA","GWO","WOA","MVO","MFO","CS","HHO","SCA","JAYA","DE". For example:

optimizer=["SSA","PSO","GA"]  

After that, Select benchmark function from the list of available ones: "F1","F2","F3","F4","F5","F6","F7","F8","F9","F10","F11","F12","F13","F14","F15","F16","F17","F18","F19". For example:

objectivefunc=["F3","F4"]  

Select number of repetitions for each experiment. To obtain meaningful statistical results, usually 30 independent runs are executed for each algorithm. For example:

NumOfRuns=10  

Select general parameters for all optimizers (population size, number of iterations). For example:

params = {'PopulationSize' : 30, 'Iterations' : 50}

Choose whether to Export the results in different formats. For example:

export_flags = {'Export_avg':True, 'Export_details':True, 'Export_convergence':True, 'Export_boxplot':True}

Now your experiment is ready to run. Enjoy!

Contribute

• Issue Tracker: https://github.com/7ossam81/EvoloPy/issues
• Source Code: https://github.com/7ossam81/EvoloPy

Useful Links

• Video Demo:https://www.youtube.com/watch?v=8t10SyrhDjQ
• Paper source: https://github.com/7ossam81/EvoloPy
• Paper: https://www.scitepress.org/Papers/2016/60482/60482.pdf
• Poster source: https://github.com/7ossam81/EvoloPy-poster
• Live Demo: http://evo-ml.com/evolopy-live-demo/

List of contributors

Reference

For more information about EvoloPy, please refer to our paper:

Faris, Hossam, Ibrahim Aljarah, Seyedali Mirjalili, Pedro A. Castillo, and Juan Julián Merelo Guervós. "EvoloPy: An Open-source Nature-inspired Optimization Framework in Python." In IJCCI (ECTA), pp. 171-177. 2016. https://www.scitepress.org/Papers/2016/60482/60482.pdf

Please include the following related citations:

• Qaddoura, Raneem, Hossam Faris, Ibrahim Aljarah, and Pedro A. Castillo. "EvoCluster: An Open-Source Nature-Inspired Optimization Clustering Framework in Python." In International Conference on the Applications of Evolutionary Computation (Part of EvoStar), pp. 20-36. Springer, Cham, 2020.
• Ruba Abu Khurma, Ibrahim Aljarah, Ahmad Sharieh, and Seyedali Mirjalili. Evolopy-fs: An open-source nature-inspired optimization framework in python for feature selection. In Evolutionary Machine Learning Techniques, pages 131–173. Springer, 2020

Support

Use the issue tracker.

evolopy's Issues

bat.py____ line62--65

Find the initial best solution

fmin = min(Fitness)
I=numpy.argmin(fmin)   
best=Sol[I,:]

Here fmin will be a scalar, so all the time I will be zero Am I right?
It should be I=numpy.argmin(Fitness)

Can you please comment on it

Urgent Question

Thanks for great repo.
I have a data of numpy array with shape = (10000,10).Can you pls tell how can i do the benchmarking on my data?

GWO Algorithm

Hi,
I have been using the GWO algorithm and noticed that the position and score for beta and delta don't get updated to the second or third best position (or previous values of alpha or beta) where the iteration constantly generates better value than current alpha.
If the obj function gives a value better than current alpha, shouldn't the current value of alpha get demoted to beta instead of getting lost?
eg. in your code
if fitness<Alpha_score :
Alpha_score=fitness; # Update alpha
Alpha_pos=Positions[i,:]

instead shouldn't this be -
if fitness<Alpha_score :
Delta_score=Beta_score # Update delta
Delta_pos=Beta_pos
Beta_score=Alpha_score # Update beta
Beta_pos=Alpha_pos
Alpha_score=fitness; # Update alpha
Alpha_pos=Positions[i,:]

Thanks and Regards,
Kingshuk

Aranha, C., Camacho Villalón, C.L., Campelo, F., Dorigo, M., Ruiz, R., Sevaux, M., Sörensen, K. and Stützle, T., 2022. Metaphor-based metaheuristics, a call for action: The elephant in the room. Swarm Intelligence, 16(1), pp.1-6.

Aranha, C., Camacho Villalón, C.L., Campelo, F., Dorigo, M., Ruiz, R., Sevaux, M., Sörensen, K. and Stützle, T., 2022. Metaphor-based metaheuristics, a call for action: The elephant in the room. Swarm Intelligence, 16(1), pp.1-6.

where is EvoloPy-FS?

can you help to find
EvoloPy-FS: An Open-Source Nature-Inspired Optimization Framework in Python for Feature Selection

Hybrid HHO-MFO (FCMHMD)

This is a wonderful project. Thanks so much. Please can you implement also the FCMHMD algorithm (per 'A hybrid Harris hawks-moth-flame optimization algorithm including fractional-order chaos maps and evolutionary population dynamics' by Mirjalili et al.

A question

Hi,

Thanks for your efforts on this great repo.

In CS.py in the main loop, I understood the following:

new_nest=get_cuckoos(nest,bestnest,lb,ub,n,dim) is equivalent to
Move a cuckoo randomly via Levy flights in CS pseudo code.

Then, fnew,best,nest,fitness=get_best_nest(nest,new_nest,fitness,n,dim,objf) is equivalent to
Evaluate its fitness Fi

Next, new_nest=empty_nests(new_nest,pa,n,dim) is equivalent to
Randomly choose nest among n available nests

My question is:

Where is this step 👇🏼 ? I didn't find it in the code:
if (Fi > Fj) replace j by the new solution

GWO.py line65-line78

can u provide ref about your special mechanism of updating, it's looks like shifting, and it better than orignal GWO
https://github.com/mzychlewicz/GWO

运行example时报错

run(optimizer, objectivefunc, NumOfRuns, params, export_flags)
a = numpy.concatenate(
File "<array_function internals>", line 200, in concatenate
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (4,) + inhomogeneous part.

User defined-MILP

Hi, I have skimmed most of the codes in EvoloPy, and I think it outstanding from my perspective as a freshman. However, I wanna to solve a Mixed-Integer Linear Programming problem with binary variants which have over 4 subscripts. If so could you please tell me if MILP is proper here or kindly explain how that could be done? Thank you very much.

MVO Algorithm

Hello everyone,
When i try to use MVO Algorithm I have some puzzles.
In the function of "RouletteWheelSelection", it always return 0 or -1. In addition, after the following processing, the value of White_hole_index can only be 0.
"
if White_hole_index == -1:
White_hole_index = 0
White_hole_index = 0
"
I tried to modify according to my understanding, but I don’t know if it is reasonable. Please check the request.

Please can we get the multi-objective version of those metaheuristic?

Bat Algorithm

Hello everyone,
When i try to use Bat Algorithm i got this error.

EvoloPy-master\optimizers\BAT.py", line 96, in BAT
if Fnew<=fmin:

UnboundLocalError: local variable 'fmin' referenced before assignment

question about a and r1 and r2

a and r1 and r2 are random numbers while according to "S. Mirjalili, S. M. Mirjalili, A. Lewis, Grey Wolf Optimizer, Advances in Engineering Software , vol. 69, pp. 46-61, 2014, DOI: http://dx.doi.org/10.1016/j.advengsoft.2013.12.007"
they should be arrays. why these parameters are set to be random numbers in this code?

multivariables only have one constraint set?

hello, I was looking over your code, and I was wondering if all the variables in a function share the same lb and ub. It seems like that is the case. Is there a way to have it so that the user can define constraints for each variable?

For example, if:

def objf(x):
    return x[0]+x[1]

lb could be an array or length 2 (or ndim), like [-10, 20]
and the same with ub.

If I am completely mistaken and there already exists this capability, please let me know.

If not, I would love to contribute to this repo by making a PR. Let me know if that is the case.

thanks!

Code broken

I tested the MothFlame algorithm, after hacking it to able to run, and what I figured out? The Moth Flame implementation is broken. It does not work! I t simply does not converge, at least after a certain point.

Brilliant work. Massive amounts of time wasted. And I have a small time budget left. A would like to deeply thank for being hindered by it! Two python implementations on the internet, and BOTH broken!

Hopefully this MFO code will be deleted or fixed after that. Of course, since I have been systematically IGNORED so far, I do not expect it to happen.

Constrains issue in FFA

The algorithm generates numbers larger than the upper bound

I need help can you tell me the names of benchmark functions you used

can you please tell me the names of benchmark functions defined in benchmarks.py in evolopy

Is there any detailed availble code samples

Dear sir
I am a masters degree student who wishes to use your package in the implementation of my research
however there is one simple difficulty which is the lack of any detailed code samples that give some guidence on how to use your package efficently
regards
Ayman

Regarding Evolopy framework

Sir/Madam,

How to apply our own dataset to the Evolopy framework
If I want to use the framework for feature selection.Fitness of the feature selection based on the Filter and Wrapper methods?How can customize the framework?

Thanks in advance.

PSO velocity limit default value

Hi,
the value of PSO velocity clamping constant (Vmax) should be proportional to the search space range. However, it is set to a constant value (6). This brings a lot of trouble for inexperienced users.

User defined deep learning model

activation_cnn_layer = [
'relu',
'tanh']

activation_feed_layer = [
'relu',
'tanh']

optimizers=["adam",'sgd']

map from real number [0, 1] to integer ranging [1, 3]

def swap_activation_cnn(val):
if val == 1:
return 2
return int(val * 2 + 1)

def swap_activation_feed(val):
if val == 1:
return 2
return int(val * 2 + 1)

def swap_optimzer(val):
if val == 1:
return 2
return int(val * 2 + 1)

def no_of_epoch(val):
return int(50)

map from real number [0, 1] to integer ranging [5, 15]

def no_of_batch(val):
return int(val *224 + 32)

def no_of_filters(val):
return int(val*8 + 1)

def no_neurons_hidden_layer(val):
return int(val*64+8)

def kernel_size(val):
return int(val1+10)
def dropout(val):
return int(val1+0.1)

class cnnbenchmark():
def init(self):
self.Lower = 0
self.Upper = 1
def function(self):
# our definition of fitness function
def evaluate(D, solution):
acc_cnn_layer = activation_cnn_layer[swap_activation_cnn(solution[0]-1)]
print(acc_cnn_layer)
acc_feed_layer = activation_feed_layer[swap_activation_feed((solution[1]) - 1)]
optimizer = optimizers[(swap_optimzer(solution[2])- 1)]
epochs = no_of_epoch(solution[3])
batch=no_of_batch(solution[4])
filters=no_of_filters(solution[5])
neurons=no_neurons_hidden_layer(solution[6])
kernel_s=kernel_size(solution[7])
drop_neurons=dropout(solution[8])

      accuracy = 1 - model.model_build(acc_cnn_layer,acc_feed_layer,optimizer,epochs,batch,filters,neurons,kernel_s,drop_neurons)
      scores.append([accuracy,acc_cnn_layer,acc_feed_layer,optimizer,epochs,batch,filters,neurons,kernel_s,drop_neurons])
      return accuracy
  return evaluate

class model():
def model_build(acc_cnn_layer,acc_feed_layer,
optimizer,epochs,batch,filters,neurons,kernel_s,drop_neurons):
print("Activation cnn layer :",acc_cnn_layer)
print("Activation feed layer :",acc_feed_layer)
print("optimizer :",optimizer)
print("batch :",batch)
print("epochs :",epochs)
print("dropout :",drop_neurons)
print("no of filters :",filters)
model = Sequential()
model.add(Conv2D(64, (3,3), activation=acc_cnn_layer, input_shape=input_shape))
model.add(Conv2D(128, (3,3), activation=acc_cnn_layer))
model.add(Dropout(drop_neurons))
model.add(Flatten())
model.add(Dense(6*numPCAcomponents, activation=acc_feed_layer))
model.add(Dropout(drop_neurons))
model.add(Dense(16, activation='softmax'))

# Define optimization and train method
reduce_lr = ReduceLROnPlateau(monitor='val_acc', factor=0.9, patience=25, 
                              min_lr=0.000001, verbose=1)
checkpointer = ModelCheckpoint(filepath="checkpoint.hdf5", verbose=1, 
                              save_best_only=False)
# sgd = SGD(lr=0.001, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=optimizer, 
                              metrics=['accuracy'])
# Start to train model 
history = model.fit(X_train, y_train, 
                    batch_size=batch, 
                    epochs=epochs, 
                    verbose=1, 
                    validation_data=(X_test, y_test),
                    callbacks=[reduce_lr, checkpointer],
                    shuffle=True)

model.save('HSI_model_epochs100.h5')
model = load_model('HSI_model_epochs100.h5')
score = model.evaluate(X_test, y_test, batch_size=32)
Test_Loss = score[0]
Test_accuracy = score[1]

return Test_accuracy

I want to find the optimal hyperparameters of above deep learning model using a grey wolf optimizer. Can you please help me in this process using EvoloPy library.

Issues with HH

Two questions regarding HHO:

1. EvoloPy/optimizers/HHO.py

   Line 117 in db8424f

   if objf(X1)< fitness: # improved move?

if objf(X1)< fitness:

fitness is set in the loop at the beginning.
It is the value of the last X vector, is this intentional?
Why the last one?

EvoloPy/optimizers/HHO.py

Line 92 in db8424f

X[i,:]=(Rabbit_Location - X.mean(0))-random.random()*((ub-lb)*random.random()+lb)

X[i,:]=(Rabbit_Location - X.mean(0))-random.random()*((ub-lb)*random.random()+lb)

is consistent with the paper but still looks strange.
(ub-lb)*random.random()+lb is a random phenotype vector whithin the bounds.

Think about bounds
lb = [100000]*dim
ub = [100001]*dim

then random.random()*((ub-lb)*random.random()+lb) probably doesn't do what it should.
My feeling is that the scaling should be for the genotype, not the phenotype.

In https://github.com/7ossam81/EvoloPy/blob/master/benchmarks.py all boundaries
are around 0 or have lower bound 0. May be you can include a test with shifted boundaries
for an existing test function:

f(x) = ...
shift_f(x) = f(100000 + x)

shift_f should be equivalent to f with shifted boundaries.

Cheers Dietmar

Add unit tests

Without unit tests it's difficult to know if contributions are correct or not.

Not able to use the framework with own data

How can we use our own data with these functions or load any datasets like California housing etc? It is shown in the google collab code but not sure how to use it. Is there an example code snippet?

empty_nests parameters

Hi,

Thanks for your repo.

According to the following lines, I'm wondering why you passed new_nest to empty_nests ??
I checked the original code in Matlab they passed nest

new_nest=get_cuckoos(nest,bestnest,lb,ub,n,dim)
           
# Evaluate new solutions and find best
fnew,best,nest,fitness=get_best_nest(nest,new_nest,fitness,n,dim,objf)
         
new_nest=empty_nests(new_nest,pa,n,dim) ;

Here is Matlab code:

% Generate new solutions (but keep the current best)
new_nest=get_cuckoos(nest,bestnest,Lb,Ub);   
[fnew,best,nest,fitness]=get_best_nest(nest,new_nest,fitness);

% Update the counter
N_iter=N_iter+n; 

% Discovery and randomization
new_nest=empty_nests(nest,Lb,Ub,pa) ;

F21, F21, F23 are wrong

fixed:

Error with benchmark F12

There is an error with F12. I have comment it out.

Relax python3 requirement

The code seems to work in python2. I only saw issues with PEP238, integer division behavior, in FFA.py, alpha_new. I'm not a python3 user but I think all that is needed is to add these lines at the front of each routine:

from __future__ import division
from __future__ import print_function

Or be explicit when doing floating point arithmetic, for example in alpha_new:

 delta=1.-(10.**(-4.)/0.9)**(1./NGen); 

I need help

EvoloPy/optimizers/HHO.py

Line 49 in 5f12bf5

s = solution()

In Python 3, this package doesn't have this usage。so what can I do to get the code to work?

Doubts about the code

Hi some doubts (many actually) about the code:

• What "lb = [lb] * dim" and "ub = [ub] * dim" does precisely? I know they are determine a maximum and minimum value that the moths can take (I assume that to make the algorith to find a answer faster), but what the "[]" means and way are the dim variable involved?

• Some reason line n°28: "numpy.random.uniform(0, 1, N)" is generating values between 0 and 1?

• What means "inf" on line 29 "Moth_fitness=numpy.full(N,float("inf"))". What is that necessary as the initial values?

• "Convergence_curve" (declared on line 32) is just a container for the best flame scores in each iteartion, so one can identify how the algorith is converging to a answer?

• What "objf" (on line n°73) does. Is it the function to be appoached? It is applied to an entire line of the moth matrix?

• On line 79: "fitness_sorted=numpy.sort(Moth_fitness)", does "Moth_fitness" remains sorted or only "fitness_sorted" ?

• What "argsort on" line n°80 "I=numpy.argsort(Moth_fitness)" and on line n°95 "I2 =numpy.argsort(double_fitness)" does?

• The line n°82 "sorted_population=Moth_pos[I,:]" copy all the rows from Moth_pos's line n° I to sorted_population?

• What this line (n°90 and also the 91) "double_population=numpy.concatenate((previous_population,best_flames),axis=0)" does? I mean it creates something with size 2N x dim?. And what the "axis" means?

• What this line does: "double_sorted_population[newindex,:]=numpy.array(double_population[I2[newindex],:])" It simple copies the content of "double_population" to "double_sorted_population" ?

• On line 111: "Best_flame_pos=sorted_population[0,:]" Is being copied an entire line os the positions of a certain moth?

-Can you give me an overall explanation why there are so many structures, like "previous_population", "sorted_population", "double_population", "double_sorted_population"? Plus the "Moth" structure. Can't they al be replaced by "Moth" alone? Declare it with 2x the number of moths and update it instead of creating so many other? Perhaps moth and previous_population at maximum?

BAT Optimization Algorithm

Hello everyone. When I try to use BAT to optimizers such as Ackley and Rastrigin and ext., I don't get the result near to 0. For example for the Ackley function, with 2000 iterations I get 18, compared with other algorithms(for example with GWO), by using them I get approximately 0. What can be the issue? Thanks in advance.

About a license issue...

Hi, I am a graduate student studying about optimization in Korea. I want to use this open source in a project to help student studying about optimization, but might be also used by some commercial companies. So I'm wondering about the license things like permissions, limitations, or conditions to use this open source.
Would you state about the license of the EvoloPy?

Vectorized objective function extensions added (please commit it in the master)

Dear all

I just added vectorized=False and modified all optimizer to execute in vectorized way the objective function. In additional the python implementation does not store solution individual best thus i also added it.

I am not sure, but you have to check whether all "best position" vectors get copied in python (especially for numpy objects) otherwise the alignment will only be set by reference. I didn't yet reviewed everthing up to now.

Kind regards
Adrian Egli

EvoloPy.zip

Write a CONTRIBUTING.md

For people who want to contribute new optimizers, for instance.

User defined function

I am new to coding and I was wondering if it is possible to use the Evolopy framework with a user-defined function other than the benchmark functions. If so could you please kindly explain how that could be done?

MFO Algorithm, the search agent number is hard coded.

in optimizer.py, the PopulatioinSize would not effect MFO algorithm since in optimzers/MFO.py , the N=50 overwrite the function parameter which was passed in.
I discover the issue when i was increasing the population size, and the benchmark time don't quite change

Benchmark F3 always gives 0

Please @aljarrahcs check this function. It return always 0

F12 is wrong

fixed:

ref:

please, i compare the result with the same algorithm in matlab that gave different plotting the reason is that in matlab use semilogy function to plot, when i used it i can't plot negate valu can you help?

Dataset

From where did you pick the DATA from the entire code?

Function Evaluations (FES) - to be addded.

@7ossam81

Example or document

Dear Authors,
I would like to use EvoloPy for my own dataset. Can you send me an example or documents on how I can use the EvoloPy?

Regards,
Dr Sadiq Hussain,
India

F13 is wrong

fixed:

ref:
SEYEDALI MIRJALILI(GWO)

Can't run it.

executing "python optimizer.py" returns: "No module named optimizers.PSO"
How to solve it?