ccsi-toolset / foqus Goto Github PK
View Code? Open in Web Editor NEWFOQUS: Framework for Optimization and Quantification of Uncertainty and Surrogates
Home Page: https://foqus.readthedocs.io
License: Other
FOQUS: Framework for Optimization and Quantification of Uncertainty and Surrogates
Home Page: https://foqus.readthedocs.io
License: Other
FOQUS start-up splash screen has overlapping text.
Chat history:
John Eslick added a comment - 17/Oct/14 5:44 AM - edited
Can you post a screenshot? I think this happens sometimes, but I haven't seen it. A picture may help me figure out how to fix it.
Natalia Kitch (Inactive) added a comment - 17/Oct/14 4:58 PM
I'm getting this, start-up screen overlap, all the time.
John Eslick added a comment - 23/Oct/14 8:04 AM - edited
I think its due to the option in windows that lets you increase the size of fonts other things.
For me on normal everything is fine. On medium 125% things look a bit off but its still readable. On large 150% things overlap as in the screen shot. I'm not sure this can be fixed for the release, but I have three ideas.
From JIRA:
Description:
Numerous thread safety type issues. Most noticeably some things can be changed in the GUI while something is running. There are a lot of those, and it can lead to some weird confusing behavior.
Other things are less clear, like saving a FOQUS session while optimization or UQ are running.
These are issues that aren't a big deal if you know what's going on, but are probably a major pain for a new user. This issues isn't really complicated, but it would take quite a bit of time to sort everything out.
Integration with FOQUS/Turbine going forward with DMF.
In FOQUS, provide the functionality to browse and select versions of simulations. This involves changes in FOQUS to select simulations from the DMF and not Turbine. Also involves Turbine populating the DMF with simulations to the DMF Simulations folder. This may be done when FOQUS is first loaded or when Turbine is first activated, although this is undecided, and there are concerns that it may be inefficient.
Concerns about dependency tracking.
a) Version checking between Turbine and the DMF may not be working correctly.
b) FOQUS needs to track simulations with versions from DMF
Features targeted for 2015.01.0
Features change log for 2015.03.0
Feature changes for 2015.06.00
Feature changes for 2015.10.00
Feature changes for 2016.02.00
Next release:
I am not sure if this is a SimSinter issue, but lets see. I am getting several errors from aspen (see attached figure).
The figure has 10-20, but I left the computer running all the night and this morning I had to close about 100-150.
From JIRA:
Description:
As you submit jobs to turbine, it seems to take longer and longer to submit jobs. After about 10,000 jobs this seems to lead to significant slow down on the Turbine server. I suspect and inefficient SQL query may be the cause. I think it also happens in TurbineLite. When I first install it job submission is very fast and slows down the more I use it.
Chat history:
Abdelrahman Elbashandy added a comment - 26/Jan/16 4:52 PM
Does this happen only on a single submission or in general?
Josh, should we backup the data and clean up the database periodically?
Joshua Boverhof added a comment - 26/Jan/16 9:55 PM
I suspect it is an ORM layer inefficiency. Since it's present in TurbineLite we shouldn't have too much trouble replicating the issue. Just grab a SQL Compact database file and store it with the test code.
Once we have a few unittests that we can use to easily time operations we can dig in a little and replace some of the inefficient ORM code.
From JIRA:
Description:
Run the MEMCAP simulation with FOQUS with the current Jenkins build.
Running Membrane Simulation
1.Upload files to Turbine. Refer to steps and screenshot. ◦The files are attached to jira issue TURBSCIGATE-88 in a zip. Get and extract files
◦Run FOQUS
◦Select Add/Upload simulation to turbine from the main menu
◦Click the browse button and select the sinter config file MEMCAP.json. This will automatically fill in the simulation name, and add the sinter config and simulation resources.
◦The simulation depends on a bunch of files in the AM_MEMCAP directory. Select the add files button. Go to the AM_MEMCAP directory and select all the files (you can select multiple files).
◦Select the resource files (all except the sinter config and aspen file) to be placed in the AM_MEMCAP subdirectory of the simulation working directory. You can select the third file in the file table, then hold shift and select the last file.
◦Click the resource relative path button and enter AM_MEMCAP in the dialog box that appears.
◦Upload the files to turbine by pressing okay.
2.Add a node to the flowsheet. ◦Select the flowsheet tab
◦Click the add node button
◦Click anywhere on the flowsheet
3.Set the node to use the MEMCAP simulation ◦Click the node editor button
◦Select turbine from the model type pull-down
◦select MEMCAP from the model pull-down
4.Run the model by clicking the run button. It should complete successfully, but due to the bug subsequent runs may or may not be successful.
Attachments:
From JIRA:
Description:
the "select all variables" button must select all the variables and not the "status" and "graph.error".
Another option is to remove the status and graph.error from the list of possible output variables to generate surrogate models.
setup.py
tries to bundle nssm.exe, SqlCeCmd40.exe into the foqus distribution. These executable were not imported from SVN. The question is can these be safely removed from the distribution? If not, how do we handle these dependencies?
Instead of hardcoding the product version in the setup.py, we should pull version from git tags.
$ git describe --tags
2015.10.00
Combine the 2D and 3D models and update documentation to reflect the merger.
There is a component to FOQUS/Turbine that called Turbine Hydro. It has a separate repository, but depends on the FOQUS installer to be installed. In the past we had to check it out of SVN and put it in the right place then run the FOQUS setup.py. I think we need to give Hydro its own installer then make it a dependency or optional package. I don't think its a critical part of FOQUS unless you want to use Turbine to run parallel FOQUS jobs. We will need to get some help from Josh to sort this stuff out.
From JIRA:
Description:
After running 2000 simulations, the computer shutdown. I can see the simulations are stored in the log file. Can we recover this results?
In the flowsheet "import" panel, can we include an option to import from log file? (FOQUS allows imports from "clipboard" and "csv file")
UQ currently displays the name of the variables including the name of the node + the variable name, making the variable tag/name too long to be displayed and read correctly.
I've attached two examples (figures). In the actual size of the figure you can figure out which columns belong to each variable, but, for a paper you need to reduce the size of the figure (very small), and the labels can not be read correctly.
I uploaded the AspenPlus backup file to SinterConfigGUI.
After filling out the information in the home page, I was taken to the SinterConfigGUI Variable Configuration Page.
Here, I included the reactor temperature as an input variable. I changed the name of the variable to "rxr_TEMP" (shown in the file "Heat_Integration_Bug_1"). When I clicked "Finish", SinterConfigGUI shuts down (shown in the file "Heat_Integration_Bug_2").
{
"title": "Heat_Integration_Bug",
"config-version": "1.27",
"description": "This is the file for the heat integration example.",
"model": {
"file": "Heat_Integration_Example_03_20_2017.bkp",
"DigestValue": "ae0e4b9dc0418f107de3938272570ebbc90870c2",
"SignatureMethodAlgorithm": "sha1"
},
"input-files": [],
"author": "Byron Soepyan",
"date": "4/11/2017",
"filetype": "sinterconfig",
"filetype-version": 0.3,
"application": {
"name": "Aspen Plus",
"version": "30.0",
"constraint": "AT-LEAST"
},
"inputs": {
"flash_PRES": {
"path": [
"\\Data\\Blocks\\FLASH\\Input\\PRES"
],
"type": "double",
"default": 45.0,
"description": "Lets you type the pressure. Absolute units: outlet pressure if value > 0; pressure drop \nif value <= 0. Gauge units: outlet pressure for all values. See Help.",
"units": "bar",
"min": 2.5,
"max": 50.0
},
"TEMP_flash": {
"path": [
"\\Data\\Blocks\\FLASH\\Input\\TEMP"
],
"type": "double",
"default": 32.6,
"description": "Lets you type the outlet temperature. See Help.",
"units": "degC",
"min": 25.0,
"max": 150.0
},
"rxr_PRES": {
"path": [
"\\Data\\Blocks\\REACTOR\\Input\\PRES"
],
"type": "double",
"default": 50.0,
"description": "Reactor outlet pressure or pressure drop. Absolute units: value > 0 - outlet\npressure; value <= 0 - pressure drop. Gauge units: outlet pressure.",
"units": "bar",
"min": 25.0,
"max": 50.0
},
"TEMP_rxr": {
"path": [
"\\Data\\Blocks\\REACTOR\\Input\\TEMP"
],
"type": "double",
"default": 200.0,
"description": "Reactor outlet temperature.",
"units": "degC",
"min": 200.0,
"max": 250.0
},
"split_frac": {
"path": [
"\\Data\\Blocks\\SPLITTER\\Input\\FRAC\\11"
],
"type": "double",
"default": 0.5,
"description": "Fraction of inlet stream going to this outlet stream.",
"units": null,
"min": 0.01,
"max": 0.9
},
"syngas_flow": {
"path": [
"\\Data\\Streams\\SYNGAS\\Input\\TOTFLOW\\MIXED"
],
"type": "double",
"default": 100.0,
"description": "Total flow. Required if -Frac or -Conc composition basis is used. If -Flow composition basis\nis used & total flow entered, component flows are normalized. Stdvol is standard liquid volume.",
"units": "kmol/hr",
"min": 100.0,
"max": 100.0
},
"syngas_pres": {
"path": [
"\\Data\\Streams\\SYNGAS\\Input\\PRES\\MIXED"
],
"type": "double",
"default": 10.0,
"description": "Lets you type the stream pressure.",
"units": "bar",
"min": 10.0,
"max": 10.0
},
"syngas_temp": {
"path": [
"\\Data\\Streams\\SYNGAS\\Input\\TEMP\\MIXED"
],
"type": "double",
"default": 25.0,
"description": "Lets you type the stream temperature.",
"units": "degC",
"min": 25.0,
"max": 25.0
}
},
"outputs": {
"Rxr_TEMP": {
"path": [
"\\Data\\Blocks\\REACTOR\\Output\\B_TEMP"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "degC"
},
"Rxr_Duty": {
"path": [
"\\Data\\Blocks\\REACTOR\\Output\\QNET"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "GJ/hr"
},
"Cooler1_Duty": {
"path": [
"\\Data\\Blocks\\COOLER1\\Output\\QNET"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "GJ/hr"
},
"Cooler2_Duty": {
"path": [
"\\Data\\Blocks\\COOLER2\\Output\\QNET"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "GJ/hr"
},
"Heater1_Duty": {
"path": [
"\\Data\\Blocks\\HEATER1\\Output\\QNET"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "GJ/hr"
},
"Heater2_Duty": {
"path": [
"\\Data\\Blocks\\HEATER2\\Output\\QNET"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "GJ/hr"
},
"Heater3_Duty": {
"path": [
"\\Data\\Blocks\\HEATER3\\Output\\QNET"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "GJ/hr"
},
"Cooler1_Tin": {
"path": [
"\\Data\\Streams\\1\\Output\\TEMP_OUT\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "degC"
},
"Cooler1_Tout": {
"path": [
"\\Data\\Streams\\2\\Output\\TEMP_OUT\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "degC"
},
"Cooler2_Tin": {
"path": [
"\\Data\\Streams\\7\\Output\\TEMP_OUT\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "degC"
},
"Cooler2_Tout": {
"path": [
"\\Data\\Streams\\8\\Output\\TEMP_OUT\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "degC"
},
"Heater1_Tin": {
"path": [
"\\Data\\Streams\\4\\Output\\TEMP_OUT\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "degC"
},
"Heater1_Tout": {
"path": [
"\\Data\\Streams\\5\\Output\\TEMP_OUT\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "degC"
},
"Heater2_Tin": {
"path": [
"\\Data\\Streams\\9\\Output\\TEMP_OUT\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "degC"
},
"Heater2_Tout": {
"path": [
"\\Data\\Streams\\PRODUCT\\Output\\TEMP_OUT\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "degC"
},
"Heater3_Tin": {
"path": [
"\\Data\\Streams\\11\\Output\\TEMP_OUT\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "degC"
},
"Heater3_Tout": {
"path": [
"\\Data\\Streams\\BYPRODUC\\Output\\TEMP_OUT\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "degC"
},
"Flash_Duty": {
"path": [
"\\Data\\Blocks\\FLASH\\Output\\QNET"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "GJ/hr"
},
"Flash_TEMP": {
"path": [
"\\Data\\Blocks\\FLASH\\Output\\B_TEMP"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "degC"
},
"comp1_work": {
"path": [
"\\Data\\Blocks\\COMP1\\Output\\WNET"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "kW"
},
"comp2_work": {
"path": [
"\\Data\\Blocks\\COMP2\\Output\\WNET"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "kW"
},
"comp3_work": {
"path": [
"\\Data\\Blocks\\COMP3\\Output\\WNET"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "kW"
},
"METHANOL_frac": {
"path": [
"\\Data\\Streams\\PRODUCT\\Output\\X\\METHANOL"
],
"type": "double",
"default": 0.0,
"description": null,
"units": null
},
"product_profit": {
"path": [
"\\Data\\Streams\\PRODUCT\\Output\\COST\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "$/hr"
},
"byproduct_profit": {
"path": [
"\\Data\\Streams\\BYPRODUC\\Output\\COST\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "$/hr"
},
"syngas_cost": {
"path": [
"\\Data\\Streams\\SYNGAS\\Output\\COST\\MIXED"
],
"type": "double",
"default": 0.0,
"description": null,
"units": "$/hr"
}
}
}
Currently, Jenkins creates the iREVEAL zip file via a Windows command execution script. That script should be copied into a make.bat (or some such name), checked into svn and then called from Jenkins.
This will make it possible for change to the building of that .zip file to be more easily done by developers.
Chat history:
John Eslick added a comment - 25/Feb/16 12:13 PM
There are a couple things:
A zip file without the docs would be nice. If we still want to have iREVEAL possibly as a separate stand alone product we can have an additional zip with docs.
The spaces in the documentation file name are causing a little trouble writing the WiX files. It's not a big deal, but to make things easier I rename it in the build process and replace the spaces by underscores. The FOQUS build is getting overly complicated, but maybe that doesn't matter because, we can probably merge the iREVEAL manual into the FOQUS for the next release.
From JIRA:
Description:
This new setting helps to improve the quality of the models. It works to scale the output variables, transform data to find better fitting (exp and log).
Include a new column for each output variable adding the option to transform the data (zscale, exp, log).
Changes: Each new transformation is added to the output variables information:
noutputs (outputs + #transfunctions)
maxterms (add a column for each transfunction)
Ignore (add a column for each transfunction)
tolrelmetric (add a column for each transfunction)
tolmeanerror (add a column for each transfunction)
zlabels (add a name for each transfunction "not the same output")
Include in the alamo input file ("before BEGIN_DATA"):
Example for 4 transformations:
Ntrans 4
BEGIN_TRANSFORMS
GasOutput1000
log(GasOutput)
exp(GasOutput)
GasConcentration1000
END_TRANSFORMS
In the effort to prepare the foqus repository for open source, it was necessary to remove the compile binaries for pyDRMSampling. This build has been changed to expect the python home directory and coin library directory. It is not clear whether these change will allow the project to be loaded easily into Visual Studio IDE.
echo *****************************
echo Clean and Build pyDRMSampling
echo *****************************
+MSBuild.exe /p:Configuration=Release /p:PythonDir=%PYTHON_DIR% /t:Clean pyDRMSampling\pyDRMTraining.sln || goto :error
+MSBuild.exe /p:Configuration=Release /p:CoinLibDir=%COIN_LIB_DIR% /p:PythonDir=%PYTHON_DIR% pyDRMTraining\pyDRMSampling.sln || goto :error
There are a lot of files needed by FOQUS that are not py files, like ui, qrc, svg, html.... Probably will need a MANIFEST.in file to make sure we get everything.
In the effort to prepare the foqus repository for open source, it was necessary to remove the compile binaries for pyDRMTraining. This build has been changed to expect the python home directory and coin library directory. It is not clear whether these change will allow the project to be loaded easily into Visual Studio IDE.
echo *****************************
echo Clean and Build pyDRMTraining
echo *****************************
+MSBuild.exe /p:Configuration=Release /p:PythonDir=%PYTHON_DIR% /t:Clean pyDRMTraining\pyDRMTraining.sln || goto :error
+MSBuild.exe /p:Configuration=Release /p:CoinLibDir=%COIN_LIB_DIR% /p:PythonDir=%PYTHON_DIR% pyDRMTraining\pyDRMTraining.sln || goto :error
From JIRA:
Description:
How long ACOSSO tutorial has to run, as it is running very long, and it looks like doing nothing, so that it needs to be killed, as Stop function/button doesn't stop it:
Starting ACOSSO
Exporting Data...
Inputs: ["eq.x1", "eq.x2"]
Output: ["eq.z1", "eq.z2"]
Runinng ACOSSO...
Chat history:
Natalia Kitch (Inactive) added a comment - 05/Nov/15 10:27 AM
It appears that "Rgui.exe" was set instead of "RScript.exe" in RScript Path, which caused the hanging.
It would be nice if FOQUS could give a warning in this case, and Stop button would stop the run.
Perhaps, the issue can be closed.
John Eslick added a comment - 16/Nov/15 7:21 AM
Will add checks in January release. Also will try to improve the stop button. The stop button currently stops simulations running for surrogate methods, but doesn't stop the surrogate method itself.
We need to figure out what to do about documentation. Currently the manual, which is very large, is done in LaTeX. Most people probably won't have what they need to build the docs. The primary way for getting FOQUS will be straight from github, so I'm not sure if there is a straight forward way for us to build the docs to be available on github, other than having someone build them and check them in.
I hate to suggest this, but should we convert the docs to something like sphinx? At least we know that the users should be able to build the docs maybe as part of the install. I'm not sure what's best. @ksbeattie any ideas?
Originally, *.pyd binaries compiled from pyDRMTraining and pyDRMSampling were checked in to SVN. For compatibility with open source conventions, the binaries were removed from the repository. Now these libraries need each to have a build script that generates *.pyd libraries.
The tests developed with "TestComplete" should be added to our Jenkins instance. The end goal of this issue is to have a Jenkins project which performs at least one simple "TestComplete" UI test of FOQUS.
The output of this test can just be "raw" text for now. The main goal is simply a proof of concept, the actual useful tests can be created later.
Chat history:
Natalia Kitch (Inactive) added a comment - 12/Feb/15 12:07 PM
In order to run a TestComplete test/script on Jenkins the following needs to happen first at LBL:
Download and install TestExecute:
Step 1: Log into SmartBear secured product download and customer services portal, which provides access to our/LLNL registration information and the installation password for the products we own.
URL https://my.smartbear.com/
Email [email protected]
Customer ID A46570
Password Hazel123
Note: The Customer ID and Password are case-sensitive.
Step 2: Download and install TestExecute v.10 and SmartBear License Manager.
Step 3: Use License Key (provided in the portal) to activate TestExecute.
Step 4: Run Win batch file (will be provided in svn with TestComplete test/script to be run from a command line).
Natalia Kitch (Inactive) added a comment - 19/Feb/15 12:01 PM - edited
This could be a sample of a command line (in Jenkins scripts) executing/running TestComplete Project/KeywordTests/Test1, given TestExecute is installed, for example in "C:\Program Files (x86)\SmartBear\TestExecute 10" directory, and Path to TestComplete PJS (project containing KeywordTest Test1) file is given:
C:> "C:\Program Files (x86)\SmartBear\TestExecute 10\Bin\TestExecut.exe" "C:[Path_To_TestComplete_PJS_file.pjs]" /run /project:"[name_of_project-PJS-file]" /test:"KeywordTests|Test1"
John Eslick added a comment - 28/Oct/15 6:33 AM
The bundleTest.py script in with the FOQUS bundle in SVN should probably be the first thing to run. All you need to do make a directory somewhere and put the script the bundleTestFiles directory and the FOQUS installer in it. The script will return a nonzero error code if a test fails. So far the only way to really know what happened if it fails is the standard output. It's pretty rough.
Here is what the script does.
Keith Beattie added a comment - 24/Feb/16 1:57 PM
This is waiting on me to get a VM in which to run it.
Create a Turbine Science Gateway Cluster on the NETL SciLab cluster.
Chat history:
Joshua Boverhof added a comment - 29/Jan/15 2:21 PM
Need FOQUS consumer for the NETL SciLab Cluster
Joshua Boverhof added a comment - 04/May/16 9:20 AM
The cluster needs to be updated.
We may be able to include SQL compact in the TurbineLite install or with the FOQUS bundle. I'm not sure about the details. Its not very hard to install it separately, so this isn't a big deal. If it's to much work just close the issue, and we can drop the idea.
Chat history:
Abdelrahman Elbashandy added a comment - 30/Mar/16 10:59 PM
Josh, is there a license issue with this or should I proceed with it?
The issue seems to be that FOQUS reads the flowsheet data and somehow it isnt reading the rows as integers. (temporary fix: adding i = int(i), in line 827 of ALAMO.py file).
Error log:
Traceback (most recent call last):
Steam_v2.zip
File "C:\Users\mzamarripa\Documents\FOQUS_main\foqus_lib\framework\surrogate\ALAMO.py", line 515, in run
self.writeAlamoInputFile()
File "C:\Users\mzamarripa\Documents\FOQUS_main\foqus_lib\framework\surrogate\ALAMO.py", line 828, in writeAlamoInputFile
val = np.array(res.resultElement(i, col))
File "C:\Users\mzamarripa\Documents\FOQUS_main\foqus_lib\framework\sampleResults\results.py", line 833, in resultElement
el = row[cindex]
TypeError: 'long' object has no attribute 'getitem'
From JIRA:
Description:
FOQUS.Session->Add/Update Turbin Model->create/edit SinterConfig File ->Sinter Config GUI->browse->select file->loading...
While the above is in process:
Attachments
Chat history:
Natalia Kitch (Inactive) added a comment - 16/Oct/14 4:28 PM
NK: added a snapshot.
John Eslick added a comment - 17/Oct/14 8:02 AM
I think that is normal. It could probably be done better, but the upload window is waiting on the sinter config gui to return. I'll try to fix this in the next version.
setup.py
references two pieces of SimSinter documentation. These were not found in the SVN repository. Should SimSinter become a submodule for this project if only the documentation is needed?
It would be useful to know how many simulations failed while you are running (if too many fail you can just stop all). The only way to do this is manually exploring the table of variables.
Something like 1/200, being updated every simulation.
From JIRA:
Description:
The edge editor allows the user to automatically select the variable connections by clicking the "Auto" button. But right now there is no action after clicking this button. (we have two options: either remove it, or fix it).
I would prefer if we can fix it (it could be very useful when several variables need to be connected).
Chat history:
John Eslick added a comment - 29/Jul/16 12:40 PM
Auto works on variables of the same name, connecting outputs or input in the "from" node to inputs in the "to" node. Do the variables you are trying to connect have the same name? I did a simple test and it's working for me. Can you attach the FOQUS session?
From JIRA:
Description:
Exporting data to clipboard and to cvs files is easy, but when you try to return this data the columns dont match. Some columns make the mismatch error, some of the columns are: Tags, initial, maximum time, settings, homotopy, etc.
From JIRA:
Description:
Changing a flowsheet or optimization settings while a problem is running could cause confusion or errors. Need to disable editing most things while optimization or simulation is running.
From JIRA:
Description:
I tried to stop the simulations and it looked that it was stopped, but, the simulations were running in the background. (I think the issue was that I save the session and by doing this the simulations must stop, but they keep running in the background)
From JIRA:
Description:
When you open foqus directly from the file (double clicking)... you get an error:
Unhandled Exception
File Foqus_lib\gui\surrogate\surrogateFrame.pyc", line 193, in selectTool AttributeError: 'surrogateFrame' object has no attribute 'prevTool'
Attachments:
Add into docs the instructions on where to get and how to install psuade for use within foqus
Perhaps, if possible, look into automating this for CI builds.
From JIRA:
Description:
When solving multiple node simulations, it could be useful to highlight which node its being simulated at the time.
The current installer is not signed with a digital certificate and the publisher is not identified - producing a Security Warning on launching the installer. (See screenshot).
Steps to Reproduce:
Download the version 0.2.c0 installer MSI for TSG from the project website.
Double-click the MSI to initiate the installation.
RESULT:
Security warning presented. (Can click Run to continue so no loss in functionality)
Expected Result:
Installer welcome screen is presented.
Copyright_Assertion_Approval.pdf
is distributed with (see setup.py
) with foqus. Is this needed or should it be switched to bundling LICENSE.md
?
FOQUS supports vector variables, but the have to be entered into cells like [5.0, 6.0, 7.0]. It would be nice to have an editor that would show a spreadsheet like view that would allow you to edit the value, default, min, max and number of elements in a convenient way.
From JIRA:
Description:
Betty: I remain quite baffled by the inconsistent behavior related to the min/max in the node setting and the parameter distribution definition in UQ. In fact, I am wondering if the linkages between the flow-sheet page and the UQ page are broken. From a user’s standpoint, I was expecting either (1) Node definition is optional (for information only) and UQ controls behavior of all variables, or (2) Node definition is the global control, UQ changes does not override node definition. So the current situation seems really confusing
Jeremy: I can understand the confusion. We developed this in mind with the node definition being the global control, but allowing the user to modify min and max within that global range. In this way, you can create several ensembles with different ranges. I understand right now FOQUS is not enforcing the min and max to be within the node’s global range, but that will be in future release.
Betty: A more consistent behavior between node definition and UQ in future releases will be appreciated.
Chat history:
John Eslick added a comment - 23/Oct/15 11:05 AM
Lets change the UQ form so that changing the min/max there actually changes it on the flowsheet, or ignores the flowsheet settings and lets to go outside those ranges, and only uses the flowsheet settings for the initial value.
Either way is okay with me, and it seems Betty is okay with either way. It seems I often run into problems were I want to change the UQ min or max, but am not allowed. Then I have to spend a bunch of time figuring out what flowsheet min or max I may have violated.
I'm not sure how to make these changes, but if you need my help let me know, or reassign it back to me with a bit more info on how to proceed.
From JIRA:
Description:
There is an OUU test problem in examples/OUU/Toy2/ouu_Toy2.foqus, and I'm having trouble getting it to work. This example uses ACM to do the inner optimization. The recourse variables are outputs of the ACM simulation. To set up OUU, I selected full flowsheet. The primary optimization variables are d1, d2, d3, and d4. The uncertain parameters are, omega1 - omega4. The output variable for objective is z.
When I go to run I get the error below:
2016-10-27 14:16:28,134 - DEBUG - foqus.foqus_lib.framework.sim.turbineConfiguration - Stopped all running consumers
Traceback (most recent call last):
File "C:\Users\jeslick\work\foqus\foqus_lib\gui\ouu\ouuSetupFrame.py", line 902, in analyze
optDriver = optDriver, ensOptDriver = ensembleOptDriver, plotSignal = self.p
lotSignal, endFunction=self.finishOUU)
File "C:\Users\jeslick\work\foqus\foqus_lib\framework\ouu\OUU.py", line 314, in ouu
inputLowerBounds=inputLB, inputUpperBounds=inputUB,
UnboundLocalError: local variable 'inputLB' referenced before assignment
exiting foqus listener
From JIRA:
Description:
After ALAMO execution it would be nice to visualize the model obtained. Attached (fig and python code) you can find an example of the figure that displays data samples vs surrogate model.
Other comments:
Attachments:
Python code:
import numpy as np
import matplotlib.pyplot as plt
from openpyxl import load_workbook
from math import exp, expm1, pow
wb = load_workbook('Newdata.xlsx')
ws = wb['Sheet1']
cell_range = ws['L2:L144']
tuple(ws.iter_rows('L2:L144'))
GasInT=np.array([])
GasInCO2=np.array([])
Lb=np.array([])
Dx=np.array([])
Lhx=np.array([])
SolidInT=np.array([])
Dt=np.array([])
Out_GasOutT=np.array([])
N1=np.array([])
for row in ws.iter_rows('L2:L144'):
for cell in row:
#print(cell.value)
N1=np.array([cell.value])
GasInT = np.hstack([GasInT,N1])
for row in ws.iter_rows('M2:M144'):
for cell in row:
#print(cell.value)
N1=np.array([cell.value])
GasInCO2 = np.hstack([GasInCO2,N1])
for row in ws.iter_rows('U2:U144'):
for cell in row:
#print(cell.value)
N1=np.array([cell.value])
Lb = np.hstack([Lb,N1])
for row in ws.iter_rows('V2:V144'):
for cell in row:
#print(cell.value)
N1=np.array([cell.value])
Dx = np.hstack([Dx,N1])
for row in ws.iter_rows('X2:X144'):
for cell in row:
#print(cell.value)
N1=np.array([cell.value])
Lhx = np.hstack([Lhx,N1])
for row in ws.iter_rows('P2:P144'):
for cell in row:
#print(cell.value)
N1=np.array([cell.value])
SolidInT = np.hstack([SolidInT,N1])
for row in ws.iter_rows('W2:W144'):
for cell in row:
#print(cell.value)
N1=np.array([cell.value])
Dt = np.hstack([Dt,N1])
## data samples for the output variable
for row in ws.iter_rows('AJ2:AJ144'):
for cell in row:
#print(cell.value)
N1=np.array([cell.value])
Out_GasOutT = np.hstack([Out_GasOutT,N1])
## evaluation of the first model
Int1 = np.array(Dx*GasInCO2*Lb)
Int2 = np.array(GasInCO2*GasInT*Lhx)
#GasOutT= -62 * math.pow(Dx*GasInCO2*Lb,0.1) + 0.11E3 * math.pow(GasInT*GasInCO2*Lhx,0.1)
GasOutT= -62 * np.power(Int1,0.1) + 0.11E3 * np.power(Int2,0.1)
import matplotlib.pyplot as plt
## new is the vector of sample points (from 0 to 142,
## the value depends on the number of samples)
New=np.linspace(0,142,143)
on=np.ones(143)
zmax=np.max(Out_GasOutT)*on
zmin=np.min(Out_GasOutT)*on
fig1 = plt.figure(1)
plt.subplot(211)
## plot data calculated for the surrogate model
plt.plot(New,GasOutT, 'r', marker='o',label='surrogate model')
#if you dont like the marker just remove it
## plot data from flowsheet
plt.plot(New,Out_GasOutT, 'b', marker='o', label='sampling points')
##plot the min and max values (lines)
plt.plot(New,zmax, label='max val')
plt.plot(New,zmin, label='min val')
# axis
plt.xlim(0,150)
plt.ylim(35,100)
plt.xlabel('data point')
plt.ylabel('Temperature, C')
## different types of legends for each line or data
#plt.legend(bbox_to_anchor=(1.05,1),loc=2, borderaxespad=0.)
plt.legend(bbox_to_anchor=(0., 1.02, 1., .102), loc=3,
ncol=2, mode="expand", borderaxespad=0.)
plt.annotate('R2=0.45', xy=(130,40))
## annotate uses the xy possition in the graph
##plt.show()
## equation lablels with the value of the input variables
X0=np.array(Dt); X1=np.array(Dx); X2=np.array(GasInT); X3=np.array(GasInCO2);
X4=np.array(Lb); X5=np.array(Lhx); X6=np.array(SolidInT);
#here is the solution from the surrogate models (.py document)
# Replace X[0] by X0 and then the variable X0 - match Dt (i did this manualy with Word).
GasOutT3 = - 46.367714532286690598539 * np.log(X1) - 83.437866408632501702414 * np.log(X3) - 0.74663510879062380386911E-003 * np.exp(X4) - 14.194999566158760018197 * X1**-0.50 + 10.036752805745475569665 * X2 - 274.44441969201676556622 * X3 + 2.7381620575962664076997 * X4 + 0.35628807899149789184534 * X0**2.0 - 0.44064067613854951410701E-001 * X2**2.0 + 0.97868638245329375724424 * X4**2.0 + 421.07853757885402501415 * X5**2.0 + 0.28269013654116676576011E-001 * X6**2.0 + 1980.5694273762865122990 * (X0*X6)**-0.50 + 88.403008516440280573079 * (X1*X4)**-0.50 - 158.97500139266477958699 * (X1*X6)**-0.50 - 1543.9781985794563752279 * (X2*X4)**-0.50 + 74.271482303558329363113 * (X3*X4)**-0.50 - 394.62137499673008278478 * (X5*X6)**-0.50 - 102.88516992298136187856 * (X0*X3)**0.50 - 1131.0683529837617697922 * (X1*X3)**0.50 - 47.948298804294637420753 * (X2*X4)**0.50 - 31.365953313543645464279 * (X2*X6)**0.50 - 0.66590034525512820096083 * X0*X4 - 42.414082014114946161953 * X0*X5 - 0.30036600970171623004035 * X0*X6 + 14.309742288521377417965 * X1*X2 + 8574.9231528776654158719 * X1*X3 - 2626.3504064585858941427 * X1*X5 + 14.264165134090857733895 * X1*X6 + 0.20203052761177062168763 * X2*X4 + 12.566827259345073386498 * X2*X5 + 0.85024618348708919102208E-001 * X2*X6 - 0.31137267809381263816704 * X4*X6 - 17.136337320947461648757 * X5*X6 + 78.142903526310689699130 * (X2*X3*X6)**0.20 + 98.418366025188120715939 * (X2*X4*X5)**0.20 - 99.999999999999189981281 * (X0*X1*X2)**0.50 + 80.977913059860952671443 * (X0*X1*X4)**0.50 + 32.359072432775448646680 * (X0*X2*X4)**0.50 - 82.145101968005889148117 * (X0*X2*X5)**0.50 + 3.2137416963387068769009 * (X0*X2*X6)**0.50 + 82.163437742143003106321 * (X0*X3*X6)**0.50 - 23.945260118580126373899 * (X0*X4*X6)**0.50 + 59.825752213658027756082 * (X0*X5*X6)**0.50 - 60.138949634019937207086 * (X1*X2*X4)**0.50 + 52.197952003160892786582 * (X1*X2*X6)**0.50 - 2446.3003252898884056776 * (X1*X3*X5)**0.50 + 46.543733923898436444233 * (X1*X4*X6)**0.50 - 102.03613079118160555936 * (X1*X5*X6)**0.50 - 14.820588613640460096121 * (X2*X3*X4)**0.50 - 242.84044259170326540698 * (X3*X4*X5)**0.50 + 74.547537725898649796363 * (X3*X5*X6)**0.50 + 54.761670994925850664004 * (X4*X5*X6)**0.50 - 5.2484954570220088143628 * (X0*X1*X3)**-0.50 - 130.11857183144073246694 * (X0*X1*X4)**-0.50 + 74.339697310446609890278 * (X0*X1*X5)**-0.50 - 34.742407326595404981617 * (X0*X3*X5)**-0.50 + 54.729628652936845867316 * (X0*X3*X6)**-0.50 + 60.295407976068062794184 * (X1*X2*X3)**-0.50 - 83.611048323585890784670 * (X1*X2*X5)**-0.50 + 2.5706720648855068134253 * (X1*X3*X5)**-0.50 - 72.404771170131255075830 * (X1*X3*X6)**-0.50 - 30.713342231051328212743 * (X1*X4*X5)**-0.50 - 366.43214781329913876107 * (X2*X3*X4)**-0.50 + 857.98370085127567108430 * (X4*X5*X6)**-0.50 + 2.8463312534186426816518 * X0*X1*X2 - 16.997278548110575258079 * X0*X1*X4 + 359.88910036892758625982 * X0*X1*X5 - 0.93994257788959534316575 * X0*X1*X6 - 0.52885444683342464067266 * X0*X2*X3 - 0.80166073197540835271191E-001 * X0*X2*X4 + 0.72991466693989826008959 * X0*X2*X5 - 0.45148076282008143059188E-002 * X0*X2*X6 - 1.1386884469798042029964 * X0*X3*X6 - 0.40168261776447367061849 * X0*X4*X5 + 0.66417177390234963252347E-001 * X0*X4*X6 - 0.40187298764129714134796E-001 * X0*X5*X6 + 1.8520318784146294621706 * X1*X2*X4 - 0.76472997797634389982591 * X1*X2*X6 + 9902.7879975088526407490 * X1*X3*X5 - 86.626351936904129047434 * X1*X3*X6 + 1.2097610608263216214198 * X2*X3*X4 - 0.73388239168827182346000E-001 * X2*X3*X6 - 0.86630815064513966294868 * X2*X4*X5 + 0.50448773781136269089842E-002 * X2*X4*X6 + 0.22996705161038643455695E-001 * X2*X5*X6 + 113.30038153134069034422 * X3*X4*X5 - 0.88745416361360862644858 * X3*X4*X6 - 0.42433162877840463478307 * X4*X5*X6
# the equation is in the line above (but is too long so it is hidded, I dont know why)
plt.subplot(212)
## plot data from surrogate model
plt.plot(New,GasOutT3, 'r', marker='o',label='new surrogate model')
#if you dont like the marker just remove it
## plot data from flowsheet
plt.plot(New,Out_GasOutT, 'b', marker='o', label='sampling points')
##plot the min and max values (lines)
plt.plot(New,zmax, label='max val')
plt.plot(New,zmin, label='min val')
# axis (must be estimated with the zmin and zmax)???
plt.xlim(0,150)
plt.ylim(35,100)
plt.xlabel('data point')
plt.ylabel('Temperature, C')
plt.annotate('R2=0.91', xy=(130,40))
#show figure
plt.show()
Recreated the Jenkins build for this into a script. This will also requireq that the following external reposiroties be included as git submodules.
Turb Client
REM unzip turb_client into expected dir
7za x -y CCSI_turb_client_*.zip
del CCSI_turb_client.zip
REM rename dir
move TurbineClient-* turbine_client
Turb Hydro
REM unzip turb_hydro into expected dir
7za x -y CCSI_turb_hydro_*.zip
del CCSI_turb_hydro_*.zip
move CCSI_turb_hydro_* turbine_hydro
iReveal
REM unzip the iREVEAL stuff
cd foqus_lib\framework\surrogate\iREVEAL
7za x -y iREVEAL.zip
del iREVEAL.zip
del "iREVEAL Installation Guide.pdf"
del "iREVEAL User Manual.pdf"
dmf_lib
set JAVA_HOME=C:\Program Files\Java\jdk1.8.0_65
set ANT_HOME=C:\Program Files\apache-ant-1.9.6
set PATH=%PATH%;%ANT_HOME%\bin
ant -f dmf_lib\java\build.xml
Other Commands
cd docs
ren "iREVEAL User Manual.pdf" iREVEAL_User_Manual.pdf
REM unzip the html user manual created by the foqus_unix build
cd foqus_lib\help\html
7za x -y ..\..\..\docs\FOQUS_User_Manual_HTML.zip
REM This really should be moved into a make.bat file and checked in
copy .ccsi_common\LEGAL .
copy .ccsi_common\CCSI_TE_LICENSE.txt .
REM powershell -Command "(Get-Content .ccsi_common\CCSI_TE_LICENSE.txt) | ForEach-Object { $_ -replace '\[SOFTWARE NAME & VERSION\]', 'foqus v.2016.04.00' } | Set-Content CCSI_TE_LICENSE_foqus.txt"
setup\setup_dep.py
setup.py wix
From JIRA:
Description:
When switching turbine from "local" to "remote" the user must upload the simulation again. But, we need a "warning" before launching the simulations from "UQ module" to "turbine remote", otherwise they will run without a simulation file.
From JIRA:
Description:
The button to test the remote configuration file is not working, this button is located in the Turbine Configuration window (create/edit window).
Note that the button in the Turbine Getaway works.
Attachments:
Summary
Using the "FOQUS" framework create a suite of tests for DMFLite. Please work with John to understand how to create and run these tests. The near term goal to is automate the tests and run via Jenkins after successful builds. Please document the tests you add and what they do in this JIRA issue under "work log".
Chat history:
You-Wei Cheah added a comment - 01/Dec/15 5:13 PM - edited
Checked in preliminary version of test script. Allows saving, loading of basic session file through the DMF lite. Next steps are to automate the creation of a test flowsheet with dependencies in script.
You-Wei Cheah added a comment - 02/Dec/15 12:17 PM
Location of script is in svn for foqus: trunk/dmf_lib/test/DMFLiteTest.py
To load the script in FOQUS do the following:
The old Jenkins build had the ability to compile artifacts (binaries, documentation ...etc) on several different machines (Windows, Linux). This is an unreasonable requirment for users of this repositoriy. It is now required that the foqus application be compiled on Windows machine only which means the latex documentation build needs to written so that it can be built on Windows. Currently the make file for this uses unix.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.