Installation Problem about dedop-core

[hans-permana]

This is because the documentation was out of date. In commits fcdbd5e and 59e9fe3 the documentation has been updated to include cate-cli in the package dependency. Could you try the installation again with the updated instruction?

[dedop-cotton]

OK, Thanks Hans, We’ll try again> Regards David Dr P.D. Cotton Satellite Oceanographic Consultants (SatOC) www.satoc.eu Direct: [phone] Mobile: [phone]

…

On 29 Jan 2018, at 14:35, Hans Permana ***@***.***> wrote: This is because the documentation was out of date. In commits fcdbd5e <fcdbd5e> and 59e9fe3 <59e9fe3> the documentation has been updated to include cate-cli in the package dependency. Could you try the installation again with the updated instruction? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AenaHobfqd9J_6spz4SiGXnDNml54noMks5tPdcagaJpZM4Runx2>.

[dedop-cotton]

Hi Hans, Tried it using the instructions at http://dedop-core.readthedocs.io/en/latest/user_manual/um_setup.html, but I still get the same error (see below) . I can see cate-util but not cate.cli in the libraries installed. Will put this on github too. Regards David Dr P.D. Cotton Satellite Oceanographic Consultants (SatOC) www.satoc.eu Direct: [phone] Mobile: [phone]

On 29 Jan 2018, at 14:35, Hans Permana ***@***.***> wrote: This is because the documentation was out of date. In commits fcdbd5e <fcdbd5e> and 59e9fe3 <59e9fe3> the documentation has been updated to include cate-cli in the package dependency. Could you try the installation again with the updated instruction? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AenaHobfqd9J_6spz4SiGXnDNml54noMks5tPdcagaJpZM4Runx2>.

Traceback (most recent call last): File "/Users/davidcotton/miniconda3/envs/dedop/bin/dedop", line 11, in <module> load_entry_point('dedop', 'console_scripts', 'dedop')() File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.5/site-packages/pkg_resources/__init__.py", line 572, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2755, in load_entry_point return ep.load() File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2408, in load return self.resolve() File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2414, in resolve module = __import__(self.module_name, fromlist=['__name__'], level=0) File "/Users/davidcotton/dedop-core/dedop/cli/main.py", line 23, in <module> from cate.util.cli import run_main, Command, SubCommandCommand, CommandError ImportError: No module named 'cate'

[hans-permana]

hi David,

I can reproduce your issue in MacOS 10.13.3. My investigation suggests that with the given command, Miniconda somehow fetches the wrong version of cate-util (i.e. one for Python 3.6). This results in the fetched cate-util package not available for the current environment, which is on Python 3.5.

Long term solution is to migrate the entire dedop to Python 3.6 (#33), but in the meantime I updated the doc (again!) to ensure that a specific cate-util version is installed.

[dedop-cotton]

HI Hans, Thanks managed to get a bit further today, but then it all fell over when I tried to launch the studio (See the attached doc for my experience). I am not at all familiar with code installations, so I suspect some things are not in the expected place in my directory structure (everything is currently installed under the point where my mac terminal window opens (/Users/davidcotton/….) So, if you want the installation guidance to be properly fool proof (with me as the fool), then I need precise guidance on where things should be installed, and where I need to located myself before following the instructions. At present the instructions expect a higher level of knowledge than I possess. Regards David Dr P.D. Cotton Satellite Oceanographic Consultants (SatOC) www.satoc.eu Direct: [phone] Mobile: [phone]

…

On 31 Jan 2018, at 09:02, Hans Permana ***@***.***> wrote: hi David, I can reproduce your issue in MacOS 10.13.3. My investigation suggests that with the given command, Miniconda somehow fetches the wrong version of cate-util (i.e. one for Python 3.6). This results in the fetched cate-util package not available for the current environment, which is on Python 3.5. Long term solution is to migrate the entire dedop to Python 3.6 (#33 <#33>), but in the meantime I updated the doc (again!) to ensure that a specific cate-util version is installed. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AenaHi7FyIVIbATE_rODxZw1fnqj5ia2ks5tQCwcgaJpZM4Runx2>.

[hans-permana]

Hi David,

Could you please attach the screenshot showing the error here? I could not find one from your last reply. Also, did you encounter this issue when trying to install DeDop Studio (from source, I assume) or when launching DeDop Studio after a successful install?

Thanks,
Hans

[dedop-cotton]

HI Hans, The error message was in terminal window. I didn’t take a screenshot, but the contents are pasted below and in the attached document. Yes - The error was in trying to install dedop studio from source, after dedop-core had successfully been installed. I suspect the problem could be that dedop core installation was not in the directory expected by dedop studio? Regards David

On 5 Feb 2018, at 16:06, Hans Permana ***@***.***> wrote: Hi David, Could you please attach the screenshot showing the error here? I could not find one from your last reply. Also, did you encounter this issue when trying to install DeDop Studio (from source, I assume) or when launching DeDop Studio after a successful install? Thanks, Hans — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AenaHpbrXKkUq8Gd9yurwV0ru2rnNI95ks5tRycjgaJpZM4Runx2>. ***@***.*** start /Users/davidcotton/dedop-core/dedop-studio electron app

dedop-studio App configuration successfully loaded from "/Users/davidcotton/dedop-core/dedop-studio/dedop-config.js" dedop-studio User preferences successfully loaded from "/Users/davidcotton/.dedop/dedop-prefs.json" dedop-studio appConfig: { webAPIConfig: { command: '/Users/davidcotton/miniconda3/envs/dedop/bin/dedop-webapi', servicePort: 9090, serviceAddress: '', serviceFile: 'dedop-webapi-info.json', processOptions: {}, useMockService: false }, prefsFile: null, devToolsOpened: true, devToolsExtensions: [] } dedop-studio userPrefs: { mainWindowBounds: { x: 396, y: 140, width: 1366, height: 768 }, devToolsOpened: false, isChdEditable: false, isCnfEditable: true, isCstEditable: false, isOfflineMode: false, processes: [] } dedop-studio Ready. dedop-studio Waiting for response from http://127.0.0.1:9090/ dedop-studio Waiting for DeDop backend service... dedop-studio Waiting for DeDop WebAPI service to respond after 0 ms dedop-studio Starting DeDop WebAPI service using arguments: --caller,dedop-studio,--port,9090,--address,,--file,dedop-webapi-info.json,start dedop-studio Waiting for response from http://127.0.0.1:9090/ dedop-studio Waiting for DeDop backend service... dedop-studio Waiting for DeDop WebAPI service to respond after 0 ms dedop-studio Waiting for response from http://127.0.0.1:9090/ dedop-studio Waiting for DeDop backend service... dedop-studio Waiting for DeDop WebAPI service to respond after 0 ms dedop-studio error: [Errno 2] No such file or directory: '/Users/davidcotton/.dedop/1.3.0/webapi.log' dedop-studio DeDop WebAPI service process exited with code 1. dedop-studio Quit. dedop-studio Stopping DeDop WebAPI service using arguments: --caller,dedop-studio,--port,9090,--address,,--file,dedop-webapi-info.json,stop dedop-studio Waiting for response from http://127.0.0.1:9090/ dedop-studio Waiting for DeDop backend service... dedop-studio showSplashMessage: splash not visible Waiting for DeDop backend service... dedop-studio Waiting for DeDop WebAPI service to respond after 0 ms npm ERR! code ELIFECYCLE npm ERR! errno 6 npm ERR! [email protected] start: `electron app` npm ERR! Exit status 6 npm ERR! npm ERR! Failed at the [email protected] start script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above. npm ERR! A complete log of this run can be found in: npm ERR! /Users/davidcotton/.npm/_logs/2018-02-02T17_42_28_510Z-debug.log

[hans-permana]

Hi David,
This problem is caused by a missing version-specific directory inside .dedop. For installation using an installer, this directory is automatically created at the end of the installation. I have made a fix so that this is also the case for installation by source.

If you could re-install the dedop-core by running python setup.py develop and then run dedop-studio again, this issue should be solved.

NOTE: I have also simplified the instruction to install dedop-core by source. It is now using a .yml file to list all the dependencies rather than including them individually in the conda command.

[dedop-cotton]

Thanks Hans, I am busy these next few days, but will try to reinstall according to your instructions. David

…

On 6 Feb 2018, at 09:39, Hans Permana ***@***.***> wrote: Hi David, This problem is caused by a missing version-specific directory inside .dedop. For installation using an installer, this directory is automatically created at the end of the installation. I have made a fix so that this is also the case for installation by source. If you could re-install the dedop-core by running python setup.py develop and then run dedop-studio again, this issue should be solved. NOTE: I have also simplified the instruction to install dedop-core by source. It is now using a .yml file to list all the dependencies rather than including them individually in the conda command. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AenaHqRfbYQ-LRXnaX8viQaLpIkb1eHoks5tSB2-gaJpZM4Runx2>.

[dedop-cotton]

Hi Hans, Sorry if I”m being a bit slow here. Get the same problem after (I think) following your instructions) I attach a screen shot of the entire terminal session: Regards David

…

On 6 Feb 2018, at 09:39, Hans Permana ***@***.***> wrote: Hi David, This problem is caused by a missing version-specific directory inside .dedop. For installation using an installer, this directory is automatically created at the end of the installation. I have made a fix so that this is also the case for installation by source. If you could re-install the dedop-core by running python setup.py develop and then run dedop-studio again, this issue should be solved. NOTE: I have also simplified the instruction to install dedop-core by source. It is now using a .yml file to list all the dependencies rather than including them individually in the conda command. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AenaHqRfbYQ-LRXnaX8viQaLpIkb1eHoks5tSB2-gaJpZM4Runx2>.

[hans-permana]

Hi David,

Did you start from zero this time (i.e. remove no .dedop directory, brand new conda environment, git pull both dedop-core and dedop-studio repositories)? If so, I could not figure out what is going wrong. I tested it on our iMac with OSX 10.13.3 and it worked. If you are available, I would like to diagnose this via skype (or other similar screen sharing software) so we can get to the root of this faster. What do you think?

[dedop-cotton]

Hi Hans, No, this last time I just went back to the "python setup.py develop” step. A skype call could be useful, but possibly difficult to arrange. Are you available tomorrow morning? David Dr P.D. Cotton Satellite Oceanographic Consultants (SatOC) www.satoc.eu Direct: [phone] Mobile: [phone]

…

On 8 Feb 2018, at 07:52, Hans Permana ***@***.***> wrote: Hi David, Did you start from zero this time (i.e. remove no .dedop directory, brand new conda environment, git pull both dedop-core and dedop-studio repositories)? If so, I could not figure out what is going wrong. I tested it on our iMac with OSX 10.13.3 and it worked. If you are available, I would like to diagnose this via skype (or other similar screen sharing software) so we can get to the root of this faster. What do you think? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AenaHqybjis1DOM7LUC4ApphOa702HWSks5tSqfMgaJpZM4Runx2>.

[hans-permana]

Hi David,
Yes I am available tomorrow morning from 9.00 CET.

One thing that could be the cause of this problem: if you do conda list, do you see dedop on the list of dependencies? If so, try to run this command pip uninstall dedop so that it is not in the list anymore. After that, run python setup.py develop and run dedop-studio. But anyway, I will be available to support you tomorrow morning. We can talk more about it.

Hans

[dedop-cotton]

Hi Hans, Now I’m back from holiday and back in the office and trying to install dedop again. So I went back to the beginning, removed dedop directory, installed a brand new conda environment, git pulled dedop-core repository and installed that, now moved on to dedop studio installation - following instructions to install from source (at http://dedop-core.readthedocs.io/en/latest/user_manual/um_setup.html <http://dedop-core.readthedocs.io/en/latest/user_manual/um_setup.html>). completed

git clone https://github.com/DeDop/dedop-studio.git cd dedop-studio npm install

and come to the instruction to create a dedop-config.js and include the location of dedop-webapi.exe. webAPIConfig: { command: "C:\\Miniconda3\\envs\\dedop\\Scripts\\dedop-webapi.exe", servicePort: 2999, processOptions: {} } I can’t find a dedop-webapi.exe, but I can find a binary dedop-webapi under users\davidcotton\miniconda3\envs\dedop\bin I don’t have a …\dedop\scripts sub-directory Can you advise? Thanks! David Dr P.D. Cotton Satellite Oceanographic Consultants (SatOC) www.satoc.eu Direct: [phone] Mobile: [phone]

…

On 8 Feb 2018, at 13:41, Hans Permana ***@***.***> wrote: Hi David, Yes I am available tomorrow morning from 9.00 CET. One thing that could be the cause of this problem: if you do conda list, do you see dedop on the list of dependencies? If so, try to run this command pip uninstall dedop so that it is not in the list anymore. After that, run python setup.py develop and run dedop-studio. But anyway, I will be available to support you tomorrow morning. We can talk more about it. Hans — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AenaHrj2Aep9yZPuY2zVIHmUdCcZBlPfks5tSvl9gaJpZM4Runx2>.

[hans-permana]

Hi David,

The directory that you wrote (users\davidcotton\miniconda3\envs\dedop\bin) is the right one for MacOS.

Admittedly, no one has comprehensively tested this setup step for OS other than Windows. It is good that you identified some issues (well, not good for you, but good for completing the gaps in the documentation). As a response, I have updated the docs and it should now have different configuration examples for different OS's.

[dedop-cotton]

Hi Hans, Success! - following the revised instructions I have now installed dedop-core and dedop-studio and seems to open OK. I’ll try to find to find time to test setting up the workspace, configuration, loading files, running the processing and analysing the output. One thing isn’t immediately clear to me. How do I start a new session now it is installed? Do I go to the dedop_studio directory and type “npm start”? Best regards David Dr P.D. Cotton Satellite Oceanographic Consultants (SatOC) www.satoc.eu Direct: [phone] Mobile: [phone]

…

On 21 Feb 2018, at 09:16, Hans Permana ***@***.***> wrote: Hi David, The directory that you wrote is the right one for MacOS. Admittedly, no one has comprehensively tested this setup step for OS other than Windows, so it is good that you encountered some difficulties (well, not good for you, but good for completing the gaps in the documentation). As a response, I have updated the docs and it should now have different configuration examples for different OS's. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AenaHrmD12qukOnElK9--JhNZ0Btx3c7ks5tW98CgaJpZM4Runx2>.

[dedop-cotton]

..and then failure! I tried to process the svalbard file included in the distribution (CS_OFFL_SIR1SAR_FR_20120724T000711_20120724T000724_C001.DBL.nc with the default configuration and had a processing failure (see attached screen shot). I havent tried to run dedop config upgrade as the error message indicates. David

…

On 21 Feb 2018, at 14:21, david cotton ***@***.***> wrote: Hi Hans, Success! - following the revised instructions I have now installed dedop-core and dedop-studio and seems to open OK. I’ll try to find to find time to test setting up the workspace, configuration, loading files, running the processing and analysing the output. One thing isn’t immediately clear to me. How do I start a new session now it is installed? Do I go to the dedop_studio directory and type “npm start”? Best regards David Dr P.D. Cotton Satellite Oceanographic Consultants (SatOC) www.satoc.eu <http://www.satoc.eu/> Direct: [phone] Mobile: [phone] > On 21 Feb 2018, at 09:16, Hans Permana ***@***.*** ***@***.***>> wrote: > > Hi David, > > The directory that you wrote is the right one for MacOS. Admittedly, no one has comprehensively tested this setup step for OS other than Windows, so it is good that you encountered some difficulties (well, not good for you, but good for completing the gaps in the documentation). As a response, I have updated the docs and it should now have different configuration examples for different OS's. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AenaHrmD12qukOnElK9--JhNZ0Btx3c7ks5tW98CgaJpZM4Runx2>. >

[hans-permana]

Hi @dedop-cotton,

How do I start a new session now it is installed?

When you run from the source code, you don't actually install the software. To run the software, always use npm start command inside dedop-studio directory, as you already know.

And for the failure, can you drag n drop the image on the issue? I cannot see it on your last reply.

[dedop-cotton]

Thanks Hans Here is the screen shot again (as pdf).

…

On 21 Feb 2018, at 16:22, Hans Permana ***@***.***> wrote: Hi @dedop-cotton <https://github.com/dedop-cotton>, How do I start a new session now it is installed? When you run from the source code, you don't actually install the software. To run the software, always use npm start command inside dedop-studio directory, as you already know. And for the failure, can you drag n drop the image on the issue? I cannot see it on your last reply. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AenaHoPFW4DkazwsZvA5tiZesPYCW2ERks5tXEKsgaJpZM4Runx2>.

[hans-permana]

Hi @dedop-cotton ,

As far as I know, it is not possible to attach file format in GitHub Issue other than an image. You could probably send this file to my email instead. Alternatively, you can state here the error message.

[hans-permana]

hi @dedop-cotton ,

It is indeed an issue of old configuration version. If you go to the Configuration tab, there should be a button that says upgrade. If you click it, it will activate the configuration upgrade.

[dedop-cotton]

Hi Sorry - more! I upgraded configuration - but now get another error message, as follows: process() call raised exception: "CHD file does not contain a definition for parameter: brf_sar_chd” An error (code 20) occurred while executing a background process: Traceback (most recent call last): File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.6/site-packages/cate/util/web/jsonrpchandler.py", line 192, in send_service_method_result result = future.result() File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.6/concurrent/futures/_base.py", line 398, in result return self.__get_result() File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.6/concurrent/futures/_base.py", line 357, in __get_result raise self._exception File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.6/concurrent/futures/thread.py", line 55, in run result = self.fn(*self.args, **self.kwargs) File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.6/site-packages/cate/util/web/jsonrpchandler.py", line 269, in call_service_method result = method(*method_params, monitor=monitor) File "/Users/davidcotton/dedop-core/dedop/webapi/websocket.py", line 132, in process processor = L1BProcessor(process_name, cnf_file, cst_file, chd_file, output_path, skip_l1bs=False) File "/Users/davidcotton/dedop-core/dedop/proc/sar/processor.py", line 100, in __init__ self.gap_threshold = self.chd.bri_sar * 1.5 File "/Users/davidcotton/dedop-core/dedop/conf/characterization.py", line 88, in bri_sar return 1. / self.brf_sar File "/Users/davidcotton/dedop-core/dedop/conf/auxiliary_parameter.py", line 52, in __get__ value = self._retreive_value(instance) File "/Users/davidcotton/dedop-core/dedop/conf/auxiliary_parameter.py", line 69, in _retreive_value return instance[self.name] File "/Users/davidcotton/dedop-core/dedop/conf/auxiliary_file_reader.py", line 94, in __getitem__ raise MissingParameterError(item, self._id) dedop.conf.auxiliary_errors.MissingParameterError: CHD file does not contain a definition for parameter: brf_sar_chd

[dedop-cotton]

Apologies You can ignore this email - I realised I needed to create a new configuration with the Cryosat adapted file values. It is running now. Regards David

…

On 22 Feb 2018, at 14:28, david cotton ***@***.***> wrote: Hi Sorry - more! I upgraded configuration - but now get another error message, as follows: process() call raised exception: "CHD file does not contain a definition for parameter: brf_sar_chd” An error (code 20) occurred while executing a background process: Traceback (most recent call last): File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.6/site-packages/cate/util/web/jsonrpchandler.py", line 192, in send_service_method_result result = future.result() File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.6/concurrent/futures/_base.py", line 398, in result return self.__get_result() File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.6/concurrent/futures/_base.py", line 357, in __get_result raise self._exception File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.6/concurrent/futures/thread.py", line 55, in run result = self.fn(*self.args, **self.kwargs) File "/Users/davidcotton/miniconda3/envs/dedop/lib/python3.6/site-packages/cate/util/web/jsonrpchandler.py", line 269, in call_service_method result = method(*method_params, monitor=monitor) File "/Users/davidcotton/dedop-core/dedop/webapi/websocket.py", line 132, in process processor = L1BProcessor(process_name, cnf_file, cst_file, chd_file, output_path, skip_l1bs=False) File "/Users/davidcotton/dedop-core/dedop/proc/sar/processor.py", line 100, in __init__ self.gap_threshold = self.chd.bri_sar * 1.5 File "/Users/davidcotton/dedop-core/dedop/conf/characterization.py", line 88, in bri_sar return 1. / self.brf_sar File "/Users/davidcotton/dedop-core/dedop/conf/auxiliary_parameter.py", line 52, in __get__ value = self._retreive_value(instance) File "/Users/davidcotton/dedop-core/dedop/conf/auxiliary_parameter.py", line 69, in _retreive_value return instance[self.name] File "/Users/davidcotton/dedop-core/dedop/conf/auxiliary_file_reader.py", line 94, in __getitem__ raise MissingParameterError(item, self._id) dedop.conf.auxiliary_errors.MissingParameterError: CHD file does not contain a definition for parameter: brf_sar_chd

[hans-permana]

It is good to hear that it is finally running! But on the other hand, it would not leave a good impression if every user has to go through what you just did, just to be able to process something. I hope after trying more functionalities, you could give us more feedback on the usability of DeDop (Shell and Studio).

[dedop-cotton]

Hi Yes I’ll send feedback, together with a step by step list of how I installed and ran it on my mac. David

…

On 22 Feb 2018, at 14:39, Hans Permana ***@***.***> wrote: It is good to hear that it is finally running! But on the other hand, it would not leave a good impression if every user has to go through what you just did, just to be able to process something. I hope after trying more functionalities, you could give us more feedback on the usability of DeDop (Shell and Studio). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AenaHhC1dKc8sn-wGSzELbM7ztpjDuTEks5tXXwLgaJpZM4Runx2>.