Comments (33)
lucadelu
commented on June 26, 2024
1
@kbkhand you need to set also path option to MOLA.
Try and let me know
from pymodis.
lucadelu
commented on June 26, 2024
@lpalao you have to install version 2.0. Version 1.0 is not supporting the new NASA authentication system.
Please update pyModis and let me know
from pymodis.
jamieodonnell
commented on June 26, 2024
I'm getting this same error (empty download files) and I'm on pymodis 2.0.2 (Python 3.5)
2016-12-14 13:06:29,677 - DEBUG - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-12-14 13:06:48,848 - DEBUG - http://e4ftl01.cr.usgs.gov:80 "GET /MOLT/MOD11A1.005 HTTP/1.1" 301 252
2016-12-14 13:06:49,991 - DEBUG - http://e4ftl01.cr.usgs.gov:80 "GET /MOLT/MOD11A1.005/ HTTP/1.1" 200 None
2016-12-14 13:06:57,819 - DEBUG - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-12-14 13:07:19,869 - DEBUG - http://e4ftl01.cr.usgs.gov:80 "GET /MOLT/MOD11A1.005/2016.11.01 HTTP/1.1" 301 263
2016-12-14 13:07:20,481 - DEBUG - http://e4ftl01.cr.usgs.gov:80 "GET /MOLT/MOD11A1.005/2016.11.01/ HTTP/1.1" 200 None
2016-12-14 13:07:42,286 - ERROR - Cannot download MOD11A1.A2016306.h09v05.005.2016307100736.hdf.xml. Retrying...
2016-12-14 13:08:07,564 - ERROR - Cannot download MOD11A1.A2016306.h09v05.005.2016307100736.hdf.xml. Retrying...
2016-12-14 13:08:23,915 - ERROR - Cannot download MOD11A1.A2016306.h09v05.005.2016307100736.hdf.xml. Retrying...
2016-12-14 13:08:44,003 - ERROR - Cannot download MOD11A1.A2016306.h09v05.005.2016307100736.hdf.xml. Retrying...
2016-12-14 13:09:07,509 - ERROR - Cannot download MOD11A1.A2016306.h09v05.005.2016307100736.hdf.xml. Retrying...
2016-12-14 13:09:39,360 - ERROR - Cannot download MOD11A1.A2016306.h09v05.005.2016307100736.hdf.xml. Retrying...
2016-12-14 13:09:58,341 - ERROR - Cannot download MOD11A1.A2016306.h09v05.005.2016307100736.hdf.xml. Retrying...
from pymodis.
lucadelu
commented on June 26, 2024
@jamieodonnell can you send your command line (without user password) ?
from pymodis.
jamieodonnell
commented on June 26, 2024
import os
import glob
from pymodis import downmodis
WxPython missing, no GUI enabled
dest = 'C:/PymodisDownloads/'
tiles = 'h08v07,h09v07,h07v06,h08v06,h09v06,h07v05,h08v05,h09v05'
day = "2012-06-01"
delta = 1460
day = "2016-11-01"
user = 'username'
password = 'password'
modisDown = downmodis.downModis(destinationFolder=dest, tiles=tiles, today=day, delta=delta, user = user, password = password)
modisDown.connect()
modisDown.downloadsAllDay()
I'm attempting to follow the iPython Notebook
from pymodis.
lucadelu
commented on June 26, 2024
@jamieodonnell for me it is working, maybe your user is not enabled to download from that site?
Please check the documentation and if you can download the data from the website
l /tmp/
-rw-r--r-- 1 lucadelu gis 500 Dec 15 12:31 modisMOD11A1.005.log
-rw-r--r-- 1 lucadelu gis 672K Dec 15 12:31 MOD11A1.A2016306.h07v06.005.2016307100707.hdf
-rw-r--r-- 1 lucadelu gis 6.9M Dec 15 12:31 MOD11A1.A2016306.h08v06.005.2016307100733.hdf
-rw-r--r-- 1 lucadelu gis 585K Dec 15 12:31 MOD11A1.A2016306.h09v07.005.2016307100728.hdf
-rw-r--r-- 1 lucadelu gis 9.4K Dec 15 12:31 MOD11A1.A2016306.h07v05.005.2016307100710.hdf.xml
-rw-r--r-- 1 lucadelu gis 8.8K Dec 15 12:31 MOD11A1.A2016306.h08v07.005.2016307100701.hdf.xml
-rw-r--r-- 1 lucadelu gis 8.8K Dec 15 12:31 MOD11A1.A2016306.h07v06.005.2016307100707.hdf.xml
-rw-r--r-- 1 lucadelu gis 0 Dec 15 12:31 MOD11A1.A2016306.h08v07.005.2016307100701.hdf
-rw-r--r-- 1 lucadelu gis 9.1K Dec 15 12:31 MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml
-rw-r--r-- 1 lucadelu gis 338 Dec 15 12:31 listfileMOD11A1.005.txt
from pymodis.
mrjohnston
commented on June 26, 2024
Good afternoon,
I am having the same trouble as @jamieodonnell. I have used the same command line as jaimeodonnell, for reproducibility, and I am using Python 2.7.6 and pymodis 2.0.2. I don't have any trouble downloading data from the website using your link above.
Traceback:
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 785, in downloadsAllDay
self._downloadAllDaysHTTP(days)
File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 801, in _downloadAllDaysHTTP
self.dayDownload(day, listFilesDown)
File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 763, in dayDownload
self.downloadFile(i, file_hdf, day)
File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 626, in downloadFile
self._downloadFileHTTP(filDown, filHdf, day)
File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 652, in _downloadFileHTTP
self._downloadFileHTTP(filDown, filHdf, day)
File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 652, in _downloadFileHTTP
self._downloadFileHTTP(filDown, filHdf, day)
File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 652, in _downloadFileHTTP
self._downloadFileHTTP(filDown, filHdf, day)
File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 652, in _downloadFileHTTP
self._downloadFileHTTP(filDown, filHdf, day)
(and so on), and Logfile:
2016-12-15 14:11:14,526 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-12-15 14:11:14,570 - DEBUG - Setting read timeout to 100
2016-12-15 14:11:14,611 - DEBUG - "GET /MOLT/MOD11A1.005 HTTP/1.1" 301 252
2016-12-15 14:11:14,612 - DEBUG - Setting read timeout to 100
2016-12-15 14:11:17,510 - DEBUG - "GET /MOLT/MOD11A1.005/ HTTP/1.1" 200 None
2016-12-15 14:11:18,706 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-12-15 14:11:18,751 - DEBUG - Setting read timeout to 100
2016-12-15 14:11:18,792 - DEBUG - "GET /MOLT/MOD11A1.005/2016.11.01 HTTP/1.1" 301 263
2016-12-15 14:11:18,793 - DEBUG - Setting read timeout to 100
2016-12-15 14:11:19,532 - DEBUG - "GET /MOLT/MOD11A1.005/2016.11.01/ HTTP/1.1" 200 None
2016-12-15 14:11:20,070 - ERROR - Cannot download MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml. Retrying...
2016-12-15 14:11:25,162 - ERROR - Cannot download MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml. Retrying...
2016-12-15 14:11:30,254 - ERROR - Cannot download MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml. Retrying...
2016-12-15 14:11:35,344 - ERROR - Cannot download MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml. Retrying...
2016-12-15 14:11:40,557 - ERROR - Cannot download MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml. Retrying...
2016-12-15 14:11:45,650 - ERROR - Cannot download MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml. Retrying...
2016-12-15 14:11:50,740 - ERROR - Cannot download MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml. Retrying...
(etc.).
Many thanks, in advance! I really appreciate any insight you may be able to offer.
-Miriam
from pymodis.
jamieodonnell
commented on June 26, 2024
I got it working in the end - turns out I hadn't capitalised one of the letters in my password.
Many thanks for testing @lucadelu and for your work on the plugin in general!
from pymodis.
lucadelu
commented on June 26, 2024
@mrjohnston I'm sorry but with 2.7.12 I'm not able to reproduce your error
Python 2.7.12+ (default, Nov 22 2016, 00:48:54)
Type "copyright", "credits" or "license" for more information.
IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import pymodis
In [2]: tiles = 'h08v07,h09v07,h07v06,h08v06,h09v06,h07v05,h08v05,h09v05'
...: day = "2012-06-01"
...: delta = 1460
...: day = "2016-11-01"
In [3]: user = 'myuser'
In [4]: passwd = 'mypass'
In [5]: dest='/tmp'
In [6]: modisDown = pymodis.downmodis.downModis(destinationFolder=dest, tiles=tiles, today=day, delta=delta, user = user, password = passwd)
In [7]: modisDown.connect()
In [8]: modisDown.downloadsAllDay()
ls /tmp
-rw-r--r-- 1 lucadelu gis 5.3M Dec 16 15:14 MOD11A1.A2016306.h08v05.005.2016307100724.hdf
-rw-r--r-- 1 lucadelu gis 9.0K Dec 16 15:14 MOD11A1.A2016306.h09v05.005.2016307100736.hdf.xml
-rw-r--r-- 1 lucadelu gis 5.8M Dec 16 15:14 MOD11A1.A2016306.h09v05.005.2016307100736.hdf
-rw-r--r-- 1 lucadelu gis 9.1K Dec 16 15:14 MOD11A1.A2016306.h09v07.005.2016307100728.hdf.xml
-rw-r--r-- 1 lucadelu gis 188K Dec 16 15:14 MOD11A1.A2016306.h07v05.005.2016307100710.hdf
-rw-r--r-- 1 lucadelu gis 9.1K Dec 16 15:14 MOD11A1.A2016306.h09v06.005.2016307100722.hdf.xml
-rw-r--r-- 1 lucadelu gis 8.8K Dec 16 15:14 MOD11A1.A2016306.h08v06.005.2016307100733.hdf.xml
drwxrwxrwt 23 root root 52K Dec 16 15:14 .
-rw-r--r-- 1 lucadelu gis 2.0M Dec 16 15:14 MOD11A1.A2016306.h09v06.005.2016307100722.hdf
-rw-r--r-- 1 lucadelu gis 384 Dec 16 15:14 listfileMOD11A1.005.txt
-rw-r--r-- 1 lucadelu gis 1.9K Dec 16 15:14 modisMOD11A1.005.log
Maybe it was a problem on the server? on Wednesday they usually do some maintenance works. Could you try again?
from pymodis.
lucadelu
commented on June 26, 2024
@jamieodonnell cool that you solve the problem!
from pymodis.
mrjohnston
commented on June 26, 2024
@lucadelu I just tried again, and I didn't change anything except for the username & password from the test you just ran:
import pymodis
tiles = 'h08v07,h09v07,h07v06,h08v06,h09v06,h07v05,h08v05,h09v05'
day = "2012-06-01"
delta = 1460
day = "2016-11-01"
user = 'username'
passwd = 'password'
dest='/tmp'
modisDown = pymodis.downmodis.downModis(destinationFolder=dest, tiles=tiles, today=day, delta=delta, user = user, password = passwd)
modisDown.connect()
modisDown.downloadsAllDay()
I receive the same errors:
2016-12-16 09:52:57,469 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-12-16 09:52:57,536 - DEBUG - Setting read timeout to 30
2016-12-16 09:52:57,606 - DEBUG - "GET /MOLT/MOD11A1.005 HTTP/1.1" 301 252
2016-12-16 09:52:57,608 - DEBUG - Setting read timeout to 30
2016-12-16 09:52:58,019 - DEBUG - "GET /MOLT/MOD11A1.005/ HTTP/1.1" 200 None
2016-12-16 09:52:59,203 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-12-16 09:52:59,248 - DEBUG - Setting read timeout to 30
2016-12-16 09:52:59,417 - DEBUG - "GET /MOLT/MOD11A1.005/2016.11.01 HTTP/1.1" 301 263
2016-12-16 09:52:59,418 - DEBUG - Setting read timeout to 30
2016-12-16 09:53:00,394 - DEBUG - "GET /MOLT/MOD11A1.005/2016.11.01/ HTTP/1.1" 200 None
2016-12-16 09:53:01,921 - ERROR - Cannot download MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml. Retrying...
2016-12-16 09:53:07,014 - ERROR - Cannot download MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml. Retrying...
2016-12-16 09:53:12,191 - ERROR - Cannot download MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml. Retrying...
2016-12-16 09:53:17,330 - ERROR - Cannot download MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml. Retrying...
2016-12-16 09:53:22,444 - ERROR - Cannot download MOD11A1.A2016306.h08v05.005.2016307100724.hdf.xml. Retrying...
Do you think that it's possible that the issue is my version of Python, 2.7.6 (rather than 2.7.12)? Thanks again for your help!
from pymodis.
lucadelu
commented on June 26, 2024
@mrjohnston do you know a little bit of Python? I could help you in the debug because I'm not able to reproduce the error...
from pymodis.
mrjohnston
commented on June 26, 2024
@lucadelu I'd really appreciate that; and I know a little bit but I'm just starting (natively an R person).
from pymodis.
lucadelu
commented on June 26, 2024
@mrjohnston so what you can modify to test is the file downmodis.py:
- after line 648 add
print("inside requests") - after line 654 add
print("inside urllib") - after line 655 add `import sys; print(sys.exc_info())
With this lines we can see which library is used to download the data and later check what is the real error.
Please let me know
from pymodis.
mrjohnston
commented on June 26, 2024
@lucadelu Thanks, and apologies for the delayed response - happy new year! I edited downmodis.py as suggested, with relevant section (from line 644) reproduced here:
try: # download and write the file
req = urllib.request.Request(url, headers = self.http_header)
http = urllib.request.urlopen(req)
orig_size = http.headers['Content-Length']
filSave.write(http.read())
print("inside requests")
# if local file has an error, try to download the file again
except:
try:
http = requests.get(url, timeout=self.timeout, verify=False)
orig_size = http.headers['Content-Length']
filSave.write(http.content)
print("inside urllib")
except:
import sys; print(sys.exc_info())
logging.error("Cannot download {name}. "
"Retrying...".format(name=filDown))
filSave.close()
os.remove(filSave.name)
import time
time.sleep(5)
self._downloadFileHTTP(filDown, filHdf, day)
filSave.close()
I then attempted to run the new file in python this way:
variables={'destinationFolder':'/tmp','tiles':'h08v07,h09v07,h07v06,h08v06,h09v06,h07v05,h08v05,h09v05','today':"2016-11-01",'delta':1460,'user':'user','password':'password'}
execfile("downmodis_edit.py",variables)
There were no errors, but there was also no output (either in the python terminal or in /tmp). I assume something is wrong with my python here, rather than your code? In any event, I'm happy to troubleshoot with you if you think it may be useful in general, but I've also now written my own MODIS download code -- let me know, and thanks again!
from pymodis.
lucadelu
commented on June 26, 2024
@mrjohnston Thanks a lot for your tests.
There is something wrong in your edits
try: # download and write the file
req = urllib.request.Request(url, headers = self.http_header)
http = urllib.request.urlopen(req)
orig_size = http.headers['Content-Length']
filSave.write(http.read())
print("inside requests")
# if local file has an error, try to download the file again
except:
try:
http = requests.get(url, timeout=self.timeout, verify=False)
orig_size = http.headers['Content-Length']
filSave.write(http.content)
print("inside urllib")
except:
import sys; print(sys.exc_info())
logging.error("Cannot download {name}. "
"Retrying...".format(name=filDown))
filSave.close()
os.remove(filSave.name)
import time
time.sleep(5)
self._downloadFileHTTP(filDown, filHdf, day)
filSave.close()
When you have modified this, if you are modifying the source code you should also reinstall pyModis, could you try again to download and send me the log file?
Thanks
from pymodis.
mrjohnston
commented on June 26, 2024
@lucadelu I just uninstalled pyModis, reinstalled it, and edited downmodis.py to be as above (I don't know what was going on with the indents in my last version; they showed up differently in my text editor, sorry). Then, I imported the (edited) pymodis:
$ python
Python 2.7.6 (default, Oct 26 2016, 20:30:19)
[GCC 4.8.4] on linux2
from pymodis import downmodis
dest='/tmp'
tiles = 'h08v07,h09v07,h07v06,h08v06,h09v06,h07v05,h08v05,h09v05'
delta = 1460
day = "2016-11-01"
username='user'
password='pw'
modisDown = downmodis.downModis(destinationFolder=dest, tiles=tiles, today=day, delta=delta, user = username, password = password)
modisDown.connect()
modisDown.downloadsAllDay()
I'm attaching the resulting logfile. Many thanks!
from pymodis.
lucadelu
commented on June 26, 2024
@mrjohnston sorry I put the print functions but it should be better to use the logging system.
try: # download and write the file
logging.error("inside requests")
req = urllib.request.Request(url, headers = self.http_header)
http = urllib.request.urlopen(req)
orig_size = http.headers['Content-Length']
filSave.write(http.read())
# if local file has an error, try to download the file again
except:
try:
logging.error("inside urllib")
http = requests.get(url, timeout=self.timeout, verify=False)
orig_size = http.headers['Content-Length']
filSave.write(http.content)
except:
import sys; logging.error((sys.exc_info())
logging.error("Cannot download {name}. "
"Retrying...".format(name=filDown))
filSave.close()
os.remove(filSave.name)
import time
time.sleep(5)
self._downloadFileHTTP(filDown, filHdf, day)
filSave.close()
However the problem seems to be related to authorization, instead it return a 401 code error
from pymodis.
mrjohnston
commented on June 26, 2024
@lucadelu I edited downmodis.py accordingly, except that the adding "logging.error((sys.exc_info())" caused an error in the import, so I had to omit it (downmodis.py attached).
The new code doesn't create a persistent logfile. The logfile is initialized (0 bytes) and then flashes in and out of the /tmp folder, never growing to >0 bytes. At a keyboard interrupt of the downloadsAllDay process, it disappears.
Regarding the original 401 error, I can reproduce it using the incorrect username/password. However, I'm pretty sure that the username & password I actually use in the code are correct, since they work for manual MODIS downloads.
Apologies for the mystery and the trouble!
from pymodis.
spatialexplore
commented on June 26, 2024
Hi, I am having the same problem on Windows 64bit (Server 2012), pyModis 2.0.4.
I am getting warnings
InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
``` and
``` req = urllib.request.Request(url, headers = self.http_header)
http = urllib.request.urlopen(req)
is throwing an exception "URLError(u'unknown url type: https',)"
and then
http = requests.get(url, timeout=self.timeout, verify=False)
orig_size = http.headers['Content-Length']
also throws an exception - because 'Content-Length' doesn't exist since the get request returns 401 - Access denied.
I tried adding 'auth=requests.auth.HTTPBasicAuth(self.user, self.password),' in the get request, but it still doesn't work. I can download fine from a browser with the same credentials.
from pymodis.
lucadelu
commented on June 26, 2024
@spatialexplore I also get InsecureRequestWarning but this is not causing me any problem downloading data.
Are you using python 2 or 3?
Are you sure that you download data from the same application that pyModis is using? For example are you able to download this file?
from pymodis.
lucadelu
commented on June 26, 2024
The InsecureRequestWarning should be fixed in 5b77f40, as reported in #81
from pymodis.
kbkhand
commented on June 26, 2024
@lucadelu I am having difficult time while downloading Aqua LST product (MYD11A2.006), the error is displayed as: Exception: There are some troubles with the server. The directory seems to be empty
while using this code:
dest = 'D/MODIS_LST/8-dayLST_aqua'
tiles = 'h09v05'
date = '2011.12.30'
enddate = '2011.01.01'
product = 'MYD11A2.006'
modis_down = downmodis.downModis(destinationFolder=dest, tiles=tiles, today=date, enddate=enddate, product=product, user=user, password=password)
modis_down.connect()
modis_down.downloadsAllDay()
the same code works fine for Terra LST. Thanks.
from pymodis.
kbkhand
commented on June 26, 2024
@lucadelu it worked! thanks much.
from pymodis.
Isaquedanielre
commented on June 26, 2024
Hi, I got the same problem.
My code is:
`from pymodis import downmodis
dest = "C:/Users/B2046470858/Documents/Isaque/CEF/cmscode/temp/"
tiles ="h11v12"
day="2016-01-01"
productSelect="MOD13Q1.006"
delta=45
modisDown = downmodis.downModis(destinationFolder=dest,
product=productSelect,
tiles=tiles,
today=day,
delta=delta,
user="user",
password="passw",
debug=True,)
modisDown.connect()
modisDown.downloadsAllDay()`
Any guest about?
I'm running the Anaconda python 2.7 in 64bits Windows, and the log file show the problem:
2017-08-16 14:59:42,950 - DEBUG - Starting new HTTPS connection (1): e4ftl01.cr.usgs.gov
2017-08-16 14:59:43,655 - DEBUG - https://e4ftl01.cr.usgs.gov:443 "GET /MOLT/MOD13Q1.006/2016.01.01/MOD13Q1.A2016001.h11v12.006.2016029064227.hdf HTTP/1.1" 302 522
2017-08-16 14:59:43,657 - DEBUG - Starting new HTTPS connection (1): urs.earthdata.nasa.gov
2017-08-16 14:59:44,336 - DEBUG - https://urs.earthdata.nasa.gov:443 "GET /oauth/authorize?scope=uid&app_type=401&client_id=ijpRZvb9qeKCK5ctsn75Tg&response_type=code&redirect_uri=https%3A%2F%2Fe4ftl01.cr.usgs.gov%2Foauth&state=aHR0cHM6Ly9lNGZ0bDAxLmNyLnVzZ3MuZ292L01PTFQvTU9EMTNRMS4wMDYvMjAxNi4wMS4wMS9NT0QxM1ExLkEyMDE2MDAxLmgxMXYxMi4wMDYuMjAxNjAyOTA2NDIyNy5oZGY HTTP/1.1" 401 None
2017-08-16 14:59:44,341 - WARNING - Tried to downlaod with requests but got this error (<type 'exceptions.KeyError'>, KeyError('content-length',), <traceback object at 0x000000000C6CFB88>)
2017-08-16 14:59:44,341 - ERROR - Cannot download MOD13Q1.A2016001.h11v12.006.2016029064227.hdf. Retrying...
2017-08-16 14:59:52,411 - WARNING - Tried to downlaod with urllib but got this error (<class 'future.backports.urllib.error.HTTPError'>, HTTPError(), <traceback object at 0x000000000C6DDF88>)
My installed libraries:
`C:\Users\B2046470858\Documents\Isaque\CEF\cmscode>conda list
packages in environment at C:\Users\B2046470858\AppData\Local\Continuum\Anaconda2:
_license 1.1 py27_1
affine 2.1.0 py27_0 conda-forge
alabaster 0.7.10 py27_0
anaconda custom py27_0
anaconda-client 1.6.3 py27_0
anaconda-navigator 1.6.2 py27_0
anaconda-project 0.6.0 py27_0
asn1crypto 0.22.0 py27_0
astroid 1.4.9 py27_0
astropy 1.3.2 np112py27_0
babel 2.4.0 py27_0
backports 1.0 py27_0
backports_abc 0.5 py27_0
beautifulsoup4 4.6.0 py27_0
biggus 0.15.0 py27_0 conda-forge
bitarray 0.8.1 py27_1
blaze 0.10.1 py27_0
bleach 1.5.0 py27_0
bokeh 0.12.5 py27_1
boto 2.46.1 py27_0
boto3 1.4.4 py27_0 conda-forge
botocore 1.5.84 py27_0 conda-forge
bottleneck 1.2.1 np112py27_0
bzip2 1.0.6 vc9_3 [vc9]
cartopy 0.15.1 np112py27_2 conda-forge
cdecimal 2.3 py27_2
certifi 2017.7.27.1
cf_units 1.1.3 py27_1 conda-forge
cffi 1.10.0 py27_0
chardet 3.0.3 py27_0
chardet 3.0.4
click 6.7 py27_0
click-plugins 1.0.3 py27_0 conda-forge
cligj 0.4.0 py27_0 conda-forge
cloudpickle 0.2.2 py27_0
clyent 1.2.2 py27_0
colorama 0.3.9 py27_0
comtypes 1.1.2 py27_0
conda 4.3.23 py27_0 conda-forge
conda-env 2.6.0 0 conda-forge
configparser 3.5.0 py27_0
console_shortcut 0.1.1 py27_1
contextlib2 0.5.5 py27_0
cryptography 1.8.1 py27_0
curl 7.52.1 vc9_0 [vc9]
cycler 0.10.0 py27_0
cython 0.25.2 py27_0
cytoolz 0.8.2 py27_0
dask 0.14.3 py27_1
datashape 0.5.4 py27_0
decorator 4.0.11 py27_0
descartes 1.0.2 py27_0 ioos
distributed 1.16.3 py27_0
docutils 0.13.1 py27_0
entrypoints 0.2.2 py27_1
enum34 1.1.6 py27_0
espa-api-client 1.0.0.dev27
et_xmlfile 1.0.1 py27_0
expat 2.1.0 vc9_0 [vc9] ioos
fastcache 1.0.2 py27_1
fiona 1.7.8 np112py27_0 conda-forge
flask 0.12.2 py27_0
flask-cors 3.0.2 py27_0
folium 0.2.1 py27_0 ioos
freetype 2.5.5 vc9_2 [vc9]
freexl 1.0.2 vc9_0 [vc9] ioos
funcsigs 1.0.2 py27_0
functools32 3.2.3.2 py27_0
future 0.16.0
futures 3.1.1 py27_0
GDAL 2.2.0
gdal 2.2.0 np112py27_vc9_2 [vc9] conda-forge
geojson 1.3.3 py27_0 ioos
geopandas 0.2.1 py27_4 conda-forge
geopy 1.10.0 py27_0 ioos
geos 3.5.1 vc9_1 [vc9] conda-forge
get_terminal_size 1.0.0 py27_0
gevent 1.2.1 py27_0
greenlet 0.4.12 py27_0
GribInventory 0.0.2
grin 1.2.1 py27_3
h5py 2.7.0 np112py27_1 conda-forge
hdf4 4.2.12 vc9_0 [vc9] conda-forge
hdf5 1.8.18 vc9_0 [vc9] conda-forge
heapdict 1.0.0 py27_1
html5lib 0.999 py27_0
icu 57.1 vc9_0 [vc9]
idna 2.5 py27_0
idna 2.6
imagesize 0.7.1 py27_0
ipaddress 1.0.18 py27_0
ipykernel 4.6.1 py27_0
ipython 5.3.0 py27_0
ipython_genutils 0.2.0 py27_0
ipywidgets 6.0.0 py27_0
iris 1.13.0 py27_1 conda-forge
iris_grib 0.9.0 0 scitools
isort 4.2.5 py27_0
itsdangerous 0.24 py27_0
jdcal 1.3 py27_0
jedi 0.10.2 py27_2
jinja2 2.9.6 py27_0
jmespath 0.9.3 py27_0 conda-forge
jpeg 9b vc9_0 [vc9]
jsonschema 2.6.0 py27_0
jupyter 1.0.0 py27_3
jupyter_client 5.0.1 py27_0
jupyter_console 5.1.0 py27_0
jupyter_core 4.3.0 py27_0
kealib 1.4.7 vc9_2 [vc9] conda-forge
krb5 1.14.2 vc9_0 [vc9] conda-forge
lazy-object-proxy 1.2.2 py27_0
libgdal 2.1.4 vc9_2 [vc9] conda-forge
libiconv 1.14 vc9_0 [vc9] ioos
libnetcdf 4.4.1.1 vc9_5 [vc9] conda-forge
libpng 1.6.28 vc9_1 [vc9] conda-forge
libpq 9.5.4 vc9_4 [vc9] conda-forge
libspatialindex 1.8.5 vc9_0 [vc9] ioos
libspatialite 4.3.0a vc9_15 [vc9] conda-forge
libtiff 4.0.6 vc9_3 [vc9]
libxml2 2.9.4 vc9_0 [vc9] ioos
llvmlite 0.18.0 py27_0
locket 0.2.0 py27_1
lxml 3.7.3 py27_0
markupsafe 0.23 py27_2
matplotlib 2.0.2 np112py27_0
menuinst 1.4.7 py27_0
mistune 0.7.4 py27_0
mkl 2017.0.1 0
mkl-service 1.1.2 py27_3
mplleaflet 0.0.5 py27_0 ioos
mpmath 0.19 py27_1
msgpack-python 0.4.8 py27_0
multipledispatch 0.4.9 py27_0
munch 2.1.1 py27_0 conda-forge
navigator-updater 0.1.0 py27_0
nbconvert 5.1.1 py27_0
nbformat 4.3.0 py27_0
nc_time_axis 1.0.2 py27_0 conda-forge
netcdf4 1.2.9 np112py27_0 conda-forge
networkx 1.11 py27_0
nltk 3.2.3 py27_0
nose 1.3.7 py27_1
notebook 5.0.0 py27_0
numba 0.33.0 np112py27_0
numexpr 2.6.2 np112py27_0
numpy 1.12.1 py27_0
numpy 1.13.1
numpydoc 0.6.0 py27_0
odo 0.5.0 py27_1
olefile 0.44 py27_0
openjpeg 2.1.2 vc9_3 [vc9] conda-forge
openpyxl 2.4.7 py27_0
openssl 1.0.2l vc9_0 [vc9]
owslib 0.14.0 py27_0 conda-forge
packaging 16.8 py27_0
pandas 0.20.1 np112py27_0
pandocfilters 1.4.1 py27_0
partd 0.3.8 py27_0
path.py 10.3.1 py27_0
pathlib2 2.2.1 py27_0
patsy 0.4.1 py27_0
pcre 8.39 vc9_0 [vc9] conda-forge
pep8 1.7.0 py27_0
pickleshare 0.7.4 py27_0
pillow 4.1.1 py27_0
pip 9.0.1 py27_1
ply 3.10 py27_0
proj4 4.9.3 vc9_0 [vc9] ioos
prompt_toolkit 1.0.14 py27_0
psutil 5.2.2 py27_0
psycopg2 2.7.1 py27_0 conda-forge
py 1.4.33 py27_0
pycosat 0.6.2 py27_0
pycparser 2.17 py27_0
pycrypto 2.6.1 py27_6
pycurl 7.43.0 py27_2
pyepsg 0.3.2 py27_0 conda-forge
pyflakes 1.5.0 py27_0
pygments 2.2.0 py27_0
pyke 1.1.1 py27_0 conda-forge
pylint 1.6.4 py27_1
pyModis 2.0.6
pyodbc 4.0.16 py27_0
pyopenssl 17.0.0 py27_0
pyowm 2.7.1
pyparsing 2.1.4 py27_0
pyproj 1.9.5.1 py27_0 conda-forge
pyqt 5.6.0 py27_2
pysal 1.12.0 py27_0 ioos
pyshp 1.2.11 py27_0 conda-forge
pytables 3.4.2 np112py27_1 conda-forge
pytest 3.0.7 py27_0
python 2.7.13 1
python-dateutil 2.6.0 py27_0
pytz 2017.2 py27_0
pywavelets 0.5.2 np112py27_0
pywin32 220 py27_2
pyyaml 3.12 py27_0
pyzmq 16.0.2 py27_0
qt 5.6.2 vc9_4 [vc9]
qtawesome 0.4.4 py27_0
qtconsole 4.3.0 py27_0
qtpy 1.2.1 py27_0
rasterio 1.0a9 py27_1 conda-forge
requests 2.18.4
requests 2.14.2 py27_0
requests-futures 0.9.7
rios 1.4.3 py27_3 conda-forge
rope 0.9.4 py27_1
rtree 0.8.2 py27_0 ioos
ruamel_yaml 0.11.14 py27_1
s3transfer 0.1.10 py27_1 conda-forge
scandir 1.5 py27_0
scikit-image 0.13.0 np112py27_0
scikit-learn 0.18.2 np112py27_0
scipy 0.19.0 np112py27_0
seaborn 0.7.1 py27_0
setuptools 27.2.0 py27_1
shapely 1.5.17 np112py27_3 conda-forge
simplegeneric 0.8.1 py27_1
singledispatch 3.4.0.3 py27_0
sip 4.18 py27_0
six 1.10.0 py27_0
snowballstemmer 1.2.1 py27_0
snuggs 1.4.1 py27_0 conda-forge
sortedcollections 0.5.3 py27_0
sortedcontainers 1.5.7 py27_0
sphinx 1.5.6 py27_0
spyder 3.1.4 py27_0
sqlalchemy 1.1.9 py27_0
sqlite 3.13.0 vc9_0 [vc9] ioos
ssl_match_hostname 3.4.0.2 py27_1
statsmodels 0.8.0 np112py27_0
subprocess32 3.2.7 py27_0
sympy 1.0 py27_0
tblib 1.3.2 py27_0
testpath 0.3 py27_0
tk 8.5.18 vc9_0 [vc9]
toolz 0.8.2 py27_0
tornado 4.5.1 py27_0
traitlets 4.3.2 py27_0
tuiview 1.2.0 np112py27_0 conda-forge
udunits2 2.2.23 vc9_0 [vc9] conda-forge
unicodecsv 0.14.1 py27_0
urllib3 1.22
usgs 0.2.1
vc 9 0 conda-forge
vs2008_runtime 9.00.30729.5054 0
vs2015_runtime 14.0.25123 0
wcwidth 0.1.7 py27_0
werkzeug 0.12.2 py27_0
wget 3.2
wheel 0.29.0 py27_0
widgetsnbextension 2.0.0 py27_0
win_unicode_console 0.5 py27_0
wrapt 1.10.10 py27_0
wxpython 4.0.0a3 py27_0 conda-forge
xerces-c 3.1.4 vc9_0 [vc9] ioos
xlrd 1.0.0 py27_0
xlsxwriter 0.9.6 py27_0
xlwings 0.10.4 py27_0
xlwt 1.2.0 py27_0
xz 5.2.2 0 conda-forge
zict 0.1.2 py27_0
zlib 1.2.8 vc9_3 [vc9]
`
from pymodis.
penguinnick
commented on June 26, 2024
Hi, I',m having the same problem as above:
Not sure what info you need.. I'm going after MOD17A3 data. Not too savvy in python either..
Traceback (most recent call last):
File "/usr/local/bin/modis_download_from_list.py", line 133, in
main()
File "/usr/local/bin/modis_download_from_list.py", line 119, in main
modisOgg.connect()
File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 383, in connect
raise Exception("There are some troubles with the server. "
Exception: There are some troubles with the server. The directory seems to be empty
from pymodis.
lucadelu
commented on June 26, 2024
@penguinnick could you send the command line used?
from pymodis.
penguinnick
commented on June 26, 2024
modis_download.py --url https://e4ftl01.cr.usgs.gov -p MOD17A3.055 -I -r -t h10v05,h11v05,h10v06,h11v06 -A -U <username> -P <password> ~/LVM_share/modis_grid/modis121117/
from pymodis.
spareeth
commented on June 26, 2024
Hi Luca
I have the same issue with Windows 7. I am using pyModis-master and the OSGeo4W64 way of installing. Installation went well, but on trying to download I get this error:
Traceback (most recent call last):
File "C:\Users\paree2\AppData\Roaming\GRASS7\addons/scripts/r.modis.download.py", line 269, in
sys.exit(main())
File "C:\Users\paree2\AppData\Roaming\GRASS7\addons/scripts/r.modis.download.py", line 249, in main
modisOgg.connect()
File "C:\OSGEO4~1\apps\Python27\lib\site-packages\pymodis\downmodis.py", line 383, in connect
raise Exception("There are some troubles with the server. "
Exception: There are some troubles with the server. The directory seems to be empty
I am using r.modis addon with GRASS GIS.
r.modis.download -g settings=- tiles=h22v05 startday=2016-07-02 endday=2016-07-04 folder=D:\temp
The same command works well in Linux (Ubuntu) with the same latest version of pyModis.
from pymodis.
lucadelu
commented on June 26, 2024
@penguinnick your command is working for me
-rw-r--r-- 1 lucadelu femgis 327 Dec 12 16:28 modisMOD17A3.055.log
-rw-r--r-- 1 lucadelu femgis 1.2M Dec 12 16:28 MOD17A3.A2014001.h10v06.055.2016008120516.hdf
-rw-r--r-- 1 lucadelu femgis 5.8K Dec 12 16:28 MOD17A3.A2014001.h10v05.055.2016008120515.hdf.xml
-rw-r--r-- 1 lucadelu femgis 5.8K Dec 12 16:28 MOD17A3.A2014001.h10v06.055.2016008120516.hdf.xml
-rw-r--r-- 1 lucadelu femgis 5.8K Dec 12 16:28 MOD17A3.A2014001.h11v06.055.2016008120533.hdf.xml
-rw-r--r-- 1 lucadelu femgis 5.8M Dec 12 16:28 MOD17A3.A2014001.h10v05.055.2016008120515.hdf
-rw-r--r-- 1 lucadelu femgis 0 Dec 12 16:28 MOD17A3.A2014001.h11v05.055.2016008120532.hdf
-rw-r--r-- 1 lucadelu femgis 242 Dec 12 16:28 listfileMOD17A3.055.txt
@penguinnick @spareeth did you follow the instructions and are you sure about user and password?
from pymodis.
sapkotashishir
commented on June 26, 2024
from pymodis import downmodis
dest = "/Data preparation"
tiles1 = "h27v06,h28v08,h27v08,h26v06,h28v07,h27v07"
day = "2019-10-31"
enddate1 = '2019-11-1 '
delta = 1
pass1 = "Dontthink91321"
user1 = "shishir123"
url1 = "https://ladsweb.modaps.eosdis.nasa.gov/"
product1 = 'MCD19A2.006'
modisDown = downmodis.downModis(destinationFolder = dest, password = pass1,
user = user1, url = url1, tiles = tiles, path = 'MOLT',
product = product1, today = day, enddate = enddate1,
delta = delta)
modisDown.connect()
I want to download MCD19A2 data from ladsweb . What source parameter should I keep in the downModis function
from pymodis.
DomiNando
commented on June 26, 2024
Hello @lucadelu,
I installed all the packages necessary to make pymodis run (I think). These are all my libraries:
appdirs==1.4.4
appnope==0.1.0
backcall==0.2.0
certifi==2020.4.5.1
chardet==3.0.4
decorator==4.4.2
distlib==0.3.1
filelock==3.0.12
future==0.18.2
GDAL==3.1.0
idna==2.10
ipython==7.16.1
ipython-genutils==0.2.0
jedi==0.17.1
numpy==1.18.4
parso==0.7.0
pexpect==4.8.0
pickleshare==0.7.5
prompt-toolkit==3.0.5
ptyprocess==0.6.0
Pygments==2.6.1
pyModis==2.0.9
pyproj==2.6.1.post1
requests==2.24.0
Shapely==1.7.0
six==1.15.0
traitlets==4.3.3
urllib3==1.25.9
virtualenv==20.0.25
wcwidth==0.2.5
I'm using MacOS X Catalina, 10.15.5. I'm not extremely well versed in Python, as I started playing around with it a couple of weeks ago. I basically downloaded your Jupyter Notebook Pymodis code, and tweeked it a bit to make it more compatible with the inputs I'm feeding to a Landsatxplore Python code I will merge with this one. I'm testing to see if I can establish a connection with the Modis database. I already created the .netrc file and I placed it where it's supposed to be. I'll only post the code that's giving me issues up to the trouble point. Can you please tell me why I'm getting a "There are some troubles with the server. The directory seems to be empty." exception? I might be missing something here. Also, where can I find the debug information I set the True flag for? (Apologies if my questions are somewhat basic, I'm still trying to learn how to work with all of this.)
Code:
import os
import glob
from IPython.core.display import Image
from pymodis import downmodis
from datetime import datetime
destination folder
dest = "/tmp"
tiles to download
tiles = "h18v04,h19v04"
starting day
day = "2014-08-14"
beginDate = datetime.strptime(day,"%Y-%m-%d")
#end date
day2 = "2014-08-15"
endDate= datetime.strptime(day2,"%Y-%m-%d")
number of day to download
deltaT = endDate- beginDate
delta = deltaT.days
modisDown = downmodis.downModis(destinationFolder=dest, tiles=tiles, today=day, delta=delta, debug=True)
modisDown.connect()
Exception:
Exception Traceback (most recent call last)
in
----> 1 modisDown.connect()
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pymodis/downmodis.py in connect(self, ncon)
383 self._connectHTTP(ncon)
384 if len(self.dirData) == 0:
--> 385 raise Exception("There are some troubles with the server. "
386 "The directory seems to be empty")
387
Exception: There are some troubles with the server. The directory seems to be empty
Thanks for your help!
from pymodis.
DomiNando
commented on June 26, 2024
Hello @lucadelu,
I installed all the packages necessary to make pymodis run (I think). These are all my libraries:
appdirs==1.4.4
appnope==0.1.0
backcall==0.2.0
certifi==2020.4.5.1
chardet==3.0.4
decorator==4.4.2
distlib==0.3.1
filelock==3.0.12
future==0.18.2
GDAL==3.1.0
idna==2.10
ipython==7.16.1
ipython-genutils==0.2.0
jedi==0.17.1
numpy==1.18.4
parso==0.7.0
pexpect==4.8.0
pickleshare==0.7.5
prompt-toolkit==3.0.5
ptyprocess==0.6.0
Pygments==2.6.1
pyModis==2.0.9
pyproj==2.6.1.post1
requests==2.24.0
Shapely==1.7.0
six==1.15.0
traitlets==4.3.3
urllib3==1.25.9
virtualenv==20.0.25
wcwidth==0.2.5I'm using MacOS X Catalina, 10.15.5. I'm not extremely well versed in Python, as I started playing around with it a couple of weeks ago. I basically downloaded your Jupyter Notebook Pymodis code, and tweeked it a bit to make it more compatible with the inputs I'm feeding to a Landsatxplore Python code I will merge with this one. I'm testing to see if I can establish a connection with the Modis database. I already created the .netrc file and I placed it where it's supposed to be. I'll only post the code that's giving me issues up to the trouble point. Can you please tell me why I'm getting a "There are some troubles with the server. The directory seems to be empty." exception? I might be missing something here. Also, where can I find the debug information I set the True flag for? (Apologies if my questions are somewhat basic, I'm still trying to learn how to work with all of this.)
Code:
import os
import glob
from IPython.core.display import Image
from pymodis import downmodis
from datetime import datetimedestination folder
dest = "/tmp"
tiles to download
tiles = "h18v04,h19v04"
starting day
day = "2014-08-14"
beginDate = datetime.strptime(day,"%Y-%m-%d")
#end date
day2 = "2014-08-15"
endDate= datetime.strptime(day2,"%Y-%m-%d")number of day to download
deltaT = endDate- beginDate
delta = deltaT.days
modisDown = downmodis.downModis(destinationFolder=dest, tiles=tiles, today=day, delta=delta, debug=True)
modisDown.connect()
Exception:
Exception Traceback (most recent call last)
in
----> 1 modisDown.connect()/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pymodis/downmodis.py in connect(self, ncon)
383 self._connectHTTP(ncon)
384 if len(self.dirData) == 0:
--> 385 raise Exception("There are some troubles with the server. "
386 "The directory seems to be empty")
387Exception: There are some troubles with the server. The directory seems to be empty
Thanks for your help!
Just tried it in Ubuntu 20.04, and I had the exact same issue.
from pymodis.
Related Issues (20)
- Unable to download MYDOCGA HOT 1
- netrc file no longer recognized HOT 2
- Why I am getting the error </head> doesn't have matching <head>> HOT 1
- create_mosaic/convert_modis is returning an empty list
- I am trying to download MOD09GA and MYD09GA in a for loop. MOD09GA was downloaded successfully, but MYD09GA failed, why? HOT 1
- documentation issue HOT 1
- error when running pymodis with python 3.9 and 3.10 HOT 3
- I can't get pymodis in my vscode HOT 2
- AttributeError: 'xml.etree.ElementTree.Element' object has no attribute 'getiterator' HOT 1
- Downloading MOD10A2(V5) failed using downModis()
- Pymodis website cannot be accessed HOT 1
- MODIS authorization stopped working HOT 10
- Project requires some support HOT 6
- Unable to import pymodis
- Downloading NASADEM_HGT v001 bug HOT 1
- Error when convert the MOD13Q1.006 to GTIFF format HOT 1
- KeyError: 'EV_250_RefSB'
- MODIS No Data value "NA" in new MOD11A1 v006 product error HOT 1
- Error running modis_download.py HOT 3
- Maybe it's time to release v2.4.1? HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pymodis.