lucadelu / pymodis

python library to work with MODIS data

Welcome in pyModis library

Thank you for downloading pyModis

pyModis is a Python library to download and process MODIS data from NASA server

Copyright Luca Delucchi - 2010-2020

License

pyModis is licensed under the terms of GNU GPL 2. Read COPYING for more info

Install

For how to compile or install pyModis see INSTALL file.

Executables

Some executables are distributed with the library:

• modis_download downloads MODIS data
• modis_download_from_list downloads MODIS data from NASA servers, the names of files to download have to be contained into a text file.
• modis_parse parses the XML file of MODIS data and return some variables
• modis_multiparse parses the XML file of multiple MODIS data and returns the bounding box or writes the XML file with the information of all selected tiles
• modis_mosaic creates the mosaic of multiple MODIS tiles
• modis_convert converts MODIS data from HDF format and Sinusoidal projection to other formats and projections
• modis_quality checks the quality of MODIS data using the QA layer

Other info

The logo of pyModis is pyModis.png; it is released under CC-BY-SA and it's distributed with the pyModis library

More info about pyModis at http://www.pymodis.org/

pymodis's Issues

Pymodis creatMosaic output missing

I'm trying to use createMosaic function. It's working but i dont know where the output file is stored.
Kindly check out this.
This is what i used in Ipython

convertmodis.createMosaic(listfile="E://listfileMOD11A1.005.txt", outprefix='out', mrtpath="E://", subset=False)
<pymodis.convertmodis.createMosaic instance at 0x0000000009E4E5C8>
where is the output??? Or else tell me how to use creatmosaic function in python. I need to mosaic 100 files.Kindly give ur suggestion as soon as possible.

Regards
Rajasivaranjan

SyntaxError: Missing parentheses in call to 'print' : __init__.py

Log:

import pymodis
Traceback (most recent call last):
File "", line 1, in
File "~/anaconda/lib/python3.5/site-packages/pymodis/init.py", line 10
print "WxPython missing, no GUI enabled"
^
SyntaxError: Missing parentheses in call to 'print'

Solution: for Python 3.5 change line 10 as print ("WxPython missing, no GUI enabled")

Error using modis_mosaic.py

I am getting used to the pymodis library by trying out the examples given on the page :

(http://pymodis.fem-environment.eu/examples/full_example_gdal.html#downloading-data)

Using modis_download.py I managed to successfully download the .hdh files and the .xml files along with the text file listing them. Next I tried to mosaic them as in the example using modis_mosaic.py but I get the error as :

ERROR 4: `MOD11A1.A2012340.h29v05.005.2012342062359.hdf' does not exist in the file system,
and is not recognised as a supported dataset name.

The .hdf and .xml files are present in the directory, attaching snapshot of it too.

Use destinationPath in checkFile?

It would be nice if checkFile automatically looked in destinationPath for the named hdf. Happy to send PR...

SyntaxError: invalid syntax : downmodis.py

Logs:
import pymodis
Traceback (most recent call last):
File "", line 1, in
File "/anaconda/lib/python3.5/site-packages/pymodis/init.py", line 1, in
import downmodis
File "/anaconda/lib/python3.5/site-packages/pymodis/downmodis.py", line 354
except (EOFError, ftplib.error_perm), e:

SyntaxError: invalid syntax

Solution: change "except (EOFError, ftplib.error_perm), e:" to "except (EOFError, ftplib.error_perm) as e:"

GDAL_HDF4_support

Hi PyModis Users,

Good day!

I would want to use pyMODIS to download MODIS data but I am faced with some challenges. Could I ask for some guidance?

Here is the error message that I am receiving when running the code:

runfile('E:/Python_scripts/pyModis_download.py', wdir='E:/Python_scripts')
Traceback (most recent call last):
File "", line 1, in
runfile('E:/Python_scripts/pyModis_download.py', wdir='E:/Python_scripts')
File "C:\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 580, in runfile
execfile(filename, namespace)
File "E:/Python_scripts/pyModis_download.py", line 19, in
tiles='h26v06', path='MOLT', product='MOD13Q1.005', today=2015-01-01, enddate=2015-03-24)
File "C:\Anaconda\lib\site-packages\pymodis\downmodis.py", line 272, in init
raise IOError("GDAL installation has no support for HDF4, please"
IOError: GDAL installation has no support for HDF4, please update GDAL

In the end it says: GDAL installation has no support for HDF4, please update GDAL.

Then I tried to update my Python GDAL version using

pip install GDAL --upgrade

But it seems that my GDAL version is up-to-date. With regards to my GDAL core installer from GISinternals, I just downloaded and installed it this February 2015.

Could you help me trouble shoot this issue?

This is the code that I am trying to test.

import os
import pymodis
from pymodis import downmodis
ws = 'E:/Test'

tile = 'h26v06'

prod = 'MOD11C2.005'

day = '2014-01-01'

end = '2014-01-01'

pymodis.downmodis.downModis(ws, url='http://e4ftl01.cr.usgs.gov',\
tiles='h26v06', path='MOLT', product='MOD13Q1.005', today=2015-01-01, enddate=2015-03-24)

Thanks and all the best.
-Leo

modis_mosaic.py: add support for multiple time stamps

When downloading MODIS data of several days, the list file contains all of them. While this is fine, the modis_mosaic.py command does not respect this yet as it should generate multiple mosaiks with the timestamp in the resulting filename(s).

Error when using FTP server

Line 726 of downmodis.py throws the following error. There seems to be a typo: line 726 calls self._downloadAllDaysFTP(), but the function is called _downloadsAllDayFTP(self, days) (with 's' after 'download' but 'Day' instead of 'Days'.

---

AttributeError Traceback (most recent call last)
in ()
----> 1 modisDown.downloadsAllDay()

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pymodis/downmodis.pyc in downloadsAllDay(self, clean, allDays)
724 self._downloadAllDaysHTTP(days)
725 elif self.urltype == 'ftp':
--> 726 self._downloadAllDaysFTP(days)
727
728 def _downloadAllDaysHTTP(self, days):

AttributeError: downModis instance has no attribute '_downloadAllDaysFTP'

FTP download: list index out of range

List index out of range when calling downloadsAllDayFTP:
(Debugger tells me variable 'name' is a list containing this element: ['001'] )
Note: this is for FTP server ftp://ladsweb.nascom.nasa.gov/allData/51/MOD04_L2/

Traceback (most recent call last):
File "/Users/username/Development/SunsetAnalysis/pyModisTestFromNASASite.py", line 63, in
modisDown.downloadsAllDay()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pymodis/downmodis.py", line 726, in downloadsAllDay
self._downloadAllDaysFTP(days)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pymodis/downmodis.py", line 756, in _downloadAllDaysFTP
listAllFiles = self.getFilesList()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pymodis/downmodis.py", line 448, in getFilesList
return self._getFilesListFTP()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pymodis/downmodis.py", line 525, in _getFilesListFTP
finalList = cicle_file()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pymodis/downmodis.py", line 505, in cicle_file
if self.tiles.count(name[3]) == 1 and jpeg:
IndexError: list index out of range

installation problem

Hi all,
I tried all the installation options avalable in the page and also followed the trobleshooting methods. All the time this is the error message during the installation "warning C4530: c++ exception handler used, but unwind semantics are not enabled. Specify /EHsc" . I think it related to http://msdn.microsoft.com/en-us/library/2axwkyt4.aspx

I don`t have much idea on how to enable that. Your help is highly appreciated. Hope this is the correct place for this post if not please redirect me. Thanks!

pyModis GDAL HDF4 support

Dear Luca DeLucchi,

Good day!

I would want to use pyMODIS to download MODIS data but I am faced with some challenges. Could I ask for your guidance?

Here is the error message that I am receiving when running the code:

runfile('E:/Python_scripts/pyModis_download.py', wdir='E:/Python_scripts')
Traceback (most recent call last):
File "", line 1, in
runfile('E:/Python_scripts/pyModis_download.py', wdir='E:/Python_scripts')
File "C:\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 580, in runfile
execfile(filename, namespace)
File "E:/Python_scripts/pyModis_download.py", line 19, in
tiles='h26v06', path='MOLT', product='MOD13Q1.005', today=2015-01-01, enddate=2015-03-24)
File "C:\Anaconda\lib\site-packages\pymodis\downmodis.py", line 272, in init
raise IOError("GDAL installation has no support for HDF4, please"
IOError: GDAL installation has no support for HDF4, please update GDAL

In the end it says: GDAL installation has no support for HDF4, please update GDAL.

Then I tried to update my Python GDAL version using

pip install GDAL --upgrade

But it seems that my GDAL version is up-to-date. With regards to my GDAL core installer from GISinternals, I just downloaded and installed it this February 2015.

Could you help me trouble shoot this issue?

This is the code that I am trying to test.

import os
import pymodis
from pymodis import downmodis
ws = 'E:/Test'

tile = 'h26v06'

prod = 'MOD11C2.005'

day = '2014-01-01'

end = '2014-01-01'

pymodis.downmodis.downModis(ws, url='http://e4ftl01.cr.usgs.gov',\
tiles='h26v06', path='MOLT', product='MOD13Q1.005', today=2015-01-01, enddate=2015-03-24)

Thanks and all the best in your future development for pyMODIS.

Deactivate proxy to download

Hi,
In order to download modis images, I have to deactivate the proxy server option in internet explorer otherwise I get this error:

raise Exception("There are some troubles with the server. "

Exception: There are some troubles with the server. The directory seems to be empty

Is there a way to modify the python script so that I don't have to deactivate this option everytime I need to download images?

Thanks!

Date range

When I want to download data for a some date range, I use parameters '-f' and '-e'. In documentation is written, that the '-f','--firstday' is 'the day to start download'. That's why I thought that when I want to download the data for example from 2013-01-01 to 2013-01-05 I would expect, that this command will do that:

modis_download.py -r -t h20v01 -f 2013-01-01 -e 2013-01-05 E:\tmp\pymodis

but after execute this command, in the destination folder are just files for four days (starting from 2013-01-02). Is that correct? When looking at the code, there are these lines downmodis.py>downModis>_getToday:

def _getToday(self):
        """Set the dates for the start and end of downloading"""
        ...
        # set delta
        if self.today and self.enday:
            if self.today < self.enday:
                today = self.enday
                enday = self.today
                self.today = today
                self.enday = enday
            delta = self.today - self.enday
            self.delta = abs(delta.days)

and then at [downmodis.py>downModis>getListDays]

def getListDays(self):
        """Return a list of all selected days"""
        self._getToday()

        today_s = self.today.strftime("%Y.%m.%d")
        # dirData is reverse sorted
        for i, d in enumerate(self.dirData):
            if d <= today_s:
                today_index = i
                break

        days = self.dirData[today_index:][:self.delta]

It means, that having the first day = 2013-01-01 and the end day = 2013-01-05, the self.delta variable is 4, not 5 as I would expect. And then at getListDays() method we get list of days counting from the end day and using delta = 4. That's why we have date range [2013-01-02:2013-01-05]. For me it looks strange. Should it behave like that?

modis_download.py download error

Hi Luca,

I have installed the latest version (2.0.2) in both Ubuntu Trusty and openSUSE Leap through pip.
I have confirmed my username and password by downloading data from the e4ftl01.cr.usgs.gov ftp server.

I am getting the same error on both systems:

2016-10-14 01:14:20,442 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov 2016-10-14 01:14:20,821 - DEBUG - "GET /MOLT/MOD09A1.005 HTTP/1.1" 301 252 2016-10-14 01:14:21,060 - DEBUG - "GET /MOLT/MOD09A1.005/ HTTP/1.1" 200 None 2016-10-14 01:14:22,228 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov 2016-10-14 01:14:22,627 - DEBUG - "GET /MOLT/MOD09A1.005/2016.09.05 HTTP/1.1" 301 263 2016-10-14 01:14:23,213 - DEBUG - "GET /MOLT/MOD09A1.005/2016.09.05/ HTTP/1.1" 200 None 2016-10-14 01:14:26,713 - ERROR - Cannot download MOD09A1.A2016249.h19v05.005.2016258070854.hdf.xml. Retrying... 2016-10-14 01:14:33,016 - ERROR - Cannot download MOD09A1.A2016249.h19v05.005.2016258070854.hdf.xml. Retrying... 2016-10-14 01:14:39,428 - ERROR - Cannot download MOD09A1.A2016249.h19v05.005.2016258070854.hdf.xml. Retrying... 2016-10-14 01:14:45,705 - ERROR - Cannot download MOD09A1.A2016249.h19v05.005.2016258070854.hdf.xml. Retrying... 2016-10-14 01:14:52,098 - ERROR - Cannot download MOD09A1.A2016249.h19v05.005.2016258070854.hdf.xml. Retrying..

Order of bound variables incorrect for confResample_swath

Lines 473-476 should be replaced with to ensure naming consistency:

conFile.write("OUTPUT_SPACE_UPPER_LEFT_CORNER (LONG LAT) = %f %f\n" % (bound['min_lon'],
bound['max_lat']))
conFile.write("OUTPUT_SPACE_LOWER_RIGHT_CORNER (LONG LAT) = %f %f\n" % (bound['max_lon'],
bound['min_lat']))

pyModis windows 64 work around not working

I have not been successful getting pyModis in Windows going. I have Anaconda python 2.7 installed. Tried installing OSGeo4W (and pyModis) received "package missing in current win-64 channels" error(s for both). I could not follow to completion the second windows work around. Any other windows users successfully get pyModis working?

clip raster

Give the possibility to clip a raster using a bounding box or a vector file

not downloading anything

trying to download over h30v08 tile for 2016, but seems to me it's not downloading anything. The log file continues to increase in size (more logs) but the download for first tile does not move more than 0 bytes, and it seems to me that download is failing, and it tries to download again (the loop conitnues). Any advise? Been using pymodis for a long time now, but I changed my computer and reinstall again. I am using pymodis 1.0 and using the latest package of pymodis in PyPi. here is a sample of the content of the log file:

2016-10-09 15:45:41,039 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-10-09 15:45:41,091 - ERROR - Error in connection
2016-10-09 15:45:41,091 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-10-09 15:45:41,154 - ERROR - Error in connection
2016-10-09 15:45:41,154 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-10-09 15:45:41,191 - ERROR - Error in connection
2016-10-09 15:45:41,191 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-10-09 15:45:41,892 - DEBUG - "GET /MOLT/MOD13Q1.005 HTTP/1.1" 301 252
2016-10-09 15:45:42,193 - DEBUG - "GET /MOLT/MOD13Q1.005/ HTTP/1.1" 200 None
2016-10-09 15:45:43,342 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-10-09 15:45:43,828 - DEBUG - "GET /MOLT/MOD13Q1.005/2016.06.25 HTTP/1.1" 301 263
2016-10-09 15:45:44,865 - DEBUG - "GET /MOLT/MOD13Q1.005/2016.06.25/ HTTP/1.1" 200 None
2016-10-09 15:45:46,346 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-10-09 15:45:46,901 - DEBUG - "GET /MOLT/MOD13Q1.005/2016.06.25/MOD13Q1.A2016177.h30v08.005.2016200071432.hdf HTTP/1.1" 302 520
2016-10-09 15:45:46,917 - INFO - Starting new HTTPS connection (1): urs.earthdata.nasa.gov
2016-10-09 15:45:50,795 - DEBUG - "GET /oauth/authorize?scope=uid&app_type=401&client_id=ijpRZvb9qeKCK5ctsn75Tg&response_type=code&redirect_uri=http%3A%2F%2Fe4ftl01.cr.usgs.gov%2Foauth&state=aHR0cDovL2U0ZnRsMDEuY3IudXNncy5nb3YvTU9MVC9NT0QxM1ExLjAwNS8yMDE2LjA2LjI1L01PRDEzUTEuQTIwMTYxNzcuaDMwdjA4LjAwNS4yMDE2MjAwMDcxNDMyLmhkZg HTTP/1.1" 401 None
2016-10-09 15:45:50,795 - ERROR - Cannot download MOD13Q1.A2016177.h30v08.005.2016200071432.hdf. Retrying...
2016-10-09 15:45:50,813 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-10-09 15:45:51,316 - DEBUG - "GET /MOLT/MOD13Q1.005/2016.06.25/MOD13Q1.A2016177.h30v08.005.2016200071432.hdf HTTP/1.1" 302 520
2016-10-09 15:45:51,316 - INFO - Starting new HTTPS connection (1): urs.earthdata.nasa.gov
2016-10-09 15:45:53,450 - DEBUG - "GET /oauth/authorize?scope=uid&app_type=401&client_id=ijpRZvb9qeKCK5ctsn75Tg&response_type=code&redirect_uri=http%3A%2F%2Fe4ftl01.cr.usgs.gov%2Foauth&state=aHR0cDovL2U0ZnRsMDEuY3IudXNncy5nb3YvTU9MVC9NT0QxM1ExLjAwNS8yMDE2LjA2LjI1L01PRDEzUTEuQTIwMTYxNzcuaDMwdjA4LjAwNS4yMDE2MjAwMDcxNDMyLmhkZg HTTP/1.1" 401 None
2016-10-09 15:45:53,467 - ERROR - Cannot download MOD13Q1.A2016177.h30v08.005.2016200071432.hdf. Retrying...
2016-10-09 15:45:53,467 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov
2016-10-09 15:45:54,082 - DEBUG - "GET /MOLT/MOD13Q1.005/2016.06.25/MOD13Q1.A2016177.h30v08.005.2016200071432.hdf HTTP/1.1" 302 520
2016-10-09 15:45:54,082 - INFO - Starting new HTTPS connection (1): urs.earthdata.nasa.gov
2016-10-09 15:45:56,523 - DEBUG - "GET /oauth/authorize?scope=uid&app_type=401&client_id=ijpRZvb9qeKCK5ctsn75Tg&response_type=code&redirect_uri=http%3A%2F%2Fe4ftl01.cr.usgs.gov%2Foauth&state=aHR0cDovL2U0ZnRsMDEuY3IudXNncy5nb3YvTU9MVC9NT0QxM1ExLjAwNS8yMDE2LjA2LjI1L01PRDEzUTEuQTIwMTYxNzcuaDMwdjA4LjAwNS4yMDE2MjAwMDcxNDMyLmhkZg HTTP/1.1" 401 None
2016-10-09 15:45:56,539 - ERROR - Cannot download MOD13Q1.A2016177.h30v08.005.2016200071432.hdf. Retrying...
2016-10-09 15:45:

fix documentation in master

We should fix the following issues:

• fix sentence in index.rst about quality analysis
• add examples and description in modis_quality.rst
• add an example in examples/full_example.rst of modis_quality.py

OSError: [Errno 7] Argument list too long

Hi,

I developed a python script to generated one tif file, with only one band, for each band contained in one hdf file, so:

Code part (hosted in https://github.com/lapig-ufg/sitt/blob/master/src/tools/modis_acess.py):

for filename in os.listdir(self.targetPath):
  if filename.endswith(".hdf"):
    for spectralBase in self.__createSpectralBases():

        hdfFile = path.join(self.targetPath, filename)
        tifFile = path.join(self.targetPath, self.__createTifName(filename))

        modisParse = parsemodis.parseModis(hdfFile)
        confname = modisParse.confResample(spectral=spectralBase,
            output=tifFile)

        modisCover = convertmodis.convertModis(
            hdfname=hdfFile, confile=confname,
            mrtpath=self.mrtPath)

        modisCover.run()

Image Input:

• MOD13Q1.A2010209.h11v08.006.2015209150427.hdf

Image Output:

• MOD13Q1.A2010209.h11v08.006.2015209150427.250m_16_days_blue_reflectance.tif
• MOD13Q1.A2010209.h11v08.006.2015209150427.250m_16_days_composite_day_of_the_year.tif
• MOD13Q1.A2010209.h11v08.006.2015209150427.250m_16_days_EVI.tif
• MOD13Q1.A2010209.h11v08.006.2015209150427.250m_16_days_MIR_reflectance.tif
• MOD13Q1.A2010209.h11v08.006.2015209150427.250m_16_days_NDVI.tif
• MOD13Q1.A2010209.h11v08.006.2015209150427.250m_16_days_NIR_reflectance.tif
• MOD13Q1.A2010209.h11v08.006.2015209150427.250m_16_days_pixel_reliability.tif
• MOD13Q1.A2010209.h11v08.006.2015209150427.250m_16_days_red_reflectance.tif
• MOD13Q1.A2010209.h11v08.006.2015209150427.250m_16_days_relative_azimuth_angle.tif
• MOD13Q1.A2010209.h11v08.006.2015209150427.250m_16_days_sun_zenith_angle.tif
• MOD13Q1.A2010209.h11v08.006.2015209150427.250m_16_days_view_zenith_angle.tif
• MOD13Q1.A2010209.h11v08.006.2015209150427.250m_16_days_VI_Quality.tif

BUT, when I run this script thousands and thousand of times (the goal is execute this for entire time-series, 2000 - 2015), I got this error:

Error:

Traceback (most recent call last):
  File "/data/lapig/DEV/repositories/satellite-image-time-series/src/modules/_module.py", line 49, in run
    result = self.process(message)
  File "/data/lapig/DEV/repositories/satellite-image-time-series/src/modules/modis.py", line 39, in process
    tmpPath=tmpPath, pathMrt=pathMrt);
  File "/data/lapig/DEV/repositories/satellite-image-time-series/src/modules/modis.py", line 59, in __downloadAndReproject
    modisAcess.reproject()
  File "/data/lapig/DEV/repositories/satellite-image-time-series/src/tools/modis_acess.py", line 65, in reproject
    modisCover.run()
  File "/usr/local/lib/python2.7/dist-packages/pymodis/convertmodis.py", line 85, in run
    subprocess.call([execut,'-p',self.conf])
  File "/usr/lib/python2.7/subprocess.py", line 522, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
    raise child_exception
OSError: [Errno 7] Argument list too long

It's occurs because for each iteration the method checkMRTpath, class convertmodis.py, appends a repeated value in os.environ['PATH']:

def checkMRTpath(mrtpath):
    """Function to check if MRT path it correct
       :param str mrtpath: the path to MRT directory
       :return: The path to 'bin' and 'data' directory inside MRT path
    """
    if os.path.exists(mrtpath):
        if os.path.exists(os.path.join(mrtpath, 'bin')):
            mrtpathbin = os.path.join(mrtpath, 'bin')
            os.environ['PATH'] = "{path}:{data}".format(path=os.environ['PATH'],
                                                        data=os.path.join(mrtpath, 'data'))
        else:
            raise Exception('The path {path} does not exist'.format(path=os.path.join(mrtpath, 'bin')))
        if os.path.exists(os.path.join(mrtpath, 'data')):
            mrtpathdata = os.path.join(mrtpath, 'data')
            os.environ['MRTDATADIR'] = os.path.join(mrtpath, 'data')
        else:
            raise Exception('The path {path} does not exist'.format(path=os.path.join(mrtpath, 'data')))
    else:
        raise Exception('The path {name} does not exist'.format(name=mrtpath))
    return mrtpathbin, mrtpathdata

Comparison of empty date objects

I am getting the following error:

$ modis_download.py -t h19v04,h19v05 -p MOD09A1.005 -s MOLT -O MOD09A1.005
Traceback (most recent call last):
  File "/usr/local/bin/modis_download.py", line 125, in <module>
    main()
  File "/usr/local/bin/modis_download.py", line 119, in main
    modisOgg.downloadsAllDay(clean=options.empty, allDays=options.alldays)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 603, in downloadsAllDay
    days = self.getListDays()
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 334, in getListDays
    self._getToday()
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 326, in _getToday
    if self.today < self.enday:
TypeError: can't compare datetime.date to NoneType 

Problem with downmodis "Folder to store downloaded files does not exist or is not writeable"

Hello @lucadelu first of all congratulations on PyModis it is really useful. Now my issue, I'm triying to use downmodis inside my python script, it works but always says that the destination folder is not writable, but it is writable and if I use modis_download via cmd with the exact same parameters it works perfectly, wich could by the problem?

I'm working in Windows 7. The error refers to line 236 in downmodis.py

This are the lines.

---

    if os.access(destinationFolder, os.W_OK):
        self.writeFilePath = destinationFolder
    else:
        raise IOError("Folder to store downloaded files does not exist"
                      " or is not writeable")

---

If I execute in my python console "os.access(destinationFolder, os.W_OK)" it returns "true" so there should be no error.

Any thougths on what is happening?

Thanks.

modis_parse.py: currently broken

At time modis_parse.py does not parse anything:

[mundialis@fedora23 modis]$ modis_parse.py -q lst_terra.006/MOD11A1.A2012011.h18v04.006.2016103023737.hdf 
Traceback (most recent call last):
  File "/usr/bin/modis_parse.py", line 4, in <module>
    __import__('pkg_resources').run_script('pyModis==2.0.2', 'modis_parse.py')
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 735, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1659, in run_script
    exec(script_code, namespace, namespace)
  File "/usr/lib/python2.7/site-packages/pyModis-2.0.2-py2.7.egg/EGG-INFO/scripts/modis_parse.py", line 155, in <module>

  File "/usr/lib/python2.7/site-packages/pyModis-2.0.2-py2.7.egg/EGG-INFO/scripts/modis_parse.py", line 134, in main

KeyError: 'QAStats'


[mundialis@fedora23 modis]$ modis_parse.py -a lst_terra.006/MOD11A1.A2012011.h18v04.006.2016103023737.hdf 
Traceback (most recent call last):
  File "/usr/bin/modis_parse.py", line 4, in <module>
    __import__('pkg_resources').run_script('pyModis==2.0.2', 'modis_parse.py')
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 735, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1659, in run_script
    exec(script_code, namespace, namespace)
  File "/usr/lib/python2.7/site-packages/pyModis-2.0.2-py2.7.egg/EGG-INFO/scripts/modis_parse.py", line 155, in <module>

  File "/usr/lib/python2.7/site-packages/pyModis-2.0.2-py2.7.egg/EGG-INFO/scripts/modis_parse.py", line 134, in main

KeyError: 'QAStats'

typo: downloadsAllDayFTP in downmodis.py

Line 670 of downmodis.py should read self._downloadsAllDayFTP(days) instead of self._downloadAllDayFTP(days) to be consistent with definition on line 688.

modis_convert problem

Hi
I am using pyModis on a Windows7 x86 platform. As I am new to python programing, I have faced some problems and it would be highly appreciated if you can clear these queries for me:

1. I have installed the pyModis properly and it works with the example of full process scripts which is provided in pyModis webpage, except the last step (convert to GeoTIFF) which is crucial to me. When I run modis.convert.py with the required parameters I get immediately the error saying "pymodis_convert.py: error: You have to pass the name of HDF file."
   I have double checked parameters with regard to misspelling and everything seems to be according to the documentation! here is my command which leads to this error:
   modis_convert.py -s '( 1 1 1 1 1 1 1 1 1 1 1 1 )' -m F:\Software\MRT_download_Win -o F:\test_modis\finalfile.tif -g 250 F:\test_modis\outputfile.hdf
2. I need to download the data over a specific geographical region from this ftp address : ftp://ladsweb.nascom.nasa.gov/allData/6/MYD04_L2/
   would you please kindly provide me with a sample command to download the data from above mentioned address? for example what would be the command in order to download this specific data : ftp://ladsweb.nascom.nasa.gov/allData/6/MYD04_L2/2005/004/MYD04_L2.A2005004.0925.006.2014024032926.hdf ?

Any help is greatly appreciated.
Mohamad

python or R - automatically download data MODIS

Hello,

I'm trying to download modis bulk data automatically, but unfortunately the server USGS change ftp to http and so far I could not find any info to download the information to code R, because all files available for ftp, then I incline to use python but I pymodis presented problems because although the import GDAL install python not recognize me says no function is gdal. can you help me, I need to download modis bulk data with R or python but automatically.

Add login URL at http://www.pymodis.org/info.html#userpw-label

At http://www.pymodis.org/info.html#userpw-label it is written how to register but not where to login. Please add the phrase:

If you are an already registered user, login and enable the applications at: https://urs.earthdata.nasa.gov/home

Also a typo "ollowing" --> following

add products object

We should add a product object to store information like default download url, information about quality control, and for MRT configuration file.
We could use this new object inside the scripts to add default values and simplify the work of end-user

Catch exception if input file does not exist

At time the message is a bit "wild" for standard users :-)

modis_parse.py -q bla
Traceback (most recent call last):
  File "/usr/bin/modis_parse.py", line 4, in <module>
    __import__('pkg_resources').run_script('pyModis==2.0.2', 'modis_parse.py')
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 735, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1659, in run_script
    exec(script_code, namespace, namespace)
  File "/usr/lib/python2.7/site-packages/pyModis-2.0.2-py2.7.egg/EGG-INFO/scripts/modis_parse.py", line 155, in <module>

  File "/usr/lib/python2.7/site-packages/pyModis-2.0.2-py2.7.egg/EGG-INFO/scripts/modis_parse.py", line 110, in main

  File "build/bdist.linux-x86_64/egg/pymodis/parsemodis.py", line 65, in __init__
Exception: bla does not exist

Maybe easy to catch...

RPM packaging

The Error not recognised as a supported file format

I run the modis_download.py but the size of the .hdf file is 0 K all along;
Could you please tell me why this happens and how to deal with it? Thanks a lot^_^

2016-10-14 22:57:52,059 - ERROR - `H:\Download\MOD09GA005\MOD09GA.A2016285.h14v00.005.2016287060338.hdf' not recognised as a supported file format.

Unable to download when behind a corporate proxy

Hi,
Many thanks for this excellent tool. I am being successful in using the new version with my login credentials. However, it fails when I am using my work machine behind a proxy. How do I set the proxy for pyModis to succeed?

Thanks

downmodis log and list file empty

Hello,

I am having a hard time pulling data of the LP DAAC server with the downmodis command (run in Anaconda). I've also tried to run the modis_download.py module from my command prompt and iPython but continually receive syntax errors (despite, in several cases, copying the examples from the pymodis website, with a path relevant to my computer)

down = pymodis.downmodis.downModis(destinationFolder = 'C:\modis', password = 'password', user='username', url="http://e4ftl01.cr.usgs.gov", tiles = "h26v08", path = "MOLT", product = "MOD13Q1.005", today = "2014-01-01", enddate = "2014-02-02", jpg = False, debug = True, timeout = 30)

I've replaced my actual password/username with 'password' and 'username'. When I run this, the list and log files are created but they are empty. I've played with the dates, but this has not helped.

Please advise and thanks.

pymodis.downmodis issues python: nc4file.c:2696: NC4_open: Assertion `0' failed. Aborted

Hi,

I tried to download with pymodis but it issues the following messages and simply quits iPython !

In [5]: import pymodis

In [6]: download = pymodis.downmodis.downModis("project/OUT/", product="MOD13Q1.005", 
tiles="h11v04", today="2015-04-04", delta=31)                                                                                                   

In [7]: download.connect()

In [8]: download.downloadsAllDay()
python: nc4file.c:2696: NC4_open: Assertion `0' failed.
Aborted  

The same error issues when I try to use modis_dowload.py

Have you ever meet this problem ?

download progress bar in PyModis

Hi Luca,

I don't know if this is already part of the PyModis, but it would be nice if download progress bar can be incorporated so we can see how much time we need to finish the download.

Thanks,
-Leo

Add new -l (layers) flag to modis_parse.html + modis_multiparse.html to easily show data layers of HDF file

At time it is not possible to get the data layers of a MODIS HDF file with pyModis.

A workaround is using GDAL:

# show included data layers
gdalinfo lst_terra.005/MOD11A1.A2016002.h18v04.005.2016023015602.hdf | grep "_NAME" | cut -d':' -f5 | nl
     1    LST_Day_1km
     2    QC_Day
     3    Day_view_time
     4    Day_view_angl
     5    LST_Night_1km
     6    QC_Night
     7    Night_view_time
     8    Night_view_angl
     9    Emis_31
    10    Emis_32
    11    Clear_day_cov
    12    Clear_night_cov

Such a list would be very convenient to define --subset="..." .
(example: MODIS LST: --subset="1 1 0 0 1 1 0 0 0 0 0 0")

Select tiles using a bounding box

Could be useful to add the capabilities to select the tiles to download from a bounding box or a vector data.
This could be easily implemented using ogr library and a geojson file with the tiles in the right projection system and use a touch/overlay/contains function to select tiles.

Downloading MODIS HDF files is not working when GDAL is not installed

I have tried to use master to download some files on Ubuntu Xenial.
Installation through pip fails due to GDAL, I had to remove GDAL from setup.py
But then, I find that download function is not working because of this:
https://github.com/lucadelu/pyModis/blob/master/pymodis/__init__.py#L5-L6
I had to comment out those lines to get download working again.

change order and naming of firstday and endday

1. I would propose to start the order chronologically, where first day is before endday (default: today)
   It is just more intuitive (Ithink)
2. consistent naming for these two options
   either firstdate/firstday - lastdate/lastday
   or startdate/startday - enddate/endday

modis_convert.py specifiying the subset

Hello Luca,

The pymodis library is working great. Thank you for the awesome job 👍 . I just have some trouble with the modis_convert.py scripts. On the pymodis page it says that --

"Using VRT format it create dataset of only one later, so you are forced to use -s "( 1 )" ".

So I am trying to convert the .vrt files of the first two layers ie; -s '1 1 0 0'. I have two .vrt files after mosaicking the .hdf files now if I want to create the .tif files for the corresponding layers, how should I specify the subset here ? Is it goin to be like the same -s "(1 1)" or am I wrong ?

*Thanks *

warnings in WPS

Warnings in WPS should be avoided

HDF4 mac

Hi. This tool looks really useful, but I've never been able to use it because I always get an error saying GDAL has no support for HDF4. I'm sure I'm not the only person to encounter this. Is there a way to fix this? If so it would be really useful to have a documentation on how to fix it.

Thanks!
Mitch

wx dependency is missing

I compiled pymodis from source following these instructions: http://pymodis.fem-environment.eu/info.html#compile-from-source
I have python 2.7.7, numpy 1.8.1 and gdal 1.11.1 also installed. After I was finished I tried to import pymodis:

[markus.mueller@build-gpu-p pyModis]$ python
Python 2.7.7 (default, Jun 18 2014, 14:29:08)
[GCC Intel(R) C++ gcc 4.4 mode] on linux2
Type "help", "copyright", "credits" or "license" for more information.
import pymodis
Traceback (most recent call last):
File "", line 1, in
File "pymodis/init.py", line 1, in
import downmodis, parsemodis, convertmodis, optparse_required, optparse_gui, qualitymodis, convertmodis_gdal
File "pymodis/optparse_gui.py", line 14, in
import wx
ImportError: No module named wx

It seems that wxpython is an additional but not documented dependency. As additional information: I work on a linux cluster and am not interested in the GUI.

Any help appreciated

modis.mosaic multiple layers sequence

I am using modis.mosaic to mosaic a large batch of MOD13A2 tiles together. However it seems that when including multiple layers in the -s argument the order of layers is not retained in the mosaic output. For example using '-s "1 1 1 0 0 0 0 0 0 0 1 1" etc.' the mosaic output's first band is not NDVI (though NDVI is the first band in the MOD13A2 product). Is there some method to the mosaic madness? I couldn't find anything in the pyModis documentation...

Bad formatting of log string when using ftp connection leads to KeyError

[0]
in _connectFTP() method of downModis class:

if self.debug:
    logging.debug("Open connection {url}".format(name=self.url))

leads to KeyError: 'url'

I guess should be changed to

logging.debug("Close connection {url}".format(url=self.url))

[1]
in closeFTP() method of downModis class:

def closeFTP(self):
        """Close ftp connection and close the file list document"""
        self.ftp.quit()
        self.closeFilelist()
        if self.debug:
            logging.debug("Close connection {url}".format(self.url))

leads to KeyError: 'url'

I guess should be changed to
logging.debug("Close connection {}".format(self.url))
or
logging.debug("Close connection {url}".format(url=self.url))

Parameters file generate by pyModis is different than generate by ModisTool

Hi,

we I use the function pymodis.convertmodis.processModis the image output file has row/col number and pixel size different than output produced by ModisTool.jar. The problem is the parameters files:

pyModis parameter file:

INPUT_FILENAME = MOD13Q1.A2014001.h10v09.006.2015272060600.hdf
SPECTRAL_SUBSET = ( 1 0 0 0 0 0 0 0 0 0 0 0 )
SPATIAL_SUBSET_TYPE = INPUT_LAT_LONG
SPATIAL_SUBSET_UL_CORNER = ( 0.00723066070579 -81.2341289433 )
SPATIAL_SUBSET_LR_CORNER = ( -10.0062171535 -69.729801381 )
OUTPUT_FILENAME = /home/leandro/Tmp/MODIS/006/NDVI/MOD13Q1.A2014001.h10v09.006.2015272060600.250m_16_days_NDVI.tif
RESAMPLING_TYPE = NEAREST_NEIGHBOR
OUTPUT_PROJECTION_TYPE = GEO
OUTPUT_PROJECTION_PARAMETERS = ( 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 )
DATUM = WGS84

ModisTool parameter file:

INPUT_FILENAME = MOD13Q1.A2014001.h10v09.006.2015272060600.hdf
SPECTRAL_SUBSET = ( 1 0 0 0 0 0 0 0 0 0 0 0 )
SPATIAL_SUBSET_TYPE = INPUT_LAT_LONG
SPATIAL_SUBSET_UL_CORNER = ( 0.0 -79.999999993 )
SPATIAL_SUBSET_LR_CORNER = ( -9.999999999 -71.079862825 )
OUTPUT_FILENAME = /home/leandro/Tmp/MODIS/006/NDVI/MOD13Q1.A2014001.h10v09.006.2015272060600.250m_16_days_NDVI.tif
RESAMPLING_TYPE = NEAREST_NEIGHBOR
OUTPUT_PROJECTION_TYPE = GEO
OUTPUT_PROJECTION_PARAMETERS = (  0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 )
 DATUM = WGS84

The properties SPATIAL_SUBSET_UL_CORNER and SPATIAL_SUBSET_LR_CORNER are different . This change the row/col number and pixel size of output image.

I can't solve it, so I will try use MRT directly.

Connection problem with modis_download.py

Hello!

I am attempting to download data with the modis_download.py script but whatever I try, it returns the following phrase (with that exact spelling):
"modis_download.py: error: Some problem with connection occur"
My operating system is Ubuntu 12.10.

This is the command I used:
modis_download.py -t h09v07,h09v08,h09v09,h09v10,h09v11,h10v07,h10v08,h10v09,h10v10,h10v11,h11v07,h11v08,h11v09,h11v10,h11v11,h12v07,h12v08,h12v09,h12v10,h12v11,h13v07,h13v08,h13v09,h13v10,h13v11,h14v07,h14v08,h14v09,h14v10,h14v11 -p MCD45A1 -f 2000-01-01 -e 2010-12-31 SouthAmerica/

Adding "-s MODIS_Composites/MOTA" and/or "-u http://e4ftl01.cr.usgs.gov" does not help. I also tried the examples on the documentation page and they return the same error.

According to he download_modis.py script, the error message is returned after 20 failed connection attempts to the server.

When I make a list of the files to download with reverb (MODIS web page) and try to download with wget, the following error is raised:

"Connecting to e4ftl01.cr.usgs.gov (e4ftl01.cr.usgs.gov)|152.61.133.5|:80... failed: Connection refused."

What could be the solution to this problem?

Thanks!

UPDATE: The issue was caused by the data server being down due to maintenance.

Debian packaging

Authentication problem with default server

The default http server, 'http://e4ftl01.cr.usgs.gov', has been updated to require a Nasa earthdata account when downloading files. I do not believe there are any methods in the default package for http authentication.

GDAL add compression

Add compression for TIFF file in mosaic and conversion