nrel / panel-segmentation

This open-source package provides a framework for automatically detecting and extracting metadata from solar array installations in satellite images.

Panel Segmentation

This repo contains the scripts for automated metadata extraction of solar PV installations, using satellite imagery coupled with computer vision techniques. In this package, the user can perform the following actions:

• Automatically generate a satellite image using a set of lat-long coordinates, and a Google Maps API key. To get a Google Maps API key, go to the following site and set up an account: https://developers.google.com/maps/documentation/javascript/get-api-key?utm_source=google&utm_medium=cpc&utm_campaign=FY20-Q3-global-demandgen-displayonnetworkhouseads-cs-GMP_maps_contactsal_saf_v2&utm_content=text-ad-none-none-DEV_c-CRE_460848633529-ADGP_Hybrid%20%7C%20AW%20SEM%20%7C%20BKWS%20~%20Google%20Maps%20API%20Key-KWID_43700035216023629-kwd-298247230705-userloc_1014524&utm_term=KW_google%20maps%20api%20key-ST_google%20maps%20api%20key&gclid=Cj0KCQjwit_8BRCoARIsAIx3Rj7XZb01kt1iLH3zzxGODvmM62g0K4ujEMpla5pL1p057tQXmp6MXpsaAscrEALw_wcB
• Determine the presence of a solar array in the satellite image (boolean True/False), using a classification model (VGG16_classification_model.h5).
• Perform image segmentation on the satellite image, to locate the solar array(s) in the image on a pixel-by-pixel basis, using an image segmentation model (VGG16Net_ConvTranpose_complete.h5).
• Using connected components clustering, isolate individual solar arrays in the masked image.
• Perform azimuth estimation on each solar array cluster in the masked image.
• Using an object detection model (Faster R-CNN Resnet 50 trained via transfer learning), detect and classify mounting type and configuration of solar installations in satellite imagery. This includes classification of fixed tilt and single-axis trackers, as well as the rooftop, ground, and carport mounting configurations.

To install Panel-Segmentation, perform the following steps:

1. You must have Git large file storage (lfs) on your computer in order to download the deep learning models in this package. Go to the following site to download Git lfs:

https://git-lfs.github.com/

2. Once git lfs is installed, you can now install Panel-Segmentation on your computer. We are still working on making panel-segmentation availble via PyPi, so entering the following in the command line will install the package locally on your computer:

pip install git+https://github.com/NREL/Panel-Segmentation.git@master#egg=panel-segmentation

3. When initiating the PanelDetection() class, be sure to point your file paths to the model paths in your local Panel-Segmentation folder!

panel-segmentation's Issues

Try the rectangle method.

Try out the cv2.minAreaRect() rotated rectangle function for azimuth estimation, per Tim Silverman's recommendation.

Incorporate LIDAR for tilt estimates.

Investigate whether LIDAR can be used for tilt estimation in the package (need to determine if LIDAR data is publicly available). Per Joseph Karas's recommendation.

Build totally unsupervised clustering algorithm.

Make ideal number of clusters selection automated. Sample other clustering algorithms.

Additional metadata features

Work with Stephanie Trusty to add in following features:

-Array size estimates
-Ground coverage ratio
-Terrain classification (??)

Erroring when installing lfs files via git clone.

Right now we don't have a good way to include the models, as they don't download with a pip install. Erroring when doing a git clone or pip install:

batch response: Post "https://git-lfs.lsst.codes/objects/batch": x509: certificate signed by unknown authority
error: failed to fetch some objects from 'https://git-lfs.lsst.codes'

Update model paths so they're build directly into the routine.

Deploy the panel-segmentation package on PyPi.

Make pip installable via PyPi, where you can download via command line:
pip install panel_segmentation

Deprecate connected components clustering.

Replace with object detection boxes as a form of clustering.

Can't run Jupiter example

I keep getting this error when I run

#CREATE AN INSTANCE OF THE PANELDETECTION CLASS TO RUN THE ANALYSIS
panelseg = pseg.PanelDetection(model_file_path ='./panel_segmentation/VGG16Net_ConvTranpose_complete.h5', 
                               classifier_file_path ='./panel_segmentation/VGG16_classification_model.h5',
                              mounting_classifier_file_path='./panel_segmentation/object_detection_model.pth')

#GENERATE A SATELLITE IMAGE USING THE ASSOCIATED LAT-LONG COORDS AND THE GOOGLE
#MAPS API KEY
img = panelseg.generateSatelliteImage(latitude, longitude,
                                      file_name_save,
                                      google_maps_api_key)
#Show the generated satellite image
plt.imshow(img)

SavedModel file does not exist at: ./panel_segmentation/VGG16_classification_model.h5/{saved_model.pbtxt|saved_model.pb}

Branch for mounting configuration detection.

Branch for mounting type detection.

Detect type of installation--rooftop, ground mount, carport, etc.

Make the directions for getting API key clearer.

Include this in the documentation!!! Tip courtesy of @ChristopherCampos