GithubHelp home page GithubHelp logo

jreiche / bayts Goto Github PK

View Code? Open in Web Editor NEW
26.0 5.0 18.0 4.96 MB

Set of tools to apply the probabilistic approach of Reiche et al. (2015, 2018) to combine multiple optical and/or Radar satellite time series and to detect deforestation/forest cover loss in near real-time. The package includes functions to apply the approach to both, single pixel time series and raster time series. Examples and test data are provided below.

License: Other

R 100.00%

bayts's Introduction

bayts

Set of tools to apply the probabilistic approach of Reiche et al. (2015, 2018) to combine multiple optical and/or Radar satellite time series and to detect deforestation/forest cover loss in near real-time. The package includes functions to apply the approach to both, single pixel time series and raster time series. Examples and test data are provided below.

Research version

The package includes the research version of the tools, which have a limited perforamce when applied to raster time series. The reserach version allows (i) to visualise and analyse the entire time series history and (ii) it stepwise applies the probablistic approach consecutively on each observation in the time series to emulate a near real-time scenario. It includes methods used in Reiche et al., 2015 (RSE), Reiche et al., 2018 (RSE) and Reiche et al., 2018 (Remote Sensing)

Probablistic approach

The basic version of the probabilistic approach has been published in Reiche et al., (2015). An improved version was published in Reiche et al. (2018, RSE) and was used in Reiche et al., 2018 (Remote Sensing).

Figure 1 gives an schematic overview the probabilistic approach. We considered a near real-time scenario with past (t-1), current (t) and future observations (t+1), with multiple observations possible at the same observation date. First, once a new observation of either of the input time series was available (t = current) it was converted to the conditional NF probability (sNF) using the sensor specific forest (F) and non-forest (NF) probability density functions (pdf) (The sensor specific F and NF pdfs were derived using training data). The derived conditional NF probability was added to the combined time series of conditional NF probabilities derived from the previous LNDVIn, S1VVn and P2HVn time series observations (t–i). Second, we flagged a potential deforestation event in the case that the conditional NF probability was larger than 0.5. We calculated the probability of deforestation using iterative Bayesian updating. Future observation (t+i) were used to update the probability of deforestation in order to confirm or reject the flagged deforestation event.

fig Figure 1. Probabilistic approach used to combine time series of Landsat NDVI (LNDVIn), Sentinel-1 VV (S1VVn) and ALOS-2 PALSAR-2 HV (P2HVn) observations and to detect deforestation in near real-time. (Reiche et al., under review)

Core functions

bayts - applies probabalistic approach to single pixel time series (method presented in Reiche et al., 2018, RSE)

baytsSpatial - applies bayts to raster time series

baytsDD - bayts with a priori (i) seasonal-trend model fitting to remove forest seasonality and (ii) data-driven way to derive forest and non-forest distributions (method presented in Reiche et al., 2018, Remote Sensing)

baytsDDSpatial - applies baytsDD to raster time series

Install

The package can be installed directly from github using devtools

library(devtools)
install_github('jreiche/bayts')

References

Reiche, J., Verhoeven, R.; Verbesselt, J.; Hamunyela, E.; Wielaard, N. & Herold, M. (2018) Characterizing Tropical Forest Cover Loss Using Dense Sentinel-1 Data and Active Fire Alerts. Remote Sensing, 10, 5, 777, doi:10.3390/RS10050777. (http://www.mdpi.com/2072-4292/10/5/777)

Reiche, J., Hamunyela, E., Verbesselt, J., Hoekman, D. & Herold, M. (2018): Improving near-real time deforestation monitoring in tropical dry forests by combining dense Sentinel-1 time series with Landsat and ALOS-2 PALSAR-2. Remote Sensing of Environment. https://doi.org/10.1016/j.rse.2017.10.034. (https://www.sciencedirect.com/science/article/pii/S0034425717304959)

Reiche, J., de Bruin, S., Hoekman, D. H., Verbesselt, J. & Herold, M. (2015): A Bayesian Approach to Combine Landsat and ALOS PALSAR Time Series for Near Real-Time Deforestation Detection. Remote Sensing, 7, 4973-4996. DOI:10.3390/rs70504973. (http://www.mdpi.com/2072-4292/7/5/4973)

Hamunyela, E., Verbesselt, J., & Herold, M. (2016). Using spatial context to improve early detection of deforestation from Landsat time series. Remote Sensing of Environment, 172, 126–138. http://doi.org/10.1016/j.rse.2015.11.006

Citation

@software{bayts,
  author = {Reiche, Johannes},
  title = {{bayts}},
  url = {https://github.com/jreiche/bayts},
  version = {1.1},
  date = {2019-01-12},
  doi = {10.5281/zenodo.545792}
}

For external contributors

External contributions are welcome. If you would like to contribute additional features and improvements to the package; fork the repository on gitHub, commit your changes and make a pull request. Always use the develop branch as a starting point for your work. Your contribution will be reviewed for quality, relevance and consistency with the rest of the package before being merged.

Examples

Two examples are provided. Example 1 shows how to apply the functions on singel-pixel time series (Sentinel-1 VV and Landsat NDVI). Example 2 shows how to apply the functions to raster time series.

Example 1: Single-pixel example of the core bayts function

Single-pixel example using a Sentinel-1 VV and Landsat NDVI time series, covering a deforestation event in 2016. (Source code: examples/bayts_pixel_example_v01.R)

require(bayts)

##############################################
######### load data, create & plot time series

# load example data
# single pixel Sentinel-1 VV and Landsat NDVI time series (09/2014 - 05/2016); deforestation event in early 2016
data(s1vv_lndvi_pixel)

# create time series using bfastts (bfast package)
ts1vv <- bfastts(s1vv_obs,as.Date(s1vv_date),type=c("irregular"))
tlndvi <- bfastts(lndvi_obs,as.Date(lndvi_date),type=c("irregular"))

# plot time series
plotts(tsL=list(tlndvi,ts1vv),labL=list("Sentinel-1 VV [dB]","Landsat NDVI"))
plotts(tsL=list(tlndvi,ts1vv),labL=list("Sentinel-1 VV [dB]","Landsat NDVI"),ylimL=list(c(0,1),c(-13,-6)))

fig
Figure 2. Landsat NDVI and Sentinel-1 VV time series covering a deforestation event in early 2016.

######################################
######### apply bayts and plot results

# (1) Define parameters 
# (1a) Sensor specific pdfs of forest (F) and non-foerst (NF). Used to calculate the conditional NF probability of each observation. Gaussian distribution of F and NF distribution. Distributions are described using mean and sd.
s1vv_pdf <- c(c("gaussian","gaussian"),c(-7,0.75),c(-11.5,1))    
lndvi_pdf <- c(c("gaussian","gaussian"),c(0.85,0.075),c(0.4,0.125))

# (1b) Theshold of deforestation probability at which flagged change is confirmed (chi)
chi = 0.9
# (1c) Start date of monitoring
start = 2015

# (2) apply bayts (combine original time series into a time series of NF probabilities and detect deforestation)
# (2a) apply bayts
bts <- bayts(tsL=list(tlndvi,ts1vv),pdfL=list(lndvi_pdf,s1vv_pdf),chi=chi,start=start)

# (2b) plot original time series; including flagged and detected changes
plotBayts(bts$bayts,labL=list("Landsat NDVI","Sentinel-1 VV [dB]"),ylimL=list(c(0,1),c(-13,-6)),start=start)

fig
Figure 3. Landsat NDVI and Sentinel-1 VV time series and detected deforestation events. black line = start of monitoring; dotted black line = flagged deforestation event that was not confirmed; red dotted line = flagged deforestation event; red line = confirmed deforestation event.

# (2c) plot time series of NF probabilities, including flagged and detected changes
plotBaytsPNF(bts$bayts,start=start)

fig
Figure 4. Time series of NF probabilities derived from the two original time series. black line = start of monitoring; dotted black line = flagged deforestation event that was not confirmed; red dotted line = flagged deforestation event; red line = confirmed deforestation event.

# (2d) get time of change
bts$change.flagged    # time at which change is flagged
bts$change.confirmed  # time at which change is confirmed

Example 2: Applying deseasonalizeRaster and baytsSpatial function

Example applying method presented in Reiche et al., 2018 (RSE)

SOURCE CODE: examples/bayts_raster_example_v01.R

DATA: Landsat NDVI and Sentinel-1 raster time series data from a dry forest area in Bolivia

STEP 1: Spatial normalisation (deseasonalizeRaster) to remove dry forest seasonality in Landsat NDVI and Sentinel-1 time series observations

STEP 2: Probablistic approach (bayts, baytsSpatial) used to combine Landsat and Sentinel-1 time series and to detect forest cover loss

Example 3: Applying baytsDD and baytsSpatialDD function

Example applying method presented in Reiche et al., 2018 (Remote Sensing)

SOURCE CODE: examples/baytsDD_raster_example_v01.R

DATA: Sentinel-1 raster time series data from Riau, Sumatra

Step 1: Use season-trend fitting to model and remove forest seasonality

Step 2: Data-driven way to derive F and NF distribution (pdfs) to paramterise bayts

Step 3: Probablistic approach (bayts, baytsSpatial) used to combine optical and SAR time series and to detect change in near real-time

bayts's People

Contributors

jreiche avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

bayts's Issues

Errors when trying to recreate tutorial plot (with new data)

I have a dataset similar to the tutorial and I am trying to recreate the plot as the tutorial shows, but I am unable to do so for one input series (as opposed to having 2 data streams for S1 and L8 in the tutorial). It seems this is occurring for 2 main reasons:

  1. There are numerous flags that are not captured by the algorithm
  2. There is an error when it tries to plot, as if it's trying to merge two time series

I've tried going through the source code but I have been unable to understand why I'm unable to recreate the tutorial. Any advice you can give would be appreciated.

Data

The data I'm using is attached here: testBaytsData.txt

I am using R 3.6.3

Code with associated output

Data set-up

library(data.table)
dt <- fread("../testBaytsData.txt")
pre <- dt[date <= as.Date("2018-06-30"), ]
post <- dt[date > as.Date("2018-06-30"), ]

preMean <- mean(pre[,DI])
preSD <- sd(pre[,DI])
postMean <- mean(post[,DI])
postSD <- sd(post[,DI])

chi <- 0.9
PNFmin <- 0.75
start <- 2018.5

Create the timeseries

library(bayts)
pt1ts <- bfastts(data=dt$DI, dates=dt$date, type=c("irregular"))
plotts(tsL=list(pt1ts),labL=list("DI"))

Warning message:
In plot.xy(xy.coords(x, y), type = type, ...) :
  "axes" is not a graphical parameter

Note that I receive this warning message multiple times regardless if I'm plotting or not. For example, you can see below.

pt_pdf <- c(c("gaussian","gaussian"),c(preMean,preSD),c(postMean,postSD))    

Warning messages:
1: In doTryCatch(return(expr), name, parentenv, handler) :
  "axes" is not a graphical parameter
2: In doTryCatch(return(expr), name, parentenv, handler) :
  "axes" is not a graphical parameter

Run the bayts function

  • It is notable here that the actual disturbances that occur are not flagged by the algorithm. In this case, I know the date of disturbance as 2019-01-20, and we can see this in the PNF below. However, there is no record of change in PChange or Flag.
bts <- bayts(tsL=list(pt1ts),pdfL=list(pt_pdf), chi=chi, PNFmin=PNFmin, start=start)

Warning messages:
1: In min(which(bayts$Flag == "Flag")) :
  no non-missing arguments to min; returning Inf
2: In `[.zoo`(bayts, min(which(bayts$Flag == "Flag"))) :
  NAs introduced by coercion to integer range

bts$bayts[index(bts$bayts)>2018.5] # I've truncated the results a bit for brevity
          ts1                   PNF               PChange           Flag  
2018(295) -0.0103698274580355   0.146422182935227 <NA>              0     
2018(296) -0.0109193449193541   0.146697055524646 <NA>              0     
2018(297) -0.0113541214173635   0.146921833344499 <NA>              0     
2018(298) -0.0116772473208953   0.147093080750404 <NA>              0     
2018(299) -0.217396186322773    0.9               0.608173186131117 Change
2018(300) -0.210618789323509    0.9               0.933196749881069 Change
2018(301) -0.20351408921374     0.9               <NA>              <NA>  
2018(302) -0.19614418378033     0.9               <NA>              <NA>  
2018(303) -0.188548135003774    0.9               <NA>              <NA>  
2018(304) -0.180768253936085    0.9               <NA>              <NA>  
2018(305) -0.172832891918802    0.9               <NA>              <NA>  
2018(306) -0.154140020871061    0.817288220874064 <NA>              <NA>  
2018(307) -0.146443843702741    0.766186565563637 <NA>              <NA>  
2018(308) -0.138759555931493    0.709224537071085 <NA>              <NA>  
2018(309) -0.11070331698536     0.486083974497615 <NA>              0     
2018(310) -0.103861086197172    0.436457341611282 <NA>              0     
2018(311) -0.0971769215157538   0.391975796009425 <NA>              0     
2018(312) -0.0906775752067858   0.352920450624893 <NA>              0     
2018(313) -0.084404840612845    0.319263771132842 <NA>              0     
2018(314) -0.0543654049076308   0.207896178415362 <NA>              0     
2018(315) -0.0494076443785434   0.196103846211521 <NA>              0     
2018(316) -0.0447516673303344   0.186380172564815 <NA>              0     
2018(317) -0.040412965286588    0.17840255657346  <NA>              0     
2018(318) -0.0533460185563043   0.205344014641303 <NA>              0     
2018(319) -0.0490462215152801   0.195303686360202 <NA>              0     
2018(320) -0.0450323056907596   0.186931237881271 <NA>              0     
2018(321) -0.0412974297042931   0.179948027867292 <NA>              0     
2018(322) -0.0378323412253186   0.17412039985751  <NA>              0     
2018(323) -0.011107730268842    0.146793656697783 <NA>              0     
2018(324) -0.00893639888036777  0.145753340619918 <NA>              0     
2018(325) -0.00702393039844238  0.144968361845371 <NA>              0     
2018(326) -0.00535043126629303  0.144381033717584 <NA>              0     
2018(327) -0.00389818479056342  0.143945913735848 <NA>              0     
2018(328) -0.00264926090844979  0.143626706341285 <NA>              0     
2018(329) 0.0281074515148046    0.151751670081593 <NA>              0     
2018(330) 0.0280045824120031    0.151670942057999 <NA>              0     
2018(331) 0.0277209637361161    0.151450388469223 <NA>              0     
2018(332) 0.0272769307546597    0.151111025910394 <NA>              0     
2018(333) 0.0266924577887062    0.150675317006953 <NA>              0     
2018(334) 0.0555142561135827    0.18916389178469  <NA>              0     
2018(335) 0.0985780819752415    0.344696468173231 <NA>              0     
2018(336) 0.0951602331312242    0.326372572235416 <NA>              0     
2018(337) 0.0915969816182215    0.308510307791028 <NA>              0     
2018(338) 0.0879226664279991    0.291388624170535 <NA>              0     
2018(339) 0.115503976244305     0.452536318206864 <NA>              0     
2018(340) 0.11063174244274      0.418678107807219 <NA>              0     
2018(341) 0.105690457374186     0.386605546439634 <NA>              0     
2018(342) 0.102818356792821     0.369068583240284 <NA>              0     
2018(343) 0.0977686923405921    0.340250861970259 <NA>              0     
2018(344) 0.0927368306516509    0.314087886854492 <NA>              0     
2018(345) 0.0877489812322722    0.290611216034045 <NA>              0     
2018(346) 0.0828297760317749    0.269754526640656 <NA>              0     
2018(347) 0.125774602450901     0.530035543363182 <NA>              <NA>  
2018(348) 0.119474552510288     0.481615724114275 <NA>              0     
2018(349) 0.113231809374945     0.436482584247636 <NA>              0     
2018(350) 0.107094574469821     0.395477760575791 <NA>              0     
2018(351) 0.101083572329726     0.358877755376623 <NA>              0     
2018(352) 0.0952176660770799    0.326670797475777 <NA>              0     
2018(353) 0.0894854954627703    0.298512608459057 <NA>              0     
2018(354) 0.112721613650906     0.432940325092588 <NA>              0     
2018(355) 0.106418786267728     0.391183368320544 <NA>              0     
2018(356) 0.10028734921144      0.354302236732953 <NA>              0     
2018(357) 0.0943398272266948    0.322148474858418 <NA>              0     
2018(358) 0.0885871446778369    0.29438916190609  <NA>              0     
2018(359) 0.116444942971422     0.459313551039741 <NA>              0     
2018(360) 0.109984076992457     0.414340567334918 <NA>              0     
2018(361) 0.103707841570158     0.374411415916444 <NA>              0     
2018(362) 0.0976253126928801    0.339470236415147 <NA>              0     
2018(363) 0.091744553739018     0.309225242771776 <NA>              0     
2018(364) 0.086072585346437     0.283253254318169 <NA>              0     
2018(365) 0.0806114277832553    0.261062254417706 <NA>              0     
2019(1)   0.059169109652869     0.196985914599432 <NA>              0     
2019(2)   0.0547003807389136    0.187527110668438 <NA>              0     
2019(3)   0.050473585141166     0.179610954002321 <NA>              0     
2019(4)   0.0464878746141911    0.172997740752971 <NA>              0     
2019(5)   0.04274096557831      0.167484089375414 <NA>              0     
2019(6)   0.0392291916642345    0.16289766299394  <NA>              0     
2019(7)   0.0359371306326694    0.159081039905596 <NA>              0     
2019(8)   0.0328805064069418    0.155935481319106 <NA>              0     
2019(9)   0.0300405284126067    0.153341890730969 <NA>              0     
2019(10)  0.027409020236721     0.151211223134586 <NA>              0     
2019(11)  0.0249770250841742    0.149467890850071 <NA>              0     
2019(12)  0.0227408130221291    0.148051235369864 <NA>              0     
2019(13)  0.0206781138058781    0.146899258497611 <NA>              0     
2019(14)  0.0187850639485467    0.145970303107675 <NA>              0     
2019(15)  0.0170355678793249    0.145219287869532 <NA>              0     
2019(16)  0.0154519337952479    0.144627404270771 <NA>              0     
2019(17)  0.0140065719320316    0.144159398433461 <NA>              0     
2019(18)  -0.0179575895489548   0.151147179368772 <NA>              0     
2019(19)  -0.0181030882140769   0.151257838372718 <NA>              0     
2019(20)  -0.018132785447423    0.151280519924538 <NA>              0     
2019(21)  0.010949032132212     0.14339398791346  <NA>              0     
2019(22)  0.0101846571102187    0.143249932148884 <NA>              0     
2019(23)  0.0094922407690351    0.143135679844599 <NA>              0     
2019(24)  -0.0397983170134051   0.177352217893418 <NA>              0     
2019(25)  -0.0387396181208487   0.175587900088209 <NA>              0     
2019(26)  -0.0375837253829888   0.173725343765663 <NA>              0     
2019(27)  -0.0363437672108415   0.171799845363409 <NA>              0     
2019(28)  -0.0350328273522696   0.169844055994227 <NA>              0     
2019(29)  -0.203641222732472    0.9               <NA>              <NA>  
2019(30)  -0.196506707678658    0.9               <NA>              <NA>  
2019(31)  -0.18917341734071     0.9               <NA>              <NA>  
2019(32)  -0.181667907169265    0.9               <NA>              <NA>  
2019(33)  -0.174031556013066    0.9               <NA>              <NA>  
2019(34)  -0.375599166806213    0.9               <NA>              <NA>  
2019(35)  -0.360774013153237    0.9               <NA>              <NA>  
2019(36)  -0.345727197763821    0.9               <NA>              <NA>  
2019(37)  -0.330526746612581    0.9               <NA>              <NA>  
2019(38)  -0.315243014553276    0.9               <NA>              <NA>  
2019(39)  -0.533950546121899    0.9               <NA>              <NA>  
2019(40)  -0.510839511005354    0.9               <NA>              <NA>  
2019(41)  -0.487625001844107    0.9               <NA>              <NA>  
2019(42)  -0.508716529869376    0.9               <NA>              <NA>  
2019(43)  -0.484096560068751    0.9               <NA>              <NA>  
2019(44)  -0.702635241068076    0.9               <NA>              <NA>  
2019(45)  -0.670245887585986    0.9               <NA>              <NA>  
2019(46)  -0.637962987860921    0.9               <NA>              <NA>  
2019(47)  -0.605912118126666    0.9               <NA>              <NA>  

Plot the function

plotBayts(bts$bayts)

Error in merge.zoo(zoo(coredata(x), tt), zoo(, tt2)) : 
  series cannot be merged with non-unique index entries in a series
In addition: Warning messages:
1: In min(which(bayts$Flag == "Flag")) :
  no non-missing arguments to min; returning Inf
2: In `[.zoo`(bayts, min(which(bayts$Flag == "Flag"))) :
  NAs introduced by coercion to integer range
3: In zoo(data, 1900 + as.POSIXlt(dates)$year + (yday365(dates) - 1)/365,  :
  some methods for “zoo” objects do not work if the index entries in ‘order.by’ are not unique
4: In zoo(coredata(x), tt) :
  some methods for “zoo” objects do not work if the index entries in ‘order.by’ are not unique

Plotting here does not work. The error message appears to suggest that there is a timeseries that is being merged with the one I supplied, but there are duplicates even though I've already removed the Leap Day (29 Feb 2016). This also is confusing for Error messages 3 and 4.

Question about detectBayts order

This is the last argument in the function detectBayts. From the help file, I'm understanding chi to be "Threshold of Pchange at which the change is confirmed", and PNFmin as "threshold of pNF above which the first observation is flagged." Why is the conditional for chi here taken as the first step to be labeled as change? If chi is used only for when the change is confirmed, shouldn't the first conditional be when the first observation is flagged?

On that note, is there a reason 0.5 is used here instead of PNFmin?

if ((as.double(bayts$PChange[t])) >= chi) {
              if ((as.double(bayts$PNF[t])) >= 0.5) {
                bayts$Flag[min(which(bayts$Flag == "Flag")):t] <- "Change"
                return(bayts)
    }
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.