jguetschow / unfccc_non-annexi_data Goto Github PK

Eritrea's first BUR. Some detail for 2018, almost no data for other years. Still read as recent data included.

The reading function of the South Korean 2022 inventory leaves some categories untranslated and unmapped.

3. 습지에서 배수로 인한 Non-CH4 배출
4. 초지에서 농업용 석회시용으로 인한 CH4 배출
5. 산림지에서 배수로 인한 Non-CH4 배출
6. 농경지에서 농업용 석회시용으로 인한 CH4 배출

find mapping for them or ignore them explicitly. All of these categories contain zero only time-series.

Gabon's first BUR. The NIR contains detailed inventories for 1994, 2000, 2005, 2010-2017. The NIR tables use the standard NIR format and should be easy to read.

read_new_unfccc_crf_for_year does not commit it's results because of the error

OSError: [Errno 7] Argument list too long: 'git'
This is probably from the very long list of files that are affected by the commit and passed to datalad. To get rid of this message we could commit country by country or dig deeper and find another solution. As manually committing after the error in the script work fine it's not urgent to solve this issue.

Data from Ethiopia's second national communication is available through the UNFCCC DI interface.

The data shows a spike in 1.A.4 CH4 and N2O emissions from 1997 to 2004 which is not consistent with the CO2 emissions from the sector.
The data in the DI interface is consistent with the NC2 report tables, but the time series look like there is an offset and thus the reason for this should be investigated.

Indonesia's third BUR. Detailed inventory for 2019, almost no other data. Inventory is in standard NIR table format.

The fourth National Communication contains data for 2004 - 2018 in two year intervals. However, only some sectors are present and not all are gas resolved. The data could still be a good addition to the data from BUR2

Data in BUR3/NIR for Montenegro has several issues in the reporting tables. Double check if they are all sorted out in the data read in. F-gases in Sector 2.E have the be checked first.

The energy CO2 time-series from BUR3 show peaks for certain years. As we currently only have aggregate energy CO2 time-series available for use in PRIMAP-hist which are downscaled to the necessary categorical level the peaks appear in all energy CO2 time-series. However, they actually come from flaring. Flaring data are available in the NIR to BUR3. Read the flaring time-series to make the energy CO2 data more accurate.

Uruguay's fourth BUR contains detailed data for 2019 and main sector and gas timeseries for 1990-2019 where some years have no data. More data might exist for individual sectors. Try to obtain the full inventory from a different source before reading the data.

Add data fro the submission of the Nationally Determined Contributions (NDCs) to the dataset. For quantification of the emissions implied by the NDCs as submitted by countries the emissions data submitted with the NDCs are the first priority data source.
All NDC submission are available from the NDC registry.

The format is different from the BUR and NC submissions thus new code is needed for the downloader.

TODOs are:

• Write downloader that uses two steps like the BUR and NC downloaders.
  • get list of submissions including their URLs
  • download the data and store it in the folder structure for UNFCCC data
• Add information to README
• Retrieve all submissions and add to repository

Integrate data already read and adapt code

F-gases from BUR1/NIR1 for Kyrgyzstan are currently not available. Read BUR1/NIR1

For some countries duplicate download folders exist due to different spellings of the country name in the submission database and / or errors in the treatment of special characters.

Affected countries are at least:

• Venezuela (typo)
• Micronesia, Federated State(s) of (once without "s")
• Côte_d'Ivoire (also exists with "’" instead of "'"

Lebanon's third BUR. Detailed inventory for 2015, not much data for other years. Check if detailed inventory is available somewhere before reading this BUR

Trinidad and Tobago's first BUR. Contains detailed data for 2018 and individual GHG timeseries for 2006-2018 and the main sectors. More data is possibly available for some subsectors.

Integrate data from Oman's first BUR as legacy data.

The CRF reader currently reads no table that contains the "Other" sector. This affects Switzerland and New Zealand

The 2021 Inventory report is available online, but not yet submitted to the UNFCCC. It contains data for 1990 - 2021 with good sectoral and gas resolution.
www.gir.go.kr

Integrate BUR4 data already read and adapt code for data reading.

Rwanda's first BUR contains a detailed inventory for 2018 and aggregate KyotoGHG timeseries for 2006 - 2018. Mayeb more data for specific sectors. Some tables seem to have problem with data showing up under the header for the wrong year span (e.g. Page 41)

Integrate data already read from NIR xlsx file (non-UNFCCC folder) and adapt code to publish it here.

Add information on datalad / git-annex setup to README, as in some linux distributions the default packages are utdated.

• https://neuro.debian.net/install_pkg.html?p=neurodebian

Integrate BUR4 data already read and adapt code for data reading.

This likely should read Korea (and maybe another year)?

Several country reading script contain a fix_rows function to combine rows split up in the pdf reading process (because they contained line breaks in some cells). Make one unified function and place that in the helper module to reduce code and possible confusion.

Submissions that where the fix_rows function is used are:

• Mexico BUR3,
• Taiwan 2022 NIR

Additional submission where the code is not integrated into the repository use the function

First BUR from Côte d’Ivoire. Contains some sector and gas detail for 1990 and 2014 but not much data for other years.

Lebanon's fourth BUR. Detailed inventory for 2018, not much data for other years. Check if detailed inventory is available somewhere before reading this BUR

The report resolves individual gases only for 2016, however a spreadsheet containing all data is available from inventariogei.ambiente.gob.ar.

Data is in IPCC 2006 categories with very good sectoral resolution.

Malawi's first BUR contains detailed inventories for 2010 and 2017 in IPCC2006 categories. Time series for some sectors and gases exist. Data reading will take some time as time series have to be collected from tables in different formats.

Moldova's third BUR comes with a NIR that contains a detailed 1990-2019 inventory. Data uses a mixture of IPCC1996 and 2006 sectors (Agriculture and LULUCF split). The table format is similar to the standard NIR format but with separate tables per main category.

These new submissions have been added to the UNFCCC website since July 2022 and have to be considered for inclusion into PRIMAP-hist v2.5

• Albania, NC4: full inventory for 2009-2019
• Algeria, NC3: inventory for 2020 but almost no details. Timeseries only in plots, no data shown; BUR1: full gas and sector time-series for 1990-2020; NIR: 1990-2020, very detailed but in individual tables and there might be gaps for some sectors
• Argentina, BUR5: full 1990-2020 inventory (machine readable inventories outdated, so have to read from pdf)
• Bahamas, BUR1: aggregate GHG for main sectors 2001-2018. Also splits by gas or sector but not both at the same time except for 2018. Tables likely hard to read
• Bangladesh BUR1: 2013 - 2019, data in individual sector tables
• Belize, NC3, NC4: data until 2019. coverage patchy
• Bhutan, BUR1: 1994-2020 (main sector GHG), some detail for 2016-2020
• Bosnia and Herzegovina, BUR3, NC4: full inventory for 2015, 2016, CO2 for 1990,2012,2013,2014,2015,2016, some more data
• Burkina Faso, NC3: already available from DI interface
• Burundi, BUR1: full inventory for 2005-2019
• Cambodia, NC3: inventory with some details for 2005, 2010
• Central African Republic, NC3: some data for 2011-2016, but not for all sectors, BUR1: some data for 2010-2018, but no full inventory
• Chile, BUR5: 1990-2020 (xls file available)
• China, NC4: 2017 inventory with sufficient details. Also for Hong Kong
• Côte d'Ivoire, BUR2: more data in NIR; NIR: time-series for 1990-2020, detailed inventory for 2020
• Democratic Republic of the Congo, BUR1: main sector data for 2000-2018, needs downscaling. Additional data in NIR that was uploaded later
• Ecuador, BUR2: 1994, 2000, 2006, 2010, 2012, 2014, 2016, 2018. Several xls file available with the submission
• Ethiopia, NC3: detailed inventory for 2018. 1994-2018 trends for main sectors and some subsectors
• Gabon, BUR1: full inventory for 1994, 2000, 2005, 2010-2017
• Ghana, BUR4: full inventory for 2021, some KyotoGHG time-series for 1990-2021 (7 years)
• Guatemala, BUR1: TODO check for data
• Guinea, BUR1: detailed inventories for 1990, 2000, 2010, 2019 in standard format, also NIR to check
• Guyana, BUR1: detailed inventory for 2022. also time series for 1990-2022 for CH4, CO2, N2O. IPPU sector data only minimal. BTR1 has data for 1990-2022. CRT format data to be submitted on October 2024
• Honduras, BUR1: some data for 2005-2015 (check if included in DI interface)
• India, NC3: detailed 2019 inventory, 2011-2019 aggregate GHG timeseries
• Israel, BUR2: 1996-2020 (some gaps, full inventory for some years)
• Jordan, NC4: inventory for 2017 with enough details for PRIMAP-hist. Main sector data for additional years (2000, 2006, 2010, 2012, 2016)
• Kyrgyzstan, BUR1: 1990-2018, full inventory with medium detail
• Lao People's Democratic Republic, NC3: full inventory for 2010
• Lebanon, NC4: 2019 inventory -> TODO: erratum published
• Malaysia, BUR4: very detailed inventory for 2019, very detailed trends for 1990-2019 for CO2, CH4, N2O, HFCs basket, PFCs basket, SF6, NF3
• Mali, BUR1: Data for 2013-2019 but years and details differ for sectors. likely very hard to read as several different table formats
• Micronesia (Federated States of), BUR1: 2001-2018. Several different tables, much work to read and not clear immediately if full coverage
• Mongolia, BUR2: 1990-2020 timeseries for many sectors, detailed inventory for 1990, 2020 (in NIR)
• Mozambique, BUR1: main sector data for 1994, 2000, 2005, 2010, 2012, 2014, 2016; NC2: not much data -> included in DI interface
• Nicaragua, NC4: 2015 inventory, some trends (national total, 3 gases, some subsectors); BUR1: 1996-2019 time-series for main sectors and gases, some additional details. full inventories for 1996 and 2019.
• Niger, BUR1: 1990-2019 trends (main sector / gas, not both), full inventory for 2019
• North Macedonia, NC4, NIR: detailed inventory for 1990, 2000, 2005, 2014-2019
• Panama, NC4: inventory for 2019, some data for other years (2000, 2005, 2010, 2013, 2017, 2018), but have to check if all sectors are covered (NIR is similar, check both to get most data)
• Paraguay, BUR3: full inventory for 1990, 2017. 1990-2017 time series for CO2, CH4, N2O, HFCs basket, SF6 and high sectoral detail
• Peru, BUR3: Detailed inventory for 2000, 2005, 2010, 2012, 2014, 2016, 2019
• Republic of Korea, 2023 inventory
• Republic of Moldova,Republic of Moldova. NC5: full 1990-2020 inventory
• Rwanda, BUR1: 2006-2018, several tables, coverage not immediately clear
• Saint Kitts and Nevis, BUR1: full inventory for 2018, time-series for 2008-2018, but aggregate GHG only
• Sao Tome and Principe,BUR1: 2012, 2016, 2018 inventories
• Serbia, BUR2: 1990-2020 time series for many sectors
• Seychelles, BUR1: 2020 inventory split over several tables.
• Singapore, BUR5: inventories for 1994, 2000, 2010, 2012, 2014, 2016, 2018. all but 2018 lack IPPU details
• Somalia, BUR1: not much data
• South Africa, BUR5: 2020 inventory (messed up table formatting), NIR: 2000-2020 time-series but no sector and gas resolution at the same time
• Sri Lanka, NC2, NC3: 2000-2010, but no full inventories
• Suriname, BUR1, NC3: 2000-2017, but no full inventory
• Thailand, BUR4: 2019 inventory, main category GHG trends 2000-2019 NC4: inventory for 2018,
• Togo, NC4: 2018 inventory, 1995-2018 trends (with gaps) for agg. GHG with sectoral detail (differs from available from DI interface)
• Trinidad and Tobago, other:detailed 1990 inventory in xls (crf-type) format
• Tunisia, BUR3: 2010, 2020, 2021 inventories. 2010-2021 trends for some sectors; NC4: inventories for 1990, 2010, 2021, 2022, also some time series; NIR: until 2021, maybe some additional data to BUR3 and NC4
• Uganda, NC3: 2017 inventory, some 1995-2017 trends (in several tables)
• United Arab Emirates, BUR1/NC5: data for 2021 in non-standard format
• Uruguay, NC6: more data in NIR; NIR in "other": 1990-2020 time series for main sectors. 2020 inventory. format not easy to read
• Zimbabwe, NC4: 2010 inventory (maybe a mistake in the date as trends go until 2017)

last update: March 14, 2024

Integrate data already read and adapt code

Contains data in IPCC2006 categories for 2010 - 2018 in 2 year intervals.

Integrate data already read and code

Read table from page 27 in BUR1. Data is currently not read

Vanuatu's first BUR contains detailed data for 2017 in IPCC2006 categories but not much data for other years

Some BUR reading script currently don't run. As the data have been read it's not urgent to fix, but should be done at some point.

Affected scripts are

• Colombia BUR3 (column already exists when adding aggregated categories)
• Indonesia BUR3 (similar to Colombia)
• Montenegro BUR3 (similar to Colombia)
• Morocco BUR3 (similar to Colombia)
• South Korea 2021 inventory (similar to Colombia)
• South Korea BUR4 (issue with original cat names which are not unique for a given category and have to be removed before conversion to pm2 format)
• Taiwan 2020 inventory (similar to Colombia)

for most cases the errors are in the category aggregation which is done using the process_data_for_country function in newer scripts. Modifying the scripts to also use that function should circumvent the error.

The report resolves individual gases only for 2018, for 1990-2018 gas basket time series exist. However BUR3 data was made available in spreadsheet containing all data is available from inventariogei.ambiente.gob.ar. So before reading the data from pdf check if the spreadsheet has been updated (last check: Jan 24, 2022, not updated).

Data is in IPCC 2006 categories with very good sectoral resolution.

The output of the country_info task is a bit hard to read especially for the many DI datasets.

• Include information on linking of data version to actual data in DI data display. currently date labelled versions are shown alongside hash versions and it is not clear which hash version they link to.
• Sort submissions
• Output optimization for display in docs

Integrate data already read and code.

Saint Lucia has recently updated it's first BUR. Neither the BUR nor the NIR seem to contain much emissions data.

Mauritius' BUR1 contains detailed inventories for 2000, 2005, 2010, 2014 - 2016 and KyotoGHG time series for 2000-2016. Data is contained in the NIR and using the standard NIR format.

Integrate data already read and adapt code

The NIR that was submitted with the fourth NC contains a detailed inventory for 2017 and some time series for other sectors and gases though likely not all sectors and gases.

The NIR submitted with Costa Rica's second BUR contains a detailed inventory for 2012.

Colombia's BUR3 contains detailed data for 2018 but full 1990-2018 time series only for some sectors and gases. Before reading the data check if a full inventory is available somewhere. The table format is not the standard NIR format but similar and should be easy to read for the 2018 inventory. The other time series might be more complicated.

When reading data using a UNFCCC reader script for the first time the output files do not yet exist and datalad does not know about them. In consequence they are not saved by the datalad run command as it is configured to only save output files given explicitly. As the file names are determined by the scripts (and differ by submission not following a clear pattern as sometimes we have category conversions) they are not known outside the scripts unless files already exist.

Possible solutions:

• Run 'datalad run' configured to save all files in case that no input files are available. This needs clean repo for the run.

When one of the files listing the new downloads (e.g. 00_new_downloads_bur-<date>.csv) already exists it is overwritten without warning. This happen e.g. when running a download script twice on the same day.

This should be fixed and a new filename should be chosen in this case.