kedder / ofxstatement-iso20022

ISO20022 plugin for ofxstatement

ISO-20022 plugin for ofxstatement

Plugin to read ISO-20022 formatted statements.

Plugin supports the following configuration options:

• currency: Account currency. In case currency is not specified in ISO20022 statement, you can specify it with this setting. Only statement lines with account currency will be included in OFX files.
• iban: Account IBAN. In case the IBAN is not specified in the ISO20022 statement, you need to specify this option.

Currently implementation is very trivial and naive. If it doesn't work for you, feel free to improve it or file a bug on github with sample (anonymized) state file.

ofxstatement-iso20022's Issues

Support for SPS camt.053

Swiss banks use a subset of iso20022 and created a spec around that.
camt.053.001.08 specifically is used for bank to customer statements.

Trying to convert a statement with this plugin resulted in the error below:

Traceback (most recent call last):
  File "/usr/bin/ofxstatement", line 33, in <module>
    sys.exit(load_entry_point('ofxstatement==0.8.0', 'console_scripts', 'ofxstatement')())
  File "/usr/lib/python3.10/site-packages/ofxstatement/tool.py", line 205, in run
    return args.func(args)
  File "/usr/lib/python3.10/site-packages/ofxstatement/tool.py", line 176, in convert
    statement = parser.parse()
  File "/home/marc/.local/lib/python3.10/site-packages/ofxstatement/plugins/iso20022.py", line 55, in parse
    self._parse_statement_properties(tree)
  File "/home/marc/.local/lib/python3.10/site-packages/ofxstatement/plugins/iso20022.py", line 88, in _parse_statement_properties
    assert iban is not None
AssertionError

I've gone ahead and fixed the issues I could see on my own fork. (branch "rs-master")
Conversion is working and as far as I can tell correct.
https://github.com/RealStickman/ofxstatement-iso20022/tree/rs-master

I'd like to make a PR here to get these changes upstream, if that's ok with you.
The tests in this repo passed for me.

Specification:
Swiss Payment Standards 2022
Chapter 4.1 discusses the schema and field mappings for camt.053

Sample:
I've already redacted sensitive information in the file, hopefully well enough.

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.08">
  <BkToCstmrStmt>
    <GrpHdr>
      <MsgId>CAMT053v08REDACTED</MsgId>
      <CreDtTm>2023-01-26T00:49:36.906+01:00</CreDtTm>
      <MsgPgntn>
        <PgNb>1</PgNb>
        <LastPgInd>true</LastPgInd>
      </MsgPgntn>
      <AddtlInf>SPS/2.0/PROD</AddtlInf>
    </GrpHdr>
    <Stmt>
      <Id>bc8b8e304a825671937ffa86f49ab6f1</Id>
      <ElctrncSeqNb>12</ElctrncSeqNb>
      <CreDtTm>2023-01-26T00:49:36.906+01:00</CreDtTm>
      <FrToDt>
        <FrDtTm>2023-01-25T00:00:00+01:00</FrDtTm>
        <ToDtTm>2023-01-25T23:59:00+01:00</ToDtTm>
      </FrToDt>
      <Acct>
        <Id>
          <Othr>
            <Id>0035-2915203-10-000</Id>
          </Othr>
        </Id>
        <Ccy>CHF</Ccy>
        <Ownr>
          <Nm>NAME REDACTED</Nm>
          <PstlAdr>
            <AdrLine>CITYREDACTED</AdrLine>
          </PstlAdr>
        </Ownr>
        <Svcr>
          <FinInstnId>
            <BICFI>BANK BIC</BICFI>
            <Nm>BANK NAME</Nm>
            <Othr>
              <Id>ID</Id>
              <Issr>VAT-ID</Issr>
            </Othr>
          </FinInstnId>
        </Svcr>
      </Acct>
      <Bal>
        <Tp>
          <CdOrPrtry>
            <Cd>OPBD</Cd>
          </CdOrPrtry>
        </Tp>
        <Amt Ccy="CHF">832.01</Amt>
        <CdtDbtInd>CRDT</CdtDbtInd>
        <Dt>
          <Dt>2023-01-25</Dt>
        </Dt>
      </Bal>
      <Bal>
        <Tp>
          <CdOrPrtry>
            <Cd>CLAV</Cd>
          </CdOrPrtry>
        </Tp>
        <Amt Ccy="CHF">2116.31</Amt>
        <CdtDbtInd>CRDT</CdtDbtInd>
        <Dt>
          <Dt>2023-01-25</Dt>
        </Dt>
      </Bal>
      <Bal>
        <Tp>
          <CdOrPrtry>
            <Cd>CLBD</Cd>
          </CdOrPrtry>
        </Tp>
        <Amt Ccy="CHF">2116.31</Amt>
        <CdtDbtInd>CRDT</CdtDbtInd>
        <Dt>
          <Dt>2023-01-25</Dt>
        </Dt>
      </Bal>
      <TxsSummry>
        <TtlNtries>
          <NbOfNtries>1</NbOfNtries>
          <Sum>1284.30</Sum>
          <TtlNetNtry>
            <Amt>1284.30</Amt>
            <CdtDbtInd>CRDT</CdtDbtInd>
          </TtlNetNtry>
        </TtlNtries>
        <TtlCdtNtries>
          <NbOfNtries>1</NbOfNtries>
          <Sum>1284.30</Sum>
        </TtlCdtNtries>
        <TtlDbtNtries>
          <NbOfNtries>0</NbOfNtries>
          <Sum>0.00</Sum>
        </TtlDbtNtries>
      </TxsSummry>
      <Ntry>
        <Amt Ccy="CHF">1284.30</Amt>
        <CdtDbtInd>CRDT</CdtDbtInd>
        <RvslInd>false</RvslInd>
        <Sts>
          <Cd>BOOK</Cd>
        </Sts>
        <BookgDt>
          <Dt>2023-01-25</Dt>
        </BookgDt>
        <ValDt>
          <Dt>2023-01-25</Dt>
        </ValDt>
        <AcctSvcrRef>8082230156M02SJ62/1010</AcctSvcrRef>
        <BkTxCd>
          <Domn>
            <Cd>PMNT</Cd>
            <Fmly>
              <Cd>RCDT</Cd>
              <SubFmlyCd>DMCT</SubFmlyCd>
            </Fmly>
          </Domn>
        </BkTxCd>
        <AmtDtls>
          <TxAmt>
            <Amt Ccy="CHF">1284.30</Amt>
          </TxAmt>
        </AmtDtls>
        <NtryDtls>
          <TxDtls>
            <Refs>
              <MsgId>4223-F023401-02-M03920</MsgId>
              <AcctSvcrRef>A032-J30K20-03-JF021</AcctSvcrRef>
              <PmtInfId>F213-402451-MS-10432</PmtInfId>
              <EndToEndId>9502830971</EndToEndId>
              <UETR>75e77105-a09f-4f9e-bfa8-e82e91ab209b</UETR>
            </Refs>
            <Amt Ccy="CHF">1284.30</Amt>
            <CdtDbtInd>CRDT</CdtDbtInd>
            <AmtDtls>
              <TxAmt>
                <Amt Ccy="CHF">1284.30</Amt>
              </TxAmt>
            </AmtDtls>
            <RltdPties>
              <Dbtr>
                <Pty>
                  <Nm>PAYEE</Nm>
                  <PstlAdr>
                    <AdrLine>PAYEE ADDRESS</AdrLine>
                  </PstlAdr>
                </Pty>
              </Dbtr>
            </RltdPties>
            <RmtInf>
              <Ustrd>PAYMENT INFO</Ustrd>
            </RmtInf>
          </TxDtls>
        </NtryDtls>
      </Ntry>
    </Stmt>
  </BkToCstmrStmt>
</Document>

Reads only the first statement

If there are several in the xml, the rest are ignored. Looks like a design limitation in parse / _get_statement_el.

Adding Support for CAMT.052 Format

Hello,

just tried to do a conversion of Statement in camt.052-Format which is - by my understanding - part of the ISO20022 ... but failed.

Looks like camt.052 is used by at least some banks in Baden-Wurttemberg/ Germany.

Below my minor change to the plugin to make this work. Of course not this quality code.
Maybe this would be of interest not only for me.

Chris

--- iso20022.py.orig	2021-02-05 22:12:34.452497846 +0100
+++ iso20022.py	2021-02-06 08:46:40.420497846 +0100
@@ -7,7 +7,19 @@
 from ofxstatement.statement import Statement, StatementLine
 
 
-ISO20022_NAMESPACE_ROOT = 'urn:iso:std:iso:20022:tech:xsd:camt.053.001'
+#
+# 2021-02-06	[email protected]
+#
+# Change:
+# Slightly modified to additionally support CAMT.052 that is used
+# for Example by Sparkasse Pforzheim-Calw.
+#
+#
+
+
+ISO20022_NAMESPACE_ROOT_CAMT053 = 'urn:iso:std:iso:20022:tech:xsd:camt.053.001'
+ISO20022_NAMESPACE_ROOT_CAMT052 = 'urn:iso:std:iso:20022:tech:xsd:camt.052.001'
+
 
 CD_CREDIT = 'CRDT'
 CD_DEBIT = 'DBIT'
@@ -23,9 +35,11 @@
 
 
 class Iso20022Parser(object):
+
     def __init__(self, filename, currency=None):
         self.filename = filename
         self.currency = currency
+        self.camt_type = None
 
     def parse(self):
         """Main entry point for parsers
@@ -36,14 +50,17 @@
 
         # Find out XML namespace and make sure we can parse it
         ns = self._get_namespace(tree.getroot())
-        if not ns.startswith(ISO20022_NAMESPACE_ROOT):
+        if ns.startswith(ISO20022_NAMESPACE_ROOT_CAMT053):
+            self.camt_type = 53
+        if ns.startswith(ISO20022_NAMESPACE_ROOT_CAMT052):
+            self.camt_type = 52
+        if self.camt_type is None:
             raise exceptions.ParseError(0, "Cannot recognize ISO20022 XML")
 
         self.xmlns = {
             "s": ns
         }
 
-
         self._parse_statement_properties(tree)
         self._parse_lines(tree)
 
@@ -54,7 +71,11 @@
         return m.groups()[0] if m else ''
 
     def _parse_statement_properties(self, tree):
-        stmt = tree.find('./s:BkToCstmrStmt/s:Stmt', self.xmlns)
+        if self.camt_type == 53:
+            stmt = tree.find('./s:BkToCstmrStmt/s:Stmt', self.xmlns)
+
+        if self.camt_type == 52:
+            stmt = tree.find('./s:BkToCstmrAcctRpt/s:Rpt', self.xmlns)
 
         bnk = stmt.find('./s:Acct/s:Svcr/s:FinInstnId/s:BIC', self.xmlns)
         if bnk is None:
@@ -82,7 +103,6 @@
             # Amount currency should match with statement currency
             if amt_ccy != self.statement.currency:
                 continue
-
             bal_amts[cd.text] = self._parse_amount(amt)
             bal_dates[cd.text] = self._parse_date(dt)
 
@@ -93,13 +113,21 @@
 
         self.statement.bank_id = bnk.text if bnk is not None else None
         self.statement.account_id = iban.text
-        self.statement.start_balance = bal_amts['OPBD']
-        self.statement.start_date = bal_dates['OPBD']
+        if self.camt_type == 53:
+            self.statement.start_balance = bal_amts['OPBD']
+            self.statement.start_date = bal_dates['OPBD']
+        if self.camt_type == 52:
+            self.statement.start_balance = bal_amts['PRCD']
+            self.statement.start_date = bal_dates['PRCD']
         self.statement.end_balance = bal_amts['CLBD']
         self.statement.end_date = bal_dates['CLBD']
 
     def _parse_lines(self, tree):
-        for ntry in self._findall(tree, 'BkToCstmrStmt/Stmt/Ntry'):
+        if self.camt_type == 53:
+            node = 'BkToCstmrStmt/Stmt/Ntry'
+        if self.camt_type == 52:
+            node = 'BkToCstmrAcctRpt/Rpt/Ntry'
+        for ntry in self._findall(tree, node):
             sline = self._parse_line(ntry)
             if sline is not None:
                 self.statement.lines.append(sline)

Missing FITID in generated OFX

Trying to further process OFX generated from ISO20022-camt052 (also applies to camt053 Source) with Tryton ERP (which relies - by my knowledge - on python ofxparse) gives me an Error "ofxparse.ofxparse.OfxParserException: Missing FIT id (a required field)"

May it be an idea to add generation of FITID to the plugin like the MT940 Plugin does? Or am I missing some knowledge to correctly further process the OFX files?

I've tried to add the generation of FITID by myself based on the implementation in MT940 Plugin.
The subsequently converted files can be successfully processed by ofxparse.

Changes i've made to the iso20022-Plugin:

6a7

from typing import Set, Iterator, Any, IO
11a13
from ofxstatement.statement import generate_unique_transaction_id
38a41
unique_id_set: Set[str]
42a46
self.unique_id_set = set()
154a159,163

            sline.id = generate_unique_transaction_id(sline, self.unique_id_set)
            m = re.match(r'([0-9a-f]+)(-\d+)?$', sline.id)
            assert m, "Id should match hexadecimal digits, optionally followed by a minus and a counter: '{}'".format(sline.id)

Regards
Chris

Unimported ParseError raise

ParseError is not imported. Only one argument passed, two required.

this line
raise ParseError("Cannot recognize ISO20022 XML")
Should be:
exceptions.ParseError(0, "Cannot recognize ISO20022 XML")

Import error

I tried to import my bank statement and I got the following error:

C:>ofxstatement convert -t iso20022 camt053_20170412091216.xml ynab.ofx
Traceback (most recent call last):
File "c:\users\xxx\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\users\xxx\anaconda3\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\xxx\Anaconda3\Scripts\ofxstatement.exe_main.py", line 9, in
File "c:\users\xxx\anaconda3\lib\site-packages\ofxstatement\tool.py", line 150, in run
return args.func(args)
File "c:\users\xxx\anaconda3\lib\site-packages\ofxstatement\tool.py", line 128, in convert
statement = parser.parse()
File "c:\users\xxx\anaconda3\lib\site-packages\ofxstatement\plugins\iso20022.py", line 38, in parse
self._parse_statement_properties(tree)
File "c:\users\xxx\anaconda3\lib\site-packages\ofxstatement\plugins\iso20022.py", line 46, in _parse_statement_properties
bnk = stmt.find('./s:Acct/s:Svcr/s:FinInstnId/s:BIC', XMLNS)
AttributeError: 'NoneType' object has no attribute 'find'

Bank statement in attachment. ISO-20022 camt.053
camt053_20170412091216.zip