beancount / ledger2beancount

Ledger to Beancount text-based converter

beancount: Double-Entry Accounting from Text Files

Description

A double-entry bookkeeping computer language that lets you define financial transaction records in a text file, read them in memory, generate a variety of reports from them, and provides a web interface.

Documentation

Documentation can be read at:

https://beancount.github.io/docs/

Documentation authoring happens on Google Docs, where you can contribute by requesting access or commenting on individual documents. An index of all source documents is available here:

http://furius.ca/beancount/doc/index

There's a mailing-list dedicated to Beancount, please post questions there, so others can share in the responses. More general discussions about command-line accounting also occur on the Ledger mailing-list so you might be interested in that group as well.

Download & Installation

You can obtain the source code from the official Git repository on Github:

https://github.com/beancount/beancount/

See the Installing Beancount document for more details.

Versions

There are three versions

• Version 3 (branch master): The in-development next version of Beancount since June 2020. This is unstable and you want to use version 2 below. The scope of changes is described in this document.
• Version 2 (branch v2): The current stable version of Beancount, in maintenance mode as of July 2020. This was a complete rewrite of the first version, which introduced a number of constraints and a new grammar and much more. Use this now.
• Version 1 (branch v1): The original version of Beancount. Development on this version halted in 2013. This initial version was intended to be similar to and partially compatible with Ledger. Do not use this.

Filing Bugs

Tickets can be filed at on the Github project page:

https://github.com/beancount/beancount/issues

Copyright and License

Copyright (C) 2007-2022 Martin Blais. All Rights Reserved.

This code is distributed under the terms of the "GNU GPLv2 only". See COPYING file for details.

Donations

Beancount has found itself being useful to many users, companies, and foundations since I started it around 2007. I never ask for money, as my intent with this project is to build something that is useful to me first, as well as for others, in the simplest, most durable manner, and I believe in the genuinely free and open stance of Open Source software. Though its ends are utilitarian -it is about doing my own accounting in the first order - it is also a labor of love and I take great pride in it, pride which has pushed me to add the polish so that it would be usable and understandable by others. This is one of the rare areas of my software practice where I can let my desire for perfection and minimalism run untamed from the demands of time and external constraints.

Many people have asked where they can donate for the project. If you would like to give back, you can send a donation via Wise (preferably):

https://wise.com/share/martinb4019

or PayPal at:

https://www.paypal.com/paypalme/misislavski

Your donation is always appreciated in any amount, and while the countless hours spent on building this project are impossible to match, the impact of each donation is much larger than its financial import. I truly appreciate every person who offers one; software can be a lonely endeavour, and those donations as well as words of appreciation keep reminding me of the positive impact my side projects can have on others. I feel gratitude for all users of Beancount.

Thank you!

Author

Martin Blais <[email protected]>

ledger2beancount's Issues

no longer convert per-posting comments to comment metadata

after merging #18 , and with tag_as_metadata: true (in case that matters), I no longer have "comment: ..." metadata, that were used to capture per-posting comments (that are not supported by Beancount ATM)

fails to detect posting tags in inline comments

Example:

2018-05-19 * "pool"
  Expenses:Entertainement:Pool                4.00 EUR  ; mum
  Expenses:Entertainement:Pool                2.50 EUR  ; :kid:
  Expenses:Cash

The above should behave the same of:

2018-05-19 * "pool"
  Expenses:Entertainement:Pool                4.00 EUR  ; mum
  Expenses:Entertainement:Pool                2.50 EUR
    ; :kid:
  Expenses:Cash

but it currently doesn't. In the latter case kid is properly recognized as a tag (and hence converted to a fake tags:"" metadata key, because Beancount doesn't yet support posting tags, but that's another story…). In the former case the in-line comment ; :kid: is preserved as is in the generated Beancount.

Metadata after posting not recognized

Metadata does not have to be put on the next line. It can be on the same line as the posting.

2018-03-19 * Metadata after posting
    ; test: foo bar baz
    Assets:Test                        10.00 EUR ; test: foo
    Equity:Opening-Balance            -10.00 EUR

This is treated by ledger2beancount as a comment instead of metadata.

add support for specifying arbitrary beancount options

It looks like (by personal experimentation) Beancount option directives have to be specified in the main Beancount file and cannot go in included file, similarly to what happens for plugin directives. Hence, to allow users to specify Beancount options, we need an options entry in ledger2beancount conffile, equivalent of the plugins entry we already have.

Real-life use case: adding a set of document options to the generated Beancount file, to cross-reference document from directories.

payee_match is not predictable

payee_match is a hash of regex matches. The problem with this is that the order of the hash is not fixed, so we can get different results if two matches match the same line.

We should use a list instead.

My question: for backwards compatibility, should we add code to detect whether payee_match is a hash or a list. My inclination is not to because it increases complexity and because there aren't many users yet.

plugin empty gives warning

With config:

plugins:

Can't use an undefined value as an ARRAY reference at ./bin/ledger2beancount line 613.

add support for lot notes

The script supports lot dates but not lot notes.

Add more characters not allowed in account names (_ .)

       Invalid token: '_st'

with account Assets:Te_st.

add support to include conversion notes in generated output

Use cases: document potential issues/warnings detected by the conversion machinery.

It should be optional, but likely default to true.

Handle commodities with umlauts/UTF-8

ledger:

2016-04-28 * Commodity with umlaut
  Assets:Test                       1.00 Föo
Equity:Opening balance

The commodity is not mapped, even though I have a mapping defined.

Use pushtag and poptag

Martin Blais reminded us that beancount has pushtag and poptag.

support posting dates

We should support posting dates. There are two variants: a) in brackets ([date]) and b) metadata date

See the recent conversion on the ledger list.

Examples:

2017-01-02 * Posting date using comment
    A               10.00 EUR ; [2017-03-04]
    B              -10.00 EUR ; [2017-07-08]


2017-02-03 * Posting date using metadata
    A               10.00 EUR
        ; date: [2017-04-05]
    B              -10.00 EUR
        ; date: 2017-08-09

Y2018 not working

Date without year but no Y/year directive at ./bin/ledger2beancount line 127.

Commodity mapping changes account name instead of commodity

With:

2017-03-31 * VPS at Gandi March
  Expenses:Internet:Hosting                             140616 Gandi
  Assets:Voucher:Gandi                                 -140616 Gandi

I get:

2017-03-31 * "VPS at Gandi March"
  Expenses:Internet:Hosting                             140616 GANDI
  Assets:Voucher:GANDI                                 -140616 Gandi

tag_as_metadata creates invalid syntax when tags are on multiple lines

Given the ledger file:

2018-03-17 * Test
    ; :foo:bar:baz:
    ; :test:
    Assets:Test                        10.00 EUR
    Equity:Opening-Balance            -10.00 EUR

the script produces:

2018-03-17 * "Test"
  tags: "foo, bar, baz"
  tags: "test"
...

which beancount rejects:

Duplicate metadata field on entry: KeyValue(key='tags', value='test')

Please support more ledger input date formats

While beancount requires YYYY-MM-DD (ISO 8601) ledger is much more flexible. ledger2beancount should support this.

Zack says:

it'd be easy to support with a custom regexp with named groups for year/month/day

I don't think it's that easy because iirc there's some other code that assumes the date contains dashes but it should be fairly straight forward.

report collisions when we rename accounts and commodities

Zack had the idea to report when collisions happen when we rename accounts and commodities.

This partly depends on #57

consider creating a single entry point bin/ledger2beancount

The bin/ scripts are split only for historical reasons. And we are starting to see overlaps among them, e.g., the -prices script is already partially duplicated with some regex in -txns.

We should consider have a single bin/ledger2beancount entry point as the main UI.

In terms of implementation, that could either mean fold everything in a single script, or have separate (Perl?) modules for the various parts.

non-indented block comments lead to Assertion failed (Carp::Assert::assert("") called at ... line 755)

Hello,

I'm trying to convert my hledger file into beancount, but get the following:

$ledger2beancount personal.ledger > personal.beancount
Assertion failed!
 at /usr/share/perl5/vendor_perl/Carp/Assert.pm line 282.
	Carp::Assert::assert("") called at /home/gour/bin/ledger2beancount line 755

so wonder if you have any tip what might be wrong since, based on line 755 it does not look as the date-parsing problem?

Arch packaging / AUR: yaourt obsolete? doc update needed?

@wzyboy I read this article which says yaourt is no longer developed. docs/manual.md currently says:

yaourt -S ledger2beancount

Should that be updated or removed?

support Beancount accounts starting with a digit

Beancount now allows account names starting with digits.

• Update the converter.
• Document supported beancount version?

commodity mapping not applied to commodity in @price

This:

2017-03-31 * VPS at Gandi March
  Expenses:Internet:Hosting          140616 Gandi @ Gandi
  Assets:Voucher:Gandi              -140616 Gandi @ Gandi

leads to

2017-03-31 * "VPS at Gandi March"
  Expenses:Internet:Hosting          140616 GANDI @ Gandi
  Assets:Voucher:Gandi              -140616 GANDI @ Gandi

Rewrite inline math for beancount

ledger:

2016-06-23 * Test
  Assets:Test         44.06 USD @ (1/1.362 GBP)            
  Assets:Test        -32.35 GBP

has to become:

2016-06-23 * "Test"
  Assets:Test         44.06 USD @ (1/1.362) GBP            
  Assets:Test        -32.35 GBP

i.e. brackets without the commodity.

use "note" for account notes instead of "description"?

ledger allow note entries associated to accounts (and maybe other directives? I haven't checked), like this:

account Expenses:Membership
    note organization membership, non-profits, political parties, etc.

ledger2beancount currently ignores note entries.

We should support them somewhat, at least to avoid losing information. For instance, we can convert them to line-long comments.

Support implicit conversions

Ledger allows implicit conversions. It would be great if ledger2beancount would recognise them and make them explicit so they don't cause errors in beancount.

2018-04-27 * Implicit conversion
    Assets:Test                 10.00 EUR
    Assets:Test                 -8.79 GBP

After conversion you currently get:

b:5:       Transaction does not balance: (10.00 EUR, -8.79 GBP)

Use or remove Carp::Assert

After all the refactoring, we're now actually using Carp::Assert anymore. Should I remove it or do you think it makes sense to add some assert statements (and if so, where)?

emit missing account definitions

ledger allows you to define accounts but it's optional whereas it's required in beancount. ledger2beancount should create a list of accounts in use and declare them.

Full-line comments in transactions now allowed by beancount / how to handle unreleased beancount changes

Full-line comments in transactions are now allowed in beancount. We have to update ledger2beancount accordingly.

The question is how to do this since people might run the latest released version of beancount and not the current version from hg.

I asked Zack "Maybe change it on a branch and merge once a new release is cut? Or just work against hg?"

Zack said "i guess we can use an aggressive default, where we use proper comments, a have an option to fallback to current behavior for people using an "old" beancount"

And my reply: "hm I'm not sure I want an option. Although I guess an option "using released beancount vs hg beancount" might be ok. and once the feature is released in a new beancount version we remove the if"

But then the conversation stopped, so let's continue it here.

One related question is whether we can get beancount from hg installed on Travis. I know Zack had trouble even getting rc releases installed. I think pip can install from Git repositories directly but I'm not sure if that will work with hg.

Inline math doesn't like negative amounts

Inline math doesn't like negative amounts:

2013-07-10 * Flight redemption: GLA - MUC
    ; :reward:ba:
    Expenses:Travel:Airfare                            (2 * 15.00 GBP)
    Expenses:Travel:Airfare                            (2 * 4500 Avios)
    Assets:Rewards                                    (2 * -4500 Avios)
    Liabilities:BA                                     (2 * -15.00 GBP)

results in

  Expenses:Travel:Airfare                            (2 * 15.00) GBP
  Expenses:Travel:Airfare                            (2 * 4500) AVIOS
  Assets:Rewards                                    (2) X -4500 Avios)
  Liabilities:BA                                     (2) X -15.00 GBP)

It beleves 2 * is the amount and commodity.

move the bulk of documentation away from top-level README

there's too much stuff in the top-level README, we should move most of the documentation to a separate file, and leave in the top-level README only (and preferably in this order):

1. overview
2. usage, i.e.:

• CLI invocation
• configuration file location
• pointer to separate doc file for more info

3. dependencies
4. authors
5. license

As all that remains would be the current features section, the separate doc file should probably be named doc/features.md.

transform ledger account names into legal beancount names

ledger2beancount offers an account mapping but it would be good to offer regex too, for example to replace all withspace with dashes.

Create valid beancount commodities

Since we rewrite account names to create valid beancount names, it would be nice to do the same for commodities.

Considers account flags as part of the account name

ledger and beancount allow a flag right before the account name. The script thinks its part of the account name:

2018-03-17 Test
    * Assets:Test                        10.00 EUR
    ! Equity:Opening balance            -10.00 EUR

Tags can have commands associated

tag Project
    check value =~ /^(0ad|ankur|aptosid|archlinux|chakra|debian)$/

2016-01-01 * Test - correct tag
    Expenses:Hosting                    20.00 USD
        ; Project: debian
    Assets:Cash                        -20.00 USD

After ledger2beancount:

15:      Invalid token: 'check'

add option to specify non default configuration file

use case:

ledger2beancount -c l2b-accounts.yml < accounts.ledger > accounts.beancount
ledger2beancount -c l2b-transactions.yml < current.ledger > current.beancount
ledger2beancount -c l2b-prices.yml < pricedb.ledger > pricedb.beancount

proposed syntax:

• getopt with both -c and --conf
  proposed semantics:
• if -c/--conf is given, it replaces the default places where ledger2beancount.yml is looked for, ignoring it, and using the given conf file instead

stdin should be treated as UTF-8

When matching the pound symbol in payee_match I had to use two ... It wouldn't match the pound directly or only one ..

provide sane defaults for configuration file options

Currently most configuration file options are mandatory: they should be specified even if unused, because no default at all is provided in ledger2beancount. We should fix this.

Note that there are two use cases served by defaults:

• provide sane defaults, when no customization is needed, reducing the size/complexity of configuration file
• provide sane upgrade paths: when we add an option to ledger2beancount, users should not be forced to change their configuration (unless it's a major semantics change, of course)

AUR installation fails

Hi

Any ideas what could be the issue here?

OS: Arch Linux 4.17.9

0 aur/ledger2beancount 1.1-1 (0)
    Ledger to Beancount text-based converter

Type numbers to install. Separate each number with a space.
Numbers: 0

Aur Targets    (5): perl-hash-merge-simple perl-getopt-long-descriptive perl-config-onion perl-carp-assert ledger2beancount

Proceed with installation? [Y/n] y
Edit perl-hash-merge-simple PKGBUILD with $EDITOR? [Y/n] n
==> Making package: perl-hash-merge-simple 0.051-1 (Thu 26 Jul 2018 04:31:09 PM CEST)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Missing dependencies:
  -> perl-test-most>=0
==> ERROR: Could not resolve all dependencies.
The build failed.
Edit perl-getopt-long-descriptive PKGBUILD with $EDITOR? [Y/n] n
==> Making package: perl-getopt-long-descriptive 0.100-1 (Thu 26 Jul 2018 04:31:25 PM CEST)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Missing dependencies:
  -> perl-test-warnings>=0.005
==> ERROR: Could not resolve all dependencies.
The build failed.
Edit perl-config-onion PKGBUILD with $EDITOR? [Y/n] n
==> Making package: perl-config-onion 1.007-2 (Thu 26 Jul 2018 04:31:31 PM CEST)
==> Checking runtime dependencies...
==> Missing dependencies:
  -> perl-hash-merge-simple
==> Checking buildtime dependencies...
==> ERROR: Could not resolve all dependencies.
The build failed.
Edit perl-carp-assert PKGBUILD with $EDITOR? [Y/n] n
==> ERROR: provides contains invalid characters: '::'
The build failed.
Dependencies for `ledger2beancount' are not met, not building...

Assertion failed when date_match doesn't match date

With a regex in date_match that doesn't match the date in the ledger file I get:

2018-03-19 * Test
Assertion failed!
 at /usr/share/perl5/Carp/Assert.pm line 282, <> line 3.
	Carp::Assert::assert(0) called at ./bin/ledger2beancount-txns line 454

offer ability to treat payee_match (et al) case insensitive

payee_match (et al) should be case insensitive. Plus update docs.

produce invalid syntax with lot cost and lot date

This leads to invalid syntax:

2018-03-19 * Test
    Assets:Test                1.00 EUR {0.90 GBP} [2018-03-18]
    Equity:Opening-Balance

I think it's not caught by the lot code for some reason.

continuous integration

given we have an automated test suite, we should set up some CI integration (e.g., Travis)

expenses:purchase converted to expenses:Purchase

We don't uppercase the first letter

allow commodity to be treated as currency for conversion, i.e. don't keep costs

We need the opposite of commodities3char, i.e. treat a commodity as if it was a currency, i.e. don't keep the cost around. This is useful for e.g. air miles.

chokes on transactions without description

ledger seems happy with this transaction in my ledger file:

 2005/02/08
     Expenses:Books                         EUR 60.42
     Assets:Current Assets:ING:Checking Account

but ledger2beancount chokes on it:
Cannot process transaction header: 2005/02/08

Adding some spaces to the end of the date works around the issue.

document how to install on Windows

Guys hi,

How ledger2beancount can be installed on Windows?
Installation manual provides directions for Debian, but not for Windows users.

does not strip ledger assertion from generated beancount, making it invalid

this Ledger input:

2018-01-01 * foo bar baz
    ; Label: some text here
    Liabilities:Mortgage                        1.00 EUR = -42.0 EUR
    Expenses:Interest:Mortgage                  2.00 EUR
    Expenses:Insurance:Mortgage                 3.00 EUR
    Assets:Checking                            -4.00 EUR

gives this Beancount output:

2018-01-01 * "foo bar baz"
  bank-label: "some text here"
  Liabilities:Mortgage                        1.00 EUR
  Liabilities:Mortgage                        1.00 EUR = -42.0 EUR
  Expenses:Interest:Mortgage                  2.00 EUR
  Expenses:Insurance:Mortgage                 3.00 EUR
  Assets:Checking                            -4.00 EUR

2018-01-02 balance Liabilities:Mortgage  -42.0 EUR

the balance assertion is correct, but the = ledger assertion remains, inducing a syntax error.

This used to work, I'm not sure when it got broken. We need a regression test to avoid it happens again.

add the ability to selectively ignore input lines

While ledger2beancount can translate most ledger directives to equivalent beancount ones, it is sometime desirable to selectively ignore input lines. For instance, specific include directives (but not others) in a Ledger setup might be unneeded in the resulting beancount setup.

To support this I propose to replicate a mechanism that is popular for linting tools: support inline comments that tell the tool to look away, e.g.:

include foo.ledger
include bar.ledger  ; NoL2B
include baz.ledger

will be translated to:

include "foo.beancount"
include "baz.beancount"

To be fully generic this will also need a block-level equivalent, e.g.

; NoL2B_begin
stuff in between these two will be completely
; NoL2B_end

but that is not strictly needed to make an initial version of this feature useful.

Ledger allows number as metadata key but beancount doesn't

Test case:

2013-03-03 * Test
    ; 2013: 50
    A          10.00 EUR
    B

Please support virtual costs

Would be good to support virtual costs, i.e. (@) and (@@).