GithubHelp home page GithubHelp logo

pyscaffold / configupdater Goto Github PK

View Code? Open in Web Editor NEW
64.0 4.0 9.0 509 KB

๐ŸŒด Parser like ConfigParser but for updating configuration files

Home Page: https://configupdater.readthedocs.io/

License: Other

Python 100.00%
python configuration-files updating

configupdater's Introduction

Configupdater logo

CI/CD Tests Coverage Publish Package GitHub Sponsors
Package PyPI - Version Conda - Version PyPI - Downloads PyPI - Python Version
Details Hatch project Linting - Ruff test - pytest Pre-Commit Types - Mypy License - MIT Docs - RTD

The sole purpose of ConfigUpdater is to easily update an INI config file with no changes to the original file except the intended ones. This means comments, the ordering of sections and key/value-pairs as wells as their cases are kept as in the original file. Thus ConfigUpdater provides complementary functionality to Python's ConfigParser, which is primarily meant for reading config files and writing new ones.

Features

The key differences to ConfigParser are:

  • minimal invasive changes in the update configuration file,
  • proper handling of comments,
  • only a single config file can be updated at a time,
  • the original case of sections and keys are kept,
  • control over the position of a new section/key

Following features are deliberately not implemented:

  • interpolation of values,
  • propagation of parameters from the default section,
  • conversions of values,
  • passing key/value-pairs with default argument,
  • non-strict mode allowing duplicate sections and keys.

Usage

First install the package with either:

pip install configupdater

or:

conda install -c conda-forge configupdater

Now we can simply do:

from configupdater import ConfigUpdater

updater = ConfigUpdater()
updater.read("setup.cfg")

which would read the file setup.cfg that is found in many projects.

To change the value of an existing key we can simply do:

updater["metadata"]["author"].value = "Alan Turing"

At any point we can print the current state of the configuration file with:

print(updater)

To update the read-in file just call updater.update_file() or updater.write(open('filename','w')) to write the changed configuration file to another destination. Before actually writing, ConfigUpdater will automatically check that the updated configuration file is still valid by parsing it with the help of ConfigParser.

Many of ConfigParser's methods still exists and it's best to look them up in the module reference. Let's look at some examples.

Adding and removing options

Let's say we have the following configuration in a string:

cfg = """
[metadata]
author = Ada Lovelace
summary = The Analytical Engine
"""

We can add an license option, i.e. a key/value pair, in the same way we would do with ConfigParser:

updater = ConfigUpdater()
updater.read_string(cfg)
updater["metadata"]["license"] = "MIT"

A simple print(updater) will give show you that the new option was appended to the end:

[metadata]
author = Ada Lovelace
summary = The Analytical Engine
license = MIT

Since the license is really important to us let's say we want to add it before the summary and even add a short comment before it:

updater = ConfigUpdater()
updater.read_string(cfg)
(updater["metadata"]["summary"].add_before
                                .comment("Ada would have loved MIT")
                                .option("license", "MIT"))

which would result in:

[metadata]
author = Ada Lovelace
# Ada would have loved MIT
license = MIT
summary = Analytical Engine calculating the Bernoulli numbers

Using add_after would give the same result and looks like:

updater = ConfigUpdater()
updater.read_string(cfg)
(updater["metadata"]["author"].add_after
                            .comment("Ada would have loved MIT")
                            .option("license", "MIT"))

Let's say we want to rename [summary]{.title-ref} to the more common `description`:

updater = ConfigUpdater()
updater.read_string(cfg)
updater["metadata"]["summary"].key = "description"

If we wanted no summary at all, we could just do del updater["metadata"]["summary"].

Adding and removing sections

Adding and remove sections just works like adding and removing options but on a higher level. Sticking to our Ada Lovelace example, let's say we want to add a section options just before metadata with a comment and two new lines to separate it from metadata:

updater = ConfigUpdater()
updater.read_string(cfg)
(updater["metadata"].add_before
                    .section("options")
                    .comment("Some specific project options")
                    .space(2))

As expected, this results in:

[options]
# Some specific project options


[metadata]
author = Ada Lovelace
summary = The Analytical Engine

We could now fill the new section with options like we learnt before. If we wanted to rename an existing section we could do this with the help of the name attribute:

updater["metadata"].name = "MetaData"

Sometimes it might be useful to inject a new section not in a programmatic way but more declarative. Let's assume we have thus defined our new section in a multi-line string:

sphinx_sect_str = """
[build_sphinx]
source_dir = docs
build_dir = docs/_build
"""

With the help of two ConfigUpdater objects we can easily inject this section into our example:

sphinx = ConfigUpdater()
sphinx.read_string(sphinx_sect_str)
sphinx_sect = sphinx["build_sphinx"]

updater = ConfigUpdater()
updater.read_string(cfg)

(updater["metadata"].add_after
                    .space()
                    .section(sphinx_sect.detach()))

The detach method will remove the build_sphinx section from the first object and add it to the second object. This results in:

[metadata]
author = Ada Lovelace
summary = The Analytical Engine

[build_sphinx]
source_dir = docs
build_dir = docs/_build

Alternatively, if you want to preserve build_sphinx in both ConfigUpdater objects (i.e., prevent it from being removed from the first while still adding a copy to the second), you call also rely on stdlib's copy.deepcopy function instead of detach:

from copy import deepcopy

(updater["metadata"].add_after
                    .space()
                    .section(deepcopy(sphinx_sect)))

This technique can be used for all objects inside ConfigUpdater: sections, options, comments and blank spaces.

Shallow copies are discouraged in the context of ConfigUpdater because each configuration block keeps a reference to its container to allow easy document editing. When doing editions (such as adding or changing options and comments) based on a shallow copy, the results can be unreliable and unexpected.

For more examples on how the API of ConfigUpdater works it's best to take a look into the unit tests and read the references.

Notes

ConfigUpdater was mainly developed for PyScaffold.

configupdater's People

Contributors

abravalheri avatar ctheune avatar delgan avatar florianwilhelm avatar hrnciar avatar maresb avatar pliablepixels avatar pre-commit-ci[bot] avatar pseonghoon avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

configupdater's Issues

Potentially invalid string produced for `option = None`

Description of your problem

ConfigUpdater seems to potentially generate a file with invalid syntax when option = None.
This issue was first reported in pyscaffold/pyscaffold#506.

Please provide a minimal, self-contained, and reproducible example.

>>> from tempfile import NamedTemporaryFile
>>> from configupdater import ConfigUpdater
>>> example = """\
... [section]
... opt1 = 42
... """
>>> file = NamedTemporaryFile()
>>> file.write(bytes(example, "utf-8"))
20

>>> file.flush()
>>> cfg = ConfigUpdater().read_file(open(file.name))
>>> str(cfg)
'[section]\nopt1 = 42\n'

>>> cfg['section']['opt2'] = None
>>> str(cfg)
'[section]\nopt1 = 42\nopt2\n'

>>> cfg.update_file()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".venv/lib/python3.8/site-packages/configupdater/configupdater.py", line 172, in update_file
    self.validate_format(**self._parser_opts)
  File ".venv/lib/python3.8/site-packages/configupdater/document.py", line 97, in validate_format
    parser.read_string(str(self))
  File "/usr/lib/python3.8/configparser.py", line 723, in read_string
    self.read_file(sfile, source)
  File "/usr/lib/python3.8/configparser.py", line 718, in read_file
    self._read(f, source)
  File "/usr/lib/python3.8/configparser.py", line 1113, in _read
    raise e
configparser.ParsingError: Source contains parsing errors: '<string>'
        [line  3]: 'opt2\n'

Source contains parsing errors: '<string>'
        [line  3]: 'opt2\n'

The same error appears if we simply try to use ConfigUpdater to parse the generated string:

>>> ConfigUpdater().read_string(str(cfg))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".venv/lib/python3.8/site-packages/configupdater/configupdater.py", line 149, in read_string
    return self._parser().read_string(string, source, self)
  File ".venv/lib/python3.8/site-packages/configupdater/parser.py", line 302, in read_string
    return self.read_file(sfile, source, into)
  File ".venv/lib/python3.8/site-packages/configupdater/parser.py", line 278, in read_file
    self._read(f, source, document)
  File ".venv/lib/python3.8/site-packages/configupdater/parser.py", line 516, in _read
    raise e
configparser.ParsingError: Source contains parsing errors: '<string>'
        [line  3]: 'opt2\n'

Source contains parsing errors: '<string>'
        [line  3]: 'opt2\n'

Please provide any additional information below.

Current Behaviour:

>>> cfg = ConfigUpdater()
>>> cfg.add_section('section')
>>> cfg.set('section', 'opt1', None)
<ConfigUpdater [
    <Section: 'section' [
        <Option: opt1 = None>
    ]>
]>

>>> str(cfg)
'[section]\nopt1\n'

If we want to be resilient, I would say the expected behaviour is the following (if allow_no_value is set to False):

>>> str(cfg)
'[section]\nopt1=\n'

(adding a = without a value)

Another possibility is to raise an error in __str__ and tell the user to set allow_no_value to True.

Anyway, I also advise adding something along the lines:

def validate_format(self, **kwargs):
    return super().validate_format(**{**self._parser_opts, **kwargs})

to the ConfigUpdater class, to improve consistency.

Versions and main components

  • ConfigUpdater Version: 3.0
  • Python Version: 3.8
  • Operating system: Ubuntu 18.04 (LTS)
  • How did you install ConfigUpdater: pip

New section does not save on new line

Description of your problem

Given a config file that does not end with a newline character, calling add_section() and then update_file() results in an incorrectly formatted file.

If we start with the following example config, where the final line does not end with a newline character:

[section1]
option1 = value1
option2 = value2

then running the following code:

conf = ConfigUpdater()
conf.read("config.ini")
if not conf.has_section("section2"):
    conf.add_section("section2")
    conf.update_file()

will result in the new section being appended to the value of option2:

[section1]
option1 = value1
option2 = value2[section2]

If the code is run a second time, the new section will be parsed as part of the value for option2, and the resulting config file will be:

[section1]
option1 = value1
option2 = value2[section2]
[section2]

Note that many popular text editors will display two files identically if the only difference is the presence or absence of a newline character at the end, so this bug may be difficult to reproduce if such an editor is used. Though many automatically add a newline to the end of a file when saving for compatibility reasons, this is not always the case (and is how I first encountered this bug).

Since the parser does not throw an exception when reading a file that does not end with a newline, it is unexpected for it to fail when saving that file. I believe that the issue could be solved if the parser assumed an implicit newline at the end of a file even if none is present. Though this would technically result in a change to the file only from calling read() followed by update_file(), this same behaviour is shared by many text editors by default. Despite adding a character to the end of the file, it would not change the formatting in any visible way (adding/removing spaces/comments/etc) and so I believe this would fit with the intent of the library.

Versions and main components

  • ConfigUpdater Version: 3.1.1
  • Python Version: 3.11.4
  • Operating system: Ubuntu 22.04
  • How did you install ConfigUpdater: pip

Nested square brackets inside sections are incorrectly parsed

Description of your problem

Hi!

I noticed difference between standard configparser and configupdater parsing.

Please provide a minimal, self-contained, and reproducible example.

Given this INI file:

# example.ini
[[]]

Using configparser:

import configparser

config = configparser.ConfigParser()
config.read('example.ini')
sections = config.sections()
print(sections)  # => ['[]']

Using configupdater:

from configupdater import ConfigUpdater

updater = ConfigUpdater()
updater.read("example.ini")
sections = updater.sections()
print(sections)  # => ['[']

Please provide the full traceback.
N/A

Please provide any additional information below.
It seems it's interpreted as being a comment.
I'll try to open a PR if I can figure it ouy.

Versions and main components

  • ConfigUpdater Version: master
  • Python Version: 3.11.5
  • Operating system: Linux
  • How did you install ConfigUpdater: pip

Empty lines in file are not added as Space object

Hi!
Thank you for your ConfigUpdater!

I am having an issue with lines/spaces when reading my .ini file.

"The basic idea of ConfigUpdater is that a configuration file consists of
three kinds of building blocks: sections, comments and spaces for separation."

May I ask, what are lines? Are they only empty lines in comments or key values?

My issue is that when I read my .ini file, the empty lines are not added to the structure as a Space object (as I believe they should). Am I doing something wrong in my example?

>>> test = ConfigUpdater()
>>> test.read('test.ini')
>>> test.structure
[<Section: test>]
>>> test.add_section('test2')
>>> test.structure
[<Section: test>, <Section: test2>]
>>> test['test2'].add_before.space()
<configupdater.configupdater.BlockBuilder object at 0x7fe8f1875668>
>>> test.structure
[<Section: test>, , <Section: test2>]
>>> test.update_file()
>>> test.read('test.ini')
>>> test.structure
[<Section: test>, <Section: test2>]

~ $ cat test.ini
[test]
key1 = one
key2 = two

[test2]

Indentation of section name isn't always preserved

Description of your problem

Please provide a minimal, self-contained, and reproducible example.

import configupdater

cfg = """
[You can use comments]
# like this
; or this

# By default only in an empty line.
# Inline comments can be harmful because they prevent users
# from using the delimiting characters as parts of values.
# That being said, this can be customized.

    [Sections Can Be Indented]
        can_values_be_as_well = True
        does_that_mean_anything_special = False
        purpose = formatting for readability
        multiline_values = are
            handled just fine as
            long as they are indented
            deeper than the first line
            of a value
        # Did I mention we can indent comments, too?
"""


updater = configupdater.ConfigUpdater()
updater.read_string(cfg)
print(updater)

Please provide the full traceback.
N/A

Please provide any additional information below.
Hi!

So I was comparing configupdater against configparser using some examples from the documentation and I noticed the above section lose it's indentation when parsed. It results with:

[You can use comments]
# like this
; or this

# By default only in an empty line.
# Inline comments can be harmful because they prevent users
# from using the delimiting characters as parts of values.
# That being said, this can be customized.

[Sections Can Be Indented]
        can_values_be_as_well = True
        does_that_mean_anything_special = False
        purpose = formatting for readability
        multiline_values = are
            handled just fine as
            long as they are indented
            deeper than the first line
            of a value
        # Did I mention we can indent comments, too?

It happens because the [You can use comments] section is empty (except for the comments).

Not a big deal, though. ;)

Versions and main components

  • ConfigUpdater Version: 3.1
  • Python Version: 3.10.5
  • Operating system: Arch Linux
  • How did you install ConfigUpdater: pip

SyntaxError: invalid syntax

Hi!
I'm trying to use ConfigUpdater.
Just doing import ConfigUpdater yet raises a SyntaxError: invalid syntax pointing to *, at line 546, in Python2 as well as Python3.

image

Any idea what I'm doing wrong and how to fix it?

Version on pypi seems out of date

The version of configupdater on pypi seems to be significantly older than the code in this repository, most obviously the current pypi version will not allow the input Options.set_values(โ€ฆ prepend_newline=False).

Are we awaiting an update, or is there some issue?

Handle unindented comments in multi-line values

Description of your problem

Following config is accepted by ConfigParser but not by Configupdater:

[header]
install_requires =
    importlib-metadata; python_version<"3.8"
#Adding some comments here that are perfectly valid.
    some-other-dependency

Please provide the full traceback.

self = <Parser: {'allow_no_value': False, 'delimiters': ('=', ':'), 'comment_prefixes': ('#', ';'), 'inline_comment_prefixes': (), 'strict': True, 'empty_lines_in_values': True, 'space_around_delimiters': True}>
line = '    some-other-dependency\n'

    def _add_option_line(self, line: str):
        last_section = self._last_block
        if not isinstance(last_section, Section):  # pragma: no cover
            msg = f"{last_section!r} should be Section"
            raise InconsistentStateError(msg, self._fpname, self._lineno, line)
        # if empty_lines_in_values is true, we later will merge options and whitespace
        # (in the _check_values_with_blank_lines function called at the end).
        # This allows option values to have empty new lines inside them
        # So for now we can add parts of option values to Space nodes, than we check if
        # that is an error or not.
        last_option = last_section.last_block
        if not isinstance(last_option, (Option, Space)):  # pragma: no cover
            msg = f"{last_option!r} should be Option or Space"
>           raise InconsistentStateError(msg, self._fpname, self._lineno, line)
E           configupdater.parser.InconsistentStateError: <Comment: '#Adding some comments here that are perfectly valid.\n'> should be Option or Space
E           <string>(5): '    some-other-dependency\n'

../../.mambaforge/envs/pyscaffold/lib/python3.9/site-packages/configupdater/parser.py:364: InconsistentStateError

Please provide any additional information below.

This was first found as an update issue of PyScaffold 4.1.

Versions and main components

  • ConfigUpdater Version: 3.0
  • Python Version: 3.7
  • Operating system: Linux
  • How did you install ConfigUpdater: (pip)

Problems with inline/indented comments in sections

Hello, I am just using this issue to document some limitations of ConfigUpdater.
Maybe if they cannot be fixed, it would be nice to add them to the docs?

No support for indented comment on beginning of a section

>>> from configupdater import ConfigUpdater
>>> cu = ConfigUpdater()

>>> sample1 = """
... [section1]
...     # indented comment
... a = 42
... """

>>> cu.load_string(sample1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/abravalheri/projects/pyscaffold/.venv/lib/python3.8/site-packages/configupdater/configupdater.py", line 732, in read_string
    self.read_file(sfile, source)
  File "/home/abravalheri/projects/pyscaffold/.venv/lib/python3.8/site-packages/configupdater/configupdater.py", line 722, in read_file
    self._read(f, source)
  File "/home/abravalheri/projects/pyscaffold/.venv/lib/python3.8/site-packages/configupdater/configupdater.py", line 912, in _read
    raise e
configparser.ParsingError: Source contains parsing errors: '<string>'
        [line  3]: '   # indented comment\n'

Here ConfigParser seems to handle the case fine.

>>> from configparser import ConfigParser
>>> cp = ConfigParser() 
>>> cp.load_string(sample1)
>>> cp['section1']['a']
42

Inconsistent behaviour on inline comments for section titles

>>> sample2 = """
... [section2]  # inline comment
... a = 42
... """

>>> cu.load_string(sample2)
>>> print(cu)
[section2]  # inline comment
a = 42

>>> cu['section2'].name = 'different'
>>> print(cu)
[different]
a = 42

Here the comment is gone once we change the title of the section.

Versions and main components

  • ConfigUpdater Version: 2.0
  • Python Version: 3.8
  • How did you install ConfigUpdater: pip

Section comments are bound to the preceding section rather than the following

Description of your problem

Please provide a minimal, self-contained, and reproducible example.

from configupdater import ConfigUpdater

INPUT = '''\
[section1]
sec1key = SEC1VAL
'''

OUTPUT = '''\
[section1]
sec1key = SEC1VAL

[section2]
sec2key = SEC2VAL

# section3 comment
[section3]
sec3key = SEC3VAL
'''

def test_configupdater():
    conf = ConfigUpdater()
    conf.read_string(INPUT)

    (conf["section1"]
     .add_after
     .comment("section3 comment")
     .section("section3"))
    conf["section3"]["sec3key"] = "SEC3VAL"

    (conf["section3"]
     .add_before
     .section("section2")
     .space(2))
    conf["section2"]["sec2key"] = "SEC2VAL"

    assert str(conf) == OUTPUT

Please provide the full traceback.

$ pytest -rA -vv tests/test_configupdater.py
=================================== test session starts ===================================
platform linux -- Python 3.11.3, pytest-7.4.0, pluggy-1.1.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /home/intelfx/devel/proj/build.py
configfile: pyproject.toml
plugins: anyio-3.7.1
collected 1 item

tests/test_configupdater.py::test_configupdater FAILED                              [100%]

======================================== FAILURES =========================================
___________________________________ test_configupdater ____________________________________

    def test_configupdater():
        conf = ConfigUpdater()
        conf.read_string(INPUT)

        (conf["section1"]
         .add_after
         .comment("section3 comment")
         .section("section3"))
        conf["section3"]["sec3key"] = "SEC3VAL"

        (conf["section3"]
         .add_before
         .section("section2")
         .space(2))
        conf["section2"]["sec2key"] = "SEC2VAL"

>       assert str(conf) == OUTPUT
E       AssertionError: assert '[section1]\nsec1key = SEC1VAL\n# section3 comment\n[section2]\nsec2key = SEC2VAL\n\n\n[section3]\nsec3key = SEC3VAL\n' == '[section1]\nsec1key = SEC1VAL\n\n[section2]\nsec2key = SEC2VAL\n\n# section3 comment\n[section3]\nsec3key = SEC3VAL\n'
E           [section1]
E           sec1key = SEC1VAL
E         -
E         + # section3 comment
E           [section2]
E           sec2key = SEC2VAL
E
E         - # section3 comment
E         +
E           [section3]
E           sec3key = SEC3VAL

tests/test_configupdater.py:38: AssertionError
================================= short test summary info =================================
FAILED tests/test_configupdater.py::test_configupdater - AssertionError: assert '[section1]\nsec1key = SEC1VAL\n# section3 comment\n[section2]\...
==================================== 1 failed in 0.06s ====================================

Please provide any additional information below.

This report might be seen as an RFE rather than a bug, since the expected behavior was never guaranteed in the documentation. However, this example implies that ConfigUpdater's authors recognize a comment immediately preceding a section as a comment pertaining to that section. As such, the principle of least surprise is probably being broken here.

Versions and main components

  • ConfigUpdater Version: python-configupdater 3.1.1
  • Python Version: Python 3.11.3
  • Operating system: Arch
  • How did you install ConfigUpdater: distribution package

Allowing sections/options to be copied from one document to the other

Right now it is not possible to simply copy a section or option object from one container to another.
When we try to add the copied object to the other container a AlreadyAttechedError is raised.

This is a symptom of the fact that every block contains a reference to their containers and when they are copied, the copy reference keeps pointing to the previous container.

I think copies should always be created in a "detached" mode, i.e., the reference to the parent container should be None.
To solve this problem we also have to rely on deepcopy, because when copying a section, the references inside its options should also be clear (in my understanding it makes very little sense to implement shallow copies in ConfigUpdater).

Add unittests setting a custom optionxform

Describe your use-case

Since the implementation of optionxform changed quite a bit, it could make sense to test it more thoroughly in the unit tests by choosing a custom optionxform.

Create an attribute linesep

Since PyScaffold uses internally \n for line separation in its templates it makes sense that linesep is configurable in PyScaffold. It should be by default os.linesep but can be changed during initialization.

Add support for "global" properties (before any section)?

Describe your use-case

Hi again.

I implemented a library on top of configupdater and someone reported it didn't work with EditorConfig configuration files. This is because such file can contain "global" properties which declared before any section:

root = true

[section]
foo = bar
baz = 42

There are no definitive specifications on the acceptable syntax of an INI file. In that respect, configupdater is consistent with configparser since both reject such file and raise MissingSectionHeaderError error.

Given that there are certain variants of the INI format that accept global parameters (also Apache), what do you think about extending configupdater support to such files?

Describe the solution you would like to see for your use-case

I imagine a new option to the ConfigUpdater class, such as allow_global_parameters defaulting to False. If True, the global parameters would be added to a section with None name or something like that.

I can try to open a PR if you're interested with such feature (don't know how technically feasible it is yet).

Otherwise, I'll just hack a workaround in my own library. But since it may be beneficial to others, I wanted to discuss it upstream first.

For Options, add a flag to not add the seperator + indent on the first item in a multi-line value

Describe your use-case

I am writing a Python script that parses the typical pylintrc file. In particular, this script modifies the ignore option under the MASTER section of the pylintrc file. You can view the file here if you would like.

The ignore option can be either in-line or multi-line. My script essentially checks if a particular name is in the value and if not adds it to the value. I detect if the option is a multi-line value by try/except for the AssignMultilineValueError. When the AssignMultilineValueError occurs, I am appending the value.

What bothered me enough to make this issue feature request is the following. Say I have the following pylintrc file setting below:

ignore=some_value1,
       some_value2

If my script modifies this setting to add the value some_value3, I get the following result:

ignore=
       some_value1,
       some_value2,
       some_value3

As you can see, the first item in the list is appended with the indent and separator. This is due to this line of code.

Describe the solution you would like to see for your use-case

Going off of the sample above, it would be nice if there were some way to not append the indent and separator to the first element in the list. The ideal result for the pylintrc file above, would be that when adding some_value3, I get the following result:

ignore=some_value1,
       some_value2,
       some_value3

This was achievable locally by modifying this line of code to be the following:

self._value = values[0] + separator.join(values[1:])

This solution seems to work even if the values: Iterable[str] provided to the set_values function is an empty list or a list with one element. In either case, it seems the empty string is being added to the front of the values list.

This change would likely need to be a new boolean option that defaults to False to preserve the original functionality for any backward compatibility. I have read the CONTRIBUTING and would be more than happy to assist in this change if it is accepted. I wanted to start a discussion to see if this idea had any potential of being accepted and if so, what the best way to implement it would be.

Rework `container` object of blocks

Check if it would make more sense to have the container object be the parent not the structure. This would simplify the passing of optionxform (no longer in constructor but direct ref) and make it possible to implement insert_at(...) in the section block more easily.

Dependency description for setup.py is not automatically resolvable

Installing configupdater through pip (e.g. in a virtualenv) doesn't automatically cause a good version of setuptools to be installed but only errors out after installation through pip if setuptools isn't new enough (can happen on older systems and does happen on Travis for me):

require('setuptools>=38.3')

I'd rather use setup(..., install_requires=['setuptools>=38.3']) but it looks like the pyscaffold thing is doing something very meta - but apparently that breaks regular installation patterns.

How do I obtain a list of options for a section?

I've looked over the documentation as well as some of the source code but I don't see how to obtain a list of options within a section. Is this possible?

Versions and main components

  • ConfigUpdater Version:
  • Python Version: 3.1.1
  • Operating system: Debian
  • How did you install ConfigUpdater: pip

Extend the Sphinx docs by a page about the concepts of ConfigUpdater

Describe your use-case

Right now many features of ConfigUpdater are only explained by some examples in Spinx Usage page and thus it's cumbersome for new users to really understand how to use ConfigUpdater.

Describe the solution you would like to see for your use-case

Add a Sphinx page that explains the concepts of ConfigUpdater and which methods should be used in which context. The concept is basically that there are Container objects holding Block objects. ConfigUpdater itself is a Container and also Section. Option, Space, Comment but also Section are Block objects.

Now there are several methods for:

  1. navigation, e.g. first/last_block of a container or next/previous_block of a block
  2. creation, e.g. add_before/after.comment/option/space
  3. removal, e.g. block.remove
  4. inspection, e.g. option_blocks
  5. etc.

3.1: sphinx warnings

Looks like lates sphinx shows some warning

+ /usr/bin/python3 setup.py build_sphinx -b man --build-dir build/sphinx
running build_sphinx
Running Sphinx v4.5.0
Creating file /home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/api/configupdater.rst.
Creating file /home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/api/modules.rst.
making output directory... done
[autosummary] generating autosummary for: api/configupdater.rst, api/modules.rst, authors.rst, changelog.rst, contributing.rst, index.rst, license.rst, usage.rst
building [mo]: targets for 0 po files that are out of date
building [man]: all manpages
updating environment: [new config] 8 added, 0 changed, 0 removed
reading sources... [100%] usage
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/option.py:docstring of configupdater.configupdater.Option.key:1: WARNING: duplicate object description of configupdater.configupdater.Option.key, other instance in api/configupdater, use :noindex: for one of them
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/option.py:docstring of configupdater.configupdater.Option.updated:1: WARNING: duplicate object description of configupdater.configupdater.Option.updated, other instance in api/configupdater, use :noindex: for one of them
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/option.py:docstring of configupdater.configupdater.Option.value:1: WARNING: duplicate object description of configupdater.configupdater.Option.value, other instance in api/configupdater, use :noindex: for one of them
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/section.py:docstring of configupdater.configupdater.Section.name:1: WARNING: duplicate object description of configupdater.configupdater.Section.name, other instance in api/configupdater, use :noindex: for one of them
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/section.py:docstring of configupdater.configupdater.Section.updated:1: WARNING: duplicate object description of configupdater.configupdater.Section.updated, other instance in api/configupdater, use :noindex: for one of them
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/option.py:docstring of configupdater.option.Option.key:1: WARNING: duplicate object description of configupdater.option.Option.key, other instance in api/configupdater, use :noindex: for one of them
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/option.py:docstring of configupdater.option.Option.updated:1: WARNING: duplicate object description of configupdater.option.Option.updated, other instance in api/configupdater, use :noindex: for one of them
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/option.py:docstring of configupdater.option.Option.value:1: WARNING: duplicate object description of configupdater.option.Option.value, other instance in api/configupdater, use :noindex: for one of them
/usr/lib64/python3.8/configparser.py:docstring of configupdater.parser.MissingSectionHeaderError.filename:1: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/usr/lib64/python3.8/configparser.py:docstring of configupdater.parser.ParsingError.filename:1: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/section.py:docstring of configupdater.section.Section.name:1: WARNING: duplicate object description of configupdater.section.Section.name, other instance in api/configupdater, use :noindex: for one of them
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/section.py:docstring of configupdater.section.Section.updated:1: WARNING: duplicate object description of configupdater.section.Section.updated, other instance in api/configupdater, use :noindex: for one of them
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
writing... python-configupdater.3 { usage contributing license authors changelog api/modules api/configupdater } /home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/builder.py:docstring of configupdater.builder.BlockBuilder.section:: WARNING: more than one target found for cross-reference 'Section': configupdater.configupdater.Section, configupdater.section.Section
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/option.py:docstring of configupdater.option.Option:: WARNING: more than one target found for cross-reference 'Section': configupdater.configupdater.Section, configupdater.section.Section
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/option.py:docstring of configupdater.option.Option.section:: WARNING: more than one target found for cross-reference 'Section': configupdater.configupdater.Section, configupdater.section.Section
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/section.py:docstring of configupdater.section.Section.add_option:: WARNING: more than one target found for cross-reference 'Option': configupdater.configupdater.Option, configupdater.option.Option
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/section.py:docstring of configupdater.section.Section.get:: WARNING: more than one target found for cross-reference 'Option': configupdater.configupdater.Option, configupdater.option.Option
/home/tkloczko/rpmbuild/BUILD/configupdater-3.1/docs/../src/configupdater/section.py:docstring of configupdater.section.Section.get:: WARNING: more than one target found for cross-reference 'Option': configupdater.configupdater.Option, configupdater.option.Option
done
build succeeded, 18 warnings.

How to wrap values in quotes?

I need to make config file like this:

[common]
skipSaveTrainName = "true"
vendor = "Delphi"
skipCheckSignatureAndVariant = "true"
region = "Europe"
region2 = "RoW"
region3 = "USA"

As you can see values in quotes. When I create it with ConfigUpdater I get:

[common]
skipSaveTrainName = true
vendor = Delphi
skipCheckSignatureAndVariant = true
region = Europe
region2 = RoW
region3 = USA

How can I add quotes to values?

Case of options is lost when using file

Hi,
The section Feature in the doc says :

  • the original case of sections and keys are kept,

That's no true for keys (option in the code).
When reading file, the case of all keys is changed to lower case (line 789).

Removing all the calls to optionxform() in configupdater.py seems to do the job.
They are 6 of them :
configupdater.py:386: # option = self._container.optionxform(option)
configupdater.py:789: # optname = self.optionxform(optname.rstrip())
configupdater.py:972: # option = self.optionxform(option)
configupdater.py:1012: #option = self.optionxform(option)
configupdater.py:1027: # option = self.optionxform(option)
configupdater.py:1048: # option = self.optionxform(option)

Another way would be to transform optionxform() in a do nothing function (maybe with an option in configupdater.__init__())
configupdater.py:639: def optionxform(self, optionstr):

I haven't tested updating or writing file yet.

Regards

Indentation in multiline values is lost when modifying `value`

Let the contents of the file setup.cfg be as follows:

[metadata]
classifiers =
    Development Status :: 3 - Alpha
    #Development Status :: 4 - Beta
    #Development Status :: 5 - Production/Stable
    Programming Language :: Python :: 3 :: Only
    Programming Language :: Python :: 3
    Programming Language :: Python :: Implementation :: CPython
    Programming Language :: Python :: Implementation :: PyPy
    License :: OSI Approved :: MIT License

Then:

>>> from configupdater import ConfigUpdater
>>> cfg = ConfigUpdater().read("setup.cfg")
>>> print(str(cfg))
[metadata]
classifiers =
    Development Status :: 3 - Alpha
    #Development Status :: 4 - Beta
    #Development Status :: 5 - Production/Stable
    Programming Language :: Python :: 3 :: Only
    Programming Language :: Python :: 3
    Programming Language :: Python :: Implementation :: CPython
    Programming Language :: Python :: Implementation :: PyPy
    License :: OSI Approved :: MIT License

>>> print(cfg["metadata"]["classifiers"].value)

Development Status :: 3 - Alpha
#Development Status :: 4 - Beta
#Development Status :: 5 - Production/Stable
Programming Language :: Python :: 3 :: Only
Programming Language :: Python :: 3
Programming Language :: Python :: Implementation :: CPython
Programming Language :: Python :: Implementation :: PyPy
License :: OSI Approved :: MIT License
>>> cfg["metadata"]["classifiers"].value += "\nTopic :: Utilities"
>>> print(cfg["metadata"]["classifiers"].value)

Development Status :: 3 - Alpha
#Development Status :: 4 - Beta
#Development Status :: 5 - Production/Stable
Programming Language :: Python :: 3 :: Only
Programming Language :: Python :: 3
Programming Language :: Python :: Implementation :: CPython
Programming Language :: Python :: Implementation :: PyPy
License :: OSI Approved :: MIT License
Topic :: Utilities
>>> print(str(cfg))
[metadata]
classifiers =
Development Status :: 3 - Alpha
#Development Status :: 4 - Beta
#Development Status :: 5 - Production/Stable
Programming Language :: Python :: 3 :: Only
Programming Language :: Python :: 3
Programming Language :: Python :: Implementation :: CPython
Programming Language :: Python :: Implementation :: PyPy
License :: OSI Approved :: MIT License
Topic :: Utilities

>>> cfg.update_file()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jwodder/.local/virtualenvwrapper/venvs/tmp-756f8f491897ace/lib/python3.9/site-packages/configupdater/configupdater.py", line 180, in update_file
    self.validate_format()
  File "/Users/jwodder/.local/virtualenvwrapper/venvs/tmp-756f8f491897ace/lib/python3.9/site-packages/configupdater/configupdater.py", line 195, in validate_format
    return super().validate_format(**{**self._parser_opts, **kwargs})
  File "/Users/jwodder/.local/virtualenvwrapper/venvs/tmp-756f8f491897ace/lib/python3.9/site-packages/configupdater/document.py", line 103, in validate_format
    parser.read_string(str(self))
  File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/configparser.py", line 723, in read_string
    self.read_file(sfile, source)
  File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/configparser.py", line 718, in read_file
    self._read(f, source)
  File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/configparser.py", line 1096, in _read
    raise DuplicateOptionError(sectname, optname,
configparser.DuplicateOptionError: While reading from '<string>' [line  7]: option 'programming language' in section 'metadata' already exists

Observe that the option's value is returned with leading indentation stripped, though something remembers that the indentation is there, as seen when stringifying cfg. However, when the value is modified, all memory of the indentation is lost, leading to a validation error when trying to save the updated file.

Versions and main components

  • ConfigUpdater Version: 3.0.1
  • Python Version: 3.9.10
  • Operating system: macOS 11.6
  • How did you install ConfigUpdater: pip

Extend examples in Sphinx docs

Describe your use-case

Right now the usage Sphinx docs show only a few examples. This makes it difficult to really understand ConfigUpdater.

Describe the solution you would like to see for your use-case

Extend the examples to more typical use-cases.

Error on parsing options with comments

Error on parsing options with comments

Thanks for your work in this lib, I've just found some errors when we have comments in our options. See the example bellow:

setup.cfg:

[flake8]

exclude =
  # Trash and cache:
  .git
  __pycache__
  .venv
  .eggs
  *.egg
  temp
  # Bad code that I write to test things:
  ex.py
>>> from configupdater import ConfigUpdater
>>> updater = ConfigUpdater()
>>> updater.read('/lambdas/setup.cfg')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/.cache/pypoetry/virtualenvs/lambdas-swHUnyqi-py3.9/lib/python3.9/site-packages/configupdater/configupdater.py", line 618, in read
    self._read(fp, filename)
  File "/.cache/pypoetry/virtualenvs/lambdas-swHUnyqi-py3.9/lib/python3.9/site-packages/configupdater/configupdater.py", line 826, in _read
    raise e
configparser.ParsingError: Source contains parsing errors: '/lambdas/setup.cfg'
	[line 10]: '  lambdas/contrib/mypy/*\n'
	[line 39]: '  .git\n'
	[line 40]: '  __pycache__\n'
	[line 41]: '  .venv\n'
	[line 42]: '  .eggs\n'
	[line 43]: '  *.egg\n'
	[line 44]: '  temp\n'
	[line 46]: '  ex.py\n'

Other error I've found, sometimes we didn't get the ParsingError with comments but our options is parsed wrongly. See the example below:

setup.cfg:

[flake8]

exclude =
  .git
  __pycache__
  .venv
  .eggs
  *.egg
  temp
  ex.py

per-file-ignores =
  # Disable imports in `__init__.py`:
  lambdas/__init__.py: WPS226, WPS413
  lambdas/contrib/mypy/lambdas_plugin.py: WPS437
  # There are multiple assert's in tests:
  tests/*.py: S101, WPS226, WPS432, WPS436, WPS450
  # We need to write tests to our private class:
  tests/test_math_expression/*.py: S101, WPS432, WPS450
>>> from configupdater import ConfigUpdater
>>>
>>> updater = ConfigUpdater()
>>> updater.read('/Projects/lambdas/setup.cfg')
>>>
>>> per_file_ignores = updater['flake8']['per-file-ignores']
>>> per_file_ignores
<Option: per-file-ignores = >
>>> per_file_ignores.lines
['per-file-ignores =\n']
>>>
>>> # While `exclude` is fine
>>>
>>> exclude = updater['flake8']['exclude']
>>> exclude
<Option: exclude = 
.git
__pycache__
.venv
.eggs
*.egg
temp
ex.py>
>>> exclude.lines
['exclude =\n', '  .git\n', '  __pycache__\n', '  .venv\n', '  .eggs\n', '  *.egg\n', '  temp\n', '  ex.py\n']

Please provide any additional information below.

Refs andreoliwa/nitpick#296

Versions and main components

  • ConfigUpdater Version: 1.1.3
  • Python Version: 3.9
  • Operating system: Linux
  • How did you install ConfigUpdater: pip

Logo proposal ๐Ÿ˜

During the weekend, I was playing around with ConfigUpdater so I decided to draw something:

configupdater

It would be nice once we update the docs (maybe even using the same theme as PyScaffold uses now), to have a nice image.

I have used the opensource Monoid typeface (Andreas Larsen and contributors, 2015).
The colors used are:

accent: #008c70 (teal-ish)
pure black: #000000
pure white: #FFFFFF
30% gray: #b3b3b3 (box shadow)

I have the SVG files available here if that is interesting for the project ๐Ÿ˜„

ConfigUpdater.get returns string literal if fallback provided

Description of your problem

The output of the get() method is a different type depending on whether or not it returns the fallback option, meaning that it cannot be reliably assumed to be an object of type Option as expected.

import configupdater
conf = configupdater.ConfigUpdater()
conf.read("config.ini")
example1 = conf.get("section", "existingKey", "fallback").value
print(example1)
conf["section"].setdefault("nonexistantKey", "some value")
example2 = conf.get("section", "nonexistantKey", "fallback").value
print(example2)
example3 = conf.get("section", "nonexistantKey2", "fallback").value
print(example3)

Output:

value of key "existingKey"
some value
Traceback (most recent call last):
  File "example.py", line 9, in <module>
    example3 = conf.get("section", "nonexistantKey2", "fallback").value
AttributeError: 'str' object has no attribute 'value'

Please provide any additional information below.
According to this documentation, the get method should return an object of type Option. However, if a fallback value is provided, the literal value of this fallback is returned. I would expect the behaviour of examples 2 and 3 to be the same, where if I provide a fallback value then it will be treated as an option in the respective section and returned as such. The ability to provide a fallback is pointless since I still have to check the output and do something different depending on whether or not the key was found in that section

Versions and main components

  • ConfigUpdater Version: 3.1.1
  • Python Version: 3.10.12
  • Operating system: Ubuntu 22.04
  • How did you install ConfigUpdater: pip

Recheck the whole API for consistency

"There are only two hard things in Computer Science: cache invalidation and naming things". Since ConfigUpdater doesn't make use of the former, we should focus on the latter ;-)
There is surely potential to improve here, e.g.

  1. Container has a structure holding Block objects. Should it be rather named blocks to make it clearer?
  2. From ConfigParser we "inherited" several methods containing the word items but on the other hand we also have many blocks methods. Is this always consistent and do items only return strings and values like in a dictionary versus Block objects in case of blocks methods?
  3. ...

Method whereby you can convert sections into dictionary structure

Describe your use-case

As I have with using ConfigParser, I can take a section and convert it to a dictionary. Then I pass this dict value to different functions within my program.

Describe the solution you would like to see for your use-case

I can retrieve a list of keys via list(updater[section].keys() but am unable to enumerate the values without getting the "<Option:" string for each value.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.