thombashi / pytablewriter

pytablewriter is a Python library to write a table in various formats: AsciiDoc / CSV / Elasticsearch / HTML / JavaScript / JSON / LaTeX / LDJSON / LTSV / Markdown / MediaWiki / NumPy / Excel / Pandas / Python / reStructuredText / SQLite / TOML / TSV.

pytablewriter

Summary

pytablewriter is a Python library to write a table in various formats: AsciiDoc / CSV / Elasticsearch / HTML / JavaScript / JSON / LaTeX / LDJSON / LTSV / Markdown / MediaWiki / NumPy / Excel / Pandas / Python / reStructuredText / SQLite / TOML / TSV / YAML.

Features

• Write a table in various formats:

  • Text formats:

    • AsciiDoc
    • CSV / Tab-separated values (TSV) / Space-separated values (SSV)
    • HTML / CSS
    • JSON / Line-delimited JSON(LDJSON)
    • Labeled Tab-separated Values (LTSV)
    • LaTeX: tabular/array environment
    • Markdown: CommonMark / GitHub Flavored Markdown (GFM) / kramdown
    • MediaWiki
    • reStructuredText: Grid Tables/Simple Tables/CSV Table

    • Source code (definition of a variable that represents tabular data)

      • JavaScript / NumPy (numpy.array) / Pandas (pandas.DataFrame) / Python

    • TOML
    • YAML
    • Unicode

  • Binary file formats:

    • Microsoft Excel ^TM (.xlsx/.xls file format)
    • pandas.DataFrame pickle file
    • SQLite database

  • Application-specific formats:

    • Elasticsearch

• Automatic table cell formatting:

  • Alignment
  • Padding
  • Decimal places of numbers

• Customize table cell styles:

  • Text/Background color
  • Text alignment
  • Font size/weight
  • Thousand separator for numbers: e.g. 1,000/1 000

• Configure output:

  • Write a table to a stream such as a file/standard-output/string-buffer/Jupyter-Notebook
  • Get rendered tabular text

• Data sources:

  • nested list
  • CSV
  • pandas.DataFrame / pandas.Series
  • etc.

• Multibyte character support
• ANSI color support

Installation

Installation: pip

pip install pytablewriter

Some of the formats require additional dependency packages, you can install these packages as follows:

Installation of optional dependencies

Installation example	Remark
pip install pytablewriter[es]	Elasticsearch
pip install pytablewriter[excel]	Excel
pip install pytablewriter[html]	HTML
pip install pytablewriter[sqlite]	SQLite database
pip install pytablewriter[toml]	TOML
pip install pytablewriter[theme]	pytablewriter theme plugins
pip install pytablewriter[all]	Install all of the optional dependencies

Installation: conda

conda install -c conda-forge pytablewriter

Installation: apt

sudo add-apt-repository ppa:thombashi/ppa
sudo apt update
sudo apt install python3-pytablewriter

Examples

Write tables

Write a Markdown table

Sample Code

from pytablewriter import MarkdownTableWriter

def main():
    writer = MarkdownTableWriter(
        table_name="example_table",
        headers=["int", "float", "str", "bool", "mix", "time"],
        value_matrix=[
            [0,   0.1,      "hoge", True,   0,      "2017-01-01 03:04:05+0900"],
            [2,   "-2.23",  "foo",  False,  None,   "2017-12-23 45:01:23+0900"],
            [3,   0,        "bar",  "true",  "inf", "2017-03-03 33:44:55+0900"],
            [-10, -9.9,     "",     "FALSE", "nan", "2017-01-01 00:00:00+0900"],
        ],
    )
    writer.write_table()

if __name__ == "__main__":
    main()

Output

# example_table
|int|float|str |bool |  mix   |          time          |
|--:|----:|----|-----|-------:|------------------------|
|  0| 0.10|hoge|True |       0|2017-01-01 03:04:05+0900|
|  2|-2.23|foo |False|        |2017-12-23 12:34:51+0900|
|  3| 0.00|bar |True |Infinity|2017-03-03 22:44:55+0900|
|-10|-9.90|    |False|     NaN|2017-01-01 00:00:00+0900|

Rendering Result

Rendered markdown at GitHub

Write a Markdown table with margins

Sample Code

from pytablewriter import MarkdownTableWriter

def main():
    writer = MarkdownTableWriter(
        table_name="write a table with margins",
        headers=["int", "float", "str", "bool", "mix", "time"],
        value_matrix=[
            [0,   0.1,      "hoge", True,   0,      "2017-01-01 03:04:05+0900"],
            [2,   "-2.23",  "foo",  False,  None,   "2017-12-23 45:01:23+0900"],
            [3,   0,        "bar",  "true",  "inf", "2017-03-03 33:44:55+0900"],
            [-10, -9.9,     "",     "FALSE", "nan", "2017-01-01 00:00:00+0900"],
        ],
        margin=1  # add a whitespace for both sides of each cell
    )
    writer.write_table()

if __name__ == "__main__":
    main()

Output

# write a table with margins
| int | float | str  | bool  |   mix    |           time           |
| --: | ----: | ---- | ----- | -------: | ------------------------ |
|   0 |  0.10 | hoge | True  |        0 | 2017-01-01 03:04:05+0900 |
|   2 | -2.23 | foo  | False |          | 2017-12-23 12:34:51+0900 |
|   3 |  0.00 | bar  | True  | Infinity | 2017-03-03 22:44:55+0900 |
| -10 | -9.90 |      | False |      NaN | 2017-01-01 00:00:00+0900 |

margin attribute can be available for all of the text format writer classes.

Write a GitHub Flavored Markdown (GFM) table

If you set flavor keyword argument of MarkdownTableWriter class to "github" or "gfm", the writer will output markdown tables with GitHub flavor. GFM can apply some additional styles to tables such as fg_color (text color).

Sample Code

from pytablewriter import MarkdownTableWriter
from pytablewriter.style import Style

writer = MarkdownTableWriter(
    column_styles=[
        Style(fg_color="red"),
        Style(fg_color="green", decoration_line="underline"),
    ],
    headers=["A", "B"],
    value_matrix=[
        ["abc", 1],
        ["efg", 2],
    ],
    margin=1,
    flavor="github",
    enable_ansi_escape=False,
)
writer.write_table()

Rendered results can be found at here

Apply styles to GFM table with programmatically

Applying style filters to GFM allows for more flexible style settings for cells. See also the example

Write a Markdown table to a stream or a file

Refer an example

Write a table to an Excel sheet

Sample Code

from pytablewriter import ExcelXlsxTableWriter

def main():
    writer = ExcelXlsxTableWriter()
    writer.table_name = "example"
    writer.headers = ["int", "float", "str", "bool", "mix", "time"]
    writer.value_matrix = [
        [0,   0.1,      "hoge", True,   0,      "2017-01-01 03:04:05+0900"],
        [2,   "-2.23",  "foo",  False,  None,   "2017-12-23 12:34:51+0900"],
        [3,   0,        "bar",  "true",  "inf", "2017-03-03 22:44:55+0900"],
        [-10, -9.9,     "",     "FALSE", "nan", "2017-01-01 00:00:00+0900"],
    ]
    writer.dump("sample.xlsx")

if __name__ == "__main__":
    main()

Output

Output excel file (sample_single.xlsx)

Write a Unicode table

Sample Code

from pytablewriter import UnicodeTableWriter

def main():
    writer = UnicodeTableWriter(
        table_name="example_table",
        headers=["int", "float", "str", "bool", "mix", "time"],
        value_matrix=[
            [0,   0.1,      "hoge", True,   0,      "2017-01-01 03:04:05+0900"],
            [2,   "-2.23",  "foo",  False,  None,   "2017-12-23 45:01:23+0900"],
            [3,   0,        "bar",  "true",  "inf", "2017-03-03 33:44:55+0900"],
            [-10, -9.9,     "",     "FALSE", "nan", "2017-01-01 00:00:00+0900"],
        ]
    )
    writer.write_table()

if __name__ == "__main__":
    main()

Output

┌───┬─────┬────┬─────┬────────┬────────────────────────┐
│int│float│str │bool │  mix   │          time          │
├───┼─────┼────┼─────┼────────┼────────────────────────┤
│  0│ 0.10│hoge│True │       0│2017-01-01 03:04:05+0900│
├───┼─────┼────┼─────┼────────┼────────────────────────┤
│  2│-2.23│foo │False│        │2017-12-23 12:34:51+0900│
├───┼─────┼────┼─────┼────────┼────────────────────────┤
│  3│ 0.00│bar │True │Infinity│2017-03-03 22:44:55+0900│
├───┼─────┼────┼─────┼────────┼────────────────────────┤
│-10│-9.90│    │False│     NaN│2017-01-01 00:00:00+0900│
└───┴─────┴────┴─────┴────────┴────────────────────────┘

Write a table with JavaScript format (as a nested list variable definition)

Sample Code

import pytablewriter as ptw


def main():
    writer = ptw.JavaScriptTableWriter(
        table_name="js_variable",
        headers=["int", "float", "str", "bool", "mix", "time"],
        value_matrix=[
            [0, 0.1, "hoge", True, 0, "2017-01-01 03:04:05+0900"],
            [2, "-2.23", "foo", False, None, "2017-12-23 45:01:23+0900"],
            [3, 0, "bar", "true", "inf", "2017-03-03 33:44:55+0900"],
            [-10, -9.9, "", "FALSE", "nan", "2017-01-01 00:00:00+0900"],
        ],
    )

    writer.write_table()


if __name__ == "__main__":
    main()

Output

const js_variable = [
    ["int", "float", "str", "bool", "mix", "time"],
    [0, 0.1, "hoge", true, 0, "2017-01-01 03:04:05+0900"],
    [2, -2.23, "foo", false, null, "2017-12-23 45:01:23+0900"],
    [3, 0, "bar", true, Infinity, "2017-03-03 33:44:55+0900"],
    [-10, -9.9, "", "FALSE", NaN, "2017-01-01 00:00:00+0900"]
];

Write a Markdown table from pandas.DataFrame instance

from_dataframe method of writer classes will set up tabular data from pandas.DataFrame:

Sample Code

from textwrap import dedent
import pandas as pd
import io
from pytablewriter import MarkdownTableWriter

def main():
    csv_data = io.StringIO(dedent("""\
        "i","f","c","if","ifc","bool","inf","nan","mix_num","time"
        1,1.10,"aa",1.0,"1",True,Infinity,NaN,1,"2017-01-01 00:00:00+09:00"
        2,2.20,"bbb",2.2,"2.2",False,Infinity,NaN,Infinity,"2017-01-02 03:04:05+09:00"
        3,3.33,"cccc",-3.0,"ccc",True,Infinity,NaN,NaN,"2017-01-01 00:00:00+09:00"
        """))
    df = pd.read_csv(csv_data, sep=',')

    writer = MarkdownTableWriter(dataframe=df)
    writer.write_table()

if __name__ == "__main__":
    main()

Output

| i | f  | c  | if |ifc|bool |  inf   |nan|mix_num |          time           |
|--:|---:|----|---:|---|-----|--------|---|-------:|-------------------------|
|  1|1.10|aa  | 1.0|  1|True |Infinity|NaN|       1|2017-01-01 00:00:00+09:00|
|  2|2.20|bbb | 2.2|2.2|False|Infinity|NaN|Infinity|2017-01-02 03:04:05+09:00|
|  3|3.33|cccc|-3.0|ccc|True |Infinity|NaN|     NaN|2017-01-01 00:00:00+09:00|

Adding a column of the DataFrame index if you specify add_index_column=True:

Sample Code

import pandas as pd
import pytablewriter as ptw

def main():
    writer = ptw.MarkdownTableWriter(table_name="add_index_column")
    writer.from_dataframe(
        pd.DataFrame({"A": [1, 2], "B": [10, 11]}, index=["a", "b"]),
        add_index_column=True,
    )
    writer.write_table()

if __name__ == "__main__":
    main()

Output

# add_index_column
|   | A | B |
|---|--:|--:|
|a  |  1| 10|
|b  |  2| 11|

Write a Markdown table from space-separated values

Sample Code

import pytablewriter as ptw


def main():
    writer = ptw.MarkdownTableWriter(table_name="ps")
    writer.from_csv(
        """
        USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
        root         1  0.0  0.4  77664  8784 ?        Ss   May11   0:02 /sbin/init
        root         2  0.0  0.0      0     0 ?        S    May11   0:00 [kthreadd]
        root         4  0.0  0.0      0     0 ?        I<   May11   0:00 [kworker/0:0H]
        root         6  0.0  0.0      0     0 ?        I<   May11   0:00 [mm_percpu_wq]
        root         7  0.0  0.0      0     0 ?        S    May11   0:01 [ksoftirqd/0]
        """,
        delimiter=" ",
    )
    writer.write_table()


if __name__ == "__main__":
    main()

Output

# ps
|USER|PID|%CPU|%MEM| VSZ |RSS |TTY|STAT|START|TIME|   COMMAND    |
|----|--:|---:|---:|----:|---:|---|----|-----|----|--------------|
|root|  1|   0| 0.4|77664|8784|?  |Ss  |May11|0:02|/sbin/init    |
|root|  2|   0| 0.0|    0|   0|?  |S   |May11|0:00|[kthreadd]    |
|root|  4|   0| 0.0|    0|   0|?  |I<  |May11|0:00|[kworker/0:0H]|
|root|  6|   0| 0.0|    0|   0|?  |I<  |May11|0:00|[mm_percpu_wq]|
|root|  7|   0| 0.0|    0|   0|?  |S   |May11|0:01|[ksoftirqd/0] |

Get rendered tabular text as str

dumps method returns rendered tabular text. dumps only available for text format writers.

Sample Code

import pytablewriter as ptw


def main():
    writer = ptw.MarkdownTableWriter(
        headers=["int", "float", "str", "bool", "mix", "time"],
        value_matrix=[
            [0, 0.1, "hoge", True, 0, "2017-01-01 03:04:05+0900"],
            [2, "-2.23", "foo", False, None, "2017-12-23 45:01:23+0900"],
            [3, 0, "bar", "true", "inf", "2017-03-03 33:44:55+0900"],
            [-10, -9.9, "", "FALSE", "nan", "2017-01-01 00:00:00+0900"],
        ],
    )

    print(writer.dumps())


if __name__ == "__main__":
    main()

Output

|int|float|str |bool |  mix   |          time          |
|--:|----:|----|-----|-------:|------------------------|
|  0| 0.10|hoge|True |       0|2017-01-01 03:04:05+0900|
|  2|-2.23|foo |False|        |2017-12-23 45:01:23+0900|
|  3| 0.00|bar |True |Infinity|2017-03-03 33:44:55+0900|
|-10|-9.90|    |False|     NaN|2017-01-01 00:00:00+0900|

Configure table styles

Column styles

Writers can specify Style for each column by column_styles attribute of writer classes.

Sample Code

import pytablewriter as ptw
from pytablewriter.style import Style


def main():
    writer = ptw.MarkdownTableWriter(
        table_name="set style by column_styles",
        headers=[
            "auto align",
            "left align",
            "center align",
            "bold",
            "italic",
            "bold italic ts",
        ],
        value_matrix=[
            [11, 11, 11, 11, 11, 11],
            [1234, 1234, 1234, 1234, 1234, 1234],
        ],
        column_styles=[
            Style(),
            Style(align="left"),
            Style(align="center"),
            Style(font_weight="bold"),
            Style(font_style="italic"),
            Style(font_weight="bold", font_style="italic", thousand_separator=","),
        ],  # specify styles for each column
    )
    writer.write_table()


if __name__ == "__main__":
    main()

Output

# set style by styles
|auto align|left align|center align|  bold  |italic|bold italic ts|
|---------:|----------|:----------:|-------:|-----:|-------------:|
|        11|11        |     11     |  **11**|  _11_|      _**11**_|
|      1234|1234      |    1234    |**1234**|_1234_|   _**1,234**_|

Rendering result

You can also set Style to a specific column with an index or header by using set_style method:

Sample Code

from pytablewriter import MarkdownTableWriter
from pytablewriter.style import Style

def main():
    writer = MarkdownTableWriter()
    writer.headers = ["A", "B", "C",]
    writer.value_matrix = [[11, 11, 11], [1234, 1234, 1234]]

    writer.table_name = "set style by column index"
    writer.set_style(1, Style(align="center", font_weight="bold"))
    writer.set_style(2, Style(thousand_separator=" "))
    writer.write_table()
    writer.write_null_line()

    writer.table_name = "set style by header"
    writer.set_style("B", Style(font_style="italic"))
    writer.write_table()

if __name__ == "__main__":
    main()

Output

# set style by column index
| A  |   B    |  C  |
|---:|:------:|----:|
|  11| **11** |   11|
|1234|**1234**|1 234|

# set style by header
| A  |  B   |  C  |
|---:|-----:|----:|
|  11|  _11_|   11|
|1234|_1234_|1 234|

Style filter

You can apply styles to specific cells by using style filters. Style filters will be written as Python functions. Examples of a style filter function and how you apply it are as follows:

Sample Code

from typing import Any, Optional

from pytablewriter import MarkdownTableWriter
from pytablewriter.style import Cell, Style


def style_filter(cell: Cell, **kwargs: Any) -> Optional[Style]:
    if cell.is_header_row():
        return None

    if cell.col == 0:
        return Style(font_weight="bold")

    value = int(cell.value)

    if value > 80:
        return Style(fg_color="red", font_weight="bold", decoration_line="underline")
    elif value > 50:
        return Style(fg_color="yellow", font_weight="bold")
    elif value > 20:
        return Style(fg_color="green")

    return Style(fg_color="lightblue")


writer = MarkdownTableWriter(
    table_name="style filter example",
    headers=["Key", "Value 1", "Value 2"],
    value_matrix=[
        ["A", 95, 40],
        ["B", 55, 5],
        ["C", 30, 85],
        ["D", 0, 69],
    ],
    flavor="github",
    enable_ansi_escape=False,
)
writer.add_style_filter(style_filter)
writer.write_table()

Rendered results can be found at here

Theme

Theme <https://pytablewriter.readthedocs.io/en/latest/pages/reference/theme.html#pytablewriter.style.Theme> consists of a set of style filters. The following command will install external predefined themes:

pip install pytablewriter[theme]

Themes can be set via the constructor of the writer classes or the set_theme method. The following is an example of setting the altrow theme via the constructor. altrow theme will be colored rows alternatively:

Sample Code

import pytablewriter as ptw

writer = ptw.TableWriterFactory.create_from_format_name(
    "markdown",
    headers=["INT", "STR"],
    value_matrix=[[1, "hoge"], [2, "foo"], [3, "bar"]],
    margin=1,
    theme="altrow",
)
writer.write_table()

Output

[theme] extras includes the following themes:

• pytablewriter-altrow-theme

  • Generated HTML table example

• pytablewriter-altcol-theme

  • Generated HTML table example

Make tables for specific applications

Render a table on Jupyter Notebook

All table writer class instances in pytablewriter can render in Jupyter Notebook. To render writers at notebook cells, you will require the dependency packages to be installed either by:

• pip install pytablewriter[html] or
• pip install pytablewriter[all]

Jupyter Notebook code examples can be found here:

Table rendering results of Jupyter Notebook

Multibyte character support

Write a table using multibyte character

You can use multibyte characters as table data. Multibyte characters are also properly padded and aligned.

Sample Code

import pytablewriter as ptw


def main():
    writer = ptw.RstSimpleTableWriter(
        table_name="生成に関するパターン",
        headers=["パターン名", "概要", "GoF", "Code Complete[1]"],
        value_matrix=[
            ["Abstract Factory", "関連する一連のインスタンスを状況に応じて、適切に生成する方法を提供する。", "Yes", "Yes"],
            ["Builder", "複合化されたインスタンスの生成過程を隠蔽する。", "Yes", "No"],
            ["Factory Method", "実際に生成されるインスタンスに依存しない、インスタンスの生成方法を提供する。", "Yes", "Yes"],
            ["Prototype", "同様のインスタンスを生成するために、原型のインスタンスを複製する。", "Yes", "No"],
            ["Singleton", "あるクラスについて、インスタンスが単一であることを保証する。", "Yes", "Yes"],
        ],
    )
    writer.write_table()


if __name__ == "__main__":
    main()

Output

Output of multi-byte character table

Multiprocessing

You can increase the number of workers to process table data via max_workers attribute of a writer. The more max_workers the less processing time when tabular data is large and the execution environment has available cores.

If you increase max_workers larger than one, recommend using main guarded as follows to avoid problems caused by multi-processing:

from multiprocessing import cpu_count
import pytablewriter as ptw

def main():
    writer = ptw.MarkdownTableWriter()
    writer.max_workers = cpu_count()
    ...

if __name__ == "__main__":
    main()

For more information

More examples are available at https://pytablewriter.rtfd.io/en/latest/pages/examples/index.html

Dependencies

• Python 3.7+
• Python package dependencies (automatically installed)

Optional dependencies

• logging extras

  • loguru: Used for logging if the package installed

• from extras

  • pytablereader

• es extra

  • elasticsearch

• excel extras

  • xlwt
  • XlsxWriter

• html extras

  • dominate

• sqlite extras

  • SimpleSQLite

• theme extras

  • pytablewriter-altrow-theme
  • pytablewriter-altcol-theme

• toml extras

  • toml

Documentation

https://pytablewriter.rtfd.io/

Projects using pytablewriter

• pytest-md-report

Related Projects

• pytablereader

  • Tabular data loaded by pytablereader can be written another tabular data format with pytablewriter.

Sponsors

Become a sponsor

pytablewriter's Issues

Default thousand separator

The default style thousands separator is not behaving as expected.

from pytablewriter import MarkdownTableWriter
from pytablewriter.style import Style


def main():
    table = MarkdownTableWriter(
        headers=("A", "B", "C"),
        value_matrix=[
            (
                "{:,.2%}".format(4000 / 3),
                100000,
                "{:,}".format(100000)
            )
        ],
        default_style=Style(thousand_separator=","),
        margin=1
    ).dumps(flavor="github")

    print(table)


if __name__ == '__main__':
    main()

Actual

|      A      |   B    |   C    |
| ----------- | -----: | -----: |
| 133,333.33% | 100000 | 100000 |

Expected

|      A      |   B     |   C     |
| ----------- | ------: | ------: |
| 133,333.33% | 100,000 | 100,000 |

Markdown output alignment

Greetings!

I am outputting a pandas dataframe to markdown and I fail to see (and find in the documentation) how I can define the alignment of the columns in the markdown table.

I have found some references in the code, so there is some logic to define which columns are left or right aligned, but how to define that when doing the conversion?

Best regards!

MarkdownTableWriter outputs ANSI Escape Sequences

Problem

Creating a MarkdownTableWriter and specifying a column style with font_weight="bold" results in the correct **content** being output, but also the entire column data (including the asterisks) are surrounded by ANSI escape sequences for bold text. The report I'm generating is eventually sent through pandoc which renders the ANSI sequences as ugly \xAB characters in a browser.

Expected output

A normal table cell just with ** wrapped on either side of the data. No ANSI escape sequences.

Steps to reproduce

Python 3.8.3 (default, May 17 2020, 18:15:42)
[GCC 10.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pytablewriter
>>> writer = pytablewriter.MarkdownTableWriter()
>>> writer.headers = ["Test", "Test"]
>>> writer.value_matrix = [("hello", "world"), ("goodbye", "world")]
>>> writer.column_styles = [pytablewriter.style.Style(font_weight="bold"), pytablewriter.style.Style()]
>>> import sys
>>> writer.dump(sys.stdout, close_after_write=False)
|   Test    |Test |
|-----------|-----|
|**hello**  |world|
|**goodbye**|world|

You can't see the ANSI escape sequences above, obviously, but they are there when I just tested it. I also just installed the most recent version from PyPI, and my Python version is seen above in the output.

edit: forgot to mention, this also happens when writing somewhere that is not stdout. My use-case is actually writing to a file through writer.dump. The example above was just a PoC.

Write output as rst field lists?

This is a very good module and it solves a lot of my documentation problems.

I am just wondering, if a I want to turn a JSON into restructuredtext field lists, is it currently do able?

The reason is, for documentation, if it is an HTML, I usually show the JSON with a CSV table (converted from the JSON) to show more info for the user; for PDF (outputed using sphinx with Latex), I've found that rst field lists actually are rendered better than CSV tables (since they can be very long tables).

I will show what I mean.

I basically want to convert this:

into this:

How do I do this?

Fix code scanning alert - Overly permissive regular expression range

Tracking issue for:

• https://github.com/thombashi/pytablewriter/security/code-scanning/3

[MarkDown]A way to reduce the Heading size of the table name

Hello,
I didn't find a way to reduce the heading size of the table name. Can someone suggest how to lower the size? In other words instead the default "#" to be replaced for example with "###"?

Installing optional dependencies: pytablewriter 0.41.1 does not provide the extra 'html'

• macOS High Sierra
• pip 18.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
• Python 3.7.2

https://pytablewriter.readthedocs.io/en/latest/pages/installation.html says:

Some of the formats require additional dependency packages, you can install the dependency packages as follows:

...

• HTML

  • pip install pytablewriter[html]

However, that's not installing the optional dependency (dominate) for me, note the pytablewriter 0.41.1 does not provide the extra 'html':

$ pip install pytablewriter[html]
zsh: no matches found: pytablewriter[html]

$ pip install "pytablewriter[html]"
Collecting pytablewriter[html]
  Using cached https://files.pythonhosted.org/packages/18/20/52dd8b72362cdc5d585f7f8689a432ff34a6b34b90acf014f3add6c70016/pytablewriter-0.41.1-py2.py3-none-any.whl
  pytablewriter 0.41.1 does not provide the extra 'html'
Requirement already satisfied: pathvalidate<1.0.0,>=0.23.1 in /usr/local/lib/python3.7/site-packages (from pytablewriter[html]) (0.23.1)
Requirement already satisfied: setuptools>=38.3.0 in /usr/local/lib/python3.7/site-packages (from pytablewriter[html]) (40.6.3)
Requirement already satisfied: msgfy<0.1.0,>=0.0.4 in /usr/local/lib/python3.7/site-packages (from pytablewriter[html]) (0.0.4)
Requirement already satisfied: mbstrdecoder[all]<1.0.0,>=0.6.2 in /usr/local/lib/python3.7/site-packages (from pytablewriter[html]) (0.6.2)
Requirement already satisfied: typepy[datetime]<1.0.0,>=0.3.3 in /usr/local/lib/python3.7/site-packages (from pytablewriter[html]) (0.3.3)
Requirement already satisfied: tabledata<1.0.0,>=0.3.0 in /usr/local/lib/python3.7/site-packages (from pytablewriter[html]) (0.3.0)
Requirement already satisfied: DataProperty<1.0.0,>=0.38.1 in /usr/local/lib/python3.7/site-packages (from pytablewriter[html]) (0.38.1)
Requirement already satisfied: six<2.0.0,>=1.11.0 in /usr/local/lib/python3.7/site-packages (from pytablewriter[html]) (1.11.0)
Requirement already satisfied: chardet<4.0.0,>=3.0.4; extra == "all" in /usr/local/lib/python3.7/site-packages (from mbstrdecoder[all]<1.0.0,>=0.6.2->pytablewriter[html]) (3.0.4)
Requirement already satisfied: python-dateutil<3.0.0,>=2.7.5; extra == "datetime" in /usr/local/lib/python3.7/site-packages (from typepy[datetime]<1.0.0,>=0.3.3->pytablewriter[html]) (2.7.5)
Requirement already satisfied: pytz>=2018.9; extra == "datetime" in /usr/local/lib/python3.7/site-packages (from typepy[datetime]<1.0.0,>=0.3.3->pytablewriter[html]) (2018.9)
Installing collected packages: pytablewriter
Successfully installed pytablewriter-0.41.1

Am I missing something?

Thank you!

OSError with Python 3.8 (works with 3.6)

Trying to run the example code for generating the Markdown table raises OSError

from pytablewriter import MarkdownTableWriter

writer = MarkdownTableWriter()
writer.table_name = "example_table"
writer.headers = ["int", "float", "str", "bool", "mix", "time"]
writer.value_matrix = [
    [0,   0.1,      "hoge", True,   0,      "2017-01-01 03:04:05+0900"],
    [2,   "-2.23",  "foo",  False,  None,   "2017-12-23 45:01:23+0900"],
    [3,   0,        "bar",  "true",  "inf", "2017-03-03 33:44:55+0900"],
    [-10, -9.9,     "",     "FALSE", "nan", "2017-01-01 00:00:00+0900"],
]

writer.write_table()

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/concurrent/futures/process.py", line 102, in _python_exit
    thread_wakeup.wakeup()
  File "/usr/lib/python3.8/concurrent/futures/process.py", line 90, in wakeup
    self._writer.send_bytes(b"")
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 183, in send_bytes
    self._check_closed()
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

As seen in the traceback, I'm running Python 3.8 with pytablewriter==0.46.1. I've tried versions back to 0.43.0 and it didn't help.

The example works well with Python 3.6 and pytablewriter==0.46.1. My OS is Archlinux.

Will gladly add any more info, feel free to ask.

Support line breaks in HTML

If a table element contains a new line character '\n', this should create a line break in the table output. The HTML table writer currently seems to ignore the line break. I haven't tried other output formats.

Unicode text table output from pandas dataframe: turns string objects into numbers and changes their representation

This brilliant tool is breaking when rendering a pandas table as text.

The data contains very long numbers stored as strings. The strings contain representations of long decimals and long ints with thousands separators. In the Pandas dataframe, they are stored as objects. When I output the table using a pytablewriter unicode writer, it produces a faithful rendering of the long ints, but it seems to process the strings representing long decimals as though they were numbers, and shows them as though they had been converted from strings into floats, with all the problems of string representation that floats bring: unwanted zeros after the last significant decimal digit on short decimals, and precision too short to show the whole number on long decimals.

For example:

"0.000000000000001" is represented by pytablewriter as "0.000000"
"0.001" as "0.001000"

Yet, with long numbers:

"1,000,000,000,000" is represented faithfully as "1,000,000,000,000".

The problem seems to be a general issue with decimals. Notwithstanding the fact that they are held as strings for the very purpose of ensuring their representation is as strings and not numbers, the pytablewriter output table applies different justification to the strings that represent integers and those that represent decimals. The former, it justifies left, the latter it justifies right. So it seems to be treating the strings that contain decimals as though they are numbers, converts them to float and then outputs them as numbers.

It justifies the string "1" to the right with the decimals as well.

DataProperty 0.44.0 breaks pytablewriter

DataProperty 0.44.0 breaks pytablewriter:

2020-02-05 10:27:45,002 - DEBUG -   File "/usr/local/lib/python3.6/site-packages/pytablewriter/writer/text/_markdown.py", line 34, in __init__
2020-02-05 10:27:45,003 - DEBUG - super(MarkdownTableWriter, self).__init__()
2020-02-05 10:27:45,003 - DEBUG -   File "/usr/local/lib/python3.6/site-packages/pytablewriter/writer/text/_text_writer.py", line 386, in __init__
2020-02-05 10:27:45,003 - DEBUG - super(IndentationTextTableWriter, self).__init__()
2020-02-05 10:27:45,003 - DEBUG -   File "/usr/local/lib/python3.6/site-packages/pytablewriter/writer/text/_text_writer.py", line 111, in __init__
2020-02-05 10:27:45,003 - DEBUG - super(TextTableWriter, self).__init__()
2020-02-05 10:27:45,003 - DEBUG -   File "/usr/local/lib/python3.6/site-packages/pytablewriter/writer/_table_writer.py", line 429, in __init__
2020-02-05 10:27:45,003 - DEBUG - self.line_break_handling = LineBreakHandling.NOP
2020-02-05 10:27:45,003 - DEBUG -   File "/usr/local/lib/python3.6/site-packages/pytablewriter/writer/_table_writer.py", line 346, in line_break_handling
2020-02-05 10:27:45,003 - DEBUG - if self._dp_extractor.line_break_handling == value:
2020-02-05 10:27:45,003 - DEBUG - AttributeError
2020-02-05 10:27:45,003 - DEBUG - :
2020-02-05 10:27:45,003 - DEBUG - 'DataPropertyExtractor' object has no attribute 'line_break_handling'

Trailing double quote removed from cell data

Python 3.6.12
Python 3.9.7
pytablewriter==0.61.0 (newest)

And reproducible as far back as:
pytablewriter==0.47.0

Modifying the example from https://pytablewriter.readthedocs.io/en/latest/pages/examples/basic.html to add a cell containing "Debian 11 \"Bullseye\"":

import pytablewriter as ptw


def main():
    writer = ptw.MarkdownTableWriter()
    writer.table_name = "zone"
    writer.headers = ["zone_id", "country_code", "zone_name"]
    writer.value_matrix = [
        ["1", "AD", "Debian 11 \"Bullseye\""],
        ["2", "AE", "Asia/Dubai"],
        ["3", "AF", "Asia/Kabul"],
        ["4", "AG", "America/Antigua"],
        ["5", "AI", "America/Anguilla"],
    ]

    writer.write_table()


if __name__ == "__main__":
    main()

Actual result

# zone
|zone_id|country_code|     zone_name     |
|------:|------------|-------------------|
|      1|AD          |Debian 11 "Bullseye|
|      2|AE          |Asia/Dubai         |
|      3|AF          |Asia/Kabul         |
|      4|AG          |America/Antigua    |
|      5|AI          |America/Anguilla   |

Expected result

Something like:

# zone
|zone_id|country_code|     zone_name      |
|------:|------------|--------------------|
|      1|AD          |Debian 11 "Bullseye"|
|      2|AE          |Asia/Dubai          |
|      3|AF          |Asia/Kabul          |
|      4|AG          |America/Antigua     |
|      5|AI          |America/Anguilla    |

More info

Same issue with:

        ["1", "AD", 'Debian 11 "Bullseye"'],

But it works with single quotes:

        ["1", "AD", "Debian 11 'Bullseye'"],

And:

        ["1", "AD", 'Debian 11 \'Bullseye\''],

# zone
|zone_id|country_code|     zone_name      |
|------:|------------|--------------------|
|      1|AD          |Debian 11 'Bullseye'|
|      2|AE          |Asia/Dubai          |
|      3|AF          |Asia/Kabul          |
|      4|AG          |America/Antigua     |
|      5|AI          |America/Anguilla    |

Set formatting for specific cell

Hi,

Thanks for the awesome tool. I see that there is a set_style method for setting the style for a specific column by index. Is there a way to do this for a specific cell (e.g. indexed by the [x, y] row/col pair)? What i'd like to do is set the highest value of each column to be bolded (this is common when making tables for results of experiments) - so if there is any support for doing exactly that, which I have missed, being directed to it would be greatly appreciated :).

It would be cool to pass a function that tells the writer to iterate over columns and apply it, like:

writer.iter_col_styles(max, Style(font_weight="bold"))

or something like that...

Thanks!

sqlite can't set type

writer = pytablewriter.SqliteTableWriter()
...
writer.type_hints[0] = pytablewriter.String

but after writer.write_table(), sqlite schema will display INTEGER if first column is digit.

column name is only showing default A B C D ...

The old version display column names are okay for markdown. When upgrading to the latest version. column's name does not display correctly anymore.

Incorrect AsciiDoc table header generated.

The AsciiDocTableWriter generates a table header which is not recognized by Asciidoc (but it is by asciidoctor-pdf):

An example of the table definition generated is:
[cols="<40,>5, <10, <31, <15, <59" options="header"]

Asciidoc 9.0.0rc1 does not like this:

line 29: illegal style name: cols="<40

This is fixed by separating the 'cols' and the 'options' with a comma, here:

pytablewriter/pytablewriter/writer/text/_asciidoc.py

Line 96 in 29e8e75

rows = [f'[cols="{cols}" options="header"]']

    rows = [f'[cols="{cols}", options="header"]']

This makes the resulting asciidoc look like this (note the comma before 'options') :
[cols="<40,>5, <10, <31, <15, <59", options="header"]

Table generation is slow, even for space aligned

Hello, thanks for creating this library.

I'm generating a table with around 10,900 lines, and I find that pytablewriter takes ~90 seconds to generate it, ~70 when using 4 cores. By contrast, texttable takes ~7 seconds. This is all on a Raspberry Pi 4.

I am using a SpaceAlignedTableWriter with 8 columns, and simple alignment styling for each column, nothing more.

For texttable, I set max_width=0, set all columns to the text type, and use no borders (set_deco(0)).

I understand issues like this can be annoying to receive, so sorry about that. I thought I'd let you know in case this large time difference is due to some bug.

It's also possible this has to do with parsing and formatting of values or something? I noticed strings like 1,2,3 were being changed to 123 in the output. Let me know if I should file a separate issue for that.

Prevent formatting of float strings

from pytablewriter import MarkdownTableWriter

writer = MarkdownTableWriter(
    headers=["bla"],
    value_matrix=[['1.e-10']])
writer.write_table()

results in

|    bla     |
|-----------:|
|0.0000000001|

How do you prevent this autmoatic formatting of '1.e-10'?? I want the output to be unformatted:

|    bla     |
|-----------:|
|1.e-10|

Docs should be adjusted so it is very clear how to prevent this automatic formatting.

Github Pages fail to render table

Hello,

There is a bug with the jekyll markdown renderer that github pages are using where if there is no space between the table name and the table it can't render it...
How should we approach this problem?
pages-themes/cayman#82

module 'dataproperty' has no attribute 'PropertyExtractor'

While testing with your package and the excel module, I discovered an error, which I was not able to solve:

  File "C:\Users\Name\AppData\Roaming\Python\Python39\site-packages\pytablewriter\_excel_writer.py", line 176, in __init__
    super(ExcelTableWriter, self).__init__()
  File "C:\Users\Name\AppData\Roaming\Python\Python39\site-packages\pytablewriter\_table_writer.py", line 73, in __init__
    self._prop_extractor = dataproperty.PropertyExtractor()
AttributeError: module 'dataproperty' has no attribute 'PropertyExtractor'

Is there any way to solve this Issue?

Due to google results, I already tried an older Version (0.2.1), same issue here.

xslx formula injection

There is a formula injection bug in pytablewriter:

from pytablewriter import ExcelXlsxTableWriter, JsonTableWriter, HtmlTableWriter, ExcelXlsTableWriter

VALUES = [
    # These ones are OK:
    (HtmlTableWriter, "html"),
    (JsonTableWriter, "json"),
    (ExcelXlsTableWriter, "xls"),
    # Injection on this class:
    (ExcelXlsxTableWriter, "xlsx"),
]

for (Writer, extension) in VALUES:
    writer = Writer()
    writer.table_name = "test"
    writer.headers = ["x"]
    writer.value_matrix = [["=cmd|' /C notepad'!'A1'"]]
    writer.dump("test." + extension)

Note how the content of test.xslx is different of the content of the other generated files (including test.xsl): the input is expanded into a formula instead of being treated a plain text.

This is a security issue: it can be abused to trigger shell command injection or data exfiltration from Excel/LibreOffice. See [1,2,3] for examples of data exfiltration and shell command execution. The examples are in the context of CSV injection but the same
payloads can be used here.

[1] https://www.notsosecure.com/data-exfiltration-formula-injection/
[2] http://georgemauer.net/2017/10/07/csv-injection.html
[3] https://youtu.be/C1o5uVOaufU?t=364

Example for TableWriterFactory

I've come across the TableWriterFactory, and I think it might be quite useful for my project.

However, I can't seem to make it work and I could not find an example. Can you tell me how to use it?

I've tried

from pytablewriter import TableWriterFactory
writer = TableWriterFactory().create_from_file_extension('.csv')

But it raises an EmptyTableDataError.

I've also tried to fill the value_matrix etc but it didn't help.

Get string output for markdown writer without printing to stdout

Hello,

I am trying to get the raw string output from a MarkdownTableWriter without printing
to stdout. I have also tried redirecting output using the with io.StringIO() as buf, redirect_stdout(buf): pattern but that was unsuccessful. Trying to write to a file using
the example of the Readme leads to AttributeError: 'MarkdownTableWriter' object has no attribute 'open'. Any hints on how to resolve this? Essentially I just want a variable to hold all the contents of writer.write_table()

Error when executing example

When I try to run the first MarkDown example (https://github.com/thombashi/pytablewriter#write-tables) I get the following error:

RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

So I added the import statement

from multiprocessing import freeze_support

and moved the code into the main part:

if __name__ == '__main__':
    freeze_support()
    ...

This solved the initial problem and prints the table as markdown on the console. However now I get the following error:

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\Users\hohm\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\process.py", line 101, in _python_exit
    thread_wakeup.wakeup()
  File "C:\Users\hohm\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\process.py", line 89, in wakeup
    self._writer.send_bytes(b"")
  File "C:\Users\hohm\AppData\Local\Programs\Python\Python37\lib\multiprocessing\connection.py", line 183, in send_bytes
    self._check_closed()
  File "C:\Users\hohm\AppData\Local\Programs\Python\Python37\lib\multiprocessing\connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

I am running the example on a Windows 10 machine, with Python version 3.7.3 and pytablewriter version 0.46.1

Markdown save result as file fails

Problem

In version 0.64.0 and version 0.63.0, whenever I want to save the markdown result in a file I get an error, I tried using python3.6 and python 3.8. The error comes when I use write.dumps or save file using stream and write table as it can be seen in the code below

preview of code that generates error

with open("sample.md", "w") as f:                                                                                           
writer.stream = f                                                                                                       
writer.write_table()  

Error

 File "result.py", line 51, in <module>
    main()
  File "result.py", line 42, in main
    publish_report(report, gral_values, "DIETClassifier")
  File "result.py", line 20, in publish_report
    writer.write_table()
  File "/home/.local/lib/python3.8/site-packages/pytablewriter/writer/text/_markdown.py", line 140, in write_table
    self._write_table(**kwargs)
  File "/home/.local/lib/python3.8/site-packages/pytablewriter/writer/text/_text_writer.py", line 322, in _write_table
    self._preprocess()
  File "/home/.local/lib/python3.8/site-packages/pytablewriter/writer/_table_writer.py", line 1193, in _preprocess
    self._preprocess_table_dp()
  File "/home/.local/lib/python3.8/site-packages/pytablewriter/writer/_table_writer.py", line 1130, in _preprocess_table_dp
    self._table_value_dp_matrix = self._dp_extractor.to_dp_matrix(
  File "/home/.local/lib/python3.8/site-packages/dataproperty/_extractor.py", line 483, in to_dp_matrix
    return self.__to_dp_matrix_st(value_matrix)
  File "/home/.local/lib/python3.8/site-packages/dataproperty/_extractor.py", line 633, in __to_dp_matrix_st
    zip(
  File "/home/.local/lib/python3.8/site-packages/dataproperty/_extractor.py", line 635, in <genexpr>
    _to_dp_list_helper(
  File "/home/.local/lib/python3.8/site-packages/dataproperty/_extractor.py", line 799, in _to_dp_list_helper
    extractor._to_dp_list(data_list, type_hint=type_hint, preprocessor=preprocessor),
  File "/home/.local/lib/python3.8/site-packages/dataproperty/_extractor.py", line 696, in _to_dp_list
    dataprop = self.__to_dp(
  File "/home/.local/lib/python3.8/site-packages/dataproperty/_extractor.py", line 568, in __to_dp
    return self.__to_dp_raw(
  File "/home/.local/lib/python3.8/site-packages/dataproperty/_extractor.py", line 619, in __to_dp_raw
    value_dp = DataProperty(
  File "/home/.local/lib/python3.8/site-packages/dataproperty/_dataproperty.py", line 89, in __init__
    data, no_ansi_escape_data = preprocessor.preprocess(data)
  File "/home/.local/lib/python3.8/site-packages/dataproperty/_preprocessor.py", line 68, in preprocess
    data, no_ansi_escape_data = self.__preprocess_string(
  File "/home/.local/lib/python3.8/site-packages/dataproperty/_preprocessor.py", line 105, in __preprocess_string
    data = self.__dequote(data)
  File "/home/.local/lib/python3.8/site-packages/dataproperty/_preprocessor.py", line 135, in __dequote
    if (s[0] == s[-1]) and s.startswith(("'", '"')):
  KeyError: 0

Stable version

From version 0.62.0 and lower the files were saved without any problem

Add styling support for Excel (binary) format

I really like the patterns setup here. Looks like the various styling options for tables are not supported for Excel. The default font is WAY too small and I want also to BOLD headers.
I tried using the patterns laid out (and it is even stated they aren't supported).
Is there a plan for adding this?

Any suggestions as to how I can force this to a bigger font?
I'll dig into the code more but I think this would be a huge value.
Thanks
jim

0.63.0 - Quoted string quote mark not printed

Hi all,

Since v0.63.0 I can not print quote marks.

In v0.62.0 the output was this:

|     PARAMETER      |                               DESCRIPTION                                |REQUIRED|DEFAULT| TYPE  |
|--------------------|--------------------------------------------------------------------------|:------:|-------|------:|
|**base-revision**   |The long (40-character) git SHA of the build prior to the one being built.|   No   |'foba' | string|
|**all-files**       |Run pre-commit on all files or just on the latest commit                  |   No   |false  |boolean|
|**extra-dependency**|Install extra dependency like 'pip3 install boto3'                        |   No   |' '    | string|

In v0.63.0 the output became this:

|     PARAMETER      |                               DESCRIPTION                                |REQUIRED|DEFAULT| TYPE  |
|--------------------|--------------------------------------------------------------------------|:------:|-------|------:|
|**base-revision**   |The long (40-character) git SHA of the build prior to the one being built.|   No   |foba   | string|
|**all-files**       |Run pre-commit on all files or just on the latest commit                  |   No   |false  |boolean|
|**extra-dependency**|Install extra dependency like 'pip3 install boto3'                        |   No   |       | string|

Sample code to reproduce the error:

import io
from pytablewriter import MarkdownTableWriter
from pytablewriter.style import Style
from pytablewriter.typehint import String

TABLE_HEADER = ["PARAMETER", "DESCRIPTION", "REQUIRED", "DEFAULT", "TYPE"]

value_matrix = [
['base-revision', 'The long (40-character) git SHA of the build prior to the one being built.', 'No', "'foba'", 'string'],
['all-files', 'Run pre-commit on all files or just on the latest commit', 'No', 'false', 'boolean'],
['extra-dependency', "Install extra dependency like 'pip3 install boto3'", 'No', "' '", 'string']
]

writer = MarkdownTableWriter(
    headers=TABLE_HEADER,
    value_matrix=value_matrix,
    type_hints= [String, String, String, String, String],
    column_styles=[
        Style(align="left", font_weight="bold"),
        Style(align="left"),
        Style(align="center"),
        Style(align="left"),
        Style(align="right"),
    ],
)
output = io.StringIO()
writer.dump(output, close_after_write=False)
print(output.getvalue()[:-1])

Leading zeros are missing for strings in value_matrix

Hi,
first of all thanks a lot for creating this library!

I've just experienced that leading zeros of strings passed as part of the value_matrix are not printed when creating a Markdown table. Is this intended behaviour or do I miss something here?

Thanks a lot in advance,
Jonas

Fix Markdown table alignment row to conform to GFM

Currently, Markdown tables look like this:

| foo | foobar | baz |
|:----|--------|----:|
| aaa | bb     | c   |

According to GitHub, there should be a space between each pipe and the alignment string, like so:

| foo | foobar | baz |
| :-- | ------ | --: |
| aaa | bb     | c   |

This discrepancy causes linters (e.g. preettier) which do conform to GFM to constantly change the header row for tables that have been generated with this library.

Support line break handling for RST output

RST tables can handle cells with line breaks in them. It would be nice if pytablewriter supported them, rather than discarding them all.

0.47: quotes added around strings with NumpyTableWriter

With pytablewriter 0.47, I'm seeing quotes added around some strings with NumpyTableWriter and PandasDataFrameWriter.

Code

import pytablewriter
from pytablewriter import (
    HtmlTableWriter,
    MarkdownTableWriter,
    NumpyTableWriter,
    PandasDataFrameWriter,
    RstSimpleTableWriter,
)


def _tabulate(data, format="markdown"):
    """Return data in specified format"""

    format_writers = {
        "markdown": MarkdownTableWriter,
        "rst": RstSimpleTableWriter,
        "html": HtmlTableWriter,
        "pandas": PandasDataFrameWriter,
        "numpy": NumpyTableWriter,
    }

    writer = format_writers[format]()
    if format != "html":
        writer.margin = 1

    writer.header_list = sorted(set().union(*(d.keys() for d in data)))
    writer.value_matrix = data

    if format == "numpy":
        return writer.tabledata.as_dataframe().values
    elif format == "pandas":
        return writer.tabledata.as_dataframe()
    return writer.dumps()


if __name__ == "__main__":
    print(pytablewriter.__version__)
    data = [{"category": "without_mirrors", "downloads": 4593356}]

    print("numpy:")
    output = _tabulate(data, format="numpy")
    print(output)

    print("pandas:")
    output = _tabulate(data, format="pandas")
    print(output)

Before

• DataProperty-0.43.3 pytablewriter-0.46.3 tabledata-0.9.1

$ python --version
Python 3.8.1
$ pip install DataProperty==0.43.3 pytablewriter==0.46.3 tabledata==0.9.1
...
$ p 1.py
0.46.3
numpy:
[['without_mirrors' 4593356]]
pandas:
          category  downloads
0  without_mirrors    4593356

After

• DataProperty-0.45.0 pytablewriter-0.47.0 tabledata-0.9.2

$ pip install -U pytablewriter
Collecting pytablewriter
  Using cached pytablewriter-0.47.0-py2.py3-none-any.whl (70 kB)
Collecting DataProperty<2,>=0.45.0
  Using cached DataProperty-0.45.0-py2.py3-none-any.whl (25 kB)
Requirement already satisfied, skipping upgrade: mbstrdecoder<2,>=0.8.3 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from pytablewriter) (0.8.4)
Requirement already satisfied, skipping upgrade: typepy[datetime]<2,>=0.6.4 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from pytablewriter) (0.6.4)
Requirement already satisfied, skipping upgrade: setuptools>=38.3.0 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from pytablewriter) (45.1.0)
Collecting tabledata<2,>=0.9.2
  Using cached tabledata-0.9.2-py2.py3-none-any.whl (11 kB)
Requirement already satisfied, skipping upgrade: pathvalidate<3; python_version >= "3.5" in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from pytablewriter) (2.1.0)
Requirement already satisfied, skipping upgrade: six<2.0.0,>=1.10.0 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from pytablewriter) (1.14.0)
Requirement already satisfied, skipping upgrade: msgfy<1,>=0.0.6 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from pytablewriter) (0.0.7)
Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.4 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from mbstrdecoder<2,>=0.8.3->pytablewriter) (3.0.4)
Requirement already satisfied, skipping upgrade: pytz>=2018.9; extra == "datetime" in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from typepy[datetime]<2,>=0.6.4->pytablewriter) (2019.3)
Requirement already satisfied, skipping upgrade: python-dateutil<3.0.0,>=2.8.0; extra == "datetime" in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from typepy[datetime]<2,>=0.6.4->pytablewriter) (2.8.1)
Installing collected packages: DataProperty, tabledata, pytablewriter
  Attempting uninstall: DataProperty
    Found existing installation: DataProperty 0.43.3
    Uninstalling DataProperty-0.43.3:
      Successfully uninstalled DataProperty-0.43.3
  Attempting uninstall: tabledata
    Found existing installation: tabledata 0.9.1
    Uninstalling tabledata-0.9.1:
      Successfully uninstalled tabledata-0.9.1
  Attempting uninstall: pytablewriter
    Found existing installation: pytablewriter 0.46.3
    Uninstalling pytablewriter-0.46.3:
      Successfully uninstalled pytablewriter-0.46.3
Successfully installed DataProperty-0.45.0 pytablewriter-0.47.0 tabledata-0.9.2
$ p 1.py
0.47.0
numpy:
[['"without_mirrors"' 4593356]]
pandas:
            category  downloads
0  "without_mirrors"    4593356

Styling HTML tables?

Hi @thombashi,

I was wondering I could I style an HTML table?

The table is rendering without borders and I just wanted to add them, but can't find where.

Thanks
@stezz

[Bug] ModuleNotFoundError: No module named 'dominate'

The following code is taken from my project where it has been working for a few years:

import pytablewriter # v
df = pd.DataFrame.from_dict(
    {'my': {0: 9, 1: 8, 2: 8},
     'nice': {0: 51, 1: 56, 2: 58},
     'columns': {0: '2.3.0', 1: '2.3.0', 2: '2.3.0'}}
)
writer = pytablewriter.MarkdownTableWriter()
writer.table_name = "Metadata"
writer.header_list = list(df.columns.values)
writer.value_matrix = df.values.tolist()
print(str(writer))

Output

Metadata

my	nice	columns
9	51	2.3.0
8	56	2.3.0
8	58	2.3.0

But strangely, when I import that same code as a function from a separate file I get the error mentioned in the title, but only in a Jupyter notebook.

Steps to reproduce

1. Create conda environment conda create -y -n ptw python=3.10 && conda activate ptw
2. pip install pytablewriter pandas ipykernel
   • same result with pip install "pytablewriter<1.0.0" pandas==1.5.3 ipykernel
3. Install ipykernel via python -m ipykernel install --user --name=ptw
4. Run a fresh Jupyter notebook, activate the ptw kernel
5. Create the file external.py next to the notebook containing

   import pytablewriter
   
   def write_pytable(df):
       writer = pytablewriter.MarkdownTableWriter()
       writer.table_name = "Metadata"
       writer.header_list = list(df.columns.values)
       writer.value_matrix = df.values.tolist()
       return writer

6. In a notebook cell, execute

   from external import write_pytable
   import pandas as pd
   
   df = pd.DataFrame.from_dict(
       {'my': {0: 9, 1: 8, 2: 8},
        'nice': {0: 51, 1: 56, 2: 58},
        'columns': {0: '2.3.0', 1: '2.3.0', 2: '2.3.0'}}
   )
   
   write_pytable(df)

Traceback

~/miniconda3/envs/ptw/lib/python3.10/site-packages/pytablewriter/writer/text/_html.py:26: UserWarning: dependency packages for html not found.
you can install the dependencies with 'pip install pytablewriter[html]'

  warnings.warn(import_error_msg_template.format("html"))

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
File ~/miniconda3/envs/ptw/lib/python3.10/site-packages/IPython/core/formatters.py:344, in BaseFormatter.__call__(self, obj)
    342     method = get_real_method(obj, self.print_method)
    343     if method is not None:
--> 344         return method()
    345     return None
    346 else:

File ~/miniconda3/envs/ptw/lib/python3.10/site-packages/pytablewriter/writer/_table_writer.py:250, in AbstractTableWriter._repr_html_(self)
    240 writer = HtmlTableWriter(
    241     table_name=self.table_name,
    242     headers=self.headers,
   (...)
    246     enable_ansi_escape=self.enable_ansi_escape,
    247 )
    248 writer._dp_extractor = self._dp_extractor
--> 250 return writer.dumps()

File ~/miniconda3/envs/ptw/lib/python3.10/site-packages/pytablewriter/writer/text/_text_writer.py:313, in TextTableWriter.dumps(self, **kwargs)
    311 try:
    312     self.stream = io.StringIO()
--> 313     self.write_table(**kwargs)
    314     tabular_text = self.stream.getvalue()
    315 finally:

File ~/miniconda3/envs/ptw/lib/python3.10/site-packages/pytablewriter/writer/text/_html.py:77, in HtmlTableWriter.write_table(self, **kwargs)
     61 def write_table(self, **kwargs) -> None:
     62     """
     63     |write_table| with HTML table format.
     64 
   (...)
     74         - |None| values will be replaced with an empty value
     75     """
---> 77     tags, raw = _get_tags_module()
     78     write_css = kwargs.get("write_css", False)
     80     with self._logger:

File ~/miniconda3/envs/ptw/lib/python3.10/site-packages/pytablewriter/writer/text/_html.py:21, in _get_tags_module()
     19 def _get_tags_module() -> Tuple:
     20     try:
---> 21         from dominate import tags
     22         from dominate.util import raw
     24         return tags, raw

ModuleNotFoundError: No module named 'dominate'

Error occurs with both 0.64.2 and 1.0.0, but only in a notebook, not in a Python console. Also, it seems like it does not occur when installing pytabledata[html].

Update: I could narrow it down a bit further, the error ocurs only when displaying the writer. So markdown = write_pytable(df) does not raise, but then having a cell containing only markdown does, meaning that it's caused by the ._repr_html_() method that Jupyter tries for displaying. .__repr__() works perfectly.

Header cells ignore style

It appears that the header row discards the column_styles value. I also tried forcing a style via add_style_filter (see snippet below), but it was ignored. I ran into this on MarkdownTableWriter, but based on the code, I think it affects the base text writer (and subclassed writers) as well.

def force_header_style(cell, **kwargs):
    if cell.row == -1: 
        return None

    return Style(align="left", font_weight="bold")

writer = MarkdownTableWriter(
    headers=["a", "b", "c"],
    margin=1,
    value_matrix = [
        ["foo bar", "bar bax", "baz qux"],
    ].
)

writer.add_style_filter(force_header_style)

print(writer.dumps())

Expected Output:

| **a**   | **b**   | **c**   |
| :------ | :------ | :------ |
| foo bar | bar baz | baz qux |

Actual Output

|    a    |    b    |    c    |
| :------ | :------ | :------ |
| foo bar | bar baz | baz qux |

Add TOP, BOTTOM align as option for formatting Markdown table cells

Hello, thank you for your work on this it saved me a bunch of time.

I'm creating a table w rows of multiple cells containing lists of items.
Each item in the list is separated by
.

The length of these cell lists varies across cells,
while the largest list cell obviously consumes the most space,
adjacent smaller cells (lists) are aligned vertically as centered, which isn't great visually.
I would prefer being able to align to TOP.

I realize that Markdown doesn't support such a concept,
wondering if there any ways to style a column as Align.TOP, somehow getting renderer to add
style="vertical-align: top;" somehow into the html.

Considering the various flavors of markdown, I guess this wouldn't work.
Any ideas?

writer.dumps() fails due to multiprocessing if __main__ is not protected

I'm using a script to call a table-writer method in another module.

However, due to multiprocessing usage, this fails, as the new processes import the unprotected script.

Can be reproduces like this:

from pytablewriter import HtmlTableWriter
# no __name__=='__main__' statement
writer = HtmlTableWriter()
writer.value_matrix  = [[1,2,3,4]]    
string = writer.dumps()

I'm a bit surprised that this package needs multiprocessing, are there such heavy computations to be made?

Might it be an idea to switch to joblib.Parallel, as in my experience it doesn't suffer from this problem? Or to disable multiprocessing/switch to multithreading (no reimport of main module necessary)?

(It's a Windows problem, as there is no fork on winx)

Make pytest-runner a conditional requirement

thombashi/DataProperty#20

New version ignores `header_list`

Whereas this code used to work perfectly fine:

import pytablewriter
import pandas as pd


df = pd.DataFrame.from_dict(
    {'my': {0: 9, 1: 8, 2: 8},
     'nice': {0: 51, 1: 56, 2: 58},
     'columns': {0: '2.3.0', 1: '2.3.0', 2: '2.3.0'}}
)

writer = pytablewriter.MarkdownTableWriter()
writer.header_list = list(df.columns.values) # <-- this line is without consequence.
writer.value_matrix = df.values.tolist()

producing as output

my	nice	columns
9	51	2.3.0
8	56	2.3.0
8	58	2.3.0

the new version 1.0.0 ignores the header_list and produces

A	B	C
9	51	2.3.0
8	56	2.3.0
8	58	2.3.0

instead. I will pin my library to 0.64.2 which was working perfectly fine but thought I'll let you know that such a change to the API breaks other peoples' workflows -- and unnecessarily AFAICT. That being said, thanks a lot for maintaining the package!

Emoji causing misalignment on Linux

pytablewriter 0.36.0

Here's a table using emoji:

#!/usr/bin/env python
#  encoding: utf-8
import pytablewriter

writer = pytablewriter.MarkdownTableWriter()
# writer = pytablewriter.SpaceAlignedTableWriter()

writer.header_list = ["int", "float", "str", "bool", "mix", "time"]
writer.value_matrix = [
    ["✅", 0.1, "hoge", True, 0, "2017-01-01 03:04:05+0900"],
    [None, -9.9, "", "FALSE", "nan", "2017-01-01 00:00:00+0900"],
    ["🔺", "-2.23", "foo", False, None, "2017-12-23 45:01:23+0900"],
    [None, -9.9, "", "FALSE", "nan", "2017-01-01 00:00:00+0900"],
]
writer.margin = 1  # add a whitespace for both sides of each cell

writer.write_table()

It renders fine on macOS High Sierra / Python 3.7.0 / iTerm:

But on our CI (Linux / Python 3.4.3) the emoji are causing some misalignment:

https://travis-ci.org/hugovk/test/jobs/442039626#L363

Version 0.30.0 import failure

Some testing infa uses this package! Its really nice, but it appears version 0.30.0 is in a failing state.

Reproduce
In new environment
pip install pytablewriter==0.30.0

>>> import pytablewriter
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/keith/.virtualenvs/ispytablewriterbroken/lib/python2.7/site-packages/pytablewriter/__init__.py", line 17, in <module>
    from ._factory import TableWriterFactory
  File "/home/keith/.virtualenvs/ispytablewriterbroken/lib/python2.7/site-packages/pytablewriter/_factory.py", line 14, in <module>
    from ._table_format import FormatAttr, TableFormat
  File "/home/keith/.virtualenvs/ispytablewriterbroken/lib/python2.7/site-packages/pytablewriter/_table_format.py", line 48, in <module>
    class TableFormat(enum.Enum):
  File "/home/keith/.virtualenvs/ispytablewriterbroken/lib/python2.7/site-packages/pytablewriter/_table_format.py", line 54, in TableFormat
    [CsvTableWriter().format_name], CsvTableWriter,
  File "/home/keith/.virtualenvs/ispytablewriterbroken/lib/python2.7/site-packages/pytablewriter/writer/_csv.py", line 32, in __init__
    super(CsvTableWriter, self).__init__()
  File "/home/keith/.virtualenvs/ispytablewriterbroken/lib/python2.7/site-packages/pytablewriter/writer/_text_writer.py", line 100, in __init__
    super(TextTableWriter, self).__init__()
  File "/home/keith/.virtualenvs/ispytablewriterbroken/lib/python2.7/site-packages/pytablewriter/writer/_table_writer.py", line 221, in __init__
    self._dp_extractor.type_value_mapping[Typecode.NONE] = ""
AttributeError: 'DataPropertyExtractor' object has no attribute 'type_value_mapping'

Handle colours

It would be nice to be able to use the table with colours for output in a terminal. My use case is to report some data on a CI, and show good or bad values in green or red.

import pytablewriter

from termcolor import colored

writer = pytablewriter.MarkdownTableWriter()
# writer = pytablewriter.SpaceAlignedTableWriter()

writer.header_list = ["int", "float", "str", "bool", "mix", "time"]
writer.value_matrix = [
    [colored(0, "red"), 0.1, "hoge", True, 0, "2017-01-01 03:04:05+0900"],
    [colored(2, "yellow"), "-2.23", "foo", False, None, "2017-12-23 45:01:23+0900"],
    [colored(3, "green"), 0, "bar", "true", "inf", "2017-03-03 33:44:55+0900"],
    [-10, -9.9, "", "FALSE", "nan", "2017-01-01 00:00:00+0900"],
]
writer.margin = 1  # add a whitespace for both sides of each cell

writer.write_table()

Produces this (colours not shown):

|    int     | float | str  | bool  |   mix    |           time           |
|------------|------:|------|-------|---------:|--------------------------|
| 0 |  0.10 | hoge | True  |        0 | 2017-01-01 03:04:05+0900 |
| 2 | -2.23 | foo  | False |          | 2017-12-23 45:01:23+0900 |
| 3 |  0.00 | bar  | True  | Infinity | 2017-03-03 33:44:55+0900 |
|        -10 | -9.90 |      | False |      NaN | 2017-01-01 00:00:00+0900 |

Terminal screenshot (macOS, iTerm):

It's accounting for the length of the string, which of course includes non-visible ANSI escape sequences.

Would it be possible or sensible to use the visible text when calculating the widths?

Here's one way: https://stackoverflow.com/a/2187024/724176.

Add custom stylesheet to html table

Is it possible to add a custom stylesheet to html table?
Looked through the docs but wasn't able to see any clear instructions

Feature request: return pandas DataFrame

Right now, PandasDataFrameWriter returns a string defining the source code to get a pandas DataFrame.

• https://pytablewriter.readthedocs.io/en/latest/pages/examples/table_format/text/sourcecode/pandas_dataframe.html

To turn that string into a DataFrame for further processing, we need to exec it:

>>> import pytablewriter
>>> writer = pytablewriter.PandasDataFrameWriter()
>>> writer.table_name = "example_table"
>>> writer.headers = ["int", "float", "str", "bool", "mix", "time"]
>>> writer.value_matrix = [
    [0,   0.1,      "hoge", True,   0,      "2017-01-01 03:04:05+0900"],
    [2,   "-2.23",  "foo",  False,  None,   "2017-12-23 45:01:23+0900"],
    [3,   0,        "bar",  "true",  "inf", "2017-03-03 33:44:55+0900"],
    [-10, -9.9,     "",     "FALSE", "nan", "2017-01-01 00:00:00+0900"],
]
>>>
>>> output = writer.dumps()
>>> output
'example_table = pd.DataFrame([\n    [0, 0.1, "hoge", True, 0, "2017-01-01 03:04:05+0900"],\n    [2, -2.23, "foo", False, None, "2017-12-23 45:01:23+0900"],\n    [3, 0, "bar", True, np.inf, "2017-03-03 33:44:55+0900"],\n    [-10, -9.9, "", False, np.nan, "2017-01-01 00:00:00+0900"],\n], columns=["int", "float", "str", "bool", "mix", "time"])\n'
>>> type(output)
<class 'str'>
>>> exec(output)
>>> example_table
   int  float   str   bool  mix                      time
0    0   0.10  hoge   True  0.0  2017-01-01 03:04:05+0900
1    2  -2.23   foo  False  NaN  2017-12-23 45:01:23+0900
2    3   0.00   bar   True  inf  2017-03-03 33:44:55+0900
3  -10  -9.90        False  NaN  2017-01-01 00:00:00+0900
>>> type(example_table)
<class 'pandas.core.frame.DataFrame'>

To avoid needing to exec an arbitrary string, it would nice to be able to have an actual pandas DataFrame returned directly, instead of a string.

This would allow further processing of the DataFrame, for example to plot charts (hugovk/pypistats#74).

Would it be possible to add support for this?

---

Similarly, it would also be good to be able to get an actual NumPy array from NumpyTableWriter, and not only a string defining one.

• https://pytablewriter.readthedocs.io/en/latest/pages/examples/table_format/text/sourcecode/numpy.html

Thank you!

Comma-separate thousands or custom formatting in a column?

import pytablewriter

data = [
    {"category": "2.6", "date": "2018-08-15", "downloads": 51},
    {"category": "2.7", "date": "2018-08-15", "downloads": 63749},
    {"category": "3.2", "date": "2018-08-15", "downloads": 2},
    {"category": "3.3", "date": "2018-08-15", "downloads": 40},
    {"category": "3.4", "date": "2018-08-15", "downloads": 6095},
    {"category": "3.5", "date": "2018-08-15", "downloads": 20358},
    {"category": "3.6", "date": "2018-08-15", "downloads": 35274},
    {"category": "3.7", "date": "2018-08-15", "downloads": 6595},
    {"category": "3.8", "date": "2018-08-15", "downloads": 3},
    {"category": "null", "date": "2018-08-15", "downloads": 1019},
]

writer = pytablewriter.MarkdownTableWriter()
writer.margin = 1

writer.header_list = sorted(set().union(*(d.keys() for d in data)))
writer.value_matrix = data

writer.write_table()

Produces:

category	date	downloads
2.6	2018-08-15	51
2.7	2018-08-15	63749
3.2	2018-08-15	2
3.3	2018-08-15	40
3.4	2018-08-15	6095
3.5	2018-08-15	20358
3.6	2018-08-15	35274
3.7	2018-08-15	6595
3.8	2018-08-15	3
null	2018-08-15	1019

I'd like the downloads to have comma-separated thousands to increase readability:

category	date	downloads
2.6	2018-08-15	51
2.7	2018-08-15	63,749
3.2	2018-08-15	2
3.3	2018-08-15	40
3.4	2018-08-15	6,095
3.5	2018-08-15	20,358
3.6	2018-08-15	35,274
3.7	2018-08-15	6,595
3.8	2018-08-15	3
null	2018-08-15	1,019

I can preprocess the data:

new_data = []

for row in data:
    row["downloads"] = "{:,}".format(row["downloads"])
    new_data.append(row)

data = new_data

category	date	downloads
2.6	2018-08-15	51
2.7	2018-08-15	63,749
3.2	2018-08-15	2
3.3	2018-08-15	40
3.4	2018-08-15	6,095
3.5	2018-08-15	20,358
3.6	2018-08-15	35,274
3.7	2018-08-15	6,595
3.8	2018-08-15	3
null	2018-08-15	1,019

And then explicitly right-format the column:

from pytablewriter import Align
writer.align_list = [Align.AUTO, Align.AUTO, Align.RIGHT]

# Or more generically:
writer.align_list = [Align.AUTO] * len(writer.header_list)
download_index = writer.header_list.index("downloads")
writer.align_list[download_index] = Align.RIGHT

category	date	downloads
2.6	2018-08-15	51
2.7	2018-08-15	63,749
3.2	2018-08-15	2
3.3	2018-08-15	40
3.4	2018-08-15	6,095
3.5	2018-08-15	20,358
3.6	2018-08-15	35,274
3.7	2018-08-15	6,595
3.8	2018-08-15	3
null	2018-08-15	1,019

But is there a neater way? Would some option to format numbers with comma-separated thousands be possible or sensible? Or perhaps custom formatting?

from pytablewriter import Format
writer.format_list = [None, None, Format.Comma]
writer.format_list = [None, None, "{:,}"]

That could cover cases that format numbers like 12 345.6 or 12.345,6, and also allow other formatting options.

Thank you.

Merged rows support

Originally posted by @wxng in #40

Show how to write to file in examples

It's not obvious to all programmers (me) how to get the output from the writer object to, say, a markdown file.