alttch / rapidtables

Super fast list of dicts to pre-formatted tables conversion library for Python 2/3

rapidtables

rapidtables is a module for Python 2/3, which does only one thing: converts lists of dictionaries to pre-formatted tables. And it does the job as fast as possible.

rapidtables is focused on speed and is useful for applications which dynamically refresh data in console. The module code is heavily optimized and written purely in Python.

And unlike other similar modules, rapidtables can output pre-formatted generators of strings or even generators of tuples of strings, which allows you to colorize every single column.

Install

pip install rapidtables

Example

# if you need to keep strict column ordering, use OrderedDict for the rows
data = [
    { 'name': 'John', 'salary': 2000, 'job': 'DevOps' },
    { 'name': 'Jack', 'salary': 2500, 'job': 'Architect' },
    { 'name': 'Diana', 'salary': None, 'job': 'Student' },
    { 'name': 'Ken', 'salary': 1800, 'job': 'Q/A' }
]

from rapidtables import format_table, FORMAT_GENERATOR_COLS
from termcolor import colored

header, rows = format_table(data, fmt=FORMAT_GENERATOR_COLS)
spacer = '  '
print(colored(spacer.join(header), color='blue'))
print(colored('-' * sum([(len(x) + 2) for x in header]), color='grey'))
for r in rows:
    print(colored(r[0], color='white', attrs=['bold']) + spacer, end='')
    print(colored(r[1], color='cyan') + spacer, end='')
    print(colored(r[2], color='yellow'))

Pretty cool, isn't it? Actually, it was the most complex example, you can work with header + table rows already joined:

from rapidtables import format_table, FORMAT_GENERATOR

header, rows = format_table(data, fmt=FORMAT_GENERATOR)
print(colored(header, color='blue'))
print(colored('-' * len(header), color='grey'))
for r in rows:
    print(colored(r, color='yellow'))

Or you can use make_table function to return the table out-of-the-box (or print_table to instantly print it), and print it in raw:

print_table(data)

name  salary  job
----  ------  ---------
John    2000  DevOps
Jack    2500  Architect
Ken     1800  Q/A

Quick API reference

format_table

Formats a table. Outputs data in raw, generator of strings (one string per row) or generator of tuples of strings (one tuple per row, one string per column):

• fmt=rapidtables.FORMAT_RAW raw string
• fmt=rapidtables.FORMAT_GENERATOR generator of strings
• fmt=rapidtables.FORMAT_GENERATOR_COLS generator of tuples of strings

Align columns:

• align=rapidtables.ALIGN_LEFT align all columns to left
• align=rapidtables.ALIGN_NUMBERS_RIGHT align numbers to right (default)
• align=rapidtables.ALIGN_RIGHT align all columns to right
• align=rapidtables.ALIGN_CENTER align all columns to center
• align=rapidtables.ALIGN_HOMOGENEOUS_NUMBERS_RIGHT align numbers to right but consider the table is homogeneous and check col values only to first number or string (works slightly faster)

To predefine aligns, set align to tuple or list:

align=(rapidtables.ALIGN_LEFT, rapidtables.ALIGN_RIGHT, ....)

number of items in list must match number of columns in table.

You may also customize headers, separators etc. Read pydoc for more info.

make_table

Generates a ready to output table. Supports basic formats:

table = rapidtables.make_table(data, tablefmt='raw')

name  salary  job
-----------------------
John     2000  DevOps
Jack     2500  Architect
Diana          Student
Ken      1800  Q/A

table = rapidtables.make_table(data, tablefmt='simple')

name   salary  job
----   ------  ---------
John     2000  DevOps
Jack     2500  Architect
Diana          Student
Ken      1800  Q/A

table = rapidtables.make_table(data, tablefmt='md') # Markdown

| name  | salary | job       |
|-------|--------|-----------|
| John  |   2000 | DevOps    |
| Jack  |   2500 | Architect |
| Diana |        | Student   |
| Ken   |   1800 | Q/A       |

table = rapidtables.make_table(data, tablefmt='rst') # reStructured, simple

=====  ======  =========
name   salary  job
=====  ======  =========
John     2000  DevOps
Jack     2500  Architect
Diana          Student
Ken      1800  Q/A
=====  ======  =========

table = rapidtables.make_table(data, tablefmt='rstgrid') # reStructured, grid

+-------+--------+-----------+
| name  | salary | job       |
+=======+========+===========+
| John  |   2000 | DevOps    |
+-------+--------+-----------+
| Jack  |   2500 | Architect |
+-------+--------+-----------+
| Diana |        | Student   |
+-------+--------+-----------+
| Ken   |   1800 | Q/A       |
+-------+--------+-----------+

print_table

The same as make_table, but prints table to stdout.

Benchmarks

(Python 3.7)

Enjoy!

rapidtables's Issues

Reduce number of for loops and use generators

I was the guy bugging you on reddit. I was going to make a pull request that incorporated generators more, but got a little confused by some of the logic in format_table. This for loop could really benefit from being turned in to a generator:

rapidtables/rapidtables/__init__.py

Lines 58 to 78 in e96d9d6

	# dig
	for ki, k in enumerate(keys):
	v = ()
	alpha = False
	if generate_header:
	klen = len(headers[ki]) if headers else len(k)
	for ri, r in enumerate(table):
	value = r.get(k)
	if not alpha:
	if value is not None:
	try:
	float(value)
	except:
	alpha = True
	v += (str(value) if value is not None else '',)
	if generate_header:
	key_lengths += (max(klen, len(max(v, key=len))),)
	else:
	key_lengths += (len(max(v, key=len)),)
	if not calign: key_isalpha += (alpha,)
	vals += (v,)

However, the outer for loop is on the header keys and more importantly the key_lengths logic requires looking at every value to be printed. Before I understood what the key_lengths stuff was actually doing I was wondering if it could be moved. Now that I see what you're doing (finding the width of each column), I'm not sure there is a better way.

Here's what I was going to try to contribute:

table_iter = iter(table)
first_row = next(table_iter)
keys = tuple(first_row)
len_keys = len(keys)

formatted_tuples = format_values(first_row, table_iter)  # generator

if generate_header:
    ...
else:
    ...

result = body_generator(formatted_tuples)
... returns ...

I realized that your README says you aren't planning on being fast for large numbers of rows, but I think something like this could still perform well. This way it would only iterate over the entire input data once.

Also, you may want to test moving the body_generator outside the function and pass all of the variables used. I'm not sure on the performance differences between local variables and outer scope variables, but I would think local variables would be faster.

Request: Asciidoctor table

Asciidoctor is the new kid In town or an old one reborn. Would you be interested to integrate this format to the project?

Feel free to reject
Felix

Multiline values or text wrapping support.

Given a multiline value, or a value which is too long, the table doesn't format correctly

In [30]: data = [ 
    ...:     { 'name': 'John', 'salary': 2000, 'job': 'DevOps' }, 
    ...:     { 'name': 'Jack', 'salary': 2500, 'job': 'Multi\nLine\nJob' }, 
    ...:     { 'name': 'Diana', 'salary': None, 'job': 'Student' }, 
    ...:     { 'name': 'Ken', 'salary': 1800, 'job': 'Q/A' } 
    ...: ]                                                                                                                                                                   

In [31]: rapidtables.print_table(data,tablefmt="md")                                                                                                                         
| name  | salary | job            |
|-------|--------|----------------|
| John  |   2000 | DevOps         |
| Jack  |   2500 | Multi
Line
Job |
| Diana |        | Student        |
| Ken   |   1800 | Q/A            |

In [32]: data = [ 
    ...:     { 'name': 'John', 'salary': 2000, 'job': 'DevOps' }, 
    ...:     { 'name': 'Jack', 'salary': "Multi\nLine\nSalary", 'job': 'Developer' }, 
    ...:     { 'name': 'Diana', 'salary': None, 'job': 'Student' }, 
    ...:     { 'name': 'Ken', 'salary': 1800, 'job': 'Q/A' } 
    ...: ]                                                                                                                                                                   

In [33]: rapidtables.print_table(data,tablefmt="md")                                                                                                                         
| name  | salary            | job       |
|-------|-------------------|-----------|
| John  | 2000              | DevOps    |
| Jack  | Multi
Line
Salary | Developer |
| Diana |                   | Student   |
| Ken   | 1800              | Q/A       |

In [34]: data = [ 
    ...:     { 'name': 'John', 'salary': 2000, 'job': 'DevOps' }, 
    ...:     { 'name': 'Jack', 'salary': 2500, 'job': 'a'*200}, 
    ...:     { 'name': 'Diana', 'salary': None, 'job': 'Student' }, 
    ...:     { 'name': 'Ken', 'salary': 1800, 'job': 'Q/A' } 
    ...: ]                                                                                                                                                                   

In [35]: rapidtables.print_table(data,tablefmt="md")                                                                                                                         
| name  | salary | job                                                                                                                                                                                                      |
|-------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| John  |   2000 | DevOps                                                                                                                                                                                                   |
| Jack  |   2500 | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa |
| Diana |        | Student                                                                                                                                                                                                  |
| Ken   |   1800 | Q/A                                                                                                                                                                                                      |

The a*200 table looks good on github since it just creates this horizontal scroll view, but looks bad in a terminal cause terminals wrap text.

(This is not specific to md btw)
Im not sure what the solution can be since multiline cells differ between the styles of tables.
md tables need to use <br> instead of \n for new lines within the cell, while that would make them look bad printed, they would render correctly in a markdown viewer.

rst is even more complicated, not sure how multiline works there.

If you have any idea for a solution you want to do, id be happy to try and help to implement it.

`body_generator() takes 0 positional arguments` when using 0 fmt

rapidtables.format_table(table=data, fmt=0)

    result += '\n'.join(body_generator(False))
TypeError: body_generator() takes 0 positional arguments but 1 was given

Happens here, body_generator is called with "False".

object of type 'int' has no len()

Just using print_table(data) in Python3.74 I get the error below:

File "/home/krypterro/PyApps/cbstats/lib/python3.7/site-packages/rapidtables/init.py", line 442, in print_table
print(make_table(*args, **kwargs))
File "/home/krypterro/PyApps/cbstats/lib/python3.7/site-packages/rapidtables/init.py", line 428, in make_table
wrap_text=wrap_text)
File "/home/krypterro/PyApps/cbstats/lib/python3.7/site-packages/rapidtables/init.py", line 141, in format_table
hklen = len(headers[ki]) if headers else len(k)
TypeError: object of type 'int' has no len()