Report&Alert Format about superset

[rusackas]

Hmm... I don't really use the text-based reports, so I'll CC @yousoph here for more context/info. Did they ever support these formatters, or is it just the raw data from the database? The date formatters are picked up by the various visualizations to use as they need to... if we need to add support for formatters in text-based reports where it didn't exist before, then that seems more like a new feature request.

[liangliangGit]

Hmm... I don't really use the text-based reports, so I'll CC @yousoph here for more context/info. Did they ever support these formatters, or is it just the raw data from the database? The date formatters are picked up by the various visualizations to use as they need to... if we need to add support for formatters in text-based reports where it didn't exist before, then that seems more like a new feature request.

Yes, the data is retrieved from the database, and some formats have been implemented while others haven't. Here's a summary of my findings:
,d (12345.432 => 12,345) NG
.1s (12345.432 => 10k) NG
.3s (12345.432 => 12.3k) NG
,.1% (12345.432 => 1,234,543.2%) OK
.2% (12345.432 => 1234543.20%) OK
.3% (12345.432 => 1234543.200%) OK
.4r (12345.432 => 12350) NG
,.1f (12345.432 => 12,345.4) OK
,.2f (12345.432 => 12,345.43) OK
,.3f (12345.432 => 12,345.432) OK
"+, (12345.432 => +12,345) " NG
$,.2f (12345.432 => $12,345.43) NG
Duration in ms (66000 => 1m 6s) NG
Duration in ms (1.40008 => 1ms 400µs 80ns) NG
%d/%m/%Y | 14/01/2019 NG
%m/%d/%Y | 01/14/2019 NG
%Y-%m-%d | 2019-01-14 NG
%Y-%m-%d %H:%M:%S | 2019-01-14 01:32:10 OK
%d-%m-%Y %H:%M:%S | 14-01-2019 01:32:10 NG
%H:%M:%S | 01:32:10 NG
Prefix ¥ (JPY) NG
Suffix ¥ (JPY) NG

[rusackas]

Oh, interesting... I didn't know it was a mixed bag... I'm not sure why some are being picked up, but not others.

CC @eschutho in case you happen to know which gear isn't meshing.

[liangliangGit]

Below is a section of Python backend formatting code (found within the 'post_processing.py' file). It is deemed accurate if the Python format method aligns exactly with the d3format syntax, for instance, when using .1%.

def table(
    df: pd.DataFrame,
    form_data: dict[str, Any],
    datasource: Optional[  # pylint: disable=unused-argument
        Union["BaseDatasource", "Query"]
    ] = None,
) -> pd.DataFrame:
    """
    Table.
    """
    # apply `d3NumberFormat` to columns, if present
    column_config = form_data.get("column_config", {})
    for column, config in column_config.items():
        if "d3NumberFormat" in config:
            format_ = "{:" + config["d3NumberFormat"] + "}"
            try:
                df[column] = df[column].apply(format_.format)
            except Exception:  # pylint: disable=broad-except
                # if we can't format the column for any reason, send as is
                pass

    return df