datapane / datapane

Build and share data reports in 100% Python

Docs | Examples |

Note: Datapane is no longer actively maintained.

We've made the difficult decision to stop driving this project and therefore we will no longer actively respond to issues or pull requests. If you would like to take over maintaining this project independently, please let us know so we can add a link to your forked project here.

The final release, 0.17.0, has been optimized to focus on free local report saving, with unused features and analytics removed, and libraries updated to allow usage for as long as possible.

Thank You.

Overview

Build interactive reports in seconds using Python.

Datapane makes it simple to build interactive reports in seconds using Python.

Import Datapane's Python library into your script or notebook and build reports programmatically by wrapping components such as:

• Pandas DataFrames
• Plots from Python visualization libraries such as Bokeh, Altair, Plotly, and Folium
• Markdown and text
• Files, such as images, PDFs, JSON data, etc.
• Interactive forms which run backend Python functions

Datapane reports are interactive and can also contain pages, tabs, drop downs, and more. Once created, reports can be exported as HTML, shared as standalone files, or embedded into your own application, where your viewers can interact with your data and visualizations.

Gallery

Check out example reports:

• https://github.com/datapane/examples

Getting Started

Check out our Quickstart to build a report in 3m.

Installing Datapane

The best way to install Datapane is through pip or conda.

pip

$ pip3 install -U datapane

conda

$ conda install -c conda-forge "datapane>=0.17.0"

Datapane also works well in hosted Jupyter environments such as Colab or Binder, where you can install as follows:

!pip3 install --quiet datapane

Examples

📊 Share plots, data, and more as reports

Create reports from pandas DataFrames, plots from your favorite libraries, and text.

import altair as alt
from vega_datasets import data
import datapane as dp

df = data.iris()
fig = (
    alt.Chart(df)
    .mark_point()
    .encode(
        x="petalLength:Q",
        y="petalWidth:Q",
        color="species:N"
    )
)
view = dp.Blocks(
    dp.Plot(fig),
    dp.DataTable(df)
)
dp.save_report(view, path="my_app.html")

🎛 Layout using interactive blocks

Add dropdowns, selects, grid, pages, and 10+ other interactive blocks.

...

view = dp.Blocks(
    dp.Formula("x^2 + y^2 = z^2"),
    dp.Group(
        dp.BigNumber(
            heading="Number of percentage points",
            value="84%",
            change="2%",
            is_upward_change=True
        ),
        dp.BigNumber(
            heading="Simple Statistic", value=100
        ), columns=2
    ),
    dp.Select(
        dp.Plot(fig, label="Chart"),
        dp.DataTable(df, label="Data")
    ),
)
dp.save_report(view, path="layout_example.html")

Next Steps

• Quickstart - build a report in 3m
• View Examples
• Read the documentation

datapane's Issues

Unable to install through conda-forge on Ubuntu 20.04

Describe the bug
Hello, I'm unable to get Datapane installed on my machine. When I run the command conda install -c conda-forge datapane it progresses through a few "Solving Environment" steps, but fails each time. As a first workaround step, I tried adding conda-forge to my Conda channels and then simply running conda install datapane, but that results in the same problem. I've installed other packages using conda-forge on this machine, so I do think the issue is specific to Datapane and not a general Conda problem on my end (unless there's a specific version of Conda I should be running?). Any help is appreciated!

Desktop (please complete the following information):

• OS: Ubuntu 20.04 LTS
• Conda version: 4.8.4

Option to hide action menu (e.g. "View Source") in Altair plots

Is your feature request related to a problem? Please describe.
Altair plots always get rendered with the action menu (e.g. "View Source")

Describe the solution you'd like
Include option to hide this part

Describe alternatives you've considered
The alternative is to not use Datapane and rather rely on Altair's embed options (see here)

Additional context
n/a

CI Failed.

Describe the bug
CI failed in #28 .

To Reproduce
https://github.com/datapane/datapane/actions/runs/322515583

See Run Tests l.113 in datapane-multi-env (3.6, ubuntu-latest).

dp.Report failed when using docker image

Following the datapane configuration guidelines, I created a giswqs/geemap-datapane docker image based on nstack/datapane-python-runner. The Python script failed at the last line dp.Report(dp.Plot(Map)).publish(name='geemap_folium_map') with a TypeError: can't pickle weakref objects. Any advice? Thanks.

https://datapane.com/giswqs/scripts/geemap_test/

Dockerfile

from nstack/datapane-python-runner:latest
RUN pip3 install --user -U geemap

datapane.yaml

name: geemap_test
script: geemap_test.py # this could also be ipynb if it was a notebook
container_image_name: giswqs/geemap-datapane

geemap.test.py

import folium
import datapane as dp 
import geemap.eefolium as geemap
import ee
import os

secret = dp.Variable.get(name='EARTHENGINE_TOKEN')
ee_token = secret.value

credential = '{"refresh_token":"%s"}' % ee_token
credential_file_path = os.path.expanduser("~/.config/earthengine/")
os.makedirs(credential_file_path, exist_ok=True)
with open(credential_file_path + 'credentials', 'w') as file:
    file.write(credential)

# m = folium.Map(location=[45.5236, -122.6750])
ee.Initialize()
Map = geemap.Map(center=[40,-100], zoom=4)

# Add Earth Engine dataset
image = ee.Image('USGS/SRTMGL1_003')

# Set visualization parameters.
vis_params = {
  'min': 0,
  'max': 4000,
  'palette': ['006633', 'E5FFCC', '662A00', 'D8D8D8', 'F5F5F5']}

# Print the elevation of Mount Everest.
xy = ee.Geometry.Point([86.9250, 27.9881])
elev = image.sample(xy, 30).first().get('elevation').getInfo()
print('Mount Everest elevation (m):', elev)

# Add Earth Engine layers to Map
Map.addLayer(image, vis_params, 'DEM')
Map.addLayer(xy, {'color': 'red'}, 'Mount Everest')

# Center the map based on an Earth Engine object or coordinates (longitude, latitude)
# Map.centerObject(xy, 4)
Map.setCenter(86.9250, 27.9881, 4)

# Map.addLayerControl() # This line is not needed for ipyleaflet-based Map.
Map.setControlVisibility()

dp.Report(dp.Plot(Map)).publish(name='geemap_folium_map')

Can't run tutorial under Debian 10 in WSL2

Describe the bug
New user; just signed up. Logged into https://datapane.com/home/ and followed instructions.

To Reproduce

1. Start Debian 10 under WSL2 on Windows 10
2. goto https://datapane.com/home/
3. Copy script to rept.py
4. Do this:

sudo apt update
sudo apt install -y python3-pip
sudo apt install build-essential libssl-dev libffi-dev python3-dev
pip3 install datapane
python3 ./rept.py

4. Get error

Expected behavior
Per the instructions, I should be able to refresh that page and see results.

Screenshots
Error from running rept.py

Connected successfully to https://datapane.com as cek
Publishing report and associated data - please wait..
Traceback (most recent call last):
  File "./rept.py", line 26, in <module>
    r.publish(name=f'COVID-19 in {" ".join(locations.iso_code)}', open=True, tweet=True, description=f'COVID-19 in {", ".join(locations.location)}')
  File "/home/tig/.local/lib/python3.7/site-packages/datapane/client/api/report.py", line 384, in publish
    res = Resource(self.endpoint).post_files(dict(attachments=attachments), document=report_str, **kwargs)
  File "/home/tig/.local/lib/python3.7/site-packages/datapane/client/api/common.py", line 204, in post_files    e = MultipartEncoder(fields=fields)
  File "/home/tig/.local/lib/python3.7/site-packages/requests_toolbelt/multipart/encoder.py", line 125, in __init__
    self._prepare_parts()
  File "/home/tig/.local/lib/python3.7/site-packages/requests_toolbelt/multipart/encoder.py", line 246, in _prepare_parts
    self.parts = [Part.from_field(f, enc) for f in self._iter_fields()]
  File "/home/tig/.local/lib/python3.7/site-packages/requests_toolbelt/multipart/encoder.py", line 246, in <listcomp>
    self.parts = [Part.from_field(f, enc) for f in self._iter_fields()]
  File "/home/tig/.local/lib/python3.7/site-packages/requests_toolbelt/multipart/encoder.py", line 495, in from_field
    return cls(headers, body)
  File "/home/tig/.local/lib/python3.7/site-packages/requests_toolbelt/multipart/encoder.py", line 488, in __init__
    self.len = len(self.headers) + total_len(self.body)
  File "/home/tig/.local/lib/python3.7/site-packages/requests_toolbelt/multipart/encoder.py", line 432, in total_len
    if hasattr(o, 'len'):
  File "/home/tig/.local/lib/python3.7/site-packages/requests_toolbelt/multipart/encoder.py", line 573, in len
    return total_len(self.fd) - self.fd.tell()
  File "/home/tig/.local/lib/python3.7/site-packages/requests_toolbelt/multipart/encoder.py", line 441, in total_len
    return os.fstat(fileno).st_size
FileNotFoundError: [Errno 2] No such file or directory

Desktop (please complete the following information):

• OS: Windows 10
• Browser: Edge
• Python: 3.7.3

Markdown Table not rendering

Originally posted by @yelotmani in #46

Hi,
I’ve been using datapane for about a month now and i really like it. I established a daily report that uses a mix of Tables, Plots and markdown text and markdown table. it’s been working great until today where the markdown table have stopped rendering, ie instead of getting a table in the browser I get the raw markdown text.
Below the python code I’ve been using for the past month applied on a simple example:
Data

field1,field2,field3,field4
Name1,1,2,3
Name2,1,2,3
Name3,1,2,3
Name4,1,2,3
Name5,1,2,3
Name6,1,2,3
Name7,1,2,3
Name8,1,2,3
Name9,1,2,3

Python code:

my_df = pd.read_csv(‘test.csv’)
#Define report
r = dp.Report(
dp.Markdown(my_df.to_markdown()),
)
#Save HTML
r.save(path = ‘report.html’)

This generates the following HTML code

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <script src="https://use.fontawesome.com/dfa0b8a2ce.js"></script>
    <link
      href="https://fonts.googleapis.com/css?family=Inter:400,500,600,700|Inconsolata|Karla|Rubik&display=swap"
      rel="stylesheet"
    />
    <script src="https://cdnjs.cloudflare.com/ajax/libs/plotly.js/1.54.7/plotly.min.js"></script>
    <script>
      window.dpLocal = true;
    </script>
    
      <link href="https://datapane.com/static/base.css" rel="stylesheet"/>
      <link href="https://datapane.com/static/report-base.css" rel="stylesheet"/>
      <script src="https://datapane.com/static/base.js"></script>
      <script src="https://datapane.com/static/report-base.js"></script>
    
    <style>
      body {
        background-color: #f4f5f7;
      }

    </style>
  </head>
  <body>
    <div class="max-w-screen-xl mx-auto pb-6 sm:px-6 lg:px-8">
      <div class="px-4 py-6 sm:px-0">
        <div class="shadow-lg">
          <div id="report-root"></div>
        </div>
      </div>
    </div>
    <script>
      const report = { document: `<Report version="1" xml:id="_29731c5fe2a24ae1b5d3b7d954be021b">
  <Meta>
    <Author>Anonymous</Author>
    <CreatedOn>2020-12-07T15:19:43Z</CreatedOn>
    <Title>Local Report</Title>
    <Description><![CDATA[Description]]></Description>
  </Meta>
  <Main full_width="false">
    <Blocks rows="0" columns="1" id="block-1149">
      <Text id="block-1148"><![CDATA[|    | field1   |   field2 |   field3 |   field4 |
|---:|:---------|---------:|---------:|---------:|
|  0 | Name1    |        1 |        2 |        3 |
|  1 | Name2    |        1 |        2 |        3 |
|  2 | Name3    |        1 |        2 |        3 |
|  3 | Name4    |        1 |        2 |        3 |
|  4 | Name5    |        1 |        2 |        3 |
|  5 | Name6    |        1 |        2 |        3 |
|  6 | Name7    |        1 |        2 |        3 |
|  7 | Name8    |        1 |        2 |        3 |
|  8 | Name9    |        1 |        2 |        3 |]]></Text>
    </Blocks>
  </Main>
</Report>
` };
      const { React, ReactDOM, ReportComponent } = UMDModules;
      const ReportFactory = React.createElement.bind(null, ReportComponent);
      ReactDOM.render(
        ReportFactory({ report }),
        document.querySelector("#report-root")
      );
    </script>
  </body>
</html>

And renders as raw markdown text instead of table. This happened over the weekend so wondering if something changed in the datapane stylers imported at the beginning of the generated HTML report?

Any advice is welcome.

Thanks!

Report layout customisation

Overview

As someone building a report, I want to be able to customise the layout in a grid, instead of just having a sequential list of blocks, so that I can have more flexible reports.

This has come up with a good chunk of current users, and is something that should be made a lot easier by the recent migration to an XML-backed report structure.

Implementation Ideas

There are a few ways we could structure the end-user API. I've chatted to a few users, and the simplest to start with seems to be having some column and row primitives, but I'm open to suggestions.

We could wrap the Tailwind flex/grid html classes into some Python classes, like this:

dp.Report(
  dp.Row(
    dp.Column(dp.Table(df), width='1/3'),
    dp.Column(dp.Table(df), width='2/3')
  )
)

Other options considered

The other options I considered were: allowing the user to actually write HTML/CSS, but many people writing Python reports may not know these, and it becomes very complex. Alternatively, I considered having the properties on the blocks themselves...

dp.Report(
    dp.Table(df, width='200px'),
    dp.Table(df, width='2/3') # can't see how this would work
)

But I feel this is the wrong level of control, as we'd rather push to a proper grid system.

Notes

• Some of the blocks we support are not responsive (i.e. Altair) - so we would need to document this. How do we deal with overflow?

Getting ImportError for 'json_item' when running dp.Plot in the Datapane demo script

System Information

• OS: Win 10
• Python version: 3.8.8
• Python environment: conda
• Error occurs when running the script in JupyterLab or Spyder
• Datapane version: 0.10.3

Bug / Issue

Getting ImportError for 'json_item' when running dp.Plot in the Datapane demo script

Here is the script:

import altair as alt
import pandas as pd
import datapane as dp
from datetime import date

# Sign-in with your unique token
dp.login(token="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")

dataset = pd.read_csv('https://covid.ourworldindata.org/data/owid-covid-data.csv')
locations = dataset[['location', 'iso_code']].sample(3) # Replace with you own locations to customize!
df = dataset[dataset.iso_code.isin(locations.iso_code)]

plot = alt.Chart(df).mark_area(opacity=0.4, stroke='black').encode(
    x='date:T', y=alt.Y('new_cases_smoothed_per_million:Q', stack=None),
    color=alt.Color('location:N', scale=alt.Scale(scheme='set1')), tooltip='location:N'
).interactive().properties(width='container')

# Create report
r = dp.Report(
    f'### Comparing COVID cases in {", ".join(locations.location)}',
    f'_Built using data from [Our World in Data](https://ourworldindata.org/) on {date.today()}_',
    dp.Plot(plot),
    dp.DataTable(df),
)

# Publish
r.publish(name=f'COVID-19 in {" ".join(locations.iso_code)}', open=True, description=f'COVID-19 in {", ".join(locations.location)}')

Here is the error traceback:

Connected successfully to https://datapane.com as dreme
Traceback (most recent call last):

  File "C:\Users\emeau\OneDrive\IT\Python\Scripts\Datapane demo.py", line 30, in <module>
    dp.Plot(plot),

  File "C:\Users\emeau\anaconda3\envs\py38\lib\site-packages\datapane\client\api\report\blocks.py", line 642, in __init__
    out_fn = self._save_obj(data, as_json=False)

  File "C:\Users\emeau\anaconda3\envs\py38\lib\site-packages\datapane\client\api\report\blocks.py", line 578, in _save_obj
    from ..files import save

  File "C:\Users\emeau\anaconda3\envs\py38\lib\site-packages\datapane\client\api\files.py", line 20, in <module>
    from .files_optional import Axes, BFigure, BLayout, Figure, Map, PFigure, json_item

ImportError: cannot import name 'json_item' from 'datapane.client.api.files_optional' (C:\Users\emeau\anaconda3\envs\py38\lib\site-packages\datapane\client\api\files_optional.py)

Here is a conda list of the python modules in the env

(py38) C:\Users\emeau>conda list
# packages in environment at C:\Users\emeau\anaconda3\envs\py38:
#
# Name                    Version                   Build  Channel
abseil-cpp                20200923.3           h0e60522_0    conda-forge
alabaster                 0.7.12                     py_0    conda-forge
altair                    4.1.0                      py_1    conda-forge
anyio                     2.2.0            py38haa244fe_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
argh                      0.26.2          pyh9f0ad1d_1002    conda-forge
argon2-cffi               20.1.0           py38h294d835_2    conda-forge
arrow-cpp                 3.0.0           py38had945b9_9_cpu    conda-forge
astroid                   2.5.1            py38haa244fe_0    conda-forge
async_generator           1.10                       py_0    conda-forge
atomicwrites              1.4.0              pyh9f0ad1d_0    conda-forge
attrs                     20.3.0             pyhd3deb0d_0    conda-forge
autopep8                  1.5.6              pyhd8ed1ab_0    conda-forge
aws-c-cal                 0.4.5                hd123ed8_8    conda-forge
aws-c-common              0.5.2                h8ffe710_0    conda-forge
aws-c-event-stream        0.2.7                h828c5a6_1    conda-forge
aws-c-io                  0.9.1                h92767cf_1    conda-forge
aws-checksums             0.1.11               h92767cf_3    conda-forge
aws-sdk-cpp               1.8.151              ha9f9437_1    conda-forge
babel                     2.9.0              pyhd3deb0d_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.1                      py_0    conda-forge
bcrypt                    3.2.0            py38h294d835_1    conda-forge
black                     20.8b1                     py_1    conda-forge
bleach                    3.3.0              pyh44b312d_0    conda-forge
blosc                     1.21.0               h0e60522_0    conda-forge
boltons                   20.2.1             pyh9f0ad1d_0    conda-forge
brotli                    1.0.9                h0e60522_4    conda-forge
brotlipy                  0.7.0           py38h294d835_1001    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
c-ares                    1.17.1               h8ffe710_1    conda-forge
ca-certificates           2020.12.5            h5b45459_0    conda-forge
certifi                   2020.12.5        py38haa244fe_1    conda-forge
cffi                      1.14.5           py38hd8c33c5_0    conda-forge
chardet                   4.0.0            py38haa244fe_1    conda-forge
click                     7.1.2              pyh9f0ad1d_0    conda-forge
click-spinner             0.1.10             pyh9f0ad1d_0    conda-forge
cloudpickle               1.6.0                      py_0    conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
colorlog                  4.7.2            py38haa244fe_0    conda-forge
confuse                   1.4.0              pyhd3deb0d_0    conda-forge
cryptography              3.4.6            py38hd7da0ea_0    conda-forge
curl                      7.75.0               hf1763fc_0    conda-forge
cycler                    0.10.0                     py_2    conda-forge
dacite                    1.6.0              pyhd3deb0d_0    conda-forge
dataclasses               0.8                pyhc8e2a94_1    conda-forge
datapane                  0.10.3             pyhd8ed1ab_0    conda-forge
decorator                 4.4.2                      py_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
diff-match-patch          20200713           pyh9f0ad1d_0    conda-forge
docutils                  0.16             py38haa244fe_3    conda-forge
dominate                  2.4.0                      py_0    conda-forge
double-conversion         3.1.5                h6538335_2    conda-forge
eigen                     3.3.9                h2d74725_1    conda-forge
entrypoints               0.3             pyhd8ed1ab_1003    conda-forge
et_xmlfile                1.0.1                   py_1001    conda-forge
expat                     2.2.10               h39d44d4_0    conda-forge
face                      20.1.1                     py_0    conda-forge
ffmpeg                    4.3.1                ha925a31_0    conda-forge
flake8                    3.8.4                      py_0    conda-forge
flit-core                 3.0.0                      py_1    conda-forge
freetype                  2.10.4               h546665d_1    conda-forge
furl                      2.1.0                      py_0    conda-forge
future                    0.18.2           py38haa244fe_3    conda-forge
gflags                    2.2.2             ha925a31_1004    conda-forge
gl2ps                     1.4.2                h0597ee9_0    conda-forge
glew                      2.1.0                h39d44d4_2    conda-forge
glob2                     0.7                        py_0    conda-forge
glog                      0.4.0                h0174b99_3    conda-forge
glom                      20.11.0            pyhd3deb0d_0    conda-forge
grpc-cpp                  1.36.4               hd4140ee_0    conda-forge
hdf4                      4.2.13            h0e5069d_1004    conda-forge
hdf5                      1.10.6          nompi_h5268f04_1114    conda-forge
helpdev                   0.7.1              pyhd8ed1ab_0    conda-forge
htmlmin                   0.1.12                     py_1    conda-forge
icu                       68.1                 h0e60522_0    conda-forge
idna                      2.10               pyh9f0ad1d_0    conda-forge
imagehash                 4.2.0              pyhd8ed1ab_0    conda-forge
imagesize                 1.2.0                      py_0    conda-forge
importlib-metadata        3.7.3            py38haa244fe_0    conda-forge
importlib_metadata        3.7.3                hd8ed1ab_0    conda-forge
importlib_resources       5.1.2            py38haa244fe_0    conda-forge
intel-openmp              2020.3             h57928b3_311    conda-forge
intervaltree              3.0.2                      py_0    conda-forge
ipyevents                 0.8.2              pyhd8ed1ab_0    conda-forge
ipykernel                 5.5.0            py38hc5df569_1    conda-forge
ipympl                    0.6.3              pyhd8ed1ab_0    conda-forge
ipython                   7.21.0           py38hc5df569_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.6.3              pyhd3deb0d_0    conda-forge
isort                     5.8.0              pyhd8ed1ab_0    conda-forge
jdcal                     1.4.1                      py_0    conda-forge
jedi                      0.17.2           py38haa244fe_1    conda-forge
jinja2                    2.11.3             pyh44b312d_0    conda-forge
joblib                    1.0.1              pyhd8ed1ab_0    conda-forge
jpeg                      9d                   h8ffe710_0    conda-forge
json5                     0.9.5              pyh9f0ad1d_0    conda-forge
jsoncpp                   1.9.4                h2d74725_1    conda-forge
jsonschema                3.2.0              pyhd8ed1ab_3    conda-forge
jupyter-packaging         0.7.12             pyhd8ed1ab_0    conda-forge
jupyter_client            6.1.12             pyhd8ed1ab_0    conda-forge
jupyter_contrib_core      0.3.3                      py_2    conda-forge
jupyter_contrib_nbextensions 0.5.1              pyhd8ed1ab_2    conda-forge
jupyter_core              4.7.1            py38haa244fe_0    conda-forge
jupyter_highlight_selected_word 0.2.0           py38haa244fe_1002    conda-forge
jupyter_latex_envs        1.4.6           pyhd8ed1ab_1002    conda-forge
jupyter_nbextensions_configurator 0.4.1            py38haa244fe_2    conda-forge
jupyter_server            1.4.1            py38haa244fe_0    conda-forge
jupyterlab                3.0.12             pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.1.2              pyh9f0ad1d_0    conda-forge
jupyterlab_server         2.3.0              pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        1.0.0              pyhd8ed1ab_1    conda-forge
keyring                   23.0.0           py38haa244fe_0    conda-forge
kiwisolver                1.3.1            py38hbd9d945_1    conda-forge
krb5                      1.17.2               hbae68bd_0    conda-forge
lazy-object-proxy         1.5.2            py38h294d835_1    conda-forge
lcms2                     2.12                 h2a16943_0    conda-forge
libblas                   3.9.0                     8_mkl    conda-forge
libcblas                  3.9.0                     8_mkl    conda-forge
libclang                  11.1.0          default_h5c34c98_0    conda-forge
libcurl                   7.75.0               hf1763fc_0    conda-forge
libiconv                  1.16                 he774522_0    conda-forge
liblapack                 3.9.0                     8_mkl    conda-forge
libnetcdf                 4.7.4           nompi_h3a9aa94_107    conda-forge
libogg                    1.3.4                h8ffe710_1    conda-forge
libpng                    1.6.37               h1d00b33_2    conda-forge
libpq                     13.1                 h4f54205_2    conda-forge
libprotobuf               3.15.6               h7755175_0    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libspatialindex           1.9.3                h39d44d4_3    conda-forge
libssh2                   1.9.0                h680486a_6    conda-forge
libtheora                 1.1.1             h62dcd97_1004    conda-forge
libthrift                 0.14.1               h636ae23_1    conda-forge
libtiff                   4.2.0                hc10be44_0    conda-forge
libutf8proc               2.6.1                hcb41399_0    conda-forge
libxml2                   2.9.10               hf5bbc77_3    conda-forge
libxslt                   1.1.33               h65864e5_2    conda-forge
llvmlite                  0.36.0           py38h57a6900_0    conda-forge
loguru                    0.5.3            py38h32f6830_2    conda-forge
lxml                      4.6.2            py38h292cb97_1    conda-forge
lz4-c                     1.9.3                h8ffe710_0    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
markupsafe                1.1.1            py38h294d835_3    conda-forge
matplotlib                3.3.4            py38haa244fe_0    conda-forge
matplotlib-base           3.3.4            py38h34ddff4_0    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
micawber                  0.5.2              pyhd8ed1ab_0    conda-forge
missingno                 0.4.2                      py_1    conda-forge
mistune                   0.8.4           py38h294d835_1003    conda-forge
mkl                       2020.4             hb70f87d_311    conda-forge
mock                      4.0.3            py38haa244fe_1    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
munch                     2.5.0                      py_0    conda-forge
mypy_extensions           0.4.3            py38haa244fe_3    conda-forge
nbclassic                 0.2.6              pyhd8ed1ab_0    conda-forge
nbclient                  0.5.3              pyhd8ed1ab_0    conda-forge
nbconvert                 6.0.7            py38haa244fe_3    conda-forge
nbformat                  5.1.2              pyhd8ed1ab_1    conda-forge
nest-asyncio              1.4.3              pyhd8ed1ab_0    conda-forge
networkx                  2.5                        py_0    conda-forge
nodejs                    15.12.0              h57928b3_0    conda-forge
notebook                  6.2.0            py38haa244fe_0    conda-forge
numba                     0.53.0           py38h5c177ec_1    conda-forge
numexpr                   2.7.3            py38h4c96930_0    conda-forge
numpy                     1.20.1           py38h0cc643e_0    conda-forge
numpydoc                  1.1.0                      py_1    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
openpyxl                  3.0.7              pyhd8ed1ab_0    conda-forge
openssl                   1.1.1k               h8ffe710_0    conda-forge
orderedmultidict          1.0.1                      py_0    conda-forge
packaging                 20.9               pyh44b312d_0    conda-forge
pandas                    1.2.3            py38h4c96930_0    conda-forge
pandas-datareader         0.9.0              pyh44b312d_0    conda-forge
pandas-profiling          2.11.0             pyhd8ed1ab_0    conda-forge
pandoc                    2.12                 h8ffe710_0    conda-forge
pandocfilters             1.4.2                      py_1    conda-forge
paramiko                  2.7.2              pyh9f0ad1d_0    conda-forge
parquet-cpp               1.5.1                         2    conda-forge
parso                     0.7.0              pyh9f0ad1d_0    conda-forge
pathspec                  0.8.1              pyhd3deb0d_0    conda-forge
patsy                     0.5.1                      py_0    conda-forge
pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
phik                      0.11.2             pyhd8ed1ab_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    8.1.2            py38h9273828_0    conda-forge
pip                       21.0.1             pyhd8ed1ab_0    conda-forge
pivottablejs              0.9.0            py38haa95532_0
pluggy                    0.13.1           py38haa244fe_4    conda-forge
proj                      7.2.0                h1cfcee9_2    conda-forge
prometheus_client         0.9.0              pyhd3deb0d_0    conda-forge
prompt-toolkit            3.0.17             pyha770c72_0    conda-forge
psutil                    5.8.0            py38h294d835_1    conda-forge
psycopg2                  2.8.6            py38hd8c33c5_2    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pugixml                   1.11.4               h0e60522_0    conda-forge
pyarrow                   3.0.0           py38ha37a76c_9_cpu    conda-forge
pycodestyle               2.6.0              pyh9f0ad1d_0    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pydocstyle                6.0.0              pyhd8ed1ab_0    conda-forge
pyflakes                  2.2.0              pyh9f0ad1d_0    conda-forge
pygments                  2.8.1              pyhd8ed1ab_0    conda-forge
pylint                    2.7.2            py38haa244fe_0    conda-forge
pyls-black                0.4.6              pyh9f0ad1d_0    conda-forge
pyls-spyder               0.3.2              pyhd8ed1ab_0    conda-forge
pynacl                    1.4.0            py38h31c79cd_2    conda-forge
pyopenssl                 20.0.1             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pyqt                      5.12.3           py38haa244fe_7    conda-forge
pyqt-impl                 5.12.3           py38h885f38d_7    conda-forge
pyqt5-sip                 4.19.18          py38h885f38d_7    conda-forge
pyqtchart                 5.12             py38h885f38d_7    conda-forge
pyqtwebengine             5.12.1           py38h885f38d_7    conda-forge
pyrsistent                0.17.3           py38h294d835_2    conda-forge
pysocks                   1.7.1            py38haa244fe_3    conda-forge
pytables                  3.6.1            py38h153c448_3    conda-forge
python                    3.8.8           h7840368_0_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python-jsonrpc-server     0.4.0              pyh9f0ad1d_0    conda-forge
python-language-server    0.36.2             pyhd8ed1ab_0    conda-forge
python_abi                3.8                      1_cp38    conda-forge
pytoml                    0.1.21                     py_0    conda-forge
pytz                      2021.1             pyhd8ed1ab_0    conda-forge
pywavelets                1.1.1            py38h347fdf6_3    conda-forge
pywin32                   300              py38h294d835_0    conda-forge
pywin32-ctypes            0.2.0           py38haa244fe_1003    conda-forge
pywinpty                  0.5.7            py38h32f6830_1    conda-forge
pyyaml                    5.4.1            py38h294d835_0    conda-forge
pyzmq                     22.0.3           py38h09162b1_1    conda-forge
qdarkstyle                2.8.1              pyhd8ed1ab_2    conda-forge
qt                        5.12.9               h5909a2a_4    conda-forge
qtawesome                 1.0.2              pyhd8ed1ab_0    conda-forge
qtconsole                 5.0.3              pyhd8ed1ab_0    conda-forge
qtpy                      1.9.0                      py_0    conda-forge
re2                       2020.11.01           h0e60522_0    conda-forge
regex                     2021.3.17        py38h294d835_0    conda-forge
requests                  2.25.1             pyhd3deb0d_0    conda-forge
requests-toolbelt         0.9.1                      py_0    conda-forge
rope                      0.18.0             pyh9f0ad1d_0    conda-forge
rtree                     0.9.7            py38h8b54edf_1    conda-forge
scipy                     1.6.1            py38h5f893b4_0    conda-forge
seaborn                   0.11.1               hd8ed1ab_1    conda-forge
seaborn-base              0.11.1             pyhd8ed1ab_1    conda-forge
send2trash                1.5.0                      py_0    conda-forge
setuptools                49.6.0           py38haa244fe_3    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
snappy                    1.1.8                ha925a31_3    conda-forge
sniffio                   1.2.0            py38haa244fe_1    conda-forge
snowballstemmer           2.1.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.3.0              pyhd8ed1ab_0    conda-forge
sphinx                    3.5.3              pyhd8ed1ab_0    conda-forge
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    1.0.3                      py_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.4                      py_0    conda-forge
spyder                    4.2.3            py38haa244fe_0    conda-forge
spyder-kernels            1.10.2           py38haa244fe_0    conda-forge
sqlite                    3.34.0               h8ffe710_0    conda-forge
statsmodels               0.12.2           py38h347fdf6_0    conda-forge
stringcase                1.2.0                      py_0    conda-forge
tabulate                  0.8.9              pyhd8ed1ab_0    conda-forge
tangled-up-in-unicode     0.0.7              pyhd8ed1ab_0    conda-forge
tbb                       2020.2               h2d74725_4    conda-forge
tbb-devel                 2020.2               h2d74725_4    conda-forge
terminado                 0.9.3            py38haa244fe_0    conda-forge
testpath                  0.4.4                      py_0    conda-forge
textdistance              4.2.1              pyhd8ed1ab_0    conda-forge
three-merge               0.1.1              pyh9f0ad1d_0    conda-forge
tk                        8.6.10               h8ffe710_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
toolz                     0.11.1                     py_0    conda-forge
tornado                   6.1              py38h294d835_1    conda-forge
tqdm                      4.59.0             pyhd8ed1ab_0    conda-forge
traitlets                 5.0.5                      py_0    conda-forge
typed-ast                 1.4.2            py38h294d835_0    conda-forge
typing_extensions         3.7.4.3                    py_0    conda-forge
ujson                     4.0.2            py38h885f38d_0    conda-forge
urllib3                   1.26.4             pyhd8ed1ab_0    conda-forge
utfcpp                    3.1.2                         0    conda-forge
validators                0.18.2             pyhd3deb0d_0    conda-forge
vc                        14.2                 hb210afc_4    conda-forge
visions                   0.6.0              pyh9f0ad1d_0    conda-forge
vs2015_runtime            14.28.29325          h5e1d092_4    conda-forge
vtk                       9.0.1           no_osmesa_py38h3499093_104    conda-forge
watchdog                  1.0.2            py38haa244fe_1    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
widgetsnbextension        3.5.1            py38haa244fe_4    conda-forge
win32_setctime            1.0.3                      py_0    conda-forge
win_inet_pton             1.1.0            py38haa244fe_2    conda-forge
wincertstore              0.2             py38haa244fe_1006    conda-forge
winpty                    0.4.3                         4    conda-forge
wrapt                     1.12.1           py38h294d835_3    conda-forge
xlrd                      2.0.1              pyhd8ed1ab_3    conda-forge
xz                        5.2.5                h62dcd97_1    conda-forge
yaml                      0.2.5                he774522_0    conda-forge
yapf                      0.30.0             pyh9f0ad1d_0    conda-forge
zeromq                    4.3.4                h0e60522_0    conda-forge
zipp                      3.4.1              pyhd8ed1ab_0    conda-forge
zlib                      1.2.11            h62dcd97_1010    conda-forge
zstd                      1.4.9                h6255e5f_0    conda-forge

Datapane Report templates

Is your feature request related to a problem? Please describe.

• Users should have a sample report code available when using the CLI

Describe the solution you'd like

• A new command, datapane report init that generates a sample report, that upon running generates a simple report for local / publishing usage

Describe alternatives you've considered

• We could tie this into GH template repos to pull down templates from hosted repos that can be updated / improved by the wider community

Datapane 0.9.0 Jupyter Previewing

Originally posted by @aleiyoo in #52

I updated datapane to 0.9.0 and then I could not preview the report in Jupyter

ValueError                                Traceback (most recent call last)
<ipython-input-3-7ae3c71b24af> in <module>
----> 1 report.preview()

~\AppData\Roaming\Python\Python38\site-packages\datapane\client\api\report\core.py in preview(self, width, height, standalone)
    283 
    284             # NOTE - iframe must be relative path
--> 285             iframe_src = self._tmp_report.relative_to(Path("..").absolute())
    286             return IFrame(src=str(iframe_src), width=width, height=height)
    287         else:

D:\anaconda3\lib\pathlib.py in relative_to(self, *other)
    902         if (root or drv) if n == 0 else cf(abs_parts[:n]) != cf(to_abs_parts):
    903             formatted = self._format_parsed_parts(to_drv, to_root, to_parts)
--> 904             raise ValueError("{!r} does not start with {!r}"
    905                              .format(str(self), str(formatted)))
    906         return self._from_parsed_parts('', root if n == 1 else '',

ValueError: 'C:\\Users\\mi\\dp-cache\\dp-tmp-lkga1svb\\dp-tmp-td5nsr44.html' does not start with 'C:\\Users\\mi\\..'```

UnicodeEncodeError: 'charmap' codec can't encode character

Hi am currently encountering an error when I try to run one of the codes in the tutorial.

import pandas as pd
import datapane as dp
import numpy as np


df = pd.DataFrame(
    {"A": np.random.normal(-1, 1, 5000), "B": np.random.normal(1, 2, 5000),}
)

table = dp.Table(df)
report = dp.Report(table)
report.save(path="test.html")

I get this error

UnicodeEncodeError: 'charmap' codec can't encode character '\ue6c6' in position 232206: character maps to

Machine & Installed Libraries

OS: Windows 10

Python versions tried

• Python 3.8.3
• Python 3.6.8

Modules installed:

• datapane 0.6.2
• pandas 1.0.3
• numpy 1.18.2

Attach new data to the previously created report

Hello,

I have a function that creates a report dynamically when the if the condition is satisfied, now in the else I have written a code which has to attach to the previous report. Is there a way to do that?

MPL figures are not responsive in reports

Describe the bug
If I create a matplotlib figure, it is not responsive. It used to be, but the zoom element seems to have removed this functionality. As our MPL figures are vectors, this means a user may see only 10% of the figure (on wider graphs), and has to zoom out to see the rest.

Expected behavior
The MPL figure is responsive width (i.e. width: 100%; height: auto)

Overwrite existing report

I would like to use datapane to publish Interactive Google Earth Engine (GEE) maps. However, GEE data layers are only valid for two to three days. After the token expire, users will need to regenerate GEE data layers.

If I use the following code publish the report the second time, datapane will generate a new URL with 8 random characters at the end (e.g., geemap_folium_map_ea074e4f). Is it possible to overwrite the existing report and keep the same URL (geemap_folium_map)? The reason for this is that if I already embed the link (geemap_folium_map) in a social media post, I could not keep going back to update the URL.

dp.Report(dp.Plot(Map)).publish(name='ee_folium_map')

The original report (GEE data layers already expired. It won't show up on the map)
https://datapane.com/giswqs/reports/geemap_folium_map/

The republished report with GEE layers, but the link has changed.
https://datapane.com/giswqs/reports/geemap_folium_map_ea074e4f/

Windows & Anaconda install improvements

Overview
Many windows users are using Anaconda, which has had occasional issues with installing.

Issues seen are:

• pip3 install datapane not installing pyarrow from the recommended wheel, instead trying to compile it from source (which usually fails due to missing libraries)
• Datapane not appearing in path post-install (i.e. you can run with python -m datapane.client but it's not in path)

Although we have windows environments in CI, these are based on the default windows Python package (i.e. what you get when you install Python from the Windows store), and not Anaconda.

We should investigate how to support all common combinations of Windows (10+) and Anaconda.

Notes

• Some users are on 32bit Anaconda Python - should we recommend 64bit only?
• The PATH issue may occur on default Windows install, but requires more testing
• Some Anaconda users are installing without issue, so we need to figure out why some are not pulling down the pyarrow wheel correctly

Add E2E tests for macOS and Windows

We currently only test E2E on linux version of the client - we should also run on mac and windows to make sure we run correctly on common platforms.

table generated using DataTable, shows column value as time duration(0 days 00:00:00.000000068)

Describe the bug
For below data, column value being shows as time duration on generated html report. Its happening only to that column. If i convert to int or some string that canot be convert to int(like adding any alpha character) its being shown expected.

converting below dictionary( json array) to DF and trying to generate table and it seems that one particular column being showed as time duration in report.
data = [{'a': 0, 'b': 8.222222222222221, 'c': -107.0, 'd': "68"}]
df = pd.DataFrame.from_dict(ch_table)

in the report i see 'd' column being displayed as time duration(0 days 00:00:00.000000068).

Add more supported plot formats

Overview

Reports currently support Matplotlib, Seaborn, Altair, and Bokeh.

In addition to these, Plotly has some amazing interactive plots, and it would be good to have support for maps and geoplots too. For this, folium (which exports to leaflet.js) provides a really nice full stack solution.

Fix windows support

Window support is currently broken, as we shell out to certain libraries (such as gzip). We should use Python alternatives instead.

Can't install with Python 3.9 under Windows 10

Describe the bug
Newish Windows 10 installation. I have Python 3.9.0 installed. Running pip3 install datapane fails with a massive error output and an error.

To Reproduce

0. Install Python 3.9.0 on Win10 and ensure it's working.
1. Start Windows 10 cmd prompt.
2. Run pip3 install datapane
3. Get error

Expected behavior
It installs

Screenshots

error: Command "C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\VC\Tools\MSVC\14.28.29515\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -DNPY_NEEDS_MINGW_TIME_WORKAROUND -Inumpy\core\include -Ibuild\src.win-amd64-3.9\numpy\core\include/numpy -Inumpy\core\src\common -Inumpy\core\src -Inumpy\core -Inumpy\core\src\npymath -Inumpy\core\src\multiarray -Inumpy\core\src\umath -Inumpy\core\src\npysort -Ic:\python39\include -Ic:\python39\include -Ibuild\src.win-amd64-3.9\numpy\core\src\common -Ibuild\src.win-amd64-3.9\numpy\core\src\npymath -Ibuild\src.win-amd64-3.9\numpy\core\src\common -Ibuild\src.win-amd64-3.9\numpy\core\src\npymath -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\VC\Tools\MSVC\14.28.29515\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\VC\Tools\MSVC\14.28.29515\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt /Tcnumpy\random\mtrand\mtrand.c /Fobuild\temp.win-amd64-3.9\Release\numpy\random\mtrand\mtrand.obj" failed with exit status 2
      ----------------------------------------
  ERROR: Command errored out with exit status 1: 'c:\python39\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\charlie\\AppData\\Local\\Temp\\pip-install-ocppwjbp\\numpy\\setup.py'"'"'; __file__='"'"'C:\\Users\\charlie\\AppData\\Local\\Temp\\pip-install-ocppwjbp\\numpy\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\charlie\AppData\Local\Temp\pip-record-3r3z4ygi\install-record.txt' --single-version-externally-managed --prefix 'C:\Users\charlie\AppData\Local\Temp\pip-build-env-lvrb5b68\overlay' --compile --install-headers 'C:\Users\charlie\AppData\Local\Temp\pip-build-env-lvrb5b68\overlay\Include\numpy' Check the logs for full command output.
  ----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\python39\python.exe' 'c:\python39\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\charlie\AppData\Local\Temp\pip-build-env-lvrb5b68\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'cython >= 0.29' 'numpy==1.14.5; python_version<'"'"'3.7'"'"'' 'numpy==1.16.0; python_version>='"'"'3.7'"'"'' setuptools setuptools_scm wheel Check the logs for full command output.

Desktop (please complete the following information):

• OS: Windows 10

Style formatted Pandas Dataframe with ,

Hi ,

I am using datapane table populated with formatted pandas dataframe. The issue is out of 6 column , one column transforms to date. Rest of the 5 columns display correctly with , formatted.

Can’t deserialize Blocks at page level

My reports were working until last week. This week when I returned , it’s not working any more. When I open the html page it shows blank. On chrome I see error as attached stating

Can’t deserialize Blocks at page level

While debugging on Edge it shows https://data pane.com/static/ReactToastify.css.map could not be loaded.

token and graph plotting issues

So, the issue that I am facing is that I am not able to publish the report on the data pane server.
it is showing the token is invalid but that's not the case I have rechecked it and the token seems fine.

The second issue is when I am trying to display graphs only the first dp.Plot() method is working and other graphs are shown blank.
I am using google colab with python version 3.6.7

Hope, to get the fix of this issue.

Print only seems to render portion of screen for printing rather than whole document

Describe the bug

When trying to print out the demo's, print only seems to render the portion of the screen for printing rather than the whole document.

Reason for trying this: My aim was to see whether a report can be printed to PDF.

To Reproduce
From browser choose print, then preview or print.

Expected behavior
The whole document should be printed.

• OS: Windows 10
• Browser: Chrome
• Version 84.0.4147.125

Allow local report assets to be loaded via CDN

Is your feature request related to a problem? Please describe
We bundle all of npm packages inside local reports whether they're needed or not. This leads to reports being unnecessarily bloated -- see #24 and this thread for examples.

Describe the solution you'd like
We provide a use_cdn option on the Report constructor that allows the user to load our entire report bundle (report source code + npm assets) via our own CDN. This bundle can be code-split as our hosted instance is, to prevent unnecessary packages being sent over.

Describe alternatives you've considered

• Inspecting the report XML prior to local report generation, then including only the required JS chunks needed by the assets inside each specific report. However parcel expects code-split chunks to be loaded over HTTP, and I can't see anything on their docs to change that.
• Compiling the report bundle on-the-fly inside Report.save so that we can shake off unused packages. However this would make the save method unacceptably slow (>1 minute on most machines), and it means we'd have to ship parcel with the API.

Additional context
Adding the option to use a Datapane CDN means we'd need to track versions on our report builds. I.e. if a report bundle is built with v0.7.6, then it should always point to /datapane-js/v0.7.6.js.

dp.Table sorting the columns in my dataframe.

Describe the bug
In the previous version when passing my dataframe into dp.Table the order of the columns remain intact.
After upgraded to 0.8.5, sorting of the columns.

Expected behavior
I would expect the ordering of the columns to stay the same after it is passed into dp.Table.

Screenshots

This is the order of the columns in the dataframe right before I passed it into dp.Table.

This is the order of the columns after I generated the report.

Desktop (please complete the following information):

• OS: Linux
• Browser Chrome
• Version 0.8.5

Unsupported Feature Error: DataTable

System Information

• OS: Windows 10
• Python version: 3.7.4 via Anaconda
• Python environment: conda
• Using jupyter: true
• Datapane version: 0.10.2

Bug / Issue

I was attempting to use the Report module to export a standalone HTML with a table, but received the following error: "UnsupportedFeature: DataTable component not supported when saving locally, please publish to a Datapane Server or use dp.Table Please visit www.github.com/datapane/datapane to raise issue / discuss if error repeats"

The code is simple and mostly from the DataPane examples page, so I reproduced it below.

import datapane as dp
import numpy as np
import pandas as pd

df = pd.DataFrame({
    'A': np.random.normal(-1, 1, 5),
    'B': np.random.normal(1, 2, 5),
})

table = dp.DataTable(df)
report = dp.Report(table)
report.save(path='test.html')

Bloated html.

Describe the bug
An observation more than a bug.

I ran a simple demo report which generated a 'test' HTML file which weighed in at nearly 10mb. However, when I downloaded the data to csv it was just 191kb.

I personally would use anywhere from 10 to 50,000 rows for a generalized report. I'm sure there would be cases for larger sizes too.

Steps to reproduce the behavior:

Run this demo file, download the csv from the generated report then check the size of the csv

import pandas as pd
import datapane as dp

df = pd.DataFrame({
    'A': np.random.normal(-1, 1, 5000),
    'B': np.random.normal(1, 2, 5000),
})

table = dp.Table(df)
report = dp.Report(table)
report.save(path='test.html')

Not possible to undo sorting operation in the .html report

Describe the bug
It is not a real bug but a strange behaviour.
When I create the .html report and I click one of the columns in order to sort data, it is not possible anymore to undo the sorting operation and go back to the original state.

Is it normal?

DataTable: No objects to concat

While executing below code:

import pandas as pd
import altair as alt
import datapane as dp

df = pd.read_csv('https://query1.finance.yahoo.com/v7/finance/download/GOOG?period2=1585222905&interval=1mo&events=history')

chart = alt.Chart(df).encode(
x='Date:T',
y='Open'
).mark_line().interactive()

r = dp.Report(dp.DataTable(df), dp.Plot(chart))
r.save(path='report.html', open=True)

Getting error:
ValueError: No objects to concatenate

ValueError Traceback (most recent call last)
in
10 ).mark_line().interactive()
11
---> 12 r = dp.Report(dp.DataTable(df), dp.Plot(chart))

cache problem

I am getting this problem after upgrading to datapane 0.7.1

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\shaka\Desktop\C\Drafts\dp-cache\dp-tmp-p4ivkaca\dp-tmp-muexg9zv.arrow.gz'

Surfacing errors to users

Overview

Although we expose some errors to the end-user, a lot get lost (it seems that 500s do). We want to expose as much as possible, so that people can debug both their own code and our product.

Acceptance Criteria

All relevant errors around which are available on the server logs are exposed to the end-user, including, but not limited to:

• Deploying a script (bundle size too big)
• Script building failed
• Script bombed out with a 500 due to import error

@mands can chime in on how this will work in practice and what we really want to expose.

Notes

• This is probably a separate ticket around cleaning up these errors -- i.e. right now you just get a 403 when you've deployed too many scripts.
• It would also be good to surface logs to users via the CLI running

Build Script Exception with Jupyter Notebook

I have attempted to deploy a Jupyter notebook to the DataPane web service using the following datapane.yaml config as below:

name: imdb_movies
script: imdb_movies.ipynb
repo: https://github.com/WayneLambert/library-python/blob/master/data_science/notebooks
parameters:
  - name: query
    description: Analysis of iMDB Movie Dataset
    required: False
    type: string

Running the script, I received the following traceback:

Unhandled Exception

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/datapane/runner/exec_script.py", line 40, in run
    res_scope = exec_mod(script.script)
  File "/usr/local/lib/python3.7/site-packages/datapane/runner/exec_script.py", line 164, in exec_mod
    res_scope = runpy.run_path(str(script_path), init_globals=globalscope, run_name=RUN_NAME)
  File "/usr/lib64/python3.7/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/usr/lib64/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib64/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "imdb_movies.py", line 44, in <module>
    get_ipython().run_line_magic('matplotlib', 'inline')
NameError: name 'get_ipython' is not defined

It appears to suggest that an imdb_movies.py file is required. The file I input for the script was called imdb_movies.ipynb.

Following a discussion with Leo, it appears as though the offending cell in my notebook is:

%matplotlib inline

Is there any way that the deployment script could work around this? If it isn't fundamental to the functionality of the HMTL report, it would be ideal ignoring it and allowing the notebook to still build.

Python 3.9 install failing

Using Python3.9 and pip, the installation failing when installing pyarrow build dependencies on windows 10 pro, please help me out

Deploying from CLI

When you run the command datapane script deploy --help for guidance, it returns the following output:

Usage: datapane script deploy [OPTIONS]

  Package and deploy a Python script or Jupyter notebook as a Datapane
  Script bundle

Options:
  --config PATH
  --script PATH
  --name TEXT
  --visibility [PUBLIC|ORG|PRIVATE]
  --help                          Show this message and exit.

It suggests that the --config flag accepts a path argument, however, for my project when I'm in the project's src directory on the command line and run either the command:

$  datapane script deploy --config data_science/notebooks/datapane.yaml
# OR
$  datapane script deploy --config ./data_science/notebooks/datapane.yaml

it returns:

Please run from source directory

If I navigate to the folder containing the yaml file and run the script from there, then I can successfully run the deploy script.

Is it possible to run a deploy script from your project's src directory by passing the the relative path to the command as a flag argument?

Bokeh Plot Not Rendering

System Information

System Information

• OS: Windows 10
• Python version: 3.7.4 via Anaconda
• Python environment: conda
• Using jupyter: true
• Datapane version: 0.10.2

Bug / Issue

I ran the following code (motivated from the Datapane examples). The Bokeh plot does not render in Google Chrome, but the table does render. This is not an issue for Matplotlib. Can this issue be reproduced?

from bokeh.plotting import figure, output_file, show
import datapane as dp 
import numpy as np
import pandas as pd

df = pd.DataFrame(data=np.random.rand(10,3), columns=['one','two','three'])

bokeh_chart = figure()
bokeh_chart.circle([10,1],[10,1])

report = dp.Report(
    dp.Plot(bokeh_chart),
    dp.Table(df)
)

report.save(path='report.html', open=True)

Bad filename results in cryptic error

If you call report.save with a super long (or invalid filename), you get a really weird error. At first I thought something was wrong with my plots, but nope it was just a filename issue.

Error validating report document:

.....

validate_report_doc
    rng_validator.assertValid(xml_doc)
  File "src/lxml/etree.pyx", line 3623, in lxml.etree._Validator.assertValid
lxml.etree.DocumentInvalid: Error validating datatype string

Partially initialized module 'datapane' has no attribute

System Information

• OS: Windows10
• Python version: 3.8.3
• Python environment: pip
• Using jupyter: false
• Datapane version: 0.10.2

Bug / Issue

I tried to display a table using datapane but it always reminded me that "Partially initialized module 'datapane' has no attribute xxx". I thought it could be the login issue but later when I deleted "dp.login", the error became "AttributeError: partially initialized module 'datapane' has no attribute 'Report' (most likely due to a circular import)". It looks like datapane works nowhere.

Here is my snippet:

import pandas as pd
import datapane as dp

# # Sign-in with your unique token
dp.login(token="my token string")

dataset = pd.read_excel('C:\\Users\\13107\\Downloads\\10by10.xlsx')

# Create report
r = dp.Report(
    dp.DataTable(dataset),
)

# Publish
r.publish(name='10by10 Grid')

Report structure API / DX

Overview

A report is currently a list of components, such as Markdown, Plots, Tables etc. It looks similar to React code without JSX

In the real world, reports end up looking a bit like this:

from datapane import Report, Plot, Table, Markdown, Asset

report = [
    Markdown(f"""### Analysing clogged toilets with Google Trends
You may often find yourself wondering about the seasonality and geopraphy of toilet clogging. Which US states are the biggest culprits? Is it worse at a certain time of year?
Thanks to the marvels of modern data, we can pull this information from the Google Trends API and find out..."""),
    Table.create(df),
    Markdown("Let's plot this, both over time and per month (using the Python library Altair).."),
    Plot.create(all_trends),
    Markdown("Toilet clogging looks pretty seasonal, with a big spike around Christmas, which makes sense (I guess?) It seems to be generally growing over time, and we are at an all time high, presumably due to folks isolating and working from home.\n\nLet's use Facebook's Prophet forecasting library to try and forecast for the coming years, and analyse the nature of the seasonality."),
    Table.create(forecast),
    Plot.create(fig1),
    Plot.create(fig2),
    Markdown("Looks like 2021 is going to be a big year for clogged toilets."),
    Markdown("### The geography of cloggage\nWe can also query Google Trends to find which states are the biggest culprits, and plot it on an interactive map."),
    Plot.create(loo_map),
    Table.create(geo),
    Markdown("""### Conclusion\n
I hope you found these important analyses and visualisations informative. You can find the notebook which generated this report [on Github]().
If there's enough interest, I'd be happy to upload it to Datapane as an interactive script which works with other keywords.
""")
]

Generally, it's a series of plots/tables/etc. interspersed with Markdown/text. As text is such an important primitive, we could remove the need for a markdown block, so your report would just be:

report = [
      f"""### Analysing clogged toilets with Google Trends
You may often find yourself wondering about the seasonality and geopraphy of toilet clogging. Which US states are the biggest culprits? Is it worse at a certain time of year?
Thanks to the marvels of modern data, we can pull this information from the Google Trends API and find out..."""),
    Table.create(df),
   "Let's plot this, both over time and per month (using the Python library Altair)..",
    Plot.create(all_trends),
    "Toilet clogging looks pretty seasonal, with a big spike around Christmas, which makes sense (I guess?) It seems to be generally growing over time, and we are at an all time high, presumably due to folks isolating and working from home.\n\nLet's use Facebook's Prophet forecasting library to try and forecast for the coming years, and analyse the nature of the seasonality.",
    Table.create(forecast),
    Plot.create(fig1),
    Plot.create(fig2),
    "Looks like 2021 is going to be a big year for clogged toilets.",
    "### The geography of cloggage\nWe can also query Google Trends to find which states are the biggest culprits, and plot it on an interactive map.",
    Plot.create(loo_map),
    Table.create(geo),
    """### Conclusion\n
I hope you found these important analyses and visualisations informative. You can find the notebook which generated this report [on Github]().
If there's enough interest, I'd be happy to upload it to Datapane as an interactive script which works with other keywords.
"""
]

A more left of field idea is that, as Markdown/text is such an integral part of any report, we could invert this, and base reports more around Markdown -- with other components as format strings. This could be easier to:

a. Write a report -- I even could preview and write in a regular Markdown editor, and bring it into Python to add plots etc.

b. Visualise the end result when writing a report

That said, I'm not sure of the technical implications.

f"""
### Analysing clogged toilets with Google Trends

You may often find yourself wondering about the seasonality and geopraphy of toilet clogging. Which US states are the biggest culprits? Is it worse at a certain time of year?
Thanks to the marvels of modern data, we can pull this information from the Google Trends API and find out...

{Table.create(df)}

Let's plot this, both over time and per month (using the Python library Altair)..

{Plot.create(all_trends)}

Toilet clogging looks pretty seasonal, with a big spike around Christmas, which makes sense (I guess?) It seems to be generally growing over time, and we are at an all time high, presumably due to folks isolating and working from home.\n\nLet's use Facebook's Prophet forecasting library to try and forecast for the coming years, and analyse the nature of the seasonality.

{Table.create(forecast)}
{Plot.create(fig1)}
{Plot.create(fig2)}

Looks like 2021 is going to be a big year for clogged toilets.

### The geography of cloggage

We can also query Google Trends to find which states are the biggest culprits, and plot it on an interactive map.

{Plot.create(loo_map)}
{Table.create(geo)}

### Conclusion

I hope you found these important analyses and visualisations informative. You can find the notebook which generated this report [on Github](). If there's enough interest, I'd be happy to upload it to Datapane as an interactive script which works with other keywords.
"""

Export in HTML failed in windows

Describe the bug
in python code , it fails

import pandas as pd
import altair as alt
import datapane as dp

dataset = pd.read_csv('https://covid.ourworldindata.org/data/owid-covid-data.csv')
df = dataset.groupby(['continent', 'date'])['new_cases_smoothed_per_million'].mean().reset_index()

plot = alt.Chart(df).mark_area(opacity=0.4, stroke='black').encode(
    x='date:T',
    y=alt.Y('new_cases_smoothed_per_million:Q', stack=None),
    color=alt.Color('continent:N', scale=alt.Scale(scheme='set1')),
    tooltip='continent:N'
).interactive().properties(width='container')

dp.Report(
    dp.Plot(plot), 
    dp.Table(df)
).save(path='report.html', open=True)

due to jinja cannot find Path.
Maybe, due to windows path.

Nested dp.Select not working

Describe the bug
In the docs concerning Select, there is a note:

Select expects a list of Blocks, e.g. a Plot or Table, but also including Select or Groups themselves,
but if a Python object is passed, e.g. a Dataframe, Datapane will attempt to convert it automatically.

However, I am not able to get nested selects working. Instead, the outer selection method (tab or dropdown doesn't matter) shows, and the inner one is missing, in its place being an empty doc symbol:

To Reproduce
Execute the following repro script:

import datapane as dp


def create_plot_selection() -> dp.Select:
    test_dict = {
        "outer1": {
            "inner11label": "inner11",
            "inner12label": "inner12"
        },
        "outer2": {
            "inner21label": "inner21",
            "inner22label": "inner22",
            "inner23label": "inner23"
        }
    }

    outer_choice_blocks = []
    for outer_choice_label, inner_choices in test_dict.items():
        inner_choice_blocks = []
        for label, test_item in inner_choices.items():
            inner_choice_blocks.append(dp.Text(test_item, label=label))

        outer_choice_blocks.append(dp.Select(blocks=inner_choice_blocks, label=outer_choice_label, type=dp.SelectType.TABS))

    return dp.Select(blocks=outer_choice_blocks, type=dp.SelectType.DROPDOWN)


report = dp.Report(blocks=["Should see more than a document icon after selecting an outer choice:", create_plot_selection()])
report.save(path='report.html')

Expected behavior
When opening the generated report, I'd expect to be able to use an inner select

Desktop (please complete the following information):

• OS: Linux Mint 20,
• Browser: Firefox on Linux,
• Version 84.0.1 (64-bit) on Linux, on android

Smartphone (please complete the following information):

• Device: Redmi 7
• OS: Android 10
• Browser: Firefox and Chrome
• Version: firefox 84.1.2 and chrome 87.0.4280

Report creation API / DX

Overview

A single report can be created with Report.create([components], [options]), such as:

report = [
    Plot.create(my_plot),
    Markdown("# My Markdown"),
    Table.create(my_table)
]

# Alternatively: LocalReport.create(*report)
Report.create(*report)

This is different from the DX of how you create a report dynamically via running a script on Datapane (right now on Datapane.com, soon to be on the open source offering too). I believe this is due to the fact that the actual report generation happens outside of the user's code when running on the server.

Currently we support two options:

1. Assigning to a magic report object, which Datapane will look for and pick out:

Pros

• Shorter for smaller scripts and examples
• Easier to immediately understand with a lower cognitive overhead

Cons

• Magic and implicit
• Pollutes the namespace

report = [
    Plot.create(my_plot),
    Markdown("# My Markdown"),
    Table.create(my_table)
]

2. Adding a render function, which returns a list of components (i.e. what is passed to Report.create)

Pros

• Doesn't polute global object namespace
• Doesn't run with script in regular circumstances, so more portable

Cons

• Harder to understand on first glance
• Currently introduces a larger divergeence between ad-hoc report generation, and script/dynamic report generation

def render():
    return [
        Plot.create(my_plot),
        Markdown("# My Markdown"),
        Table.create(my_table)
    ]

---

Notes

• When developing locally, you want to be able to preview your report output regularly, without having to run the script -- a user would probably always be developering using Report.create, and then switch to the server version before deploying. From my POV, the ideal DX would be to be able to use the same code.
  • Alternatively, we could just add a helper to the datapane library, like dp.preview_report() which calls render and creates a LocalReport, or looks for the report object and does the same.

---

Questions

• Do we keep both or one?
• What is a solution that unifies the local dev story with deploying the script?
• Is there a third option? i.e. dp.report = [], or using the same Report.create syntax, but just making it work on the server.

Add datapane to conda-forge?

Thank you for developing this great package! I plan to add it as a dependency for my geemap Python package. Do you plan to make datapane available on conda-forge? Thanks.

client is out-of-date but unable to update

System Information

• OS: Windows 10
• Python version: 3.8.5
• Python environment: conda
• Using jupyter: true
• Datapane version: 0.9.2

Bug / Issue

I received the error: Your client is out-of-date (version 0.9.2) and may be causing errors, please upgrade to version 0.10.1 Please visit www.github.com/datapane/datapane to raise issue / discuss if error repeats after keying in my API token to login.

However, when I tried conda install -c conda-forge "datapane>=0.10.1", there are too many conflicts with existing packages listed and I am unsure what my next steps of action should be. I've attached the conflicts in the following txt file for your reference.
datapane update conflicts.txt

Much thanks!

Date column coercion bug

Originally posted by @ehsen in #50:

I have a pandas dataframe with date column in 'd-m-Y' format, the date column is string not datetime. But when I create datapane report it automatically converts my date to ISO format. Similarly I have another column with int datatype, datapane also converts it into ISO date (which is very weird).

Can you please tell me how can I fix this.

Update to pyarrow 1.0.0

Is your feature request related to a problem? Please describe.
The conda package is pinned to pyarrow 0.17.*, but I'd like to use pyarrow 1.0.0 functionality.

Describe the solution you'd like
Is it possible to remove the pin on 0.17.* and update the dependency to 1.0.0? I'm happy to help with testing if you need an extra set of hands.

Table function changing dates (-1) from Pandas DataFrame upon execution

Hello,

I am using DataPane to create a simple report to track historical metric values, day over day. The data is being read in via Pandas excel function to a DataFrame and then I am creating a graph in plotly to visualize it. I am trying to leverage the table function in DataPane to also give the raw data along with the graph. I noticed that the dates are shifting with respect to the data. The first date in my file is September 1st, yet it is shifting everything back 1 day so that the start date of the RAW Data table output is August 31st.

Does anyone see anything that I can do, or is this a potential bug? I know i could probably convert the dates to a string, but it seems like i shouldn't have to do that if the table function is going to assume data types.

DataPaneTest.xlsx

`
import pandas as pd
import datapane as dp
import plotly.graph_objects as go
import plotly.io as pio

HistoricalData = pd.read_excel(r'mypath\DataPaneTest.xlsx')
HistoricalData['DataDate'] = HistoricalData['DataDate'].astype('datetime64[ns]')

pio.templates.default = "plotly_dark"

fig = go.Figure()
fig.add_trace(go.Scatter(x=HistoricalData['DataDate'], y=HistoricalData['col1'],
mode='lines+markers',
name='t1'))
fig.add_trace(go.Scatter(x=HistoricalData['DataDate'], y=HistoricalData['col2'],
mode='lines+markers',
name='t2'))
fig.add_trace(go.Scatter(x=HistoricalData['DataDate'], y=HistoricalData['col3'],
mode='lines+markers',
name='t3'))
fig.add_trace(go.Scatter(x=HistoricalData['DataDate'], y=HistoricalData['col4'],
mode='lines+markers',
name='t4'))
fig.update_layout(
title_text='t0',title_x=0.5)

dp.Report(
dp.Markdown("# Stuff"),
dp.Markdown(" "),
dp.Plot(fig),
dp.Markdown(" "),
dp.Markdown("## RAW Data"),
dp.Table(HistoricalData)

).save(path=r'mypath\DataPaneTest.html', open=True)
`

Originally posted by @JCP281 in #39

ruamel-yaml (distutils) install issue

Originally posted by @ahull002 in #80

System: O365/WIN10

IDE: JupyterLab 3.0.7

Language: Python version 3.8.5

Library: datapane

Issue:
Hello, I keep getting package errors when trying to install the datapane. The errors surround ruamel-yaml (distutils) and DLL.

Steps taken:

I have tried installing this two ways via PiP and Conda:

1. Via PiP:

"ERROR: Cannot uninstall 'ruamel-yaml.' It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall."

2. Via Conda:
ImportError: DLL load failed while importing lib: The specified procedure could not be found.

Any assistance or light to help me navigate what these issues are and how to get around them would be awesome. Thank you.

Windows dependency on Visual C++

No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
      error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/

Trying to have captions with styled dataframes

Describe the bug
A clear and concise description of what the bug is.

When trying to make a caption for a styled dataframe, all that is shown is a blank caption tag.

<caption>Some caption here</caption>

Datapane table

To Reproduce
Steps to reproduce the behavior:

1. Make a styled pandas DataFrame
2. Render it with datapane
3. The caption tag should be above as raw html.