datalens-tech / datalens Goto Github PK

We are ready to test your product. Our analysts noticed that there is no possibility of uploading to Excel. How to fix it ?

Hello!

I decided to scan docker images, that have been creted after docker-compose up -d command. So, Snyk've found multiple vulnerabilities in datalens images. In attachment you can find full scan results. The great concern is about critical vulnerabilities with RCE. Are they affect to current version of datalnes? Is there any plans to update old components in the future?

Thank you!
datalens-us_0.96.0.txt
datalens-ui_0.795.0.txt
datalens-data-api_0.2037.0.txt
datalens-control-api_0.2037.0.txt

In Power BI, it is possible to create report templates to use as a basis for other reports.
I would like to see similar functionality in DataLens.
https://learn.microsoft.com/en-us/power-bi/create-reports/desktop-templates

Please advice how to eneble XLSX export option. CSV export not working well also. Any selected encoding shows incorrect results when open in Excel.

Currently our containers are running entry-point container processes using root user, which is far from ideal. We should switch to non-privileged users in all our containers:

• data-api
• control-api
• us
• ui

This non-root users should have minimum possible access to the files in the container.

Currently, I have to drag-and-drop it. But I spent 10 seconds trying to guess it. Firstly, I tried a single and double click.

C:\Windows\system32>git clone https://github.com/datalens-tech/datalens
Cloning into 'datalens'...
remote: Enumerating objects: 35, done.
remote: Counting objects: 100% (35/35), done.
remote: Compressing objects: 100% (27/27), done.

Receiving objects: 100% (35/35), 13.48 KiB | 299.00 KiB/s, done.
Resolving deltas: 100% (15/15), done.

C:\Windows\system32>cd datalens

C:\Windows\System32\datalens>set HC=1

C:\Windows\System32\datalens>docker compose up
[+] Running 1/0
✔ Network datalens_default Created 0.0s

• Container datalens-pg-us-1 Creating 0.0s
• Container datalens-pg-compeng-1 Creating 0.0s
• Container datalens-postgres-connection-1 Creating 0.0s
  Error response from daemon: mkdir C:\Windows\System32\datalens\us-data: Access is denied.

In cloud version MS SQL is available, but not in open source. We have MS SQL in private network - the only way to connect is to use DataLens self-hosted.

It would be great if MS SQL connector become public.

Great thanks to your team.

Enable Bitrix24 connector

Prerequisites:

• Enable RQE (remote-query-executor) or allow direct connection to Bitrix
• Enable caches (optional; without RQE caches Bitrix connector can be very slow)

Enable Greenplum connector

Hey there,

I’d love to test DataLens with Snowflake. Any plans adding snowflake support?

Hi there!
Are there any docs (besides those that are in Yandex.Cloud docs) covering the creation of datasets and possible error codes? I'm trying to create a dataset with an SQL query for a connected Postgres connection, but I'm stuck with this:

Relation requires at least one condition
ERR.DS_API.FORMULA
{
    "title": "ERR.DS_API.FORMULA",
    "description": "Relation requires at least one condition"
}

ERR.DS_API.VALIDATION.ERROR
[
    {
        "id": "340a4261-635d-11ee-85cb-db624a6af76d",
        "type": "avatar_relation",
        "errors": [
            {
                "level": "error",
                "code": "ERR.DS_API",
                "message": "Relation requires at least one condition",
                "details": {}
            }
        ]
    }
]

Unfortunately, nothing was found on the Internet, except this page, but it doesn't help :(

The query works independently and it has conditions.

Thanks!

Steps to reproduce: move the mouse pointer to the ? icon and click on it quickly.

After installation I can't create connections. When I click "Create connection" I only see a white sheet.

In the cmopose file all parameters are standard.

Привет.
После установки ошибка.
Нет коллекций и воркбуков.

ECONNABORTED
{
"title": "Ошибка",
"description": "Превышено время ожидания ответа"
}

[paulmer@docker-01 ~]$ cat /etc/os-release
NAME="Fedora Linux"
VERSION="38 (Thirty Eight)"
ID=fedora
VERSION_ID=38
VERSION_CODENAME=""
PLATFORM_ID="platform:f38"
PRETTY_NAME="Fedora Linux 38 (Thirty Eight)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:38"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f38/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=38
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=38
SUPPORT_END=2024-05-14
[paulmer@docker-01 ~]$

We want to create connector from DL to Datahub, please create API endpoint to get metadata from DL.

Please add export/import configurations for Dashboard/Chart/Connections all this stuff.. in some file format (JSON/YAML/...), like grafana does.

When connecting to Yandex Cloud Managed Service for PostgreSQL, it gives error. DataLens is deployed locally, loading a certificate file via Advanced connection settings does not help.

Through dbeaver I can connect to the database from this machine.

OS: Linux version 6.2.0-34-generic (buildd@lcy02-amd64-025) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~23.04) 12.3.0, GNU ld (GNU Binutils for Ubuntu) 2.40)
CPU: AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx

Error text:
Database error. ERR.DS_API.DB { { "data": { { "code": "ERR.DS_API.DB", "details": { "db_message": "connection to server at "..." (...), port 6432 failed: could not open certificate file \"/root/.postgresql/postgresql.crt\": Permission denied\n"" }, "message": "Database error." "debug": { "db_message": "connection to server at "..." (...), port 6432 failed: could not open certificate file \"/root/.postgresql/postgresql.crt\": Permission denied\n". "query": "select 1" } } }

The bottom border of an input field does not show.

Hello there.

Could you please enable ClickHouse connector option for conecction without user and password? It's now strongly required.

Добрый день! Пытаюсь построить линейную диаграмму в DataLens: при выборе графика - такого нет. Прикладываю скрин экрана:

Захожу сюда:

У меня ошибка:

Подскажите, может я что то делаю не то.

I'm following the readme on the Ubuntu machine, and for some reason, I have to type docker-compose instead of docker compose.

#36

I have PostgreSQL and Clickhouse databases running on a server. Both are accessible via DBeaver or via http
But when I try to make a connection in Datalens, connection attempt fails, with the error like this(for clickhouse):
Data source refused connection. ERR.DS_API.DB.SOURCE_CONNECT_ERROR { "data": { "details": { "db_message": "HTTPSConnectionPool(host='192.168.1.224', port=8123): Max retries exceeded with url: /?query_id=e7c0bdde-b9f7-11ee-bdd0-0242ac120006&database=system&join_use_nulls=1&distributed_ddl_task_timeout=280&send_progress_in_http_headers=0&output_format_json_quote_denormals=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f43f6287130>, 'Connection to 192.168.1.224 timed out. (connect timeout=1.0)'))" }, "message": "Data source refused connection.", "debug": { "db_message": "HTTPSConnectionPool(host='192.168.1.224', port=8123): Max retries exceeded with url: /?query_id=e7c0bdde-b9f7-11ee-bdd0-0242ac120006&database=system&join_use_nulls=1&distributed_ddl_task_timeout=280&send_progress_in_http_headers=0&output_format_json_quote_denormals=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f43f6287130>, 'Connection to 192.168.1.224 timed out. (connect timeout=1.0)'))", "query": "select 1" }, "code": "ERR.DS_API.DB.SOURCE_CONNECT_ERROR" } }
The same problem appears while running Datalens container on the server or on the other machine in network.

Could you please tell me, what may be the issue?

Currently, I haven't succeeded yet, will try later.

Hi, datalens community!
Is it possible to make a custom Datalens plugin chart?
Any guides or information will be appreciated.
Best regards!

I am facing issue while connecting my database(PostgreSQL). I am running this (datalens project) on my wsl2. How can I able to connect my PostgreSQL database to it ? Should I run PostgreSQL server on wsl2 as well? Or I can connect this via remote connection to my Windows machine already running PostgreSQL server on it.
I've tried both and getting errors, still unable to connect to DB :(

Hello!

Please give info how to add mssql connection.

Thanks!

When I create a new connection to the "Managed Service for PostgreSQL" database for use in collections, I select the root certificate; it is not used for the connection. The following error appears.

connection to server at "***********.mdb.yandexcloud.net" (158.XX.XX.XX), port 6432 failed: could not open certificate file "/root/.postgresql/postgresql.crt": Permission denied

Despite the fact that the US service is connected to the same database using a similar connection string METADATA_POSTGRES_DSN_LIST="postgres://{user}:{password}@{host}:{port}/{database}?sslmode=verify-full&sslrootcert=/certs/ root.crt"

I tried to upload the root certificate into the container, changed the rights to the root directory (for testing), this helps to find the certificate, but then I get another error:

connection to server at "***********.mdb.yandexcloud.net" (158.XX.XX.XX), port 6432 failed: could not open certificate file "/root/.postgresql/postgresql.key"

When trying to connect to metadata PostgreSQL DB with TLS/SSL, the following error appears when running scripts/preflight.sh (I was trying to connect to Yandex Managed Service for PostgreSQL):

error: odyssey: c3577ff1eb926: SSL is required

Although the correct root certificate is present in /certs/root.crt, it does not help.

It seems like here certificate path is not passed to initPosgresDB() (although I could not find the signature of this function since gravity-ui/postgreskit is probably a private repo).

The solution that worked for me was to add the following parameters to DSN:
postgres://{user}:{password}@{host}:{port}/{database}?sslmode=verify-full&sslrootcert=/certs/root.crt.

I think that this solution may be added to README.

Enable BigQuery connector

Will you provide Chinese translation ? Or how could I make a pull request on translation

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=ru

уберите, пожалуйста, запуск образов из под root

You can see that if we've moved the table up just a little, there will be no need for a scroll bar.

Will it be added in the future?

Enable Metrica/AppMetrica connectors

Open questions:

• Token field in connection & listing counters/applications
  • allow setting up own OAuth application via configs & using Metrica API to get token
  • OR allow only manual mode for these fields
• Whether to include a demo dashboard

I see that you're recently added Oracle connector, as well as some other ones. Would be nice to have MySQL connector in open-source version.

Thanks in advance!

I defined a dataset on a table with 21 billion records.
When I tried to create my first chart, it did this:

![Screenshot_20230926_190229](https://github.com/datalens-tech/datalens/assets/18581488/d81e4044-9b75-4eff-8107-98b6f15dd4ce)

Running this query on ClickHouse:

SELECT `t1`.`created_at` AS `res_0` FROM `default`.`twitter` AS `t1` GROUP BY `res_0` LIMIT 1000001

The query will process nicely, but the aggregation is performed by every unique time point, and it requires 94.5 seconds to finish on a service in ClickHouse Cloud.

#36

Broken demo «Интерактивный пример встроенного чарта» in a documentation post

https://cloud.yandex.ru/blog/posts/2021/06/datalens-digest-june-2021

добрый день, была бы полезна функциональность печати дашбоардов и экспорта как pdf

Guys, a very big thanks for sharing DataLens! It would be nice to have map-charts in open source version.

After Windows 10 Update KB4074588, some ports are reserved by Windows and applications cannot bind to these ports. 50500 is in the blocked range.

You can use netsh interface ipv4 show excludedportrange protocol=tcp to list the reserved ranges.

Please add instructions for backing up and restoring datalens configuration data (connections, datasets, charts, dashboards). For example, I want to move datalens to a new server. Please also add update instructions. I want to update DataLens to a new version and am afraid of losing my data.

Hello there.

Do you have plans to add Trino connector or custom JDBC connector for your BI-system?

After executing command - docker compose down&&docker compose up -d I am loosing all my settings. I tried to attach volumes to containers pg-compeng, pg-us like this:

   volumes:
        - pg-compeng:/var/lib/postgresql/data
    volumes:
        - pg-us:/var/lib/postgresql/data
 volumes:
  pg-compeng:
  pg-us:       

Can anyone help me make the docker-compose file resistant to reboots?

Everything needed to enable file connector as a part of DataLens OSS is present across repositories.

Steps required:

• Add relevant package dependencies
• Make relevant changes to env and configs
• Include services necessary for file-uploader to work:
  • S3 (file storage)
  • Redis (task queue & meta storage)
  • ClickHouse (data processing & retrieval)
• Implement, include and configure file-uploader services:
  • file-uploader-api
  • file-uploader-worker
• Make new services optional (maybe)