GithubHelp home page GithubHelp logo

josephmachado / analytical_dp_with_sql Goto Github PK

View Code? Open in Web Editor NEW
37.0 2.0 15.0 405 KB

Code for my "Efficient Data Processing in SQL" book.

Home Page: https://josephmachado.gumroad.com/l/analyticalsql

Makefile 100.00%
big-data olap sql trino

analytical_dp_with_sql's People

Contributors

josephmachado avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

analytical_dp_with_sql's Issues

Unable to connect to Docker

Hi, I am a little stuck and not sure how to resolve this issue of not being able to connect to Docker. Appreciate your help thank you!!

Screenshot 2023-09-04 000653

[Docker] Unable to setup Docker containers in Macbook pro m1

Describe the bug
I have a macbook pro m1 pro and when i am trying to follow the setup instructions for I keep getting the following message when i do make up for invoking docker container. Please help:

To Reproduce
Steps to reproduce the behavior:

  1. Run make up on a Macbook pro m1

Expected behavior
All the required containers must spin up.

Screenshots

Screenshot 2023-02-11 at 1 41 23 PM

Desktop (please complete the following information):

  • OS: [e.g. iOS] Macbook Pro M1

Unknown shorthand flag f in -d

When trying to run first command make up I receiving this output

make up
docker volume rm --force minio-data
minio-data
docker compose up -d --build
unknown shorthand flag: 'd' in -d
See 'docker --help'.

MacOS 13.4 M1,
Docker version 24.0.2, build cb74dfcd85

I am using colima for containers runtimes

[MariaDB] Hive metastore not working with some versions of MariaDB

Describe the bug
When using latest version of the MariaDB docker image, the hive-metastore container keep shutting down because of the following error : java.sql.SQLException : Unknown system variable 'transaction_isolation'

To Reproduce
Steps to reproduce the behavior:

  1. Run make up

Expected behavior
All the required containers must spin up.

Screenshots
image

Desktop (please complete the following information):

  • OS: MacBook Pro Intel, MacOS Ventura

Additional context
I've found this thread that mentions the error. They suggest that the error doesn't happen with version 10.3.32 of mariadb.
I've tried restricting the image to that version and haven't encountered any issues yet.

It also seems to be an open issue on MariaDB's repo.

I thought I'd let you know @josephmachado ๐Ÿ˜Š

Trino container not running on mac M2 2023

Hi, i followed the steps to run the docker container on mac m2 ,after make up command I cannot enter into trino cli.

When i looked up for the running containers i found that trino and minio containers are not running.

I opened up the logs for the trino container and found this below error message


2023-03-06T16:59:22.514Z ERROR main io.trino.server.Server No factory for connector 'hive_hadoop2'. Available factories: [hive, ignite, delta_lake, memory, kudu, blackhole, kinesis, redis, accumulo, gsheets, elasticsearch, jmx, postgresql, redshift, sqlserver, localfile, example_http, tpch, iceberg, mysql, bigquery, prometheus, mongodb, pinot, tpcds, hudi, oracle, mariadb, druid, cassandra, clickhouse, kafka, singlestore, atop, raptor_legacy, trino_thrift, phoenix5]
java.lang.IllegalArgumentException: No factory for connector 'hive_hadoop2'. Available factories: [hive, ignite, delta_lake, memory, kudu, blackhole, kinesis, redis, accumulo, gsheets, elasticsearch, jmx, postgresql, redshift, sqlserver, localfile, example_http, tpch, iceberg, mysql, bigquery, prometheus, mongodb, pinot, tpcds, hudi, oracle, mariadb, druid, cassandra, clickhouse, kafka, singlestore, atop, raptor_legacy, trino_thrift, phoenix5]
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:435)
at io.trino.connector.DefaultCatalogFactory.createCatalog(DefaultCatalogFactory.java:118)
at io.trino.connector.LazyCatalogFactory.createCatalog(LazyCatalogFactory.java:45)
at io.trino.connector.StaticCatalogManager.lambda$loadInitialCatalogs$1(StaticCatalogManager.java:158)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
at java.base/java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:184)
at io.trino.util.Executors.executeUntilFailure(Executors.java:41)
at io.trino.connector.StaticCatalogManager.loadInitialCatalogs(StaticCatalogManager.java:152)
at io.trino.server.Server.doStart(Server.java:142)
at io.trino.server.Server.lambda$start$0(Server.java:90)
at io.trino.$gen.Trino_409____20230306_165916_1.run(Unknown Source)
at io.trino.server.Server.start(Server.java:90)
at io.trino.server.TrinoServer.main(TrinoServer.java:38)

2023-03-06T16:59:22.514Z INFO Thread-76 io.airlift.bootstrap.LifeCycleManager JVM is shutting down, cleaning up
2023-03-06T16:59:22.514Z INFO Thread-76 io.airlift.bootstrap.LifeCycleManager Life cycle stopping...
2023-03-06T16:59:22.514Z INFO Thread-76 io.airlift.bootstrap.LifeCycleManager Life cycle stopped
parvatheesamputikalapudi@parvathesamsMBP analytical_dp_with_sql %

Misc - alternative editor to trino cli for intuitition purpose

Describe the bug
Hi, novice sql learner here - I appreciate your effort setting up the repo with seamless makefile steps to consume the book with practical code on hand.
The problem is that coming from other native SQL editors (SSMS, VS, notepad++ etc) I find that the CLI compared to them is much more cramped and unintuitive to edit / explore the input SQL code.

I think being able to do the following on an editor can massively help consume the code and thus the book core concept :

  • partial execution - being able to highlight part of the code and execute away to quickly inspect the input
  • accessibility to source SQL code between executions : it's neat having the execution results in the same section, but at the same time it is pushing the source sql code out of the terminal view - which after some executions makes it impractical to retrieve the earlier queries. (as opposed to (SSMS) a separate editor that stays put regardless of the execution result.)
  • (should have) better syntax highlighting - way it is now the base clause such as SELECT / FROM / WHERE / GROUP BY etc are in plain black and bold which can be easy to misread along with the rest of the code.

Expected behavior
would you suggest an alternative / some tips with the CLI to achieve the above?

remote access project via SSH

Describe the bug
I decided to use my home machine as development server for this project while sending commands via ssh from a client device. The setup went through until make trino that it threw the following error below (pardon my lack of pasting, I am figuring out how to access clipboards in ssh)
image

Is there additional configurations to be setup with SSH ? I assume that it's missing port forwarding.

To Reproduce
Steps to reproduce the behavior:

  1. make up
  2. wait for 60s
  3. make trino
  4. See error

Expected behavior
I can proceed to next steps with trino gui.

Screenshots
please see list of running containers:
image

Desktop (please complete the following information):

  • OS: linux mint 21.1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.