GithubHelp home page GithubHelp logo

memgraph / mgconsole Goto Github PK

View Code? Open in Web Editor NEW
44.0 7.0 7.0 200 KB

mgconsole is a command-line interface (CLI) used to interact with Memgraph from any terminal or operating system.

License: GNU General Public License v3.0

CMake 2.82% C++ 94.49% Shell 2.70%
cli memgraph

mgconsole's Introduction

Actions Status

mgconsole

mgconsole is a command line interface for Memgraph database.

mgconsole

Building and installing

To build and install mgconsole from source you will need:

  • CMake version >= 3.4
  • OpenSSL version >= 1.0.2
  • C compiler supporting C11
  • C++ compiler supporting C++20

To install compile dependencies on Debian / Ubuntu:

apt-get install -y git cmake make gcc g++ libssl-dev

On RedHat / CentOS / Fedora:

yum install -y git cmake make gcc gcc-c++ openssl-devel libstdc++-static

On MacOS, first make sure you have XCode and Homebrew installed. Then, in the terminal, paste:

brew install git cmake make openssl

On Windows, you need to install the MSYS2. Just follow the instructions, up to step 6. In addition, OpenSSL must be installed. You can easily install it with an installer. The Win64 version is required, although the "Light" version is enough. Both EXE and MSI variants should work. Then, you'll need to install the dependencies using the MSYS2 MINGW64 terminal, which should be available from your Start menu. Just run the following command inside the MSYS2 MINGW64 terminal:

pacman -Syu --needed base-devel git mingw-w64-x86_64-toolchain mingw-w64-x86_64-cmake mingw-w64-x86_64-openssl

Once everything is in place, create a build directory inside the source directory and configure the build by running CMake from it as follows:

  • on Linux:
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
  • on MacOS:
mkdir build
cd build
cmake -DOPENSSL_ROOT_DIR="$(brew --prefix openssl)" -DCMAKE_BUILD_TYPE=Release ..
  • on Windows, from the MSYS2 MINGW64 terminal:
mkdir build
cd build
cmake -G "MinGW Makefiles" -DCMAKE_BUILD_TYPE=Release ..

After running CMake, you should see a Makefile in the build directory. Then you can build the project by running:

make

This will build the mgconsole binary. To install it, run:

make install

This will install to system default installation directory. If you want to change this location, use -DCMAKE_INSTALL_PREFIX option when running CMake.

NOTE: If you have issues compiling mgconsole using your compiler, please try to use Memgraph official toolchain. In case you encouner any problem, please create a new Github issue.

Example usage

$ mgconsole --host 127.0.0.1 --port 7687 --use-ssl=false
mgconsole 0.1
Type :help for shell usage
Quit the shell by typing Ctrl-D(eof) or :quit
Connected to 'memgraph://127.0.0.1:7687'
memgraph> :help
In interactive mode, user can enter cypher queries and supported commands.

Cypher queries can span through multiple lines and conclude with a
semi-colon (;). Each query is executed in the database and the results
are printed out.

The following interactive commands are supported:

        :help    Print out usage for interactive mode
        :quit    Exit the shell

memgraph>
memgraph> MATCH (t:Turtle) RETURN t;
+-------------------------------------------+
| t                                         |
+-------------------------------------------+
| (:Turtle {color: "blue", name: "Leo"})    |
| (:Turtle {color: "purple", name: "Don"})  |
| (:Turtle {color: "orange", name: "Mike"}) |
| (:Turtle {color: "red", name: "Raph"})    |
+-------------------------------------------+
4 rows in set (0.000 sec)
memgraph> :quit
Bye

Export & import into Memgraph

An interesting use-case for mgconsole is exporting and importing data. You can close the loop by running the following example queries:

# Export to cypherl formatted data file
echo "DUMP DATABASE;" | mgconsole --output-format=cypherl > data.cypherl

# Import from cypherl file
cat data.cypherl | mgconsole

Batched and parallelized import (EXPERIMENTAL)

Since Memgraph v2 expects vertices to come first (vertices has to exist to create an edge), and serial import can be slow, the goal with batching and parallelization is to improve the import speed when ingesting queries in the text format.

To enable faster import, use --import-mode="batched-parallel" flag when running mgconsole + put Memgraph into the STORAGE MODE IN_MEMORY_ANALYTICAL; (could be part of the .cypherl file) to be able to leverage parallelism in the best possible way.

cat data.cypherl | mgconsole --import-mode=batched-parallel
# STORAGE MODE IN_MEMORY_ANALYTICAL; is optional

IMPORTANT NOTE: Inside the import file, vertices always have to come first because mgconsole will read the file serially and chunk by chunk.

Additional useful runtime flags are:

  • --batch-size=10000
  • --workers-number=64

Memgraph in the TRANSACTIONAL mode

In TRANSACTIONAL mode, batching and parallelization might help, but since there are high chances for serialization errors, the execution times might be similar or even slower compared to the serial mode.

Memgraph in ANALYTICAL mode

In ANALYTICAL mode, batching and parallelization will mostly likely help massively because serialization errors don't exist, but since Memgraph will accept any query (e.g., on edge create failure, vertices could be created multiple times), special care is required:

  • queries with pure create vertices have to be specified first
  • please use only import statements using simple MATCH, CREATE, MERGE statements.

If you encounter any issue, please create a new mgconsole Github issue.

mgconsole's People

Contributors

antaljanosbenjamin avatar antonio2368 avatar ciglenecki avatar darych avatar gitbuda avatar imilinovic avatar jbajic avatar jmatak avatar kostasrim avatar mferencevic avatar the-joksim avatar tomicm avatar vpavicic avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

mgconsole's Issues

CYPHERL output format

I've tried exporting cypherl with mgconsole using DUMP DATABASE:
mgconsole < <(echo "DUMP DATABASE;") > output.cypherl
But it does not produce a valid .cypherl file as data is in a tabular format.

If it's not possible, I guess another output format should then be added - cypherl to be able to get clean cypher queries.

Mgconsole exits on interrupt

It is so annoying that mgconsole exits when I want to interrupt the current command. Please make it exit on double ctrl + c, or not at all.
How to reproduce: press ctrl + c.

F35: cannot find -lstdc++

Hello,
on recently upgraded Fedora 35.
receiving :

[100%] Linking CXX executable mgconsole
/usr/bin/ld: cannot find -lstdc++
collect2: error: ld returned 1 exit status
make[2]: *** [src/CMakeFiles/mgconsole.dir/build.make:103: src/mgconsole] Error 1
make[1]: *** [CMakeFiles/Makefile2:950: src/CMakeFiles/mgconsole.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

How to reproduce:

as root:

dnf install cmake make gcc gcc-c++ openssl-devel
keybase                                                                                                                  23 kB/s | 3.3 kB     00:00    
Package cmake-3.22.0-2.fc35.x86_64 is already installed.
Package make-1:4.3-6.fc35.x86_64 is already installed.
Package gcc-11.2.1-1.fc35.x86_64 is already installed.
Package gcc-c++-11.2.1-1.fc35.x86_64 is already installed.
Package openssl-devel-1:1.1.1l-2.fc35.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!

exit out of root.
as a regular user:

pwd
/home/az/XXXXX/mgconsole/build
ls -l
total 0
cmake -DCMAKE_BUILD_TYPE=Release ..
-- The C compiler identification is GNU 11.2.1
-- The CXX compiler identification is GNU 11.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/lib64/ccache/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/lib64/ccache/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMake build type: Release
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Found OpenSSL: /usr/lib64/libcrypto.so (found version "1.1.1l")  
-- Configuring done
-- Generating done
-- Build files have been written to: /home/az/XXXX/mgconsole/build

make 
< outputs a lot of stuff>
Install the project...
-- Install configuration: "Release"
-- Installing: /home/az/XXXX/mgconsole/build/replxx/lib64/libreplxx.a
-- Installing: /home/az/XXXX/mgconsole/build/replxx/share/cmake/replxx/replxx-targets.cmake
-- Installing: /home/az/XXXX/mgconsole/build/replxx/share/cmake/replxx/replxx-targets-release.cmake
-- Installing: /home/az/XXXX/mgconsole/build/replxx/share/cmake/replxx/replxx-config-version.cmake
-- Installing: /home/az/XXXX/mgconsole/build/replxx/share/cmake/replxx/replxx-config.cmake
-- Installing: /home/az/XXXX/mgconsole/build/replxx/include/replxx.hxx
-- Installing: /home/az/XXXX/mgconsole/build/replxx/include/replxx.h
[ 85%] Completed 'replxx-proj'
[ 85%] Built target replxx-proj
[ 89%] Building CXX object src/utils/CMakeFiles/utils.dir/utils.cpp.o
[ 92%] Linking CXX static library libutils.a
[ 92%] Built target utils
[ 96%] Building CXX object src/CMakeFiles/mgconsole.dir/main.cpp.o
[100%] Linking CXX executable mgconsole
/usr/bin/ld: cannot find -lstdc++
collect2: error: ld returned 1 exit status
make[2]: *** [src/CMakeFiles/mgconsole.dir/build.make:103: src/mgconsole] Error 1
make[1]: *** [CMakeFiles/Makefile2:950: src/CMakeFiles/mgconsole.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

Please adivse.

Building from source error

Getting next error while trying to build mgconsole. Tried with memgraph toolchain and without, same error

[ 76%] Building CXX object src/utils/CMakeFiles/utils.dir/thread_pool.cpp.o
In file included from /home/roma/mgconsole/src/utils/thread_pool.hpp:25,
                 from /home/roma/mgconsole/src/utils/thread_pool.cpp:16:
/home/roma/mgconsole/src/utils/synchronized.hpp:18:10: fatal error: concepts: No such file or directory
   18 | #include <concepts>
      |          ^~~~~~~~~~
compilation terminated.
make[2]: *** [src/utils/CMakeFiles/utils.dir/build.make:90: src/utils/CMakeFiles/utils.dir/thread_pool.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1019: src/utils/CMakeFiles/utils.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

Ubuntu 20.04 (Windows 10, WSL)

Add batched and parallel import

  • Automatic retries in case of serialization errors
  • Add detailed description how to run Memgraph in the ANALYTICS mode + run mgconsole; inside mgconsole/README.md page

Flags:

  • --import-mode="batched-parallel"
  • --batch-size=N
  • --workers-number=N

Not done in the end:

  • Write n info back to the user where the import has stopped
    • Single-thread case
    • Parallel case

minor: autocomplete is not working for some words

This is just minor issue regarding autocomplete. It is only just to say you know about it.

Some words cannot be auto-completed like: "constraint" or "version" (there may be more words like that), see below:

memgraph> SHOW co
coNTAINS  coUNT     coS       coNTAINS  coUNTER
memgraph> SHOW version;
+----------+
| version  |
+----------+
| "2.12.1" |
+----------+
1 row in set (round trip in 0.001 sec)

This does not have any impact. It is here just for you to know.

Batched parallel not working with UNWIND

Import the following dataset with batched parallel in-memory analytical mode and notice you have 300k nodes and 280k edges, while the dataset has 300k nodes and 300k edges (can be confirmed with classic mgconsole import with in-memory transactional mode). To be exact, there are 20k relationships of type DRIVES missing.

Here is the dataset.

can't compile mgconsole on Fedora 37

I'm trying to compile mgconsole on fedora 37 x64 bit (no arm) but after writing make (so after cmake -DCMAKE_BUILD_TYPE=Release .. successfull) I get this error:

image

Look at the last three rows. It's the same error as this one.

Implement concurrent import of Cypher queries

The biggest challange is to first create nodes because some queries might depend on that. Order queries based on query clauses or a comment indicating start of edge queries. Add something like --concurrent as a runtime flag.

Debug and improve mgclient session inside mgconsole

Sometimes in the batched-parallel import mode, mgclient session can end up in a bad state:

  • inside ExecuteBatchesParallel, taka a look at the MG_SESSION_BAD check and figure out why is happening
  • if backoff is not there -> "mg_raw_transport_recv: Bad file descriptor"

Mgconsole installation is not working on MAC

When following the instructions, I had to set additional parameter in CMake
-DCMAKE_INSTALL_PREFIX=/usr/local/ in order for the command make install to complete. Otherwise, an error:

CMake Error at src/cmake_install.cmake:41 (file):
  file INSTALL cannot copy file
  "/Users/memgraph/Memgraph-Repos/mgconsole/build/src/mgconsole" to
  "/usr/bin/mgconsole": Operation not permitted.
Call Stack (most recent call first):
  cmake_install.cmake:42 (include)

would occur.

Triggers does not get backuped on database dump

Can you please advise how to backup Triggers to cypher query language?

As DUMP DATABASE; backups also CONSTRAINT, but not TRIGGER.

As saving Triggers using show triggers; is not recoverable via cypher query language.

Not even this works:

echo "show triggers;" | mgconsole --output-format=cypherl

Improve editing of multiline pastes

When a user pastes command from, e.g. docs:

LOAD CSV FROM "/path-to/people_nodes.csv" WITH HEADER AS row
CREATE (n:Person {id: row.id, name: row.name});

Screenshot from 2021-10-13 08-46-09

it's possible to edit only the last line. A solution is to paste line by line, but that's very inconvenient when there are many lines (3+).

More informative query time

Query time consists of the total time needed to create a session in the client, execute the query and fetch the results via Bolt protocol. As a result, I would like to configure to see a more granular breakdown of query time, with more information about query processing and execution without taking the time needed to fetch it from the server.

Change the order of execution times info

When you get execution times in mgconsole, they are outputted in this order:

Query PARSING time: 0.000175982 sec
Query PLAN EXECUTION time: 0.0154524 sec
Query PLANNING time: 8.054e-05 sec

But it would make more sense if it was:

Query PARSING time: 0.000175982 sec
Query PLANNING time: 8.054e-05 sec
Query PLAN EXECUTION time: 0.0154524 sec

Because that's the order in which things happen?

'--output-format=cypherl' seems broken

Hello Memgraph's community,

can we look forward to 'cypherl' option in '--output-format' to gets working some time?
See this result on the latest Memgraph (2.14.0-1):

echo "match (n) return n;" | mgconsole --output-format=cypherl
Segmentation fault (core dumped)

While using: mgconsole --output-format=csv
everything works fine in every case.

There is no impact on this issue, just keeping current information about the status.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.