GithubHelp home page GithubHelp logo

memgraph / mgmigrate Goto Github PK

View Code? Open in Web Editor NEW
20.0 7.0 3.0 5.75 MB

mgmigrate is a tool for migrating data from MySQL or PostgreSQL to Memgraph and between Memgraph instances.

Home Page: https://github.com/memgraph/mgmigrate

License: GNU General Public License v3.0

CMake 0.02% C++ 0.17% Python 0.02% Cypher 99.78%
memgraph mysql postgresql migration-tool migration cypher

mgmigrate's Issues

Compilation Error during 'make' Step on Linux System

Issue Description

I am encountering an issue while trying to install the migration script. The problem arises during the make step, where the code fails to compile correctly. Specifically, I'm experiencing errors when configuring mysqlconnector-proj. Any help regarding this issue is greatly appreciated.

Environment

Operating System: Pop!_OS version 22.04
Virtualization software: VirtualBox
Host operating system: Windows 10

Steps to Reproduce

  1. Start the virtual machine with Pop!_OS version 22.04.

  2. Open the terminal and navigate to the desktop using the following command:

cd ~/Desktop
  1. Clone the mgmigrate repository from GitHub:
git clone https://github.com/memgraph/mgmigrate.git
  1. Change into the mgmigrate directory:
cd mgmigrate
  1. Install the necessary dependencies by running the following commands:
sudo apt-get install -y git cmake make gcc g++ python3 libpq-dev postgresql-server-dev-all
sudo apt install python3-pip
pip3 install pymgclient psycopg2 mysql-connector-python 
  1. Create a build directory and navigate into it:
mkdir build
cd build
  1. Configure the build using CMake:
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=. ..
  1. Install the libreadline development package to resolve an error encountered during the 'make' step:
sudo apt-get install libreadline-dev
  1. Finally, run the 'make' command to compile the code:
make

Error Screenshot

Please refer to the screenshots below, which illustrates the error I encountered during the 'make' step.

The following screenshot shows the tail end of the error.
image

In the same error I received an Openssl version issued that is screenshotted below:
image

Can't find table in schema

Hi, I'm trying to use mgmigrate with PostreSQL but after running the src/mgmigrate command I'm getting a weird error:

F20230220 15:54:34.233558 4138902784 schema_info.cpp:16] Couldn't find table 'pg_aggregate' in schema 'pg_catalog'!

I'm pretty sure the pg_aggregate table and pg_catalog schema exist so not sure what's going on. Is there something I'm missing here?

Instructions for compiling on Windows don't work

I've followed the instructions from README file and I wasn't able to compile the app on Windows 11 64 bit.

I've tried using 32 and 64 bit version of MSYS2. I got an error that there was no cmake and yet the install process has listed it.

I've added cmake package one more time using pacman -Syu cmake. Now I stop receiving message that I'm missing cmake but I got a new error:

$ cmake -DCMAKE_BUILD_TYPE=Release -G"MSYS Makefiles" -DCMAKE_INSTALL_PREFIX=. ..
CMake Error: Could not create named generator MSYS Makefiles

Generators
* Unix Makefiles               = Generates standard UNIX makefiles.
  Ninja                        = Generates build.ninja files.
  Ninja Multi-Config           = Generates build-<Config>.ninja files.
  CodeBlocks - Ninja           = Generates CodeBlocks project files.
  CodeBlocks - Unix Makefiles  = Generates CodeBlocks project files.
  CodeLite - Ninja             = Generates CodeLite project files.
  CodeLite - Unix Makefiles    = Generates CodeLite project files.
  Eclipse CDT4 - Ninja         = Generates Eclipse CDT 4.0 project files.
  Eclipse CDT4 - Unix Makefiles= Generates Eclipse CDT 4.0 project files.
  Kate - Ninja                 = Generates Kate project files.
  Kate - Unix Makefiles        = Generates Kate project files.
  Sublime Text 2 - Ninja       = Generates Sublime Text 2 project files.
  Sublime Text 2 - Unix Makefiles = Generates Sublime Text 2 project files.

Maybe some option is missing. Can you please check?

Additional dependencies for Debian/Ubuntu install

I couldn't install on Ubuntu, and I was receiving this error:

CMake Error at /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
  Could NOT find PostgreSQL (missing: PostgreSQL_TYPE_INCLUDE_DIR) (found
  version "14.1")
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.16/Modules/FindPostgreSQL.cmake:247 (find_package_handle_standard_args)
  CMakeLists.txt:47 (find_package)

Then I looked at CMakeLists.txt, looked into what find_package(PostgreSQL REQUIRED) actually does (it runs a CMake built-in script to find Postgres) and encountered this StackOverflow post about the same problem with that script: https://stackoverflow.com/a/40027295.

In short, the user should also be asked to apt-get install libpq-dev postgresql-server-dev-all.

Postgres found by cmake, but not by make (Mac Ventura)

attempting to build on M1 Mac, I was able to resolve some, but not all errors.

To make it find postgres on my machine, as well older version of OpenSSL which it seems to require:

cmake -DPostgreSQL_LIBRARY=/usr/local/opt/[email protected]/lib -DPostgreSQL_INCLUDE_DIR=/usr/local/opt/[email protected]/include -DCMAKE_BUILD_TYPE=Release -DOPENSSL_ROOT_DIR=/usr/local/opt/openssl -DCMAKE_INSTALL_PREFIX=. ..

To resolve warnings (below) and also find system includes on MacOS, I modified CMakeLists.txt to add the following lines right below the version check:

cmake_policy(SET CMP0114 NEW)
cmake_policy(SET CMP0135 NEW)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/")

cmake runs without error:

-- The C compiler identification is AppleClang 12.0.5.12050022
-- The CXX compiler identification is AppleClang 12.0.5.12050022
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMake build type: Release
-- Found OpenSSL: /usr/local/opt/openssl/lib/libcrypto.dylib (found version "3.0.7")  
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Found PostgreSQL: /usr/local/opt/[email protected]/lib (found version "9.5.25")  
-- Configuring done
-- Generating done
-- Build files have been written to: .../mgmigrate/build

when I run make, I see this error:

CMake Error at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find PostgreSQL (missing: PostgreSQL_LIBRARY
  PostgreSQL_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/FindPostgreSQL.cmake:272 (find_package_handle_standard_args)
  cmake/config.cmake:26 (find_package)
  CMakeLists.txt:27 (include)

then the make process continues until it halts at:

[ 64%] No install step for 'postgresql-proj'
[ 65%] Completed 'postgresql-proj'
[ 65%] Built target postgresql-proj
make: *** [all] Error 2

How do I make it so it finds Postgres in make?

warning (resolved with changes to CMakeLists.txt)

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:52 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3075 (message):
  The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
  not set.  The policy's OLD behavior will be used.  When using a URL
  download, the timestamps of extracted files should preferably be that of
  the time of extraction, otherwise code that depends on the extracted
  contents might not be rebuilt if the URL changes.  The OLD behavior
  preserves the timestamps from the archive instead, but this is usually not
  what you want.  Update your project to the NEW behavior or specify the
  DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
  robustness issue.
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
  CMakeLists.txt:52 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:73 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:94 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:122 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:145 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:173 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:202 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

Fix --version

mgmigrate --version returns just mgmigrate. Return the proper version string.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.