GithubHelp home page GithubHelp logo

nebula-java's Introduction

nebula-java

LICENSE GitHub release GitHub release date codecov

Nebula Java is a Java client for developers to connect their projects to Nebula Graph.

Please be noted that nebula-JDBC(based on nebula-java) is the JDBC implementing for nebula graph.

NOTE: Nebula Java is not thread-safe.

Two main branches of this repository

In this repository, you can find two branches for the source code of Nebula Java of different versions.

The master branch

The master branch is for Nebula Java v2.0, which works with Nebula Graph v2.0 nightly.

This README file provides Java developers with instructions on how to connect to Nebula Graph v2.0.

The v1.0 branch

In the v1.0 branch, you can find source code of these:

  • Nebula Java v1.0, which works with Nebula Graph v1.1.0 and earlier versions only.
  • Nebula Graph Exchange, Nebula Spark Connector, Nebula Flink Connector, and nebula-algorithm.

For more information, see README of v1.0.

The v2.0.0-rc branch

The v2.0.0-rc branch works with Nebula Graph v2.0.0-beta and v2.0.0-rc1, but not for the latest nightly Nebula Graph.

Prerequisites

To use this Java client, do a check of these:

Modify pom.xml

If you use Maven to manage your project, add the following dependency to your pom.xml file. Replace 3.0-SNAPSHOT with an appropriate Nebula Java version. For more versions, visit releases.

<dependency>
    <groupId>com.vesoft</groupId>
    <artifactId>client</artifactId>
    <version>3.0-SNAPSHOT</version>
</dependency>

There are the version correspondence between client and Nebula:

Client version Nebula Version
1.0.0 1.0.0
1.0.1 1.1.0,1.2.0
1.1.0 1.1.0,1.2.0
1.2.0 1.1.0,1.2.0,1.2.1
2.0.0-beta 2.0.0-beta
2.0.0-rc1 2.0.0-rc1
2.0.0 2.0.0,2.0.1
2.0.1 2.0.0,2.0.1
2.5.0 2.5.0,2.5.1
2.6.0 2.6.0,2.6.1
2.6.1 2.6.0,2.6.1
3.x 3.x
3.0-SNAPSHOT nightly

Graph client example

To connect to the nebula-graphd process of Nebula Graph:

NebulaPoolConfig nebulaPoolConfig = new NebulaPoolConfig();
nebulaPoolConfig.setMaxConnSize(10);
List<HostAddress> addresses = Arrays.asList(new HostAddress("127.0.0.1", 9669), new HostAddress("127.0.0.1", 9670));

NebulaPool pool = new NebulaPool();
pool.init(addresses, nebulaPoolConfig);

Session session = pool.getSession("root", "nebula", false);

session.execute("SHOW HOSTS;");

session.release();

pool.close();

Graph SessionPool example

To use SessionPool, you must config the space to connect for SessionPool. The SessionPool is thread-safe, and support retry(release old session and get available session from SessionPool) for both connection error, session error and execution error(caused by bad storaged server), and the retry mechanism needs users to config retryTimes and intervalTime between retrys.

And SessionPool maintains servers' status, can isolation broken graphd server and auto routing restarted graphd server when you need to execute with new session, meaning your parallel is larger than the idle session number in the session pool.

List<HostAddress> addresses = Arrays.asList(new HostAddress("127.0.0.1", 9669));
String spaceName = "test";
String user = "root";
String password = "nebula";
SessionPoolConfig sessionPoolConfig = new SessionPoolConfig(addresses, spaceName, user, password);
sessionPoolConfig.setRetryTimes(3);
sessionPoolConfig.setIntervalTime(1000);

SessionPool sessionPool = new SessionPool(sessionPoolConfig);

if (!sessionPool.init()) {
  log.error("session pool init failed.");
  return;
}

ResultSet resultSet;

try {
    
  resultSet = sessionPool.execute("match (v:player) return v limit 1;");
  System.out.println(resultSet.toString());
  
} catch (IOErrorException | ClientServerIncompatibleException | AuthFailedException | BindSpaceFailedException e) {
    
  e.printStackTrace();
  System.exit(1);
  
} finally {
    
    sessionPool.close();
    
}

nebula-java's People

Contributors

amber1990zhang avatar anyzm avatar chenxu14 avatar chenzuyibao avatar cjdxhjj avatar corvusye avatar critical27 avatar czpmango avatar darionyaphet avatar datian9966 avatar dependabot[bot] avatar dutor avatar firepation avatar harrischu avatar hebelala avatar importmengjie avatar jabinst avatar jievince avatar jude-zhu avatar jxnu-liguobin avatar laura-ding avatar nicole00 avatar nildot avatar oldlady344 avatar riverzzz avatar shinji-ikarig avatar vchangpengfei avatar wey-gu avatar wfystx avatar whitewum avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nebula-java's Issues

ping() function problem

I have 3 problems about this function:

1.when I get a session and call ping function,for example:

session = pool.getSession("root", "nebula", false);
session.ping();

the ping function:

@Override
public boolean ping() {
    try {
        client.execute(0, "YIELD 1;".getBytes());
        return true;
    } catch (TException e) {
        if (e instanceof TTransportException) {
            TTransportException te = (TTransportException) e;
            return te.getType() != TTransportException.END_OF_FILE
                    && te.getType() != TTransportException.NOT_OPEN;
        }
        return true;
    }
}

I modify a line:

@Override
public boolean ping() {
    try {
        ExecutionResponse response = client.execute(0, "YIELD 1;".getBytes());
        return true;
    } catch (TException e) {
        if (e instanceof TTransportException) {
            TTransportException te = (TTransportException) e;
            return te.getType() != TTransportException.END_OF_FILE
                    && te.getType() != TTransportException.NOT_OPEN;
        }
        return true;
    }
}

there is no expection, but the responce is:
ExecutionResponse (
error_code : E_SESSION_INVALID (E_SESSION_INVALID),
latency_in_us : 0,
error_msg : 49 6E 76 61 6C 69 64 20 73 65 73 73 69 6F 6E 20 69 64
)
is this response normal?

  1. If the response is normal, I have added a function in branch v1.1.0, the response is the same, why the master branch say:
    // Need server supported, v1.0 nebula-graph doesn't supported

  2. this function in c++ client is :

bool Connection::ping() {
    auto resp = execute(-1 /*Only check connection*/, "YIELD 1");
    if (resp.errorCode == ErrorCode::E_RPC_FAILURE || resp.errorCode == ErrorCode::E_DISCONNECTED) {
        return false;
    }
    return true;
}

any sessionId <= 0 is ok? right?

Support meta-client

We should support the features as follows:

  1. Keep the parts allocation information in local cache. (Which will be used by storage client)
  2. Support getTagID/getEdgeType RPC interfaces

Nullpoint Exception for different thead

package com.coney.microservice.dashboard;

import cn.hutool.core.util.StrUtil;
import com.facebook.thrift.TException;
import com.google.common.collect.Lists;
import com.vesoft.nebula.client.graph.*;
import com.vesoft.nebula.data.Result;
import com.vesoft.nebula.graph.RowValue;
import lombok.extern.slf4j.Slf4j;
import org.apache.poi.ss.formula.functions.T;

import java.util.List;

@slf4j
public class NebulaTest {

public static void main(String[] args) throws InterruptedException {
    GraphClientImpl graphClient = new GraphClientImpl("192.168.101.128", 3699);
    graphClient.setUser("user");
    graphClient.setPassword("password");
    try {
        graphClient.connect();
    } catch (TException e) {
        e.printStackTrace();
    }
    int code = graphClient.switchSpace("dashboard_graph");
    System.out.println(code);

    Thread t = new Thread(new Runnable() {
        @Override
        public void run() {
            try {
                String query = "GO FROM uuid(\"{}\") OVER e_user_own_dashboard " +
                        "yield " +
                        "$$.tag_dashboard.id as ID, " +
                        "$$.tag_dashboard.name AS name, " +
                        "$$.tag_dashboard.description AS description, " +
                        "$$.tag_dashboard.status AS status, " +
                        "$$.tag_dashboard.content AS content";
                ResultSet graphResult = graphClient.executeQuery(StrUtil.format(query, 1));
                List<ResultSet.Result> rows = graphResult.getResults();
                for (ResultSet.Result value : rows) {
                    log.info("ID: {}", value.getString("id"));
                    log.info("Name: {}", value.getString("name"));
                }
            } catch (ConnectionException e) {
                e.printStackTrace();
            } catch (NGQLException e) {
                e.printStackTrace();
            } catch (TException e) {
                e.printStackTrace();
            }
        }
    });

    t.start();
    t.join();
}

}

log conflict

nubula force binding slf4j with log4j, if I use logback, will be conflict

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/xx/.m2/repository/com/vesoft/client/1.1.0/client-1.1.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/xx/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

nebula java api shaded log, guava, thrift dep but not change their package name, this is a bad practice, will cause developer dependency conflict

add shaded dependency

In order to avoid such dependency conflicts , I think there should be a shaded client dependency .

List Spaces Error Code: -11 and Get tags Error: -23

Exchange 1.1.0 cannot get access to the Meta Service when Nebula Graph is deployed with Docker Compose.

Here is some information about the images:

  • Here is the image information:
REPOSITORY                      TAG                 IMAGE ID            CREATED             SIZE
vesoft/nebula-graphd            nightly             a489b524505e        3 days ago          283MB
vesoft/nebula-metad             nightly             a28667111fdf        3 days ago          288MB
vesoft/nebula-storaged          nightly             664d16ee16de        3 days ago          289MB
  • When I ran $SPARK_HOME/bin/spark-submit to import data, this error occurred:
21/01/04 16:55:52 INFO scheduler.DAGScheduler: Job 1 finished: csv at FileBaseReader.scala:86, took 0.060284 s
21/01/04 16:55:52 ERROR meta.MetaClientImpl: List Spaces Error Code: -11
21/01/04 16:55:52 ERROR meta.MetaClientImpl: Get tags Error: -23
Exception in thread "main" java.util.NoSuchElementException: key not found: courseId
	at scala.collection.MapLike$class.default(MapLike.scala:228)
	at scala.collection.AbstractMap.default(Map.scala:59)
	at scala.collection.MapLike$class.apply(MapLike.scala:141)
	at scala.collection.AbstractMap.apply(Map.scala:59)
	at com.vesoft.nebula.tools.importer.utils.NebulaUtils$$anonfun$getDataSourceFieldType$1.apply(NebulaUtils.scala:65)
	at com.vesoft.nebula.tools.importer.utils.NebulaUtils$$anonfun$getDataSourceFieldType$1.apply(NebulaUtils.scala:64)
	at scala.collection.immutable.Range.foreach(Range.scala:160)
	at com.vesoft.nebula.tools.importer.utils.NebulaUtils$.getDataSourceFieldType(NebulaUtils.scala:64)
	at com.vesoft.nebula.tools.importer.processor.VerticesProcessor.process(VerticesProcessor.scala:137)
	at com.vesoft.nebula.tools.importer.Exchange$$anonfun$main$2.apply(Exchange.scala:174)
	at com.vesoft.nebula.tools.importer.Exchange$$anonfun$main$2.apply(Exchange.scala:152)
	at scala.collection.immutable.List.foreach(List.scala:392)
	at com.vesoft.nebula.tools.importer.Exchange$.main(Exchange.scala:152)
	at com.vesoft.nebula.tools.importer.Exchange.main(Exchange.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
	at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
	at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
	at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
	at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
	at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

我在客户端看到了重试的代码,重试代码放在客户端做,高并发如何保证数据的一致性?

    int retry = executionRetry;
    while (retry-- > 0) {
        try {
            ExecutionResponse executionResponse = client.execute(sessionID, statement);
            if (executionResponse.getError_code() != ErrorCode.SUCCEEDED) {
                LOGGER.error("execute error: " + executionResponse.getError_msg());
            }
            return executionResponse.getError_code();
        } catch (TException e) {
            LOGGER.error("Thrift rpc call failed: " + e.getMessage());
            return ErrorCode.E_RPC_FAILURE;
        }
    }

Inability in usage of session-pool (NebulaPool)

As you see in all your examples, first you create a session pool with max-sessions=n, and then you pick just one Session to execute a command.

After your session creation, first, you execute "use space" and then you execute all your other commands by this Session object, the problem is that you cannot use session-pool correctly. why? because if you want to use it, per time you call session.release() it clears all states (it means it clears "use space") too. and in next time you should first call "use space; (and then your main command...)", and it increases the time of command execution (for example it increases INSERT time from 5 ms to 11 ms).

It looks like that it is better not to reset space declaration in release time and also it is helpfully to declare space name in CREATION phase of session-pool (take HostAddress, nebulaConfig, and space name)

Java tools for nebula version 2.0.0

I want to use nebula algorithm such as pagerank on database, but apparently is that correct? How can I use those algorithm on nebula version 2?

NullPointerException

ResultSet.java:34 这里会有空指针异常,返回状态码 ErrorCode.SUCCEEDED时columns可能会为空

How to use Nebula Graph Exchange in Nebula Graph v2.0

I was trying to import data from neo4j to Nebula Graph v2.0.
I found that I need Nebula Graph Exchange, but in nebula-java v2.0 branch there's no "tools" directory, so I was wondering how I can perform my task. Could you please help me how to do it (admitting that it's possible)?

can session from pool.getSession be shared in multi-threading

现在每次query 的时候
pool.getSession
使用
session.release();

但发现getSession 每次都耗时3ms 耗时挺多的,想质询下,这个getSession 是否可以做成静态变量多个线程公用,而不用每次都获取使用完释放呢

gradle Could not resolve com.vesoft:client:2.0.0-SNAPSHOT

Could not resolve 2.0.0-SNAPSHOT with gradle, details:

# build.gradle
dependencies {
    implementation 'com.vesoft:client:2.0.0-SNAPSHOT'
}
repositories {
    maven {  url 'https://oss.sonatype.org/content/repositories/snapshots/'  }
    maven {  url 'https://maven.aliyun.com/repository/public'  }
    jcenter()
}
$ gradle compileJava --stacktrace
> Task :compileJava FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Could not resolve all files for configuration ':compileClasspath'.
> Could not resolve com.vesoft:client:2.0.0-SNAPSHOT.
  Required by:
      project :
   > Could not resolve com.vesoft:client:2.0.0-SNAPSHOT.
      > Could not parse POM https://oss.sonatype.org/content/repositories/snapshots/com/vesoft/client/2.0.0-SNAPSHOT/client-2.0.0-20210511.030005-1.pom
         > Could not find com.vesoft:nebula:2.0.0-SNAPSHOT.
           Searched in the following locations:
             - https://oss.sonatype.org/content/repositories/snapshots/com/vesoft/nebula/2.0.0-SNAPSHOT/maven-metadata.xml
             - https://oss.sonatype.org/content/repositories/snapshots/com/vesoft/nebula/2.0.0-SNAPSHOT/nebula-2.0.0-SNAPSHOT.pom
             - https://oss.sonatype.org/content/repositories/snapshots/com/vesoft/nebula/2.0.0-SNAPSHOT/nebula-2.0.0-SNAPSHOT.jar
             - https://maven.aliyun.com/repository/public/com/vesoft/nebula/2.0.0-SNAPSHOT/maven-metadata.xml
             - https://maven.aliyun.com/repository/public/com/vesoft/nebula/2.0.0-SNAPSHOT/nebula-2.0.0-SNAPSHOT.pom
             - https://maven.aliyun.com/repository/public/com/vesoft/nebula/2.0.0-SNAPSHOT/nebula-2.0.0-SNAPSHOT.jar
             - https://jcenter.bintray.com/com/vesoft/nebula/2.0.0-SNAPSHOT/maven-metadata.xml
             - https://jcenter.bintray.com/com/vesoft/nebula/2.0.0-SNAPSHOT/nebula-2.0.0-SNAPSHOT.pom
             - https://jcenter.bintray.com/com/vesoft/nebula/2.0.0-SNAPSHOT/nebula-2.0.0-SNAPSHOT.jar

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException: Could not resolve all files for configuration ':compileClasspath'.
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.rethrowFailure(DefaultConfiguration.java:1054)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.access$1700(DefaultConfiguration.java:123)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$ConfigurationFileCollection.getFiles(DefaultConfiguration.java:1028)
        at org.gradle.api.internal.file.AbstractFileCollection.iterator(AbstractFileCollection.java:76)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.iterator(DefaultConfiguration.java:439)
        at org.gradle.api.internal.changedetection.state.DefaultFileSystemSnapshotter$FileCollectionVisitorImpl.visitCollection(DefaultFileSystemSnapshotter.java:257)
        at org.gradle.api.internal.file.AbstractFileCollection.visitRootElements(AbstractFileCollection.java:282)
        at org.gradle.api.internal.file.CompositeFileCollection.visitRootElements(CompositeFileCollection.java:206)
        at org.gradle.api.internal.changedetection.state.DefaultFileSystemSnapshotter.snapshot(DefaultFileSystemSnapshotter.java:142)
        at org.gradle.internal.fingerprint.impl.AbstractFileCollectionFingerprinter.fingerprint(AbstractFileCollectionFingerprinter.java:54)
        at org.gradle.internal.fingerprint.impl.DefaultCompileClasspathFingerprinter.fingerprint(DefaultCompileClasspathFingerprinter.java:46)
        at org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository.fingerprintTaskFiles(CacheBackedTaskHistoryRepository.java:358)
        at org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository.createExecution(CacheBackedTaskHistoryRepository.java:163)
        at org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository.access$100(CacheBackedTaskHistoryRepository.java:76)
        at org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository$1.getCurrentExecution(CacheBackedTaskHistoryRepository.java:123)
        at org.gradle.api.internal.changedetection.changes.DefaultTaskArtifactStateRepository$TaskArtifactStateImpl.getStates(DefaultTaskArtifactStateRepository.java:200)
        at org.gradle.api.internal.changedetection.changes.DefaultTaskArtifactStateRepository$TaskArtifactStateImpl.isUpToDate(DefaultTaskArtifactStateRepository.java:94)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:50)
        at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
        at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
        at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:51)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
        at org.gradle.execution.taskgraph.LocalTaskInfoExecutor.execute(LocalTaskInfoExecutor.java:42)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:277)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:262)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:135)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:130)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.execute(DefaultTaskPlanExecutor.java:200)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.executeWithWork(DefaultTaskPlanExecutor.java:191)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.run(DefaultTaskPlanExecutor.java:130)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
Caused by: org.gradle.internal.resolve.ModuleVersionResolveException: Could not resolve com.vesoft:client:2.0.0-SNAPSHOT.
Required by:
    project :
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.RepositoryChainComponentMetaDataResolver.resolveModule(RepositoryChainComponentMetaDataResolver.java:103)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.RepositoryChainComponentMetaDataResolver.resolve(RepositoryChainComponentMetaDataResolver.java:63)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.ComponentResolversChain$ComponentMetaDataResolverChain.resolve(ComponentResolversChain.java:94)
        at org.gradle.api.internal.artifacts.ivyservice.clientmodule.ClientModuleResolver.resolve(ClientModuleResolver.java:62)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.ComponentState.resolve(ComponentState.java:208)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.ComponentState.resolve(ComponentState.java:196)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.ComponentState.getMetadata(ComponentState.java:152)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.EdgeState.calculateTargetConfigurations(EdgeState.java:156)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.EdgeState.attachToTargetConfigurations(EdgeState.java:112)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.attachToTargetRevisionsSerially(DependencyGraphBuilder.java:315)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.resolveEdges(DependencyGraphBuilder.java:202)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.traverseGraph(DependencyGraphBuilder.java:155)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.resolve(DependencyGraphBuilder.java:126)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.DefaultArtifactDependencyResolver.resolve(DefaultArtifactDependencyResolver.java:123)
        at org.gradle.api.internal.artifacts.ivyservice.DefaultConfigurationResolver.resolveGraph(DefaultConfigurationResolver.java:167)
        at org.gradle.api.internal.artifacts.ivyservice.ShortCircuitEmptyConfigurationResolver.resolveGraph(ShortCircuitEmptyConfigurationResolver.java:89)
        at org.gradle.api.internal.artifacts.ivyservice.ErrorHandlingConfigurationResolver.resolveGraph(ErrorHandlingConfigurationResolver.java:73)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$5.run(DefaultConfiguration.java:533)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.resolveGraphIfRequired(DefaultConfiguration.java:524)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.resolveToStateOrLater(DefaultConfiguration.java:509)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.access$1800(DefaultConfiguration.java:123)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$ConfigurationFileCollection.getSelectedArtifacts(DefaultConfiguration.java:1037)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$ConfigurationFileCollection.getFiles(DefaultConfiguration.java:1025)
        ... 43 more
Caused by: org.gradle.internal.resolve.ModuleVersionResolveException: Could not resolve com.vesoft:client:2.0.0-SNAPSHOT.
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ErrorHandlingModuleComponentRepository$ErrorHandlingModuleComponentRepositoryAccess.resolveComponentMetaData(ErrorHandlingModuleComponentRepository.java:141)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ComponentMetaDataResolveState.process(ComponentMetaDataResolveState.java:75)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ComponentMetaDataResolveState.resolve(ComponentMetaDataResolveState.java:63)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.RepositoryChainComponentMetaDataResolver.findBestMatch(RepositoryChainComponentMetaDataResolver.java:138)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.RepositoryChainComponentMetaDataResolver.findBestMatch(RepositoryChainComponentMetaDataResolver.java:119)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.RepositoryChainComponentMetaDataResolver.resolveModule(RepositoryChainComponentMetaDataResolver.java:92)
        ... 70 more
Caused by: org.gradle.api.internal.artifacts.ivyservice.ivyresolve.parser.MetaDataParseException: Could not parse POM https://oss.sonatype.org/content/repositories/snapshots/com/vesoft/client/2.0.0-SNAPSHOT/client-2.0.0-20210511.030005-1.pom
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.parser.AbstractModuleDescriptorParser.parseDescriptor(AbstractModuleDescriptorParser.java:54)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.parser.AbstractModuleDescriptorParser.parseMetaData(AbstractModuleDescriptorParser.java:43)
        at org.gradle.api.internal.artifacts.repositories.metadata.DefaultMavenPomMetadataSource.parseMetaDataFromResource(DefaultMavenPomMetadataSource.java:53)
        at org.gradle.api.internal.artifacts.repositories.metadata.DefaultMavenPomMetadataSource.parseMetaDataFromResource(DefaultMavenPomMetadataSource.java:38)
        at org.gradle.api.internal.artifacts.repositories.metadata.AbstractRepositoryMetadataSource.parseMetaDataFromArtifact(AbstractRepositoryMetadataSource.java:72)
        at org.gradle.api.internal.artifacts.repositories.metadata.AbstractRepositoryMetadataSource.create(AbstractRepositoryMetadataSource.java:59)
        at org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver.resolveStaticDependency(ExternalResourceResolver.java:244)
        at org.gradle.api.internal.artifacts.repositories.resolver.MavenResolver.resolveUniqueSnapshotDependency(MavenResolver.java:135)
        at org.gradle.api.internal.artifacts.repositories.resolver.MavenResolver.doResolveComponentMetaData(MavenResolver.java:115)
        at org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver$RemoteRepositoryAccess.resolveComponentMetaData(ExternalResourceResolver.java:445)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleComponentRepository$ResolveAndCacheRepositoryAccess.resolveComponentMetaData(CachingModuleComponentRepository.java:378)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ErrorHandlingModuleComponentRepository$ErrorHandlingModuleComponentRepositoryAccess.resolveComponentMetaData(ErrorHandlingModuleComponentRepository.java:138)
        ... 75 more
Caused by: org.gradle.internal.resolve.ModuleVersionNotFoundException: Could not find com.vesoft:nebula:2.0.0-SNAPSHOT.
Searched in the following locations:
  - https://oss.sonatype.org/content/repositories/snapshots/com/vesoft/nebula/2.0.0-SNAPSHOT/maven-metadata.xml
  - https://oss.sonatype.org/content/repositories/snapshots/com/vesoft/nebula/2.0.0-SNAPSHOT/nebula-2.0.0-SNAPSHOT.pom
  - https://oss.sonatype.org/content/repositories/snapshots/com/vesoft/nebula/2.0.0-SNAPSHOT/nebula-2.0.0-SNAPSHOT.jar
  - https://maven.aliyun.com/repository/public/com/vesoft/nebula/2.0.0-SNAPSHOT/maven-metadata.xml
  - https://maven.aliyun.com/repository/public/com/vesoft/nebula/2.0.0-SNAPSHOT/nebula-2.0.0-SNAPSHOT.pom
  - https://maven.aliyun.com/repository/public/com/vesoft/nebula/2.0.0-SNAPSHOT/nebula-2.0.0-SNAPSHOT.jar
  - https://jcenter.bintray.com/com/vesoft/nebula/2.0.0-SNAPSHOT/maven-metadata.xml
  - https://jcenter.bintray.com/com/vesoft/nebula/2.0.0-SNAPSHOT/nebula-2.0.0-SNAPSHOT.pom
  - https://jcenter.bintray.com/com/vesoft/nebula/2.0.0-SNAPSHOT/nebula-2.0.0-SNAPSHOT.jar
        at org.gradle.internal.resolve.result.DefaultBuildableComponentResolveResult.notFound(DefaultBuildableComponentResolveResult.java:49)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.RepositoryChainComponentMetaDataResolver.resolveModule(RepositoryChainComponentMetaDataResolver.java:108)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.RepositoryChainComponentMetaDataResolver.resolve(RepositoryChainComponentMetaDataResolver.java:63)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ResolveIvyFactory$ParentModuleLookupResolver.resolve(ResolveIvyFactory.java:176)
        at org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolverDescriptorParseContext.resolveMetaDataArtifactFile(ExternalResourceResolverDescriptorParseContext.java:70)
        at org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolverDescriptorParseContext.getMetaDataArtifact(ExternalResourceResolverDescriptorParseContext.java:56)
        at org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolverDescriptorParseContext.getMetaDataArtifact(ExternalResourceResolverDescriptorParseContext.java:64)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.parser.GradlePomModuleDescriptorParser.parsePomForSelector(GradlePomModuleDescriptorParser.java:220)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.parser.GradlePomModuleDescriptorParser.doParsePom(GradlePomModuleDescriptorParser.java:117)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.parser.GradlePomModuleDescriptorParser.doParseDescriptor(GradlePomModuleDescriptorParser.java:94)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.parser.GradlePomModuleDescriptorParser.doParseDescriptor(GradlePomModuleDescriptorParser.java:59)
        at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.parser.AbstractModuleDescriptorParser.parseDescriptor(AbstractModuleDescriptorParser.java:48)
        ... 86 more


* Get more help at https://help.gradle.org

BUILD FAILED in 7s
1 actionable task: 1 executed

It's seem com.vesoft:nebula:2.0.0-SNAPSHOT not found on the snapshot repository.

Storage Client Error

Nebula Service Version: 2.0-rc1

### Pom:

    <groupId>com.vesoft</groupId>
    <artifactId>client</artifactId>
    <version>2.0.0-rc1</version>

### My Code:
`
import com.vesoft.nebula.client.storage.StorageClient;
import com.vesoft.nebula.client.storage.scan.ScanVertexResultIterator;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.UnsupportedEncodingException;

public class StorageClientExample {

private static final Logger LOGGER = LoggerFactory.getLogger(StorageClientExample.class);

public static void main(String[] args) throws UnsupportedEncodingException {
    StorageClient client = new StorageClient("192.168.11.20", 9779);
    try {
        client.connect();
    } catch (Exception e) {
        LOGGER.error("storage client connect error, ", e);
        System.exit(1);
    }

    ScanVertexResultIterator iterator = client.scanVertex(
            "test",
            "person");
}

}

`

### My console output:

ERROR [main] - Method name listSpaces not found
ERROR [main] - Method name listSpaces not found
Exception in thread "main" java.lang.IllegalArgumentException: Space:test does not exist.
at com.vesoft.nebula.client.meta.MetaManager.getPartsAlloc(MetaManager.java:307)
at com.vesoft.nebula.client.meta.MetaManager.getSpaceParts(MetaManager.java:289)
at com.vesoft.nebula.client.storage.StorageClient.scanVertex(StorageClient.java:272)
at com.vesoft.nebula.client.storage.StorageClient.scanVertex(StorageClient.java:154)
at com.vesoft.nebula.client.storage.StorageClient.scanVertex(StorageClient.java:113)
at com.vesoft.nebula.client.storage.StorageClient.scanVertex(StorageClient.java:80)
at com.xiao.demo.StorageClientExample.main(StorageClientExample.java:30)

Process finished with exit code 1

spark-connector读出来的列似乎存在错位的情况

我使用的nebula-spark版本:使用最新的v1.0分支代码打包的1.1.0版本

我的数据示例:
image

我的代码:

spark.read.nebula(metadAddress, space, partitionNum)
    .loadVerticesToDF(tagName, "*")
    .show

代码输出:
image

最终输出的groupID、isKey和inDegree存在错位的情况,不知道是哪里的问题,应该如何定位

AsyncGraphClient switchspace 空指针

代码库中提供的 AsyncGraphClientExample 例子,执行到 switchspace 时报空指针异常,debug 后发现是 ListeningExecutorService service 对象为空,该对象通过 AsyncAbstractClient 的 connect 方法初始化,但是 AsyncGraphClientImpl 覆写了 connect 方法,导致 service 对象没有初始化,这个是代码中的 bug 吗?

partitionID with negative number

function edgeKey & vertexKey in NebulaCodecImpl.java

 long hash = MurmurHash2.hash64(vertexId.getBytes(), vertexId.length(), SEEK);
 int partitionId = (int) (hash % getPartSize(spaceName) + 1);
 
maybe we will get negative partitionID, but we expect 1 to partition_size range

ngql查询bug

1、go FROM uuid("b3d8ff87f7e9690bbf33f7afad368a8c") OVER call BIDIRECT where call._dst==uuid("b3d8ff87f7e96978979879878787")
想要查询两个顶点是否在同一条边,如此查询报错,因为条件值后面不能接uuid。

2、go FROM uuid("b3d8ff87f7e9690bbf33f7afad368a8c") OVER use_wifi,use_device YIELD use_wifi._dst as use_wifi_id,use_device._dst as use_device_id | go FROM $-.use_wifi_id,$-.use_device_id OVER use_wifi,use_device REVERSELY
想要查询出边与uuid("b3d8ff87f7e9690bbf33f7afad368a8c") 的出边为共同顶点的顶点,且是多个类型的边,在管道中id列表使用多个变量时报错。

希望查询条件后面可以接uuid,希望id列表可以使用多个变量

为什么通过spark connector reader会读出重复的节点和边?

我通过自行构造SST文件,并调用ingest tag/edge命令来将其导入nebula,然后使用spark connector reader读取它们
导入时总计3000万个节点,读取时会读出10亿+的节点,最多的节点会出现200+条重复的记录,这些重复的记录的vertexID和其他属性完全相同
边的话则会更多,暂时还没有统计,因为我光统计读取到的数量就需要花费很长的时间。。
不知道是什么原因?

除了spark connector reader之外还有什么其他合适的统计nebula节点数量的方式么?我现在不清楚是哪一步出了问题

do client signout before the TTransport is closed

code in GraphClientImpl#close() works like this

public void close() {
    super.close();
    try {
        client.signout(sessionID);
    } catch (TException e) {
        LOGGER.error("Disconnect error: " + e.getMessage());
    } finally {
        transport.close();
    }
}

we should do client#signout before super.close(), or Disconnect error: Cannot write to null outputStream will be occur

ResultSet报NullPointerException

在nebula中执行:

go from hash("player.Tony Parker") over follow yield $$.player.name as name,follow.degree as degree

返回Execution succeeded (Time spent: 1.145/1.74 ms)
但是通过java客户端执行报错:
Exception in thread "main" java.lang.NullPointerException
at com.vesoft.nebula.client.graph.ResultSet.(ResultSet.java:35)
at com.vesoft.nebula.client.graph.GraphClientImpl.executeQuery(GraphClientImpl.java:128)
代码:

String query = "go from hash(\"player.Tony Parker\") over follow yield $$.player.name as name,follow.degree as degree";
        GraphClient client = new GraphClientImpl(addr,port);
        client.setUser("user");
        client.setPassword("password");

        try {
            client.connect();
            client.switchSpace("nba1");
        } catch (TException e) {
            e.printStackTrace();
        }
        ResultSet resultSet = null;
        try {
            resultSet = client.executeQuery(query);
        } catch (ConnectionException e) {
            e.printStackTrace();
        } catch (NGQLException e) {
            e.printStackTrace();
        } catch (TException e) {
            e.printStackTrace();
        }

执行session.execute时,普通用户“use ${space}”语句和其他查询语句一起执行提示“No space selected”

我在数据库创建了个用户admin,授权了Admin角色
执行

Session session = pool.getSession("admin", "*****", false);
session.execute("use friend_chain;fetch prop on user \"1000003127\" ;");

会提示No space selected,必须将切换图空间和查询语句分开才行,如下:

Session session = pool.getSession("admin", "*****", false);
session.execute("use friend_chain");
session.execute("fetch prop on user \"1000003127\" ;");

但是使用root用户(God角色)是没问题的。

麻烦看下这个问题,毕竟执行两次execute会增加网络延迟的

Why using ThreadLocal<GraphService.Client> in GraphClientImpl?

In the GraphClientImpl.java, I noticed that the "client" field is used to execute the real query is using the following way:

public class GraphClientImpl extends AbstractClient implements GraphClient {
    // the real thrift client is wrapped in a ThreadLocal
    private ThreadLocal<GraphService.Client> client = new ThreadLocal<>();
    @Override
    public int doConnect(List<HostAndPort> addresses) throws TException {
        GraphService.Client thriftClient = new GraphService.Client(protocol);
        client.set(thriftClient);
    }

This works well when the GraphClientImpl object is used within the same thread that create it, but if I create a new Thread, and use this GraphClientImpl object within the new thread, I cannot get the GraphService.client using the ThreadLocal.

GraphClient graphClient = new GraphClientImpl(addresses);
// this works well
graphClient.executeQuery(gql);

// this doesn't work because in new thread, graphClient cannot get the real thriftClient using ThreadLocal
new Thread(() -> {
        graphClient.executeQuery(gql);
    }).start();

I have two questions about this:

  1. Why using ThreadLocal to wrap the GraphService.Client? Is it to achieve multi-thread safe? If we don't use ThreadLocal, where there will be trouble?
  2. Is there a better way to do this? In this way, the thread that want to use GraphClientImpl object must be the thread that create this object, it is very inconvenient and easy to make mistakes.

Thanks for any help.

maven中无法下载v2.0版本的nebula

在maven中找到依赖,准备使用时,发现maven仓库中的pom,项目中无法使用它的jar包
image

在官网中按照要求使用依赖时,也无法依赖成功
image
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.