Any performance numbers? about euler

[renyi533]

Do you mean model performance or training QPS? For model performance, you can check at https://github.com/alibaba/euler/wiki/效果测试.

Our framework is optimized for large heterogeneous graphs and distributed training, we have internal performance numbers. We can add this soon after certain internal information sharing process.

For public small graphs, our training speed should be comparable with typical single node solutions.

[intoraw]

@renyi533
Thanks for your reply. Here I mean "training QPS".
Looking forward to seeing some training numbers on billion or trillion edges of graphs!

[intoraw]

@renyi533 To the best of my knowledge, it seems that Euler assumes the embeddings of all vertices can be held in the CPU memory (

euler/tf_euler/python/layers.py

Line 131 in c452251

self.embeddings = tf.get_variable(

).

For large graphs, the embeddings may exceed the CPU memory limit.
Is there any way to make Euler be able to handle this case?

[yangsiran]

@pgplus1628 This may be fixed by using partitioned variable.

[renyi533]

@pgplus1628 @yangsiran Please refer to https://stackoverflow.com/questions/47170879/what-is-partitioner-parameter-in-tensorflow-variable-scope-used-for.

You can define variable partition per variable with tf.get_variable. But this will pass the count of partitions to each inner object, which is not so elegant. That's why we do not use this way.

Instead, you can define default partitioner with tf.variable_scope. So we recommend you to use default partitioner in your main script and place "layer" under it.

Our ppi_main/reddit_main is a simple example. We can write a better example for illustration.

[renyi533]

@pgplus1628 https://github.com/alibaba/euler/wiki/性能测试 。这里添加了性能测试结果。