The entity2topic from rktamplayo

Implementation of selective disambiguation

Hi,
I've read your paper "Entity Commonsense Representation for Neural Abstractive Summarization`. Good idea. However, I'm a little confused about the implementation of the selective disambiguation module in your code.

where e_{i} is the entity embedding vector, e^{'}_{i} is the new entity vector after disambiguating encoder (where globally or locally)

I mean e^{'}_{i} is entity_vector, where d in your formula is a scalar. e_{i} is entity_proj

d = \sigma(W_d e^{'}_{i} + b_{d} equals to

                Wt = tf.get_variable("Wt", [1000, 1], initializer=xavier_initializer())
                bt = tf.Variable(tf.constant(0.0, shape=[1]))
                t = tf.nn.sigmoid(tf.tensordot(entity_vector, Wt, 1) + bt)

here f you use tanh, then
f(W_x e_{i} + b_x) equals to

                Wh = tf.get_variable("Wh", [1000, 1000], initializer=xavier_initializer())
                bh = tf.Variable(tf.constant(0.0, shape=[1000]))
                entity_proj = tf.nn.tanh(tf.tensordot(entity_proj, Wh, 1) + bh)

I'm doubting whether following implementation equals to d * f(W_x e_{i} + b_x) + (1-d)* f(W_y e^{'}_{i} + b_y)

entity_vector = t*entity_vector + (1-t)*entity_proj

I mean the correct implementation should be

                Wy = tf.get_variable("Wy", [1000, 1000], initializer=xavier_initializer())
                by = tf.Variable(tf.constant(0.0, shape=[1000]))
                entity_vector = tf.nn.tanh(tf.tensordot(entity_vector, Wy, 1) + by)

                entity_vector = t*entity_vector + (1-t)*entity_proj

code snippet in bigru_model.py

            if highway:
                # y
#                entity_proj = entity_inputs_emb[:,3:-3,:]
                Wh = tf.get_variable("Wh", [1000, 1000], initializer=xavier_initializer())
                bh = tf.Variable(tf.constant(0.0, shape=[1000]))
                entity_proj = tf.nn.tanh(tf.tensordot(entity_proj, Wh, 1) + bh)
                if not forward_only:
                    entity_proj.set_shape([self.batch_size, None, 1000])
                else:
                    entity_proj.set_shape([self.batch_size*10, None, 1000])
                
                if not forward_only:
                    entity_proj = tf.nn.dropout(entity_proj, keep_prob=0.5)
                
                # t
                Wt = tf.get_variable("Wt", [1000, 1], initializer=xavier_initializer())
                bt = tf.Variable(tf.constant(0.0, shape=[1]))
                t = tf.nn.sigmoid(tf.tensordot(entity_vector, Wt, 1) + bt)
                if not forward_only:
                    t.set_shape([self.batch_size, None, 1000])
                else:
                    t.set_shape([self.batch_size*10, None, 1000])
                self.t = t
                
                entity_vector = t*entity_vector + (1-t)*entity_proj

rktamplayo / entity2topic Goto Github PK

entity2topic's People

Contributors

Stargazers

Watchers

Forkers

entity2topic's Issues

Implementation of selective disambiguation

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

Jobs