ikostrikov / tensorflow-pointer-networks Goto Github PK
View Code? Open in Web Editor NEWTensorFlow implementation of Pointer Networks
License: MIT License
TensorFlow implementation of Pointer Networks
License: MIT License
Can I feed the shift of decoder outputs? Or just zeros?
Thank you @ikostrikov @j-min @GalDude33 @chenghuige
Could you please explain the dataset and the DataGenerator function you used?
I couldn't fully understand the problem you want to solve with this code.
Is it one of the three performance evaluation written in the original paper?
I download the code and run it with tensorflow version 1.3.0
ImportError: cannot import name core_rnn_cell_impl
>>> from tensorflow.contrib.rnn.python.ops import rnn_cell_impl
I change the line
from tensorflow.contrib.rnn.python.ops import rnn_cell_impl
to be
from tensorflow.python.ops import rnn_cell_impl
ValueError: Trying to share variable rnn/gru_cell/gates/kernel, but specified shape (64, 64) and found shape (33, 64).
It is because x = rnn_cell_impl._linear([inp, attns], cell.output_size, True)
outputs the tensor has shape (32, 64) and combine the m_state shape (32, 64)
Now I don't know how to resolve this.
Hi,
Thanks for sharing the code. I have a doubt in pointer.py. In the below code fragment:
if feed_prev and i > 0:
inp = tf.pack(decoder_inputs)
inp = tf.transpose(inp, perm=[1, 0, 2])
inp = tf.reshape(inp, [-1, attn_length, input_size])
inp = tf.reduce_sum(inp * tf.reshape(tf.nn.softmax(output), [-1, attn_length, 1]), 1)
inp = tf.stop_gradient(inp)
inps.append(inp)
Here you are computing inp from decoder_inputs, but at the test time, you wouldn't really have decoder_inputs. Shouldn't it be computed from encoder_inputs rather by indexing into them?
Is this model supposed to share weights between encoder and decoder?
When I run the code, it shows an ValueError that I attempted to reuse RNNCell.
I can either reuse it or not, but I am not sure what approach I should take.
Thanks in advance.
x = core_rnn_cell_impl._linear([inp, attns], cell.output_size, True)
which attns = array_ops.zeros(batch_attn_size, dtype=dtype)
why use zeros attns ๏ผ
Hello.
I'm trying to understand the code.
In pointer.py, there is an attns.
It is initialized as zeros and becomes an input to _linear function.
Why do you need it?
Thanks in advance.
Input a decoder_inputs
list of two elements. Is it right?
Thank you @ikostrikov @j-min @GalDude33 @chenghuige
if i run with tf 1.0.0, exception arise as tf.nn.ops have no attribution of rnn
i run the notebook with tf 0.12 got issue below:
`---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
in
----> 1 pointer_network = PointerNetwork(FLAGS.max_steps, 1, FLAGS.rnn_size, 1, 5, FLAGS.batch_size, 1e-2, 0.95)
2 dataset = DataGenerator()
3 pointer_network.step()
in init(self, max_len, input_size, size, num_layers, max_gradient_norm, batch_size, learning_rate, learning_rate_decay_factor)
59 with tf.variable_scope("decoder"):
60 outputs, states, _ = pointer_decoder(
---> 61 self.decoder_inputs, final_state, attention_states, cell)
62
63 with tf.variable_scope("decoder", reuse=True):
~/tutorial/TensorFlow-Pointer-Networks/pointer.py in pointer_decoder(decoder_inputs, initial_state, attention_states, cell, feed_prev, dtype, scope)
121
122 # Merge input and previous attentions into one vector of the right size.
--> 123 x = core_rnn_cell_impl._linear([inp, attns], cell.output_size, True)
124 # Run the RNN.
125 cell_output, new_state = cell(x, states[-1])
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py in _linear(args, output_size, bias, bias_start, scope)
749 res = math_ops.matmul(args[0], weights)
750 else:
--> 751 res = math_ops.matmul(array_ops.concat(args, 1), weights)
752 if not bias:
753 return res
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py in concat(concat_dim, values, name)
1073 ops.convert_to_tensor(concat_dim,
1074 name="concat_dim",
-> 1075 dtype=dtypes.int32).get_shape(
1076 ).assert_is_compatible_with(tensor_shape.scalar())
1077 return identity(values[0], name=scope)
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in convert_to_tensor(value, dtype, name, as_ref, preferred_dtype)
667
668 if ret is None:
--> 669 ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
670
671 if ret is NotImplemented:
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py in _constant_tensor_conversion_function(v, dtype, name, as_ref)
174 as_ref=False):
175 _ = as_ref
--> 176 return constant(v, dtype=dtype, name=name)
177
178
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py in constant(value, dtype, shape, name, verify_shape)
163 tensor_value = attr_value_pb2.AttrValue()
164 tensor_value.tensor.CopyFrom(
--> 165 tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
166 dtype_value = attr_value_pb2.AttrValue(type=tensor_value.tensor.dtype)
167 const_tensor = g.create_op(
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py in make_tensor_proto(values, dtype, shape, verify_shape)
365 nparray = np.empty(shape, dtype=np_dt)
366 else:
--> 367 _AssertCompatible(values, dtype)
368 nparray = np.array(values, dtype=np_dt)
369 # check to them.
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py in _AssertCompatible(values, dtype)
300 else:
301 raise TypeError("Expected %s, got %s of type '%s' instead." %
--> 302 (dtype.name, repr(mismatch), type(mismatch).name))
303
304
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.`
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.