freddyalfonsoboulton / crf_tutorial Goto Github PK

there is no recursive in this function.
it should like this:
for step in range(len(states)):
for roll in loglikelihoods[1:]:
for next_state in range(self.n_states):

I tried the pytorch official implement of viterbi algorithm, the result is differnt , the diffent is below:

1. the result array should be reversed
2. inside the second "for loop", in line 122 , the roll can`t be add directly, and should add roll[next_state ]

Hi, I followed your code, only changed all torch.cat to torch.stack (because torch.cat throws error:
'''
prev_alpha = torch.cat(alpha_t).view(1,-1)
RuntimeError: zero-dimensional tensor (at position 0) cannot be concatenated
''').

And one more: in your function "simulate_data" should you add:

prev_state = next_state

inside your for loop? (I added)

my results:
...
Epoch 0: Batch 97/100 loss is 5.7809
Epoch 0: Batch 98/100 loss is 6.7747
Epoch 0: Batch 99/100 loss is 4.9654
[4 0 0 0 4 5 3 4 0 4 2 2 5 3 1]
[0 0 0 1 0 0 0 0 0 0 1 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[[-0.33391526 -0.6212296 -2.148588 ]
[-0.8201128 -0.5577968 -0.9616032 ]]

Thanks

~/Downloads/crf_tutorial/CRF.py in _compute_likelihood_denominator(self, loglikelihoods)
92 alpha_t_next_state = prev_alpha + feature_function
93 alpha_t.append(self.log_sum_exp(alpha_t_next_state))
---> 94 prev_alpha = torch.cat(alpha_t).view(1, -1)
95 return self.log_sum_exp(prev_alpha)
96

RuntimeError: zero-dimensional tensor (at position 0) cannot be concatenated

When tried to run the notebook, i encountered the following error

Sorry for missing the first issue from Shaunlipy.

I used torch 0.3.0 for this post so maybe torch 0.4 is causing the issue?

Yes, it is. Also, in utils.py line 24, ll is a zero-dimensional tensor. We should use ll.data.numpy() instead of ll.data.numpy()[0].

93        alpha_t.append(self.log_sum_exp(alpha_t_next_state))
94    prev_alpha = torch.cat(alpha_t).view(1, -1)

When I run crf_demo.ipynb you provide, exactly in model = crf_train_loop(model, rolls, targets, 1, 0.001), I meet the error as below:

File "crf_tutorial-master/CRF.py", line 94, in _compute_likelihood_denominator
   prev_alpha = torch.cat(alpha_t).view(1, -1)
RuntimeError: zero-dimensional tensor (at position 0) cannot be concatenated

Obviously, alpha_t[0] is a zero-dimensional tensor, and alpha_t is a list. alpha_t[0] should use .view(1, -1) or some other methods to trans into the 2-dimensional tensor. That's to say, alpha_t[0].shape() should be torch.Size([1, 1]).

So, maybe in line 93, the code should be modified as:

93        alpha_t.append(self.log_sum_exp(alpha_t_next_state).view(1, -1))

Thanks.