Question about how to detect events in video about chat-univi

[jpthu17]

I am sorry to reply to you so late because I have been busy with other projects recently.

The relevant code is below, we are currently trying to improve this part of the code and will update our code base in the near future.

Chat-UniVi/ChatUniVi/model/arch.py

Lines 135 to 153 in f63c247

	cls_features = torch.mean(image_features, dim=1, keepdim=False).unsqueeze(0).clone()
	token_dict = {'x': cls_features,
	'token_num': cls_features.size(1),
	'idx_token': torch.arange(cls_features.size(1))[None, :].repeat(
	cls_features.size(0), 1),
	'agg_weight': cls_features.new_ones(cls_features.size(0), cls_features.size(1),
	1),
	'mask': None}
	down_dict, token_dict = self.get_model().ctm3(token_dict)
	events = OrderedDict()
	max_len = 0
	for id, i in enumerate(down_dict["idx_token"][0].tolist()):
	if i not in events:
	events[i] = [id]
	else:
	events[i].append(id)
	max_len = len(events[i]) if max_len < len(events[i]) else max_len

[xiaokj37]

Thanks for your reply.
I am now trying to reproduce the results of the video event slicing given in the paper. You have given the visualization code for the image, is it possible to give the processing code for the video and the corresponding reference video.
I'm waiting for your early reply.