chenglongchen / tensorflow-dsmm

Tensorflow implementations of various Deep Semantic Matching Models (DSMM).

tensorflow-DSMM

Ongoing project for implementing various Deep Semantic Matching Models (DSMM). DSMM is widely used for:

• duplicate detection
• sentence similarity
• question answering
• search relevance
• ...

Quickstart

Data

This project is developed with regard to the data format provided in the 第三届魔镜杯大赛.

You can see /data/DATA.md for the data format description and prepared data accordingly. Your data should be placed in the data directory. Current data directory also holds a toy data.

If you want to run a quick demo, you can download data from the above competition link. Download is allowed after registration.

Demo

python src/main.py

Supported Models

Representation based methods

• DSSM style models
  • DSSM: use FastText as encoder
  • CDSSM: use TextCNN as encoder
  • RDSSM: use TextRNN/TextBiRNN as encoder

Interaction based methods

• MatchPyramid style models
  • MatchPyramid: use identity/cosine similarity/dot product as match matrix
  • General MatchPyramid: use match matrices based on various embeddings and various match scores
    • word embeddings
      • original word embedding
      • compressed word embedding
      • contextual word embedding (use an encoder to encode contextual information)
    • match score
      • identity
      • cosine similarity/dot product
      • element product
      • element concat
• BCNN style models
  • BCNN
  • ABCNN1
  • ABCNN2
  • ABCNN3
• ESIM
• DecAtt (Decomposable Attention)

Building Blocks

Encoder layers

• FastText
• TimeDistributed Dense Projection
• TextCNN (Gated CNN and also Residual Gated CNN)
• TextRNN/TextBiRNN with GRU and LSTM cell

Attention layers

• mean/max/min pooling
• scalar-based and vector-based attention
• self and context attention
• multi-head attention

Acknowledgments

This project gets inspirations from the following projects:

• MatchZoo
• MatchPyramid-TensorFlow
• ABCNN

tensorflow-dsmm's Issues

您好！数据集没法下载，请问谁能提供一下数据集，万分感谢！

不知怎么生成split.pkl？

你好，直接跑程序，报错：
“FileNotFoundError: [Errno 2] No such file or directory: 'split.pkl'”

我看了前面两个问题的回答，但还是不知道怎么生成split.pkl。能更详细说明下吗？
谢谢！

what does a split.pkl contain ??

please provide Data set , download link is not working

Hi , i am trying to download dataset from the link you provided but looks like the link is expired. I am trying to understand your code but data set format is confusing me , so please provide either the dataset or a sample dataset which is of same format.

Thanks in advance