bunsenfeng / factkb Goto Github PK

Hi, I came across FactKB in a project to evaluate summarization and found it useful for my study. I tried to run the code and found that the pretrained model only accepts max 512 tokens which seems to be the combination for both summary and article. For my case my summary length is around 500 tokens and input is 4K tokens. I wonder if there are any recommendations to run factKB for my case, as I saw some previous papers doing sentence-to-sentence approaches for evaluation (e.g. section 3.2 of Measuring Faithfulness of Abstractive Summaries. I would appreciate any advice on whether using a sentence-to-sentence approach makes sense for factKB, or if there are any other methods that you found helpful. Thanks!