yhcc / utcie Goto Github PK
View Code? Open in Web Editor NEWThis is the code repo for the paper <UTC-IE: A Unified Token-pair Classification Architecture for Information Extraction>
This is the code repo for the paper <UTC-IE: A Unified Token-pair Classification Architecture for Information Extraction>
作者你好 我感觉crossformer的attention部分好像缺少一个步骤?vertical方向的attention算出来之后应该要transpose回去吧,不然不是对应位置相加?
@staticmethod
def apply_rotary(x, sinusoidal_pos):
sin, cos = sinusoidal_pos
x1, x2 = x[..., 0::2], x[..., 1::2]
# 如果是旋转query key的话,下面这个直接cat就行,因为要进行矩阵乘法,最终会在这个维度求和。(只要保持query和key的最后一个dim的每一个位置对应上就可以)
# torch.cat([x1 * cos - x2 * sin, x2 * cos + x1 * sin], dim=-1)
# 如果是旋转value的话,下面这个stack后再flatten才可以,因为训练好的模型最后一个dim是两两之间交替的。
return torch.stack([x1 * cos - x2 * sin, x2 * cos + x1 * sin], dim=-1).flatten(-2, -1)
在这部分代码中是应用旋转位置编码,论文中提到的Triangle position embedding is incorporated
to mark the position of token pairs in the fea-ture map, indicating whether the cell is in the upper or lower triangles. It adds to S in Eq.(3)
before Attention.这个位置编码时在那里面实现的呢?
When will you upload the code of this paper?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.