jack57lee / Diversify-MHA Public

Notifications You must be signed in to change notification settings
Fork 2
Star 18

EMNLP 2018: Multi-Head Attention with Disagreement Regularization; NAACL 2019: Information Aggregation for Multi-Head Attention with Routing-by-Agreement

18 stars 2 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 64 Commits
bin		bin
curve		curve
data		data
interface		interface
layers		layers
models		models
scripts		scripts
utils		utils
README.md		README.md
__init__.py		__init__.py

Repository files navigation

On the Diversity of Multi-Head Attention

Implementation for EMNLP 2018 paper: Multi-Head Attention with Disagreement Regularization and NAACL 2019 paper: Information Aggregation for Multi-Head Attention with Routing-by-Agreement, based on the THUMT toolkit.

More details including data and pre-trained models are coming later.

About

EMNLP 2018: Multi-Head Attention with Disagreement Regularization; NAACL 2019: Information Aggregation for Multi-Head Attention with Routing-by-Agreement

deep-learning neural-machine-transliteration multi-head-attention

Report repository

Releases

No releases published

Packages

Languages

Python 100.0%