Attention Mechanism

textgenrnnPython

A Python module for easily training character- or word-level text-generating neural networks on any dataset with minimal code.

BiDAFPython

A deep learning model for machine comprehension that uses bi-directional attention flow to answer questions about text passages.

Show, Attend and TellJupyter Notebook

TensorFlow implementation of an attention-based neural image caption generator that focuses on relevant image parts while generating words.

DNABERTPython

A pre-trained BERT model designed for DNA sequence analysis, enabling genome understanding tasks like classification and motif discovery.

R-NetPython

TensorFlow implementation of R-Net for machine reading comprehension on the SQuAD dataset.

LEDNetPython

A lightweight encoder-decoder neural network for real-time semantic segmentation on resource-constrained devices.

PaperRobot: Incremental Draft Generation of Scientific IdeasPython

An AI system that incrementally generates scientific paper drafts by predicting links between concepts and generating text sections.

kvaxPython

A FlashAttention 2 implementation for JAX with block-wise document mask optimization and context parallelism for efficient long-sequence training.

Attentive Object TrackingPython

A TensorFlow implementation of hierarchical attentive recurrent neural networks for single object tracking in videos.

Hierarchical Attention NetworksPython

TensorFlow implementation of hierarchical attention networks for document classification using GRU cells and attention mechanisms.

Unsupervised Object CountingPython

A TensorFlow implementation of Attend, Infer, Repeat (AIR), a generative model for fast scene understanding by reconstructing objects sequentially.