espnet2.sds.end_to_end.mini_omni.litgpt.model.build_rope_cache

source

espnet2.sds.end_to_end.mini_omni.litgpt.model.build_rope_cache(seq_len: int, n_elem: int, device: device | None = None, base: int = 10000, condense_ratio: int = 1) → Tuple[Tensor, Tensor]

Enhanced Transformer with Rotary Position Embedding.

Derived from: https://github.com/labmlai/ annotated_deep_learning_paper_implementations/blob/master/ labml_nn/ transformers/rope/_init_.py. MIT License: https://github.com/labmlai/ annotated_deep_learning_paper_implementations/blob/ master/license.