Mar 27, 2023 � In this paper, we propose a highly parameter-efficient approach to scaling pre-trained language models (PLMs) to a deeper model depth.

In this paper, we propose a highly parameter-efficient approach to scaling pre-trained language models (PLMs) to a deeper model depth.

Mar 27, 2023 � In this paper, we propose a highly parameter-efficient approach to scaling pre-trained language models (PLMs) to a deeper model depth.

Mar 2, 2023 � A new branch of research focusing on the parameter-efficient adaptation of PLMs, which optimizes a small portion of the model parameters while keeping the rest�...

Mar 27, 2023 � In this paper, we propose a highly parameter- efficient approach to scaling pre-trained lan- guage models (PLMs) to a deeper model depth.

Jan 5, 2024 � In this paper, we propose a highly parameter- efficient approach to scaling pre-trained lan- guage models (PLMs) to a deeper model depth.

Exploring parameter-efficient fine-tuning techniques for code generation with large language models.

This paper adopts matrix product operator (MPO, a tensor decomposition from quantum many-body physics) to reconstruct the parameter matrix in the expert�...

Recently, Mixture-of-Experts (short as MoE) architecture has achieved remarkable success in increasing the model capacity of large-scale language models.