allenai/D_post_LQK_8kv_8k_13k_SWA
Updated • 19
OlmPool
Collection of models from the paper "Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension".
-
-
allenai/B_post_LQK_32kv_4k_11k_SWA
Updated • 133 -
allenai/H_post_LQK_32kv_4k_11k
Updated • 103 -
allenai/H_post_LQK_32kv_8k_11k
Updated • 92 -
allenai/G_pre_8kv_4k_14k
Updated • 14 -
allenai/G_pre_4kv_8k_14k
Updated • 14 -
allenai/H_pre_32kv_8k_11k_SWA
Updated • 88 -
allenai/G_pre_LQK_8kv_8k_14k
Updated • 304 -
allenai/G_post_LQK_8kv_8k_14k
Updated • 89 -
allenai/I_pre_32kv_8k_12k
Updated • 15 -
allenai/G_pre_8kv_8k_14k
Updated • 14 -
allenai/J_pre_16kv_8k_14k
Updated • 14 -
allenai/D_post_LQK_8kv_8k_13k_SWA_fp8
Updated • 94 -
allenai/C_post_LQK_8kv_8k_13k_SWA
Updated • 93 -
allenai/H_post_HQK_32kv_8k_11k_SWA
Updated • 80 -
allenai/E_post_LQK_32kv_8k_11k_SWA_fp8
Updated • 94 -
allenai/F_pre_8kv_8k_14k_SWA
Updated • 15 -
allenai/H_post_LQK_32kv_8k_11k_SWA_fp8
Updated • 92 -
allenai/A_post_HQK_8kv_8k_13k_SWA_fp8
Updated • 88 -
allenai/H_post_LQK_32kv_8k_11k_SWA
Updated • 94 -
allenai/K_post_HQK_8kv_12k
Updated • 202 -
allenai/B_post_LQK_32kv_4k_11k
Updated • 93 -
allenai/G_post_LQK_8kv_4k_14k_SWA
Updated • 198 -
allenai/H_post_LQK_8kv_4k_11k_SWA
Updated • 92 -
allenai/G_pre_8kv_8k_14k_SWA
Updated • 117 -
allenai/H_pre_LQK_32kv_8k_11k_SWA
Updated • 13