Hugging Face's logo

cristuf
/
bge-base-financial-matryoshka

Sentence Similarity
sentence-transformers
Safetensors
English
bert
feature-extraction
Generated from Trainer
dataset_size:6300
loss:MatryoshkaLoss
loss:MultipleNegativesRankingLoss
Eval Results
text-embeddings-inference
Model card Files
xet
Community
bge-base-financial-matryoshka / README.md
cristuf's picture
cristuf
Add new SentenceTransformer model.
a7e5b12 verified
preview code
|
raw
history blame contribute delete
31.5 kB
metadata
language:
  - en
license: apache-2.0
library_name: sentence-transformers
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:6300
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
base_model: BAAI/bge-base-en-v1.5
datasets: []
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
widget:
  - source_sentence: >-
      Item 3—Legal Proceedings See discussion of Legal Proceedings in Note 10 to
      the consolidated financial statements included in Item 8 of this Report.
    sentences:
      - >-
        How much did the company's finance lease obligations total as of
        December 31, 2023?
      - What do Note 10 and Item 8 of the report encompass?
      - >-
        What was the basic earnings per common share attributable to Comcast
        Corporation shareholders in 2023?
  - source_sentence: >-
      Our quarterly Insurance segment earnings and operating cash flows are
      impacted by the Medicare Part D benefit Grant program, the changing
      membership composition, and the multistage plan period starting annually
      on January 1. These plan designs generally result in us sharing a greater
      portion of the responsibility for total prescription drug costs in the
      early stages and less in the latter stages.
    sentences:
      - >-
        What are the two main categories into which Ford Motor Company
        classifies its costs and expenses, excluding those related to Ford
        Credit?
      - >-
        How does the benefit design of Medicare Part D impact the quarterly
        insurance segment earnings and operating cash flows?
      - >-
        What basis is used to record HTM investment securities in Schwab's
        financial statements?
  - source_sentence: >-
      Operating Profit in the Wizards of the Coast and Digital Gaming segment
      decreased 2% to $538.3 million.
    sentences:
      - >-
        How much did the Wizards of the Coast and Digital Gaming segment's
        operating profit change in 2022?
      - >-
        What factors are considered in evaluating the lifetime losses for most
        loans and receivables?
      - >-
        How did the loss on certain U.S. affiliates impact the Company's
        effective tax rate in the fiscal fourth quarter of 2021?
  - source_sentence: >-
      In 2023, the net earnings of Johnson & Johnson were $35,153 million. The
      company also registered cash dividends paid amounting to $11,770 million
      for the year, priced at $4.70 per share.
    sentences:
      - What was the postpaid churn rate for AT&T Inc. in 2023?
      - >-
        What was the GAAP net revenue for the fiscal year ended October 31,
        2023?
      - What were the total net earnings of Johnson & Johnson in the year 2023?
  - source_sentence: >-
      During fiscal 2022, GameStop Corp increased its valuation allowances by
      approximately $70.2 million in various jurisdictions.
    sentences:
      - >-
        How much did GameStop Corp's valuation allowances increase during fiscal
        2022?
      - How does Gilead ensure an inclusive and diverse workforce?
      - >-
        What factors are considered in determining the estimated future warranty
        costs for connected fitness and Precor branded fitness products?
pipeline_tag: sentence-similarity
model-index:
  - name: BGE base Financial Matryoshka
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.7185714285714285
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.83
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8714285714285714
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.91
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7185714285714285
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.27666666666666667
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.17428571428571427
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.091
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7185714285714285
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.83
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8714285714285714
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.91
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8137967516958747
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7830442176870747
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7866777593387027
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.7114285714285714
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8314285714285714
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8728571428571429
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9142857142857143
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7114285714285714
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.27714285714285714
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.17457142857142854
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09142857142857141
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7114285714285714
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8314285714285714
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8728571428571429
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9142857142857143
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8123538841130576
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7798667800453513
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7831580648041446
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.7
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8285714285714286
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8614285714285714
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9042857142857142
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2761904761904762
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.17228571428571426
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09042857142857143
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8285714285714286
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8614285714285714
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9042857142857142
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8043112987059042
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7721706349206346
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7759026470022171
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.6857142857142857
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8071428571428572
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8571428571428571
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.8971428571428571
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.6857142857142857
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.26904761904761904
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1714285714285714
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.0897142857142857
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.6857142857142857
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8071428571428572
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8571428571428571
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.8971428571428571
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.79087795854059
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7568854875283447
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7608935817550728
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.66
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.7757142857142857
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8128571428571428
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.8671428571428571
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.66
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.25857142857142856
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.16257142857142853
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.0867142857142857
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.66
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.7757142857142857
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8128571428571428
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.8671428571428571
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.7616045249840884
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7281247165532877
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7330922421864847
            name: Cosine Map@100

BGE base Financial Matryoshka

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: BAAI/bge-base-en-v1.5
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 768 tokens
  • Similarity Function: Cosine Similarity
  • Language: en
  • License: apache-2.0

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("cristuf/bge-base-financial-matryoshka")
# Run inference
sentences = [
    'During fiscal 2022, GameStop Corp increased its valuation allowances by approximately $70.2 million in various jurisdictions.',
    "How much did GameStop Corp's valuation allowances increase during fiscal 2022?",
    'How does Gilead ensure an inclusive and diverse workforce?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Metric Value
cosine_accuracy@1 0.7186
cosine_accuracy@3 0.83
cosine_accuracy@5 0.8714
cosine_accuracy@10 0.91
cosine_precision@1 0.7186
cosine_precision@3 0.2767
cosine_precision@5 0.1743
cosine_precision@10 0.091
cosine_recall@1 0.7186
cosine_recall@3 0.83
cosine_recall@5 0.8714
cosine_recall@10 0.91
cosine_ndcg@10 0.8138
cosine_mrr@10 0.783
cosine_map@100 0.7867

Information Retrieval

Metric Value
cosine_accuracy@1 0.7114
cosine_accuracy@3 0.8314
cosine_accuracy@5 0.8729
cosine_accuracy@10 0.9143
cosine_precision@1 0.7114
cosine_precision@3 0.2771
cosine_precision@5 0.1746
cosine_precision@10 0.0914
cosine_recall@1 0.7114
cosine_recall@3 0.8314
cosine_recall@5 0.8729
cosine_recall@10 0.9143
cosine_ndcg@10 0.8124
cosine_mrr@10 0.7799
cosine_map@100 0.7832

Information Retrieval

Metric Value
cosine_accuracy@1 0.7
cosine_accuracy@3 0.8286
cosine_accuracy@5 0.8614
cosine_accuracy@10 0.9043
cosine_precision@1 0.7
cosine_precision@3 0.2762
cosine_precision@5 0.1723
cosine_precision@10 0.0904
cosine_recall@1 0.7
cosine_recall@3 0.8286
cosine_recall@5 0.8614
cosine_recall@10 0.9043
cosine_ndcg@10 0.8043
cosine_mrr@10 0.7722
cosine_map@100 0.7759

Information Retrieval

Metric Value
cosine_accuracy@1 0.6857
cosine_accuracy@3 0.8071
cosine_accuracy@5 0.8571
cosine_accuracy@10 0.8971
cosine_precision@1 0.6857
cosine_precision@3 0.269
cosine_precision@5 0.1714
cosine_precision@10 0.0897
cosine_recall@1 0.6857
cosine_recall@3 0.8071
cosine_recall@5 0.8571
cosine_recall@10 0.8971
cosine_ndcg@10 0.7909
cosine_mrr@10 0.7569
cosine_map@100 0.7609

Information Retrieval

Metric Value
cosine_accuracy@1 0.66
cosine_accuracy@3 0.7757
cosine_accuracy@5 0.8129
cosine_accuracy@10 0.8671
cosine_precision@1 0.66
cosine_precision@3 0.2586
cosine_precision@5 0.1626
cosine_precision@10 0.0867
cosine_recall@1 0.66
cosine_recall@3 0.7757
cosine_recall@5 0.8129
cosine_recall@10 0.8671
cosine_ndcg@10 0.7616
cosine_mrr@10 0.7281
cosine_map@100 0.7331

Training Details

Training Dataset

Unnamed Dataset

  • Size: 6,300 training samples
  • Columns: positive and anchor
  • Approximate statistics based on the first 1000 samples:
    positive anchor
    type string string
    details
    • min: 8 tokens
    • mean: 46.36 tokens
    • max: 439 tokens
    • min: 9 tokens
    • mean: 20.41 tokens
    • max: 51 tokens
  • Samples:
    positive anchor
    Japan's revenue for the year 2023 reached 2,367.0 million. What was the revenue attributed to Japan in the year 2023?
    Our four reportable segments are: •the Data Center segment, which primarily includes server CPUs, GPUs, APUs, DPUs, FPGAs, SmartNICs, AI accelerators and Adaptive SoC products for data centers; •the Client segment, which primarily includes CPUs, APUs, and chipsets for desktop, notebook and handheld personal computers; •the Gaming segment, which primarily includes discrete GPUs, semi-custom SoC products and development services; and •the Embedded segment, which primarily includes embedded CPUs, GPUs, APUs, FPGAs, SOMs, and Adaptive SoC products. What are the different segments that AMD reports financially?
    For detailed information about the company's legal proceedings, see Note 4 to the consolidated financial statements, included under the caption 'Contingencies' in the Annual Report on Form 10-K. Where can detailed information about the company's legal proceedings be found in its financial statements?
  • Loss: MatryoshkaLoss with these parameters:
    {
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

  • eval_strategy: epoch
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 16
  • gradient_accumulation_steps: 16
  • learning_rate: 2e-05
  • num_train_epochs: 4
  • lr_scheduler_type: cosine
  • warmup_ratio: 0.1
  • bf16: True
  • tf32: True
  • load_best_model_at_end: True
  • optim: adamw_torch_fused
  • batch_sampler: no_duplicates

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: epoch
  • prediction_loss_only: True
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 16
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 16
  • eval_accumulation_steps: None
  • learning_rate: 2e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 4
  • max_steps: -1
  • lr_scheduler_type: cosine
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.1
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: True
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: True
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: True
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch_fused
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: False
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • dispatch_batches: None
  • split_batches: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: proportional

Training Logs

Epoch Step Training Loss dim_128_cosine_map@100 dim_256_cosine_map@100 dim_512_cosine_map@100 dim_64_cosine_map@100 dim_768_cosine_map@100
0.8122 10 1.5267 - - - - -
0.9746 12 - 0.7446 0.7639 0.7765 0.7039 0.7725
1.6244 20 0.6742 - - - - -
1.9492 24 - 0.7606 0.7795 0.7828 0.7297 0.7839
2.4365 30 0.4469 - - - - -
2.9239 36 - 0.7643 0.7758 0.7834 0.7332 0.7845
3.2487 40 0.3712 - - - - -
3.8985 48 - 0.7609 0.7759 0.7832 0.7331 0.7867
  • The bold row denotes the saved checkpoint.

Framework Versions

  • Python: 3.11.8
  • Sentence Transformers: 3.0.1
  • Transformers: 4.41.2
  • PyTorch: 2.3.1+cu121
  • Accelerate: 0.30.1
  • Datasets: 2.19.1
  • Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss 

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss 

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}