Financial PhraseBank — DeepSeek-R1-Distill-Qwen 3

This benchmark was generated by LMTask from trainconf/fpb-dsr1qwen3.yml on 2026-08-08T16:03:28.602933-05:00.

Results

Primary metric: f1 (macro) = 0.8147

Metric

Average

Value

accuracy

—

0.7969

precision

micro

0.7969

recall

micro

0.7969

f1

micro

0.7969

precision

macro

0.8068

recall

macro

0.8687

f1

macro

0.8147

precision

weighted

0.8580

recall

weighted

0.7969

f1

weighted

0.8023

Per-class results

Class

Precision

Recall

F1

Support

negative

0.8649

0.9412

0.9014

34

neutral

0.9649

0.6962

0.8088

158

positive

0.5905

0.9688

0.7337

64

Invalid/unrecognized predictions: 0

Dataset

Split

Examples

Class counts

train

1752

positive=441, neutral=1076, negative=235

validation

256

neutral=157, negative=34, positive=65

test

256

neutral=158, negative=34, positive=64

Training

Field

Value

Global steps

110

Training loss

2.509641246362166

End-to-end training time

324.9 s

PEFT output

data/fpb/dsr1qwen3/model/peft

Adapter size

1291899160 bytes

Trainer diagnostics

Metric

Value

train_runtime

312.8679

train_samples_per_second

5.6

train_steps_per_second

0.352

total_flos

7265346283044864.0

train_loss

2.509641246362166

Held-out testing

Field

Value

Examples

256

Runtime

3318.5 s

Predictions

data/fpb/dsr1qwen3/benchmark/fpb_dsr1qwen3.jsonl

Environment

Field

Value

LMTask revision

51cf677d65e9b7c9deffa2878c505dba82b59d89

Git describe

v0.0.1-105-g51cf677

Dirty working tree

True

Python

3.13.13

OS

Linux-4.18.0-553.124.1.el8_10.x86_64-x86_64-with-glibc2.28

Kernel

4.18.0-553.124.1.el8_10.x86_64

CUDA_VISIBLE_DEVICES

4,5,6,7

PyTorch CUDA runtime

12.8

NVIDIA driver

580.142

GPUs visible to the benchmark process

Device

GPU

Memory

Peak allocated

Peak reserved

0

NVIDIA RTX A6000

50897289216

—

—

1

NVIDIA RTX A6000

50897289216

—

—

2

NVIDIA RTX A6000

50897289216

—

—

3

NVIDIA RTX A6000

50897289216

—

—

Package versions

Package

Version

scikit-learn

1.9.0

torch

2.10.0

transformers

5.5.4

trl

1.0.0

peft

0.18.1

accelerate

1.13.0

datasets

4.8.5

pandas

2.3.3

pyarrow

24.0.0

huggingface-hub

1.16.1

Reproducibility

The JSON record fpb_dsr1qwen3.json is the machine-readable source of truth for this report. The held-out predictions are stored in fpb_dsr1qwen3.jsonl.