Financial PhraseBank — Qwen 3

This benchmark was generated by LMTask from trainconf/fpb-qwen3.yml on 2026-08-08T15:02:34.921283-05:00.

Results

Primary metric: f1 (macro) = 0.3858

Metric

Average

Value

accuracy

—

0.2227

precision

micro

0.9500

recall

micro

0.2227

f1

micro

0.3608

precision

macro

0.9706

recall

macro

0.2625

f1

macro

0.3858

precision

weighted

0.9779

recall

weighted

0.2227

f1

weighted

0.3367

Per-class results

Class

Precision

Recall

F1

Support

negative

1.0000

0.1765

0.3000

34

neutral

1.0000

0.1266

0.2247

158

positive

0.9118

0.4844

0.6327

64

Invalid/unrecognized predictions: 196

Dataset

Split

Examples

Class counts

train

1752

positive=441, neutral=1076, negative=235

validation

256

neutral=157, negative=34, positive=65

test

256

neutral=158, negative=34, positive=64

Training

Field

Value

Global steps

110

Training loss

1.875404831496152

End-to-end training time

469.0 s

PEFT output

data/fpb/qwen3/model/peft

Adapter size

1476660944 bytes

Trainer diagnostics

Metric

Value

train_runtime

437.4713

train_samples_per_second

4.005

train_steps_per_second

0.251

total_flos

8467400973090816.0

train_loss

1.875404831496152

Held-out testing

Field

Value

Examples

256

Runtime

5379.4 s

Predictions

data/fpb/qwen3/benchmark/fpb_qwen3.jsonl

Environment

Field

Value

LMTask revision

51cf677d65e9b7c9deffa2878c505dba82b59d89

Git describe

v0.0.1-105-g51cf677

Dirty working tree

True

Python

3.13.13

OS

Linux-4.18.0-553.124.1.el8_10.x86_64-x86_64-with-glibc2.28

Kernel

4.18.0-553.124.1.el8_10.x86_64

CUDA_VISIBLE_DEVICES

4,5,6,7

PyTorch CUDA runtime

12.8

NVIDIA driver

580.142

GPUs visible to the benchmark process

Device

GPU

Memory

Peak allocated

Peak reserved

0

NVIDIA RTX A6000

50897289216

—

—

1

NVIDIA RTX A6000

50897289216

—

—

2

NVIDIA RTX A6000

50897289216

—

—

3

NVIDIA RTX A6000

50897289216

—

—

Package versions

Package

Version

scikit-learn

1.9.0

torch

2.10.0

transformers

5.5.4

trl

1.0.0

peft

0.18.1

accelerate

1.13.0

datasets

4.8.5

pandas

2.3.3

pyarrow

24.0.0

huggingface-hub

1.16.1

Reproducibility

The JSON record fpb_qwen3.json is the machine-readable source of truth for this report. The held-out predictions are stored in fpb_qwen3.jsonl.