Financial PhraseBank — Llama 3

This benchmark was generated by LMTask from trainconf/fpb-llama3.yml on 2026-08-08T13:24:56.393661-05:00.

Results

Primary metric: f1 (macro) = 0.8688

Metric

Average

Value

accuracy

—

0.8867

precision

micro

0.8867

recall

micro

0.8867

f1

micro

0.8867

precision

macro

0.9211

recall

macro

0.8338

f1

macro

0.8688

precision

weighted

0.8936

recall

weighted

0.8867

f1

weighted

0.8830

Per-class results

Class

Precision

Recall

F1

Support

negative

0.9655

0.8235

0.8889

34

neutral

0.8603

0.9747

0.9139

158

positive

0.9375

0.7031

0.8036

64

Invalid/unrecognized predictions: 0

Dataset

Split

Examples

Class counts

train

1752

positive=441, neutral=1076, negative=235

validation

256

neutral=157, negative=34, positive=65

test

256

neutral=158, negative=34, positive=64

Training

Field

Value

Global steps

110

Training loss

1.273833557692441

End-to-end training time

368.0 s

PEFT output

data/fpb/llama3/model/peft

Adapter size

1410003096 bytes

Trainer diagnostics

Metric

Value

train_runtime

353.3802

train_samples_per_second

4.958

train_steps_per_second

0.311

total_flos

9859320980029440.0

train_loss

1.273833557692441

Held-out testing

Field

Value

Examples

256

Runtime

35.6 s

Predictions

data/fpb/llama3/benchmark/fpb_llama3.jsonl

Environment

Field

Value

LMTask revision

51cf677d65e9b7c9deffa2878c505dba82b59d89

Git describe

v0.0.1-105-g51cf677

Dirty working tree

True

Python

3.13.13

OS

Linux-4.18.0-553.124.1.el8_10.x86_64-x86_64-with-glibc2.28

Kernel

4.18.0-553.124.1.el8_10.x86_64

CUDA_VISIBLE_DEVICES

4,5,6,7

PyTorch CUDA runtime

12.8

NVIDIA driver

580.142

GPUs visible to the benchmark process

Device

GPU

Memory

Peak allocated

Peak reserved

0

NVIDIA RTX A6000

50897289216

—

—

1

NVIDIA RTX A6000

50897289216

—

—

2

NVIDIA RTX A6000

50897289216

—

—

3

NVIDIA RTX A6000

50897289216

—

—

Package versions

Package

Version

scikit-learn

1.9.0

torch

2.10.0

transformers

5.5.4

trl

1.0.0

peft

0.18.1

accelerate

1.13.0

datasets

4.8.5

pandas

2.3.3

pyarrow

24.0.0

huggingface-hub

1.16.1

Reproducibility

The JSON record fpb_llama3.json is the machine-readable source of truth for this report. The held-out predictions are stored in fpb_llama3.jsonl.