Inference

Inference uses the same named task contract that was used to prepare and train the specialized model. LMTask keeps request formatting, model loading, generation, response handling, and optional caching behind that task abstraction.

For introductory command-line and Python examples, see the README.

Inference contract

Instruction tasks use a dedicated inference template rather than the training template:

lmtask_task_dataset:
  inference_template: |-
    Classify the financial sentiment of the sentence.
    Only output one label as 'negative', 'neutral', or 'positive'.
    ### Financial sentence:{{ request.instruction }}
    ### Sentiment:

The inference template expresses the same task and output contract as training, but renders request fields rather than expected output fields.

Request and response handling

Applications invoke a configured task through its request/response API. The task is responsible for applying the configured prompt contract and returning the task response in the configured form.

Depending on task configuration, LMTask can return raw text or machine-readable structured output. Structured-response handling can recover usable data from partial JSON when generation reaches a token limit.

Held-out inference

Held-out testing intentionally uses the task’s normal inference path. The shared test configuration uses:

lmtask_dataset_test_source:
  class_name: zensols.lmtask.dataset.LoadedTaskDatasetFactory
  task: 'instance: lmtask_task_dataset'
  messages_field: null

Leaving messages_field unset avoids pre-applying the tokenizer chat template to the test dataset. Formatting is instead applied for each inference request, matching application behavior more closely.

Test output preserves source/gold fields and adds the configured prediction field. Raw model output can also be retained when configured for error analysis.

This distinction is important:

• validation evaluates the training objective during specialization;

• held-out testing runs the specialized task through its inference path; and

• benchmarking scores those held-out predictions with task-specific metrics.

Application and agent use

LMTask exposes a trained capability as a normal Python task. An application or agent framework can call that task without taking ownership of model-specific prompt construction, adapter loading, response cleanup, or task semantics.

LMTask therefore supplies the specialized capability; an orchestration layer decides when and how to invoke it.

See Configuration for the task contract and Training for how the adapter is specialized.