Preference-tuned small medical LLMs
DPO and KTO pipelines for smaller models on curated clinical documents, traced end to end with Langfuse.
- DLDuy Le(Co-founder · CEO)
- MTMinh Tri Vo(Co-founder · CTO)
- NCNguyen Chi Nguyen(Co-founder · COO)
Why smaller, preference-tuned models
Domain-specific document and clinical workflows often benefit from smaller models that can be self-hosted (such as with vLLM) with predictable latency and controlled operating cost, while still matching strict domain behavior.
This ongoing line of work focuses on preference-tuning smaller LLMs over curated clinical documents rather than relying solely on prompt engineering over large general-purpose APIs.
DPO and KTO pipelines
We are building and evaluating both Direct Preference Optimization (DPO) and Kahneman-Tversky Optimization (KTO) training pipelines on curated clinical documents.
Running both formulations inside one reproducible pipeline lets us compare pairwise preference datasets against binary feedback signals on the same frozen evaluation sets.
End-to-end observability with Langfuse
Building on prior production experience shipping a medical RAG platform with LLMOps tracing, every stage of the pipeline is traced end to end in Langfuse.
That matches how we run all model work at Untern: start from real data, freeze a test set before training, and gate every iteration against measurable outputs.
- Curated clinical document datasets paired with frozen evaluation gates.
- DPO and KTO alignment pipelines targeting smaller, deployable model sizes.
- Full end-to-end run and evaluation tracing in Langfuse.