Every physician knows that a significant portion of their shift is consumed by routine data entry into Medical Information Systems (MIS). While these systems are designed to streamline workflows, the reality is often the opposite. Certain clinical data must be manually validated by a doctor, but much of the workload involves repetitive, boilerplate documentation that varies little from patient to patient.
A prime example is the “Patient Recommendations” section of an Electronic Health Record (EHR). These are mandatory instructions provided upon hospital discharge. In cases of similar diagnoses, comorbidities, or surgical interventions, the prescribed medications and post-operative care protocols follow highly consistent patterns. This repetitive task eats into “golden time” that should be spent on direct patient care.
The Solution: Domain-Specific LLM Fine-Tuning
Our core objective was to fine-tune a Large Language Model (LLM) on a proprietary dataset to generate relevant, case-specific clinical recommendations.
Dataset Overview:
- Domain: Cardiac pathology.
- Features: Sex, age, length of stay, and discharge summary.
- Target: Clinical recommendations.
- Language: Ukrainian.
- Size: 6,500 clinical cases (sufficient for specialized downstream tasks).
- Preprocessing: Full anonymization (removal of PII), and cleaning of redundant clinic-specific boilerplate and institutional headers.
Model Architecture and Training
For this project, we leveraged open-weights models from Google DeepMind:
- Gemma-3–4B-IT: A compact, 4-billion parameter model optimized for dialogue. It served as our flexible base, adapted to the Ukrainian language.
- MedGemma-4B-IT: This specialized variant extends the base architecture with deep medical domain expertise and multimodal capabilities.
Training Hardware & Configuration:
- Compute: NVIDIA L4 (24GB VRAM), 50GB RAM, 12 vCPUs.
- Methodology: Parameter-efficient fine-tuning via LoRA.
- Hyperparameters: Learning Rate:
2e-4 - Batch Size:
1(Per device) - Gradient Accumulation Steps:
8(Effective Batch Size = 8) - Epochs:
1.0
Results: Training was completed in approximately one hour per stage. A learning rate of 2e-4 provided stable loss convergence within a single epoch without signs of overfitting. The effective batch size of 8 ensured smooth weight updates, allowing the model to adapt effectively to the specialized medical syntax.
Deployment and Availability
The resulting model on Hugging Face: uaritm/gemmamed_cardio, is available on Hugging Face. We have provided a GGUF quantized version (2.49 GB), which allows for local inference on standard consumer-grade laptops without a dedicated GPU. When provided with a well-structured prompt, the model demonstrates strong performance in generating Ukrainian medical text.
Conclusion and Next Steps
While this model is not a replacement for clinical judgment, it serves as a powerful productivity tool, automating routine documentation.
Our roadmap includes:
- Expanding the dataset to 15,000 cases.
- Integrating English-language clinical data to improve cross-lingual accuracy and reasoning.
Expect updated releases in the near future…
