Oszkár Urbán
MPhil in Advanced Computer Science (Machine Learning), University of Cambridge. Research in AI safety, interpretability, and efficient ML systems.
About
I recently completed my MPhil in Advanced Computer Science (Machine Learning) at King's College, University of Cambridge, graduating ranked 3rd in a highly competitive cohort with First Class with Distinction, and receiving the prize for the best dissertation in the cohort. My dissertation on adaptive speculative decoding was carried out in collaboration with the Samsung AI Center Cambridge, supervised by Young D. Kwon, Stylianos I. Venieris, and Cecilia Mascolo.
My work sits at the intersection of AI safety, interpretability, and efficient ML systems, with a longer-term interest in deep learning for science. My current focus is alignment pretraining: improving pre- and midtraining methods so that alignment is built into base models, rather than relying on post-hoc solutions applied in post-training. I am pursuing this as a research intern at Mila in Yoshua Bengio's group, and as a fellow in the MARS AI Safety Fellowship — both supervised by Thomas Jiralerspong. Previously I co-founded two startups and held research and engineering roles spanning neuroscience, protein design, and computer vision.
News
- July 2026 Research at Mila (Yoshua Bengio's group) on alignment pretraining. Contact for more info.
- June 2026 Awarded the prize for the best dissertation in the MPhil Advanced Computer Science cohort at the University of Cambridge.
- June 2026 Graduated ranked 3rd in a highly competitive MPhil cohort, with First Class with Distinction.
- May 2026 MARS AI Safety Fellowship (Thomas Jiralerspong's stream), working on alignment pretraining.
- May 2026 AdaptiveSpec submitted to EMNLP 2026 (ACL ARR May cycle, under review).
Featured Research
Mila · Yoshua Bengio's Group · Ongoing
Alignment Pretraining for Robust Base Models
Ongoing research on improving pre- and midtraining methods for alignment — building alignment priors directly into base models rather than relying on post-hoc solutions applied in post-training. The aim is initialisations where alignment properties persist through later capabilities training, complementing and reducing reliance on post-training alignment techniques.
Samsung AI · EMNLP 2026 (under review)
Adaptive Speculative Decoding for Chain-of-Thought Reasoning Models
A training-free, per-step speculative decoding system that replaces the static draft-tree configuration of current decoders with an adaptive controller. It reads internal signals from the residual stream — token-level entropy, hidden-layer activations, and reasoning-dynamics features across the CoT trace — to gate speculative-tree expansion, relaxing during exploratory thinking and tightening during high-stakes answer phases. Built on EAGLE-3 within a forked SGLang codebase; achieves up to +56% throughput over EAGLE-3 at 93–102% accuracy retention.
University of Cambridge
Improving MCQ Performance via Activation Steering of Query Styles
Extracts persona vectors (polite, insulting, urgent, …) from internal LLM activations and steers along them at inference time, improving accuracy on MMLU Math, TruthfulQA, RACE, and Mind Your Tone by up to 12.5% while reducing prediction entropy and Expected Calibration Error.
ETH Zurich · SRI Lab
Adversarial Testing of Web Agents
PGD-based adversarial image attacks against VisualWebArena VLM agents (BLIP-2 / FLAN-T5), increasing the length of enforceable adversarial captions from 4 to 19 tokens via Adam optimisation and a custom gradient-friendly image preprocessor.
University of Cambridge
Multimodal Framework for Alzheimer's Disease Classification and Stable Attribution
Fuses 3D T1-weighted MRI with structured clinical text (3D ViT + ModernBERT, late fusion) and uses Sparse Autoencoders with UMAP geometric constraints to recover stable, clinically consistent feature attributions across in-distribution and out-of-distribution cohorts.
University of Cambridge
Transferability of Text LLMs for Protein Inverse Folding via Structural Tokenization
Discretises 3D protein backbones into 1D token strings (Foldseek's 3Di alphabet and C-α coordinate strings) and probes whether linguistic priors from a pretrained Llama-3-8B transfer to inverse folding. LoRA fine-tuning recovers meaningful sequence-recovery gains over a randomly initialised baseline, showing text-LLM inductive biases provide a usable foundation for protein design without architectural changes.
University of Cambridge
Hungarian Minimal Pairs for Negation-Induced Preverb Inversion
Introduces a Hungarian minimal-pair benchmark targeting preverb inversion under negation, evaluates monolingual and multilingual LMs across four lexical-complexity levels, and uses mechanistic analysis to attribute failures to tokenisation artifacts and attention collapse.