Saman Bahzad Khan

27100111@lums.edu.pk

I am a Computer Science undergraduate at LUMS, working toward a PhD application in December 2026. Driven by a deep enthusiasm for research, I am broadly curious across many areas of computer science — from systems and distributed computing to machine learning and AI alignment.

My current work sits at the intersection of LLM-based systems, preference optimization, and AI-driven automation. I collaborate with Dr. Ali Anwar at the University of Minnesota, Dr. Tianyin Xu at the University of Illinois Urbana-Champaign, Dr. Ali R. Butt at Virginia Tech, and Dr. Ahmad Faraz Khan at Google:

Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization (Findings of ACL 2026) — a weighted Direct Preference Optimization approach that reduces semantic drift during automated prompt optimization. It improved CLIP similarity by 8–12% and human-preference scores by 5–9% over standard DPO, and was preferred in 82.2% of pairwise human evaluations. Advisors: Dr. Ali Anwar and Dr. Ahmad Faraz Khan (Google).
LADs: Leveraging LLMs for AI-Driven DevOps (First Author, MLSys ‘27 submission, PDF) — a six-agent framework for configuration, deployment, monitoring, and automated repair. It reached 68% deployment success across 180 Dask, Redis, and Ray tasks and repaired 94% of runtime failures within three iterations. Advisors: Dr. Ali Anwar and Dr. Ali R. Butt (Virginia Tech).
SkillGraph: Reusable Reasoning Skills from LLM Agent Traces (First Author, MLSys ‘27 submission) — a GraphRAG-based skill graph for extracting and reusing diagnostic reasoning from agent trajectories, with a process-level critic that evaluates reasoning quality rather than final outcomes alone. Advisors: Dr. Tianyin Xu (UIUC) and Dr. Ali Anwar.
Agent Memory: Storage and Retrieval of LLM Reasoning Traces at Scale (MLSys ‘27 submission) — an analysis of 535 queries across seven models, with reasoning traces ranging from under 1 KB to over 71 KB per query, toward co-location and indexing for highly variable semantic workloads. Advisor: Dr. Ali Anwar.
Federated Skill Sharing for LLM Agents (MLSys ‘27 submission) — a federated framework in which clients share reusable skill templates and generalized reasoning patterns through a global indexed library while keeping private context and final adaptation local. Advisors: Dr. Ali R. Butt (Virginia Tech) and Dr. Ahmad Faraz Khan (Google).

Outside of research, I was a summer intern at Systems Limited, where I built an ERP workflow for procurement, project tracking, and reporting. Selected projects include the LLM Affordability Index and the CAPS Counselling Appointment System. I also review papers for IEEE and ACM conferences.

Research

ACL
Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering

Anas Mohamed, Azal Ahmad Khan, Xinran Wang, and 5 more authors

In Findings of the Association for Computational Linguistics: ACL 2026, 2026

Abs DOI Bib HTML PDF

Sem-DPO is a weighted Direct Preference Optimization approach that reduces semantic drift during automated prompt optimization and establishes a theoretical bound on drift in preference-tuned prompt generators. Across three benchmarks and three language-model backbones, it improves CLIP similarity by 8–12% and human-preference scores by 5–9% over standard DPO, and is preferred in 82.2% of pairwise human evaluations.
@inproceedings{khan2025semdpo, title = {Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering}, author = {Mohamed, Anas and Khan, Azal Ahmad and Wang, Xinran and Khan, Ahmad Faraz and Ge, Shuwen and Khan, Saman Bahzad and Ahmad, Ayaan and Anwar, Ali}, booktitle = {Findings of the Association for Computational Linguistics: ACL 2026}, year = {2026}, pages = {23656--23674}, doi = {10.18653/v1/2026.findings-acl.1184}, url = {https://aclanthology.org/2026.findings-acl.1184/}, }
MLSys
LADs: Leveraging LLMs for AI-Driven DevOps

Saman Bahzad Khan and others

In Conference on Machine Learning and Systems (MLSys), 2027

Under Submission. First author.

Abs Bib HTML PDF

LADs is a six-agent framework for configuration, deployment, monitoring, and automated repair. Across 180 Dask, Redis, and Ray tasks, it achieves 68% deployment success and repairs 94% of runtime failures within three iterations.
@inproceedings{khan2025lads, title = {LADs: Leveraging LLMs for AI-Driven DevOps}, author = {Khan, Saman Bahzad and others}, booktitle = {Conference on Machine Learning and Systems (MLSys)}, year = {2027}, note = {Under Submission. First author.}, }
MLSys
SkillGraph: Reusable Reasoning Skills from LLM Agent Traces

Saman Bahzad Khan and others

In Conference on Machine Learning and Systems (MLSys), 2027

Under Submission. First author.

Abs Bib HTML

SkillGraph is a GraphRAG-based skill graph for extracting and reusing diagnostic reasoning from agent trajectories. The project builds a database of reasoning traces for skill extraction and retrieval on new system incidents, and uses a process-level critic that evaluates reasoning quality rather than final outcomes alone.
@inproceedings{khan2026skillgraph, title = {SkillGraph: Reusable Reasoning Skills from LLM Agent Traces}, author = {Khan, Saman Bahzad and others}, booktitle = {Conference on Machine Learning and Systems (MLSys)}, year = {2027}, note = {Under Submission. First author.}, }
MLSys
Agent Memory: Storage and Retrieval of LLM Reasoning Traces at Scale

Saman Bahzad Khan and others

In Conference on Machine Learning and Systems (MLSys), 2027

Under Submission

Abs Bib HTML

An analysis of 535 queries across seven models finds reasoning traces ranging from under 1 KB to over 71 KB per query. The project develops co-location and indexing strategies for highly variable semantic reasoning workloads.
@inproceedings{khan2026agentmemory, title = {Agent Memory: Storage and Retrieval of LLM Reasoning Traces at Scale}, author = {Khan, Saman Bahzad and others}, booktitle = {Conference on Machine Learning and Systems (MLSys)}, year = {2027}, note = {Under Submission}, }
MLSys
Federated Skill Sharing for LLM Agents

Saman Bahzad Khan and others

In Conference on Machine Learning and Systems (MLSys), 2027

Under Submission

Abs Bib HTML

A federated framework in which clients share reusable skill templates and generalized reasoning patterns through a global indexed library while keeping private context and final adaptation local.
@inproceedings{khan2026federated, title = {Federated Skill Sharing for LLM Agents}, author = {Khan, Saman Bahzad and others}, booktitle = {Conference on Machine Learning and Systems (MLSys)}, year = {2027}, note = {Under Submission}, }