CV
Contact Information
| Name | Saman Bahzad Khan |
| Professional Title | CS Undergraduate & Researcher |
| 27100111@lums.edu.pk | |
| Website | https://samanbahzadkhan.github.io/ |
Professional Summary
Computer Science undergraduate at LUMS, working on LLM-based systems, preference optimization, and AI-driven automation. Working toward a PhD application in December 2026.
Experience
-
2024 - 2025 Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization
Advisors: Dr. Ali Anwar, University of Minnesota · Dr. Ahmad Faraz Khan, Google
Published at ACL 2026
- Developed a weighted Direct Preference Optimization approach that reduces semantic drift during automated prompt optimization and established a theoretical bound on drift in preference-tuned prompt generators.
- Improved CLIP similarity by 8–12% and human-preference scores by 5–9% over standard DPO across three benchmarks and three language-model backbones; preferred in 82.2% of pairwise human evaluations.
-
2025 - 2026 LADs: Leveraging LLMs for AI-Driven DevOps
Advisors: Dr. Ali Anwar, University of Minnesota · Dr. Ali R. Butt, Virginia Tech
First author. Submission at MLSys 2027
- Co-developed a six-agent framework for configuration, deployment, monitoring, and automated repair; achieved 68% deployment success across 180 Dask, Redis, and Ray tasks and repaired 94% of runtime failures within three iterations.
- Adapted two existing systems as baselines and substantially redesigned the evaluation and technical presentation to better establish LADs’ gains over prior approaches.
-
2026 - present SkillGraph: Reusable Reasoning Skills from LLM Agent Traces
Advisors: Dr. Tianyin Xu, University of Illinois Urbana-Champaign · Dr. Ali Anwar, University of Minnesota
First author. Submission at MLSys 2027
- Leading development of a GraphRAG-based skill graph for extracting and reusing diagnostic reasoning from agent trajectories; built a database of reasoning traces for skill extraction and retrieval on new system incidents.
- Identified shortcut behavior that corrupts learned skills and designed a process-level critic that evaluates reasoning quality rather than final outcomes alone.
-
2026 - present Agent Memory: Storage and Retrieval of LLM Reasoning Traces at Scale
Advisor: Dr. Ali Anwar, University of Minnesota
Submission at MLSys 2027
- Analyzed 535 queries across seven models, finding reasoning traces from under 1 KB to over 71 KB per query; developing co-location and indexing strategies for highly variable semantic reasoning workloads.
-
2026 - present Federated Skill Sharing for LLM Agents
Advisors: Dr. Ali R. Butt, Virginia Tech · Dr. Ahmad Faraz Khan, Google
Submission at MLSys 2027
- Developing a federated framework where clients share reusable skill templates and generalized reasoning patterns through a global indexed library while keeping private context and final adaptation local.
-
2025 - 2025 Lahore, Pakistan
Summer Intern
Systems Limited
Jun 2025 – Jul 2025
- Built an ERP workflow automating procurement, project tracking, and reporting for a construction company using Microsoft Dynamics.
Education
-
2023 - present Lahore, Pakistan
B.S.
Lahore University of Management Sciences
Computer Science
- Relevant Coursework: Data Structures & Algorithms, Operating Systems, AI/ML, Generative AI, Advanced GenAI, AI on edge devices, Network Centric Computing, Topics in LLMs
Publications
-
2026 Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering
Findings of ACL 2026
A weighted Direct Preference Optimization approach that reduces semantic drift during automated prompt optimization, with 8–12% higher CLIP similarity and 5–9% higher human-preference scores over standard DPO.
-
2026 LADs: Leveraging LLMs for AI-Driven DevOps
MLSys 2027 (Under Submission)
A six-agent framework for configuration, deployment, monitoring, and automated repair. First author. 68% deployment success across 180 Dask, Redis, and Ray tasks, and 94% of runtime failures repaired within three iterations.
-
2026 SkillGraph: Reusable Reasoning Skills from LLM Agent Traces
MLSys 2027 (Under Submission)
A GraphRAG-based skill graph for extracting and reusing diagnostic reasoning from agent trajectories, with a process-level critic for reasoning quality. First author.
-
2026 Agent Memory: Storage and Retrieval of LLM Reasoning Traces at Scale
MLSys 2027 (Under Submission)
Analysis of 535 queries across seven models, with reasoning traces from under 1 KB to over 71 KB per query, toward co-location and indexing for variable semantic workloads.
-
2026 Federated Skill Sharing for LLM Agents
MLSys 2027 (Under Submission)
A federated framework for sharing reusable skill templates and generalized reasoning patterns while keeping private context and final adaptation local.
Projects
-
LLM Affordability Index (LLAI)
Developed a scenario-aware affordability framework incorporating model performance, deployment cost, and regional accessibility; evaluated across five LLMs.
-
CAPS Counselling Appointment System
Built a Firebase-backed Android scheduling platform with Google Meet integration, AI assistance, and double-booking prevention.
Skills
Additional Experiences
- Organized Girls’ Coding Camp, Women in Computing Hackathon, Grad App Wars, and PsiFi through LUMS Women in Computing and SPADES.
- Reviewed papers for IEEE and ACM conferences; conducted Lahore air-quality analysis using PM2.5, GIS mapping, and Python.