CV

Contact Information

Name Saman Bahzad Khan
Professional Title CS Undergraduate & Researcher
Email 27100111@lums.edu.pk
Website https://samanbahzadkhan.github.io/

Professional Summary

Computer Science undergraduate at LUMS, working on LLM-based systems, preference optimization, and AI-driven automation. Working toward a PhD application in December 2026.

Experience

  • 2024 - 2025

    Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization
    Advisors: Dr. Ali Anwar, University of Minnesota · Dr. Ahmad Faraz Khan, Google
    Published at ACL 2026
    • Developed a weighted Direct Preference Optimization approach that reduces semantic drift during automated prompt optimization and established a theoretical bound on drift in preference-tuned prompt generators.
    • Improved CLIP similarity by 8–12% and human-preference scores by 5–9% over standard DPO across three benchmarks and three language-model backbones; preferred in 82.2% of pairwise human evaluations.
  • 2025 - 2026

    LADs: Leveraging LLMs for AI-Driven DevOps
    Advisors: Dr. Ali Anwar, University of Minnesota · Dr. Ali R. Butt, Virginia Tech
    First author. Submission at MLSys 2027
    • Co-developed a six-agent framework for configuration, deployment, monitoring, and automated repair; achieved 68% deployment success across 180 Dask, Redis, and Ray tasks and repaired 94% of runtime failures within three iterations.
    • Adapted two existing systems as baselines and substantially redesigned the evaluation and technical presentation to better establish LADs’ gains over prior approaches.
  • 2026 - present

    SkillGraph: Reusable Reasoning Skills from LLM Agent Traces
    Advisors: Dr. Tianyin Xu, University of Illinois Urbana-Champaign · Dr. Ali Anwar, University of Minnesota
    First author. Submission at MLSys 2027
    • Leading development of a GraphRAG-based skill graph for extracting and reusing diagnostic reasoning from agent trajectories; built a database of reasoning traces for skill extraction and retrieval on new system incidents.
    • Identified shortcut behavior that corrupts learned skills and designed a process-level critic that evaluates reasoning quality rather than final outcomes alone.
  • 2026 - present

    Agent Memory: Storage and Retrieval of LLM Reasoning Traces at Scale
    Advisor: Dr. Ali Anwar, University of Minnesota
    Submission at MLSys 2027
    • Analyzed 535 queries across seven models, finding reasoning traces from under 1 KB to over 71 KB per query; developing co-location and indexing strategies for highly variable semantic reasoning workloads.
  • 2026 - present

    Federated Skill Sharing for LLM Agents
    Advisors: Dr. Ali R. Butt, Virginia Tech · Dr. Ahmad Faraz Khan, Google
    Submission at MLSys 2027
    • Developing a federated framework where clients share reusable skill templates and generalized reasoning patterns through a global indexed library while keeping private context and final adaptation local.
  • 2025 - 2025

    Lahore, Pakistan

    Summer Intern
    Systems Limited
    Jun 2025 – Jul 2025
    • Built an ERP workflow automating procurement, project tracking, and reporting for a construction company using Microsoft Dynamics.

Education

  • 2023 - present

    Lahore, Pakistan

    B.S.
    Lahore University of Management Sciences
    Computer Science
    • Relevant Coursework: Data Structures & Algorithms, Operating Systems, AI/ML, Generative AI, Advanced GenAI, AI on edge devices, Network Centric Computing, Topics in LLMs

Publications

  • 2026
    Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering
    Findings of ACL 2026

    A weighted Direct Preference Optimization approach that reduces semantic drift during automated prompt optimization, with 8–12% higher CLIP similarity and 5–9% higher human-preference scores over standard DPO.

  • 2026
    LADs: Leveraging LLMs for AI-Driven DevOps
    MLSys 2027 (Under Submission)

    A six-agent framework for configuration, deployment, monitoring, and automated repair. First author. 68% deployment success across 180 Dask, Redis, and Ray tasks, and 94% of runtime failures repaired within three iterations.

  • 2026
    SkillGraph: Reusable Reasoning Skills from LLM Agent Traces
    MLSys 2027 (Under Submission)

    A GraphRAG-based skill graph for extracting and reusing diagnostic reasoning from agent trajectories, with a process-level critic for reasoning quality. First author.

  • 2026
    Agent Memory: Storage and Retrieval of LLM Reasoning Traces at Scale
    MLSys 2027 (Under Submission)

    Analysis of 535 queries across seven models, with reasoning traces from under 1 KB to over 71 KB per query, toward co-location and indexing for variable semantic workloads.

  • 2026
    Federated Skill Sharing for LLM Agents
    MLSys 2027 (Under Submission)

    A federated framework for sharing reusable skill templates and generalized reasoning patterns while keeping private context and final adaptation local.

Projects

  • LLM Affordability Index (LLAI)

    Developed a scenario-aware affordability framework incorporating model performance, deployment cost, and regional accessibility; evaluated across five LLMs.

  • CAPS Counselling Appointment System

    Built a Firebase-backed Android scheduling platform with Google Meet integration, AI assistance, and double-booking prevention.

Skills

Programming Languages: Python, Java, C++, C, TypeScript, SQL
Tools and Libraries: PyTorch, TensorFlow, Hugging Face, LangChain, Ollama, scikit-learn, Pandas, Git, Docker, Dask, Linux

Additional Experiences

  • Organized Girls’ Coding Camp, Women in Computing Hackathon, Grad App Wars, and PsiFi through LUMS Women in Computing and SPADES.
  • Reviewed papers for IEEE and ACM conferences; conducted Lahore air-quality analysis using PM2.5, GIS mapping, and Python.