Vaishnav Negi

Building custom LLM-based evaluation framework for benchmarking BMW's In-car Personal Assistant (IPA) using proprietary models (GPT-3.5, GPT-4o, GPT-5) and open-source models (DeepSeek) to assess multi-turn conversation quality. Developed automated pipeline for generating extensive conversation datasets with configurable participant roles (simulated users, evaluators), persona settings, and task descriptions for systematic model benchmarking.

Developed end-to-end AWS anomaly detection pipeline achieving 77% F1 score using Isolation Forest with frequency encoding. Collaborated with stakeholders to define 13 hard-coded business rules for anomaly detection, reducing rule-based anomalies from 49% to 5% of the dataset through iterative validation sessions. Designed AutoEncoder-based feature engineering pipeline with custom embedding layers, batch normalization, and dropout regularization.

Developed YOLO-based computer vision models for automated industrial inspection: digital gauge reading (95% accuracy), analog gauge reading (85% accuracy, ±5% error margin), valve state detection (90% accuracy), and liquid level measurement (95% accuracy). Built TUV safety stamp recognition system for fire extinguisher inspection tracking, achieving 75% accuracy using custom CV pipelines combining YOLO detection/segmentation with traditional image processing techniques.

Maintained health and inventory of 50+ high-performance compute clusters (Linux, Windows) for deep learning research, utilizing bash scripting, Python, and Ansible for system administration and resolving errors through log analysis. Adapted open-source VTiger CRM software for inter-department use, managing version control and deployments via GitLab.