VG

Vidit Gupta

B.Tech IT student · Machine Learning Researcher

Mumbai, India

@vidit_gupta

0 followers

Bachelor of Technology in Information Technology at Dwarkadas J. Sanghvi College of Engineering · Graduating 2028

About

B.Tech student in Information Technology at Dwarkadas J. Sanghvi College of Engineering with research experience in transformer geometry, representation learning, and spatiotemporal forecasting. Also builds PyTorch-based ML systems and participates in student research and coding communities.

Experience

  • Research Collaborator
    University of Central Florida
    Oct 2026 – Present

    Working under the guidance of Dr. Niloofar Yousefi on a paper generalizing a continuous query-key initialization technique for Transformer attention, targeting an ICLR 2027 workshop. Designing experiments to decouple the effects of width and context length on cross-scale transfer of initial attention regimes, and to remap regimes in context-length-aware sharpness coordinates. Setting up a matched-sharpness training comparison of initialization geometries against a mimetic-initialization baseline, with multi-seed 2-layer decoders and an associative-recall probe.

  • Research Collaborator
    University of Gävle
    Aug 2026 – Present

    Collaborating with Dr. R. P. Durga Prasad Bavirisetti on pedestrian intent prediction with occlusion-aware reasoning for autonomous driving. Developing novel prediction models and implementing architectures from recent literature, spanning attention-based, vision-language, and JEPA-based approaches. Surveying work on pedestrian behavior prediction, visual occlusion, and uncertainty-aware reasoning to guide model design choices.

  • Collaborative Researcher
    Advanced Vision Labs
    Aug 2026 – Present

    Collaborating on research in representation learning, investigating the geometry and structure of learned representations in modern deep neural networks. Reproducing and experimentally analyzing recent representation-learning methods, studying the effects of normalization and feature alignment across pretrained models.

  • Research Assistant
    IIT Roorkee
    May 2026 – Present

    Conducting research on geospatial time-series forecasting for district-level drought prediction across India using a proprietary multi-year dataset. Surveyed recent literature on spatiotemporal forecasting and drought prediction, analyzing Transformer-, graph-, and attention-based architectures to identify suitable modeling strategies. Designed candidate deep learning architectures integrating spatial attention with temporal sequence modeling for nationwide drought forecasting.

Education

  • Bachelor of Technology in Information Technology
    Dwarkadas J. Sanghvi College of Engineering
    Information Technology · Sep 2024 – 2028
  • High School
    Jul 2016 – Apr 2024

Skills

6 to 12 months 6 to 12 months Most of a year behind them.
<3 months <3 months Just getting started - under three months. This is also what shows when a level has not been set.

Tools / apps / platforms

<3 months <3 months Just getting started - under three months. This is also what shows when a level has not been set.
Hugging Face

Projects

  • Torch-IsoFLOPs: Chinchilla-Style Scaling Law Framework
    PyTorch, Scaling Laws

    Developed a reusable framework for IsoFLOP scaling-law experiments under fixed compute budgets using Chinchilla-style methodology. Implemented automatic FLOP estimation, experiment scheduling, Huber regression, RANSAC-based curve fitting, and resumable experiment logging. Enabled analytical estimation of optimal parameter-data tradeoffs across compute budgets with automated visualization.

  • Representation Geometry: Hidden State Analysis
    PyTorch, Hugging Face

    Developed and published a model-agnostic PyTorch library for transformer representation analysis using activation hooks, streaming covariance estimation, and reproducible artifact generation. Implemented spectral, statistical, and subspace metrics including participation ratio, stable rank, entropy rank, linear CKA, and activation-space SVD analysis. Supported Hugging Face and native PyTorch models through automatic transformer-block discovery with modular analysis and bounded-memory statistical backends.

  • Language Model Stack: From Training to Agents
    PyTorch, LLM Systems

    Trained a 50M-parameter decoder-only transformer from scratch with custom BPE tokenization, RoPE, RMSNorm, grouped-query attention, SwiGLU Mixture-of-Experts, weight tying, and cosine learning-rate decay. Built a token-level inference runtime for Qwen3 models implementing KV cache, greedy, beam, top-k, nucleus, min-p, and locally typical sampling without relying on model.generate(). Extended the inference runtime into a tool-using agent capable of interrupting decoding, invoking external tools, preserving KV cache across tool calls, and resuming generation.

  • Attention Geometry & Training Dynamics in Transformers
    PyTorch, Mech Interp

    Instrumented a 4M-parameter GPT-style transformer with 18 geometric metrics tracking attention entropy, effective rank, covariance spectra, Q/K/V statistics, and gradient dynamics across 87 epochs. Observed divergent query/key geometry despite shared projections, spectral redistribution in Wq, and increasing pre-softmax rank while post-softmax attention remained low-rank due to softmax concentration. Developed a reproducible measurement stack generating per-epoch JSONL logs and automated visualization for transformer geometry analysis.

Extracurricular activities

  • DJ InIT.ai

    Research Mentee → Vice Chairperson (Research); lead a student research club focused on machine learning, organizing technical sessions, paper reading groups, and research discussions. Coordinate research collaborations and mentor members in exploring and implementing machine learning research.

  • DJS Codestars

    Technical Member; authored competitive programming problems for weekly contests engaging 150+ participants and contributed to organizing CodeUncode, collaborating with teams on contest preparation and website development.

  • GDG DJSCE

    ML Research Member; first author of a research paper under review on redundancy reduction in transformer hidden-state representations and participant in research paper reading sessions and technical discussions on modern machine learning.

Publications

  • A Continuous Space of Attention Projection Geometries at Initialization
  • Query Expansion and Key Specialization in Transformer Attention Geometry
  • Residual-Stream Geometry in Transformer Language Model

Explore the Broxer community

Online · instant AI help
Broxer