Lead Data Engineer

About Vista Equity Partners

Vista Equity Partners is a premier global investment firm dedicated solely to enterprise software, data, and technology-driven businesses. With assets exceeding $100 billion and a portfolio comprising over 90 software product companies worldwide, Vista enhances growth by leveraging operational excellence, shared expertise, and enduring partnerships. Its growing Indian presence includes more than 45 portfolio companies and over 17,000 professionals across technology, product, customer success, and operations, establishing the region as a vital hub of innovation within the Vista ecosystem.

Through the Agentic AI Factory, Vista integrates Generative AI capabilities across its portfolio globally, enabling companies to embed intelligent and responsible AI into products, operations, and decision-making processes. This initiative is supported by organization-wide learning programs, leadership workshops, and AI hackathons, aimed at fostering innovation, building AI fluency, and accelerating practical adoption.

About Amtech

Amtech is a prominent provider of enterprise software solutions serving the packaging, printing, and manufacturing sectors. Its solutions integrate order management, production planning, scheduling, inventory management, and business analytics to enhance customer efficiency, reduce operational expenses, and improve performance. Backed by Vista’s investment and strategic guidance, Amtech combines the nimbleness of a growing tech company with the robustness, scale, and career prospects of a global software ecosystem.

Role Overview

Amtech is searching for a Lead Data Engineer to spearhead the architecture, scalability, and technical vision of the Amtech Cloud Data Platform (ACDP) including data warehouses, pipelines, tenant data stores, and feature stores that underpin analytics, personalization, and AI agent strategies.

This role blends practical engineering with leadership responsibilities, involving standard-setting, mentoring, and collaboration with architecture and product teams to guide the platform’s roadmap. The position is critical to constructing the data infrastructure supporting Amtech’s AI Agents initiative, encompassing tenant registries, data warehouses, analytics, monitoring, and event pipelines feeding the Amtech Agent Hub and wider AI Cloud.

The Lead Data Engineer will implement a hybrid AI strategy, ensuring data flows securely across cloud acceleration, hybrid AI zones combining cloud and on-premises data, and a deterministic on-premises AI control layer for sensitive and critical logic, maintaining organizational control over data privacy and risk.

Key Responsibilities

• Take ownership of the comprehensive architecture for ACDP including data warehouses, tenant data stores, lakes, and feature stores supporting analytics, machine learning, and AI agent use cases.
• Design and implement tenant registry and tenant-scoped data warehouse layers to handle multi-tenant AI workloads.
• Develop and manage the Data Agent layer enabling customer-controlled on-premises data to integrate with cloud-based LLM reasoning, incorporating explainable staged data outputs and human review mechanisms.
• Establish and enforce data classification and routing policies to secure customer IP while directing data flows to appropriate cloud, hybrid, or on-premises AI processing tiers.
• Set and uphold engineering standards for ETL/ELT design, orchestration, and data quality.
• Lead code reviews, mentor data engineering teams, and serve as a technical escalation resource.
• Promote pipeline automation and dependability, managing SLAs, observability tools, and incident response procedures.
• Maintain data security protocols, access controls, lineage tracking, and compliance across all platform tiers.
• Collaborate with data scientists and analysts to ensure platform data readiness for machine learning and predictive analytics.
• Build and sustain data and event pipelines integrating ERP systems (like EnCore) with the EnCore Agent Hub, supporting secure data flows and event triggering.
• Drive GenAI and LLM data enablement, including embedding strategies, vector database design, and retrieval-augmented generation (RAG) pipelines that support AI Cloud agent orchestration.
• Represent data engineering in cross-functional architecture boards and participate in platform roadmap planning.
• Optimize data pipelines for low-latency access by instituting standards for partitioning, caching, and indexing.

Qualifications

• Bachelor’s degree in Computer Science, Software Engineering, or related field.
• At least 10 years in data engineering or software development, with a minimum of 3 years in technical leadership or architectural roles.
• Expertise working with data warehousing solutions such as Snowflake, Redshift, or BigQuery.
• Proficiency in Python, Java, or Scala programming languages.
• Experience using orchestration tools such as Apache Airflow, dbt, or AWS Glue.
• Strong experience with large-scale cloud platforms including AWS, Azure, or GCP, along with knowledge of on-premises or private deployment models.
• Familiarity with big data processing frameworks like Spark or Flink and streaming technologies such as Kafka or Kinesis.
• Advanced skills in SQL and NoSQL databases; knowledge of time-series or graph databases is advantageous.
• Experience enforcing data validation, governance standards, and observability frameworks like Great Expectations, Prometheus, and Grafana.
• Track record mentoring engineers and leading technical teams.
• Experience designing data architectures with multi-tenant capabilities is highly valuable.
• Comprehension of data sensitivity classification and hybrid cloud/on-premises data routing for AI workloads is strongly preferred.

Preferred Qualifications

• Advanced certifications in Data Engineering, Cloud Architecture, or Industrial Data Systems.
• Experience working with ERP, MES, or manufacturing industry data systems.
• Knowledge of IoT and OT data pipelines including machine telemetry and SCADA signal processing.
• Hands-on experience with Generative AI tools like LangChain, vector databases, and RAG pipelines in production environments.
• Familiarity with agent orchestration platforms, multi-channel processing (MCP) tool routing, or event-driven agent frameworks.
• Expertise in managing hybrid AI environments that blend cloud acceleration with private on-premises control layers.