AI / Data Solution Architect

Role Overview

We are seeking an experienced AI / Data Solution Architect to design and implement scalable, high-performance, and secure data and AI solutions. This includes architecting full lifecycle data pipelines, AI models, and infrastructure automation to enable advanced analytics and machine learning capabilities.

Primary Accountabilities

• Architect comprehensive data and AI solutions focusing on scalability, performance, and security.
• Automate infrastructure provisioning using Infrastructure as Code (IaC) tools such as Terraform.
• Design and build data pipelines on cloud platforms like AWS, Azure, and Databricks to manage data ingestion, transformation, and storage.
• Develop AI applications involving Computer Vision, Natural Language Processing (NLP), and Generative AI leveraging modern cloud services.
• Incorporate best practices like Medallion Architecture, Change Data Capture (CDC), Data Lake, and Data Mesh to build batch and real-time processing pipelines.
• Create large-scale ETL/ELT pipelines utilizing platforms such as AWS Glue, Azure Data Factory, and Databricks.
• Enhance data models for optimal querying and facilitate easy data access through APIs and data products.
• Implement Data Governance using tools like AWS DataZone, Unity Catalog, and Azure Purview covering data cataloging, business glossary maintenance, lineage tracking, and role-based access control.
• Establish policies ensuring security, privacy, compliance (including GDPR, HIPAA), and controlled data accessibility.
• Design data warehouses using dimension and fact modeling with performance tuning on Snowflake, Databricks, and AWS Redshift.
• Architect and deploy MLOps pipelines addressing model training, validation, inference, monitoring, and feedback mechanisms.
• Build scalable AI solutions managing model lifecycle and exposure via APIs; maintain comprehensive service catalogs for ML models.
• Apply frameworks such as TensorFlow, XGBoost, Hugging Face, and LangChain in AI development.
• Implement Generative AI and Large Language Models (LLMs) including custom model development and fine-tuning.
• Develop AI solutions for Computer Vision tasks (image classification, object detection, OCR, segmentation) and NLP tasks (text generation, summarization, sentiment analysis, named entity recognition).
• Utilize tools like LangChain, HuggingFace, OpenAI API, Bedrock, and Azure OpenAI for AI model development.
• Set up CI/CD pipelines for data, ML, and AI models; employ infrastructure automation tools such as Terraform, CloudFormation, and Azure Bicep.
• Implement secure and scalable deployment strategies with continuous monitoring and model drift detection.
• Work with Informatica Intelligent Cloud Services (IICS) for ETL/ELT pipeline management and Snowflake for high-performance data warehousing.
• Possess expertise in Generative AI including LLMs, HuggingFace libraries, OpenAI APIs, and Azure OpenAI with focus on model fine-tuning, embeddings, and prompt engineering.

Candidate Requirements

• Minimum of 12 years experience in Data Architecture, Data Engineering, and AI solution development.
• Proficient in Infrastructure as Code (IaC) practices with tools like Terraform or equivalents.
• Strong background in constructing data pipelines across AWS, Azure, and Databricks cloud environments.
• Hands-on experience in MLOps processes encompassing model deployment and monitoring.
• Skilled in developing Computer Vision and NLP based AI applications.
• In-depth knowledge of Data Governance frameworks such as AWS DataZone, Unity Catalog, or Azure Purview.
• Experienced in Data Warehouse modeling with dimension and fact schema design.
• Consulting experience with Generative AI technologies, Large Language Models (LLMs), and model fine-tuning techniques.