Principal Engineer

Key Responsibilities:

• Infrastructure Management: Design, deploy, and manage AWS cloud infrastructure for data storage, processing, and analytics, ensuring high availability and scalability while adhering to security best practices.
• Data Pipeline Deployment: Collaborate with data engineering teams to deploy and maintain efficient data pipelines using tools like Apache Airflow, dbt, or similar technologies.
• Snowflake Administration: Implement and manage Snowflake data warehouse solutions, optimizing performance and ensuring data security and governance.
• MLOps Implementation: Collaborate with data scientists to implement MLOps practices, facilitating the deployment, monitoring, and governance of machine learning models in production environments.
• Information Security: Integrate security controls into all aspects of the data infrastructure, including encryption, access control, and compliance with data protection regulations (eg, GDPR, HIPAA).
• CI/CD Implementation: Develop and maintain continuous integration and continuous deployment (CI/CD) pipelines for data-related applications and services, including model training and deployment workflows.
• Support and Troubleshooting: Deploy updates and fixes, provide Level 2 technical support, and perform root cause analysis of production errors to resolve technical issues effectively.
• Tool Development: Build tools to reduce the occurrence of errors and improve the customer experience, and develop software to integrate with internal back-end systems.
• Automation and Visualization: Develop scripts to automate data visualization and streamline reporting processes.
• System Maintenance: Design procedures for system troubleshooting and maintenance, ensuring smooth operation of the data infrastructure.
• Monitoring and Performance Tuning: Implement monitoring solutions to track data workflows and system performance, proactively identifying and resolving issues.
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and support analytics initiatives.
• Documentation: Create and maintain documentation for data architecture, processes, workflows, and security protocols to ensure knowledge sharing and compliance.

Qualifications:

• 3 -6 + years of experience as a Data Ops /MLOps engineer or in a similar engineering role.
• Strong expertise in AWS services (eg, EC2, S3, Lambda, RDS) and cloud infrastructure best practices.
• Proficient in Snowflake, including data modeling, performance tuning, and query optimization.
• Experience with modern data technologies and tools (eg, Apache Airflow, dbt, ETL processes).
• Familiarity with MLOps frameworks and methodologies, such as MLflow, Kubeflow, or SageMaker.
• Experience in containerization and orchestration tools (eg, Docker, Kubernetes).
• Proficiency in scripting languages, including Python or similar , and automation frameworks.
• Proficiency with Git and GitHub workflows.
• Strong w orking experience of databases and SQL.
• Strong understanding of CI/CD tools and practices (eg, Jenkins, GitLab CI).
• Excellent problem-solving attitude and collaborative team spirit.
• Strong communication skills, both verbal and written.

Preferred Qualifications:

• Experience with data governance and compliance frameworks.
• Familiarity with data visualization tools (eg, Tableau, Looker).
• Knowledge of machine learning frameworks and concepts is a plus.
• Relevant security certifications (eg, CISSP, CISM, AWS Certified Security) are a plus.