Data Engineer

 Job Description

• Enterprise Data Accessibility team provides Business Intelligence reporting capability to our internal stakeholders, and works on enabling accessibility to data.

• The Business Intelligence is provided through reporting platforms like MSBI stack (SSIS, SSAS and SSRS), PowerBI, WebFocus, Tableau and Domo. We provide data insights and analytical features that helps customers to make smart business decisions.

• The purpose of this job is to lead and develop/enhance the Hadoop processes, that builds data sets required for reporting.

Role

• Develop high quality, secure and scalable scripts and data pipelines using shell script, Oracle procedures and tools like NiFi.

• Build automation and efficiency in Data ingestion, data movement, data access and data transfer workflows by innovation and collaboration.

• Understand, implement and enforce Software development standards and engineering principles in the RDBMS (Oracle) and Big Data space.

• Work closely with business stakeholders and embedded engineering teams within business teams in a collaborative manner to help them build scalable products in quick time.

• Leverage new technologies and approaches to innovate with increasingly large data sets.

• Work with project team to meet scheduled due dates, while identifying emerging issues and recommending solutions for problems.

• Perform assigned tasks and production incident independently.

• Contribute ideas to help ensure that required standards and processes are in place and actively look for opportunities to enhance standards and improve process efficiency.

• Prepare implementation plan and support during and after implementation.

• Work with the team as well as an individual contributor to drive efficient deliveries.

Responsibilities

• Work on complex enterprise-wide initiatives spanning multiple services/programs and drives resolution

• Work with business/product owners to architect and deliver on new services to introduce new products and bundles

• Drive the architectural design, including dependent services, service interactions, and policies

• Take ownership to improve the customer experience of an entire set of services/applications sold as products or bundled services

• Contribute and lead Guild initiatives by engaging and mentoring Engineers at all levels to improve the craftmanship of Software Engineering

• Simplify and improve the cost/benefit of a function/service/architecture strategy

• Apply judgment and experience to balance trade-offs between competing interests

• Venture beyond comfort zone to take on assignments across different areas of Software Engineering

• Take on organization-wide and public speaking engagements and publishes white papers and blogs on relevant and emerging technical topics

• Consult across teams and across organization lines to identify synergies and reuse opportunities

• Participate and contribute to Principal review architecture meetings and drive resolutions to enterprise-wide challenges and regulatory needs

• Write recommendations for job promotions based on an unbiased view of one's accomplishments

• Conduct technical interviews for hiring engineering staff and raising the performance bar

• Identify and raise awareness to Silo-ed behaviors within the organization and teams

All About You

• Experience/expertise in Hadoop Eco System – Spark, Python, Hive, Impala.

• Experience/expertise on Scripting languages like Unix, Python and OOPs programming languages (Scala preferrable).

• Experience in performance tuning of data processing in Hadoop, esp. with huge datasets (in TBs).

• Understanding of Exadata/Oracle/SQL server databases.

• Good to have exposure to NiFi/Kafka.

• Strong SQL programming skills.

• Reporting development in Data Warehouse environment is an added advantage.

• Understanding of Business Intelligence solutions for customers.

• Exposure and understanding of complete SDLC.