Senior Spark Architect

Job Number: 28363

Position Title: Senior Development Architect

External Description:

Our Company

Everything Informatica does begins and ends with data. Simply stated, we make great data – data that is connected, clean and safe -- ready to use so that all enterprises can be data ready and put their unique information potential to work. A data ready enterprise is decision-ready, customer-ready, application-ready, cloud-ready and regulation-ready. And by design, our Intelligent Data Platform delivers great data to enable our customers to be ready for anything.

Our Team

Cloud Data Integration Group is the innovation hub within Informatica that is responsible for delivering extensible, reliable, scalable, and innovative products. Our team specifically is responsible for Cloud Data Integration – Elastic (CDI-E), a cutting-edge cloud native data integration product built on top of the latest Spark on Kubernetes open source technology. Since our Engine heavily relies on open source code, especially Spark, your responsibility will be contributing to open source:

• Enhancing the Spark open source with new features and capabilities in order to improve our Spark Engine
• Developing Spark on Kubernetes open source code for optimal scaling and resource utilization
• Enhancing Apache Spark open source code to optimize jobs for best performance
• Enhancing Apache Spark open source code to reduce latency and processing time of Spark jobs to return instantaneous results
• Enhancing Apache Spark Open source to allow best execution and monitoring of Spark jobs and utilization of clusters for high performance and low TCO.
• Enhancing the Apache Spark Machine Learning capabilities
• Working with and enhancing latest open source technology to improve the product

Your Opportunity

Informatica Data Engine Platform has been a key constituent of Informatica’s platform and serves multiple products such as Data Integration, Data Quality, Data Masking, Profiling and MDM. Informatica products rely on the Spark Data Engine to fulfill the vision of executing data integration jobs across Clouds and on-prem in the most performant and cost effective way. You will be working with the latest open source technology on cutting edge innovative projects. You will be a key contributor to Informatica’s next generation Cloud platform and how it handles the volume, velocity and variety of structured and unstructured data. We are looking for you to look for ways to improve Apache Spark and other open source technology and contribute back to open source.

Our Ideal Candidate

You are an experienced software engineer who has made significant contributions to open source technologies especially in data processing, data management and distributed systems. You relish the opportunity to enhance  and build on top of open source technology to add innovative functionality and optimizations that improve data processing performance across different use cases that include batch and streaming.

Your Responsibilities

The successful candidate will be based in Redwood City, CA and will be responsible for:

• Contributing to key open source technologies
• Enhancing and building on top of open source technology to deliver innovative features for Spark Data Engine
• Using open source technology in incubating and designing the next generation cloud platform for complex heterogeneous systems and applications involving cutting edge technologies
• Collaborating with geographically dispersed, cross-functional teams in an Agile environment
• Mentoring junior engineers on technical, architectural, design and related issues and helping them contribute to open source
• Proactively identifying architectural weaknesses in open source technology we leverage and recommending appropriate solutions
• Providing architectural support and guidance for efficient deployments of Spark on cloud’
• Quickly learning, understanding, debugging and improving open source technology
• Taking ownership for open source component we leverage and enhancing them 

Your Qualifications

• 10+ years of production experience in Java, Scala.
• Good understanding of the open source contribution process
• Deep knowledge and understanding of Apache Spark.
• 3+ years of hands on experience as a key contributor or committer with Apache Spark
• Ability to communicate and express thoughts and ideas in a big group
• Able to work independently with little direct supervision and take initiative; willing to mentor and develop others
• Strong analytical problem solving and decision-making skills
• Machine Learning knowledge is a plus
• BS in Computer Science or a related technical discipline; advanced degree preferred

City: City

State:

Seniority Level: Director

Alternative Location(s) :

Community / Marketing Title: Senior Spark Architect

Remote LinkedIn Hashtag:

LinkedIN Hashtag: LI-JB3

Company Profile:

EEO Employer Verbiage:

At Informatica we know diversity drives innovation. We are proud to be an Equal Opportunity Employer dedicated to maintaining a work environment free from discrimination, one where all employees are treated with dignity and respect. All qualified applicants will receive consideration for employment without regard to race, color, gender, sex, sexual orientation, marital status, religion, age, disability, gender identity, veteran status or any other characteristic protected by applicable law and Informatica policy.

Travel Requirement: Limited

Location_formattedLocationLong: City, US