Sr. Data Engineer

Amazon.com Services, Inc.
 Culver City, CA

Desciption

Prime Video is changing the way millions of customers interact with video content. Our team delivers high-quality video to Amazon customers through subscriptions (Amazon Prime) as well as purchases and rentals. Amazon believes so deeply in the mission of Video that we've launched our own studio to create original and exclusive content. Every day we face the challenges of a fast-paced market and expanding technology set. You will have the freedom and encouragement to explore your own ideas and the reward of seeing your contributions benefit millions of Amazon.com customers worldwide. Amazon delivers video to customers via the web, mobile phones, tablets, smart TVs, game consoles, and set top boxes. We help our customers discover the best movies and TV shows by using advanced machine learning and data mining techniques. We obsess over big picture problems like “How do we deliver video that’s more reliable than the internet it’s delivered over?“ to low level details like “How do we squeeze maximum picture quality out of every bit delivered?“ We strive to be on the forefront of new consumer technologies like UHD TV and High Dynamic Range video. We build huge scale distributed systems on the AWS cloud to make sure our service is always reliable for our customers. We use computer vision and machine learning techniques to build rich metadata about videos, and partner closely with teams like IMDb to let customers explore deeper into the TV and movies they love. In short, we have exciting challenges in an industry that’s doubling in size every year, and you can be a part of it.

Prime Videos is looking for a Data Engineer to join its core Data Engineering team. There are millions of videos watched each day by customers. In order to evaluate the performance of the business and make the best forward looking decisions, we need to store and process Bigdata volumes related to digital video supply chain workflows, catalog, and content operation activities. The Data Engineering team presents exciting opportunities to work on very large data sets in one of the world's largest and most complex data lake and data warehouse environments. Our data warehouse is built on AWS cloud technology like EMR, S3 and Redshift for performing ETL processing on over 100+ TB of relational data in a matter of hours. Our team is serious about great design and redefining best practices with a cloud-based approach to scalability and automation.

As a data engineer in this team, you will take a leadership role in the data platform and you will solve big data warehousing problems on a massive scale. You will apply cloud-based AWS services to solve challenging problems around: big data processing, data warehouse design, and BI self-service. You will be part of a data engineering team that focuses on automation and optimization for all areas of DW/ETL maintenance and deployment. You will work closely with the business and technical teams in analysis on many non-standard and unique business problems and use creative problem solving to deliver actionable output. The role of data engineer in Amazon requires excellent technical skills in order to develop systems and tools to process data as well as, but not limited to, the ability to analyze data and develop reports. Your work will have a direct impact on the day-to-day decision making in the Prime Video team.

Prime Videos is looking for a Data Engineer to join its Global Video Supply Chain Data Engineering team. There are millions of videos watched each day by customers. In order to evaluate the performance of the business and make the best forward looking decisions, we need to store and process Bigdata volumes related to digital video supply chain workflows, catalog, and content operation activities. The GVSC Data Engineering team presents exciting opportunities to work on very large data sets in one of the world's largest and most complex data lake and data warehouse environments. Our data warehouse is built on AWS cloud technology like EMR, S3 and Redshift for performing ETL processing on over 100+ TB of relational data in a matter of hours. Our team is serious about great design and redefining best practices with a cloud-based approach to scalability and automation.

Job responsibilities

As a data engineer in this platform team, some of you responsibilities would be:

* Develop and automate large scale, high-performance data processing systems (batch and/or streaming) using cloud based technologies

* Build scalable data pipelines leveraging S3, Lambda, Kinesis, AWS Glue

* Design data models for optimal storage and retrieval. Work closely with the business and technical teams in analysis on many non-standard and unique business problems and use creative problem solving to deliver actionable output

* Contribute to shared Data Engineering tooling & standards to improve the productivity and quality of output for Data Engineers across org

* Improve data quality by using & improving internal tools to automatically detect issues

Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.

Basic Qualifications

· 5+ years of experience as a Data Engineer or in a similar role

· Experience with data modeling, data warehousing, and building ETL pipelines

· Experience in SQL

* Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field

* 5+ Years of Data Warehouse Experience with Oracle, Redshift, PostgreSQL, etc. Demonstrated strength in SQL, python/pyspark scripting, data modeling, ETL development, and data warehousing

* Extensive experience working with cloud services (AWS or MS Azure or GCS etc.) with a strong understanding of cloud databases (e.g. Redshift/Aurora/DynamoDB), compute engines (e.g. EMR/Glue), data streaming (e.g. Kinesis), storage (e.g. S3) etc.

* Experience in maintaining data warehouse systems and working on large scale data transformation using EMR, Hadoop, Hive, or other Big Data technologies

* Experience mentoring other Data Engineers

Preffered Qualifications

* 7+ years of industry experience as a Data Engineer or related specialty (e.g., Software Engineer, Business Intelligence Engineer, Data Scientist) with a track record of manipulating, processing, and extracting value from large datasets.

* Experience with hardware provisioning, forecasting hardware usage, and managing to a budget

* Strong interpersonal skills and the ability to communicate complex technology solutions to senior leadership, gain alignment, and drive progress

Support