Lead Data Engineer
Company: Together Labs
Location: Redwood City
Posted on: February 21, 2021
|
|
Job Description:
The Data and Analytics team is focused on creating competitive
advantage for Together Labs and our customers through novel data
infrastructure, metrics, insights and data services.--We are a
small but rapidly growing analysis and engineering team that builds
and leverages state-of-the-art analytics systems. Our work informs
decisions and direction for our business, while also impacting our
products. The petabytes of data we store and the hundreds of GB of
transactional data that we process each day provide a great
opportunity to apply machine learning techniques to design powerful
and meaningful experiences to our users.We are looking for a
seasoned Lead engineer to not only build data pipelines to
efficiently and reliably move data across systems, but also to
build the next generation of data tools to enable us to take full
advantage of this data. In this role, your work will broadly
influence the company's products, data consumers and data
scientists. The Role:* Architect and implement best practices of
big data tools such as Hadoop, Spark, Airflow, Kafka, Presto and
Time-series Databases * Set processes and policies for data
governance and data pipeline * Set and define the standards and
best practices in data team * Be the point of reference for solving
a challenging technical problem* Architect and implement Machine
Learning Pipelines for Data Science team* Build and expand IMVU's
Data Platform's Hadoop Open source Big data Clusters within an
agile collaboration environment* Observe the thriving open source
projects in the descriptive and predictive analytics big data space
and bring it into IMVU* Must love coding - prepare to spend 80% of
the time on hands-on development with development teams* Deliver
end-to-end applications and frameworks ranging from system
programming to micro-services to simple front-end applications*
Security-first mindset to ensure that any software built will
satisfy Visa's stringent security standards* Ability to own
complete project or a subject area delivery while leading team
members in a scrum setting* Use your knowledge of Java, Go or
Python to create new features in the cloud.* Understand and
influence logging to support our Data flow.Qualifications:*
Bachelor's or Master's--degree in Computer Science, Engineering or
related discipline * 10+ years of experience in design and
development of large-scale data platforms * 4+ years of software
development experience in Scala or Java.* Expert in tools such as
Spark, Airflow, Presto * Expert in design and implement reliable,
scalable, and performant distributed systems and data pipelines *
Expert in containerization, including Docker and Kubernetes *
Experience with running large-scale distributed computing
infrastructure such as load balancing, Zookeeper, Micro service
architecture * Experience with Columnar database such as Redshift,
Vertica or Parquet* Hand-on experience in optimizing existing Big
Data applications written in Spark, Hive. Optimizing * Experience
in building Open source projects, contributed to open source
projects and deploying into containers platform * Good
understanding of building utilization frameworks on Hadoop, AWS,
Azure or GCP (Restful services, Self-service BI and Analytics).*
Experience in one or more analytical modeling tools/workbench
(Jupyter or Zeppelin)* Worked on AWS services ---Lambda, S3,
Redshift, SNS (or Apache Kafka), SQS, Redis, Data-Lakes, and AWS
cloud deployment models* Strong communication skills with ability
to present complex ideas and document in a clear and concise way*
Quick learner; self-starter, detailed and thorough* Proactive sense
of urgency and "can do" attitude* A technical leader: you make
intuitive decisions about what services, frameworks, and
capabilities need to be in place before they are needed.* Self
Driven - able to own a project from inception to completion* Mentor
- open and active in sharing knowledge as well as excellent
communication skillsBonus Points:* Experience with
Free-to-play/social games* AWS certified: Architect/DevOps/Big
Data
Keywords: Together Labs, Redwood City , Lead Data Engineer, Engineering , Redwood City, California
Click
here to apply!
|
Didn't find what you're looking for? Search again!
Other Engineering JobsQA Engineer II Description: JOB REQUISITION QA Engineer II LOCATION SAN RAMON JOB DESCRIPTION As QA Engineer II, your specific responsibilities include: Be able to understand business process and system requirements to develop relevant (more...) Company: Robert Half Location: San Ramon Posted on: 03/3/2021 Lead DevOps Engineer (AWS, Python, Saltstack) URGENT Description: If you are a Lead DevOps Engineer with scripting experience outside Bash and Cloud exp, please read on br br Situated in the heart of Silicon Valley, where the impossible becomes possible and technology (more...) Company: Optello Location: Palo Alto Posted on: 03/3/2021 Data Engineer - Contractor Description: Position: Data EngineerPrealize Health, powers proactive healthcare, connecting tomorrow's predictive insights to today's most effective actions. Based in Palo Alto, California, the company was founded (more...) Company: Prealize Health Location: Palo Alto Posted on: 03/3/2021 Estes Express Lines, Inc.: Truck Mechanic Description: Responsible for repairs and required preventive maintenance of diesel engines, trucks and/or forklifts Inspect, trouble shoot, diagnose and perform maintenance on equipment Perform safety inspections (more...) Company: Estes Express Lines, Inc. Location: West Sacramento Posted on: 03/3/2021 Cyber Security Engineer (Carbon Black/Bit9) Description: Location: Palo Alto, CA Remote Job Posting Title: Cyber Security EngineerEstimated Company: Maxonic Consulting Pvt Ltd Location: Palo Alto Posted on: 03/3/2021 HIRED: Front-End Engineer - Remote Description: Join Hired and find your dream job as a Front-End Engineer at one of 10,000 companies looking for candidates just like you.Companies on Hired apply to you, not the other way around. You'll receive (more...) Company: HIRED Location: Novato Posted on: 03/3/2021 Android Engineer Description: Location: San Bruno, CADescription: Our client is currently seeking a Android Engineer This job will have the following responsibilities: Demonstrates Company: The Judge Group Location: San Bruno Posted on: 03/3/2021 Automation Engineer Description: Job Title: Automation Engineer Location:--Novato, CA must be local Job Category: Manufacturing / Production / OperationsHours Per Week: 40.00Type: Initial 12-24 month contract may convert after contract (more...) Company: Ascent Location: Novato Posted on: 03/3/2021 Perception Algorithm Engineer - Autonomous Driving Description: Job Description Leading the future of luxury mobility Lucid''s mission is Company: Lucid Motors Location: Newark Posted on: 03/3/2021 Full Stack Engineer Description: Stealth Startup is a stealth consumer finance company on a mission to reinvent everyday banking tools for the next generation. The goal is to help eliminate the 1T in national credit-card debt from fees, (more...) Company: ProArch Consulting Services Location: Palo Alto Posted on: 03/3/2021 |