As a senior software engineer on the Pipeline Engineering Team at Crux, you will be responsible for building the systems that support the ingestion and processing of data on the Crux platform. Data is the lifeblood here at Crux and all software applications revolve around the scalable and performant handling of financial datasets. This is a great opportunity for an experienced engineer to take a significant role in the development of data-driven applications that form the foundation of a truly cutting-edge product.
If this piques your interest and you’re ready to break out of the monotony of traditional backend services, let’s talk. You will embark on a new multidisciplinary challenge involving large-scale distributed systems, configuration driven batch and real-time data pipelines, a plethora of SQL and NoSQL datastores, and everything in between.
- Contribute to the design and development of our data workflow management platform
- Build tools to wrangle datasets of small and large volumes of data into cleaned, normalized, and enriched datasets
- Develop new distributed systems from scratchRefine processes for normalization and performance-tuning analytics
- Mentor fellow teammates on design patterns and best practices
- You love building elegant solutions that scaleYou bring deep experience in the architecture and development of quality backend production systems, specifically in Java
- You love working on high-performing teams, collaborating with team members, and improving our ability to deliver delightful experiences to our clients
- You are excited by the opportunity to solve challenging technical problems, and you find learning about data fascinating
- You possess a strong desire to work for a small, fast-paced startup which is growing rapidly
- You’re excited by the challenge of extracting order out of the chaos of messy and highly inconsistent financial datasets
- 7+ years professional experience in a production environment
- Expertise in core Java, some Python experience is an advantage
- Proficiency in multiple programming languages
- Experience with data pipeline development, ETL and/or other big data processes
- Experience working in a cloud-based environment, such as GCP or AWS
- Experience working independently, or with minimal guidance
- Strong problem solving and troubleshooting skills
- Ability to exercise judgment to make sound decisions Excellent communication and interpersonal skills, as well as a sense of humor
- BS degree or higher in a technical discipline
- Experience with popular big data / distributed computing frameworks, eg. Spark, Hive, Kafka, Map Reduce, Flink
- Familiarity with container-orchestration technologies, eg. Kubernetes, YARN
- RDBMS SQL and NOSQL, structured and unstructured data, BigQuery
Crux is a data delivery and operations company that takes on the critical, yet commoditized tasks of ingesting data and getting it ready for analysis by financial institutions. We are experiencing rapid growth from financial firms turning to Crux to process and onboard the data they need.
Crux is transforming how the financial services industry works with data. For years, data discovery, operations, and delivery have been expensive, time consuming, and frustrating for financial firms. We are solving this challenge by creating a new technology solution: the go-to platform for data delivery. Crux is in high demand and we have a clear path to success.
Founded in 2017 by a seasoned team of data professionals, Crux has grown to a team of 70 employees in San Francisco and New York. It is backed by leading industry players, including Citi, Goldman Sachs, and Two Sigma.
At Crux, diversity is valued and and treatment of employees and applicants are based on merit, talent and qualification. We encourage people from underrepresented groups to apply. We believe the key to success is bringing together unique perspectives and we do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. For qualified applicants with criminal histories, consideration will be consistent with the requirements of the San Francisco Fair Chance Ordinance. All your information will be kept confidential according to EEO guidelines.