Data Engineer with Spark
Correct Context is looking for a Data Engineer with Spark for Comscore in Poland and around.
Comscore is a global leader in media analytics, revolutionizing insights into consumer behavior, media consumption, and digital engagement.
Comscore leads in measuring and analyzing audiences across diverse digital platforms. Thrive on using cutting-edge technology, play a vital role as a trusted partner delivering accurate data to global businesses, and collaborate with industry leaders like Facebook, Disney, and Amazon. Contribute to empowering businesses in the digital era across media, advertising, e-commerce, and technology sectors.
We have multiple Java + Spark, Scala + Spark, Python + Spark teams and we may try to match you to multiple teams or just find you single best fit depends on your skills and experience.
We offer:
- Real big data projects (PB scale) 🚀
- An international team (US, PL, IE, CL) 🌎
- A small, independent team working environment 🧑💻
- High influence on working environment
- Hands on environment
- Flexible work time ⏰
- Fully remote or in-office work in Wroclaw, Poland 🏢
- 12,000 - 22,000 PLN net/month B2B 💰
- Private healthcare (PL) 🏥
- Multikafeteria (PL) 🍽️
- Free parking (PL)🚗
As a Data Engineer, you'll:
- Design, implement, and maintain petabyte-scale big data pipelines using Spark (Java, Python, or Scala - depending on the team), Apache Airflow, Kubernetes, and other technologies
- Optimize performance - working with big data is highly specific: sometimes IO-bound, sometimes CPU-bound. You’ll help figure out the most efficient approaches
- Collaborate closely with other big data teams
- Work with technologies such as AWS, Kubernetes, Airflow, EMR, Hadoop, Linux/Ubuntu, Kafka, and Spark
The candidate must have:
- Solid understanding of Spark basics, building blocks, and mechanics (the deeper the knowledge, the higher the value)
- Strong knowledge of Python, Java, or Scala (with the ability to expand to expert level)
- 1+ years of experience with Spark (commercial not required, deep understanding matters more than years)
- Good SQL skills - not necessarily writing complex queries by hand, but strong knowledge of available tools and approaches to solve data problems
- Understanding of data quality issues in large datasets (inconsistencies, missing data, imbalanced sets, etc.)
- 1+ years of experience with Linux (power-user skills are a big plus; deployment is not required, but Linux knowledge makes your life easier)
- Professional working proficiency in English (oral and written)
- Understanding of HTTP API communication patterns (HTTP/REST/RPC) and protocol itself
- Good software debugging skills (beyond print - using debuggers effectively)
- Deep understanging of at least one technical area (be ready to share your biggest "battle story" about it)
- Solid Git understanding
- Strong communication skills (ability to drive end-to-end projects and mentor team members)
If you don't have all the qualifications, but you're interested in what we do and you have a solid Linux understanding -> let's talk!
The recruitment process for the Data Engineer position has following steps:
- Technical survey - 10min
- Technical screening - 30 min video call
- Technical interview - 60-90min video call - this step can be multiplied if we speak to multiple teams (we have multiple teams that you may want to join, your choice)
- Final Interview - Technical/Managerial - 30 min video call
- Department
- Comscore
- Locations
- Ruska 3, Wrocław (near pl. Solny)
- Remote status
- Fully Remote
Ruska 3, Wrocław (near pl. Solny)
-
Products
We strongly believe that working on a specific product for longer period of time has a lot of benefits. Person working in such environment where deep understanding of a product, technology in a stable environment can render great ideas, improvements and benefit everybody. No switching projects every three months, no new manager every three months, no new customer every three months. -
Craftsmanship
Everyday we learn something. It's great when you can not only learn but use that knowledge to build great stuff and grow yourself. We are big fans of excellence and technical craftsmanship. No excuses to avoid CI, tests, QA, best practices. It's true that shipping is the most important thing, but in the end, it's your name on it, let's make it right way. -
Stable and Flexible Environment to Grow
As practice shows, small teams are best to achieve best results and we are big fans of that approach. It's easier to agree on holidays, specific time to work, plan your time to take children from school. It's easier to agree on technical solutions and easier to communicate. With all that and bunch of extra smaller things like private healthcare, trainings, flexible working hours, fully remote or office options, employment of B2B type of work, lawyer/accountant help we are creating stable and safe environment for you to grow not only on a technical level, but also as a person. -
Technical Autonomy
We trust you. All teams have the autonomy to pick best solutions, best software, best approach for problem solving. No crazy managers to tell you that mongodb is a must or everybody must work in vim. We want you to support you on the road to technical excellence and avoid artificial blockers. All structures are as flat as possible and all decision making is as close to the core as possible. I hope you enjoy this a lot.
Workplace & culture
We really love to make working software products. We are small teams that are focused on specific products. Each of us contributes to project we like the most and with trust, independence and autonomy, hands-on approach and great software craftsmanship we shape the future.
About Correct Context
Correct Context is about gathering together best IT Developers, UX and UI Designers, Product People and Ambitious Product Companies to build and grow Greatest Products that matter.
Already working at Correct Context?
Let’s recruit together and find your next colleague.