Web Scraper

About the Role

NIQ for Digital Commerce works at the intersection of e-commerce and retail analytics, turning massive volumes of web data into actionable intelligence for businesses. The team processes more than 60 billion data points every day to deliver real-time market visibility and support better commercial decisions. This is a fast-growing, international environment with a startup-style culture, strong team spirit, and room for ownership and career growth.

As a Python Developer in the scraping team, you will help improve large-scale data collection systems and contribute to a collaborative engineering group focused on quality, resilience, and innovation.

Key Responsibilities

• Build, maintain, and document dependable Scrapy spiders that can withstand site changes while keeping current crawlers stable.
• Review code carefully and validate collected data to ensure strong technical and data quality standards.
• Use your understanding of HTTP, browser behavior, and internal tooling to create advanced crawling workflows that can work around sophisticated anti-bot defenses.
• Contribute to major enhancements across several codebases, and provide clear technical documentation and training when required.

Requirements

• A master’s degree in Computer Science, Information Technology, or a similar discipline.
• At least 3 years of hands-on software engineering experience.
• Practical knowledge of BeautifulSoup or the Scrapy framework.
• Solid familiarity with HTML and JavaScript, including how single-page applications behave.
• Experience working with RESTful and/or GraphQL APIs.
• Hands-on exposure to Python web frameworks such as Django, FastAPI, or comparable tools.
• Scrapy experience is considered an advantage.
• Experience with time-series databases such as InfluxDB is a plus.
• Strong working knowledge of Docker, Git, pandas, regular expressions, Linux, and Bash scripting.
• Prior experience using major cloud platforms such as AWS, GCP, or Azure.
• A motivated, self-driven, and independent working style.
• Interest in challenging engineering problems and a willingness to keep learning new technologies.

Perks and Benefits

• Flexible work environment.
• Volunteer time off.
• Access to LinkedIn Learning.
• Employee Assistance Program (EAP).

Additional Information

NIQ may use artificial intelligence tools during different parts of recruitment, including résumé screening, assessments, interview scheduling, job matching, candidate communication, and some administrative tasks. These tools are intended to support efficiency and consistent evaluation using job-related criteria. Human oversight remains part of the process, and final hiring decisions are made by people. NIQ reviews its AI systems regularly to reduce bias and align with applicable laws. If you need an accommodation or human review where legally permitted, you may contact your local HR representative.

About NIQ

NIQ is a global consumer intelligence company focused on understanding how people buy and what drives growth. Following the 2023 combination with GfK, the company now combines broad global reach with advanced analytics and a comprehensive view of retail and consumer behavior. NIQ operates in more than 100 markets and covers over 90% of the world’s population.

Diversity, Equity, and Inclusion

NIQ is committed to building an inclusive workplace that reflects the diversity of the communities and markets it serves. The company values varied perspectives and makes employment decisions without discrimination based on protected characteristics under applicable law.