Python Developer Web Crawling (sde) Job in Sciative - The Pricing Company

Job Title: Python Developer Web Crawling (SDE)

Location: Navi Mumbai

Industry: AI, Big Data, SaaS

Company: Sciative

Employee Strength: 90+

Employment Type: Full-time

About Sciative

Sciative is on a mission to create the future of dynamic pricing powered by artificial intelligence and big data. Our Software-as-a-Service products are used globally across industries such as retail, ecommerce, travel, and entertainment. As a fast-growth startup, Sciative has a strong team of over 90 professionals based in a plush office in Navi Mumbai. Our products aim to provide unmatched customer-oriented solutions. We re looking for exceptionally talented, driven individuals to join our tech team and help us make history by building the best platform to find and buy anything online.

Job Responsibilities

• Web Scraping & Crawling: Develop, maintain, and optimize Python-based web scraping and crawling tools to extract data from various online sources.
• Architecture Design: Design and implement scalable, efficient web scraping architectures and workflows to handle complex websites with dynamic content.
• Cross-functional Collaboration: Work closely with cross-functional teams to understand data requirements and deliver actionable insights derived from scraped data.
• Anti-Scraping Mitigation: Handle dynamic and complex websites, including those with anti-scraping mechanisms, and develop workarounds.
• Data Integration: Integrate scraped data into databases, data pipelines, and analytical tools.
• Issue Resolution: Troubleshoot and resolve issues related to data extraction, ensuring data integrity and resolving performance challenges.
• Compliance & Documentation: Ensure compliance with legal and ethical standards for web scraping and data collection, and maintain proper documentation for scraping processes and tools.
• Stay Updated: Keep up with the latest trends and technologies in web scraping, crawling, and data processing.

Qualifications

• Educational Background: Bachelor's degree in Computer Science, Engineering, or a related field (or equivalent work experience).
• Professional Experience: Proven experience in Python development with a focus on web scraping and crawling.
• Technical Skills:
  • Strong proficiency with Python libraries and frameworks such as BeautifulSoup, Scrapy, Selenium, and requests.
  • Solid understanding of web technologies, including HTML, CSS, and JavaScript.
  • Experience with data handling and processing tools like Pandas, NumPy.
  • Knowledge of database systems (SQL and NoSQL) and data storage solutions.
  • Familiarity with version control systems like Git.
  • Strong problem-solving skills and a keen attention to detail.
  • Ability to work independently and manage multiple projects simultaneously.

Preferred Qualifications:

• Experience with cloud platforms like AWS or Google Cloud and containerization technologies like Docker.
• Knowledge of API integration and data visualization tools.
• Familiarity with machine learning concepts and their application in data extraction and analysis.

What We Offer

• Competitive Salary: A competitive salary package along with a comprehensive benefits package.
• Career Growth: Opportunities for career growth and professional development within a fast-growing tech company.
• Collaborative Environment: A collaborative and innovative work culture where your contributions are valued.
• Flexible Work: Flexible working hours and the possibility of remote work options.

Interested candidates can apply by submitting their resume along with relevant experience in Python development and web scraping.