Developer – Web Crawling and Data Acquisition

Location: Chennai

Experience: 1-3 Years

PipeCandy is a 'one of its kind', 'data science' driven market intelligence platform that tracks the global eCommerce landscape. Our insights are used by well known global brands and startups. We are venture funded by India, the US, and Singapore based investors.

About the Role:

We are building a complex data product that aims to revolutionize industry intelligence by applying sophisticated machine learning & AI algorithms on millions of data points.

We are looking for a Developer who will work on large scale web crawling and custom data acquisition. If you love playing with data and code, have an eye for detail and a strong client delivery mindset, we’d love to hear from you.

The business analyst will work under the general direction of the Chief Data Scientist and senior staff in the Data team. This position requires experience in technology related to web crawling and data scraping, the ability to understand business requirements, develop data acquisition code as per specification, and managing project delivery.

Key Responsibilities:

  • Design, develop, and configure web crawler software systems to support our crawling and data extraction requirements
  • Implement web scraper and data extraction for various websites as per business requiements
  • Monitor, test and maintain web scraper code to address any changes in crawled website
  • Implement best practices and patterns in design, development, deployment and software testing

Skills Required:

  • Web Crawling and data scraping experience
  • Knowledge of scraping frameworks such as Python (Request, BeautifulSoup, etc), Web- Harvest and others
  • Good to have experience in JavaScript, JQuery, PHP & Ruby on Rails
  • Strong coding experience in Python (Knowledge of Java, Javascript is a plus)
  • Experience with SQL and NoSQL databases
  • Experience with cloud based crawling platforms
  • Experience with multi- processing, multi- threading and AWS/ Azure

Competencies Required:

  • Qualification – Graduate, preferably B.E. or MCA background. Should be currently doing hands- on coding and development in related technologies
  • 1- 3 years of experience

Desired:

  • Ability to work with business and technology teams to deliver data acquisition projects
  • Ability to multi-task, solve problems and think strategically
  • Strong communication and collaboration skills

Perks

  • Flat organization structure with an opportunity to work very closely with the founderss
  • Forums for development of skills outside of the direct line of work
  • Group Insurance and Provident Fund
  • Stocked Pantry

Interested? Send your resume to openings@pipecandy.com