PipeCandy is a 'one of its kind', 'data science' driven market intelligence platform that tracks the global eCommerce landscape. Our insights are used by well known global brands and startups.
We are venture funded by India, the US, and Singapore based investors. We are building a complex data product that aims to revolutionize industry intelligence by applying sophisticated machine learning & AI algorithms on millions of data points.
We are looking for a Developer who will work on large scale web crawling and custom data acquisition. If you love playing with data and code, have an eye for detail and a strong client delivery mindset, we’d love to hear from you.
The business analyst will work under the general direction of the Chief Data Scientist and senior staff in the Data team. This position requires experience in technology related to web crawling and data scraping, the ability to understand business requirements, develop data acquisition code as per specification, and managing project delivery.
- Design, develop, and configure web crawler software systems to support our crawling and data extraction requirements
- Implement web scraper and data extraction for various websites as per business requiements
- Monitor, test and maintain web scraper code to address any changes in crawled website
- Implement best practices and patterns in design, development, deployment and software testing
- Web Crawling and data scraping experience
- Knowledge of scraping frameworks such as Python (Request, BeautifulSoup, etc), Web- Harvest and others
- Experience with SQL and NoSQL databases
- Experience with cloud based crawling platforms
- Experience with multi- processing, multi- threading and AWS/ Azure
- Qualification – Graduate, preferably B.E. or MCA background. Should be currently doing hands- on coding and development in related technologies
- 1- 3 years of experience
- Ability to work with business and technology teams to deliver data acquisition projects
- Ability to multi-task, solve problems and think strategically
- Strong communication and collaboration skills
- Flat organization structure with an opportunity to work very closely with the founderss
- Forums for development of skills outside of the direct line of work
- Group Insurance and Provident Fund
- Stocked Pantry