PipeCandy is a 'one of its kind', 'data science' driven market intelligence platform that tracks the global eCommerce landscape. Our insights are used by well known global brands and startups. We are venture funded by India, the US, and Singapore based investors. We are building a complex data product that aims to revolutionize industry intelligence by applying sophisticated ML & AI algorithms on millions of data points.
We are looking for a crawler who will work on large scale web crawling and custom data acquisition. If you love playing with data and code, have an eye for detail and a strong client delivery mindset, we’d love to hear from you.
This position requires experience in technology related to web crawling and data scraping, the ability to understand business requirements, develop data acquisition code as per specification, and managing project delivery.
- Design, develop, and configure web crawling software systems to support our crawling and data extraction requirements
- Implement web scraper and data extraction for various websites as per business requiements
- Monitor, test and maintain web scraper code to address any changes in crawled website
- Implement best practices and patterns in design, development, deployment and software testing
- Web Crawling and data scraping experience
- Knowledge of scraping frameworks such as Python (Request, BeautifulSoup, etc), Web- Harvest and others
- Experience with SQL and NoSQL databases
- Experience with cloud based crawling platforms
- Experience with multi- processing, multi- threading and AWS/ Azure
- Qualification – Graduate, preferably B.E. or MCA background
- Hands-on experience in coding & development in related technologies
- 1- 3 years of experience
- Ability to work with business and technology teams to deliver data acquisition projects
- Ability to multi-task, solve problems and think strategically
- Strong communication and collaboration skills
- Flat organization structure with an opportunity to work very closely with the founderss
- Access to learning, training sessions outside of your immediate line of work
- Access to group kindle account with latest titles
- Stocked pantry, of course