Python scraper based on AI

🤖/👨‍🦰 Detect bots/crawlers/spiders using the user agent string

Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt

API definition, resources and reference implementation of URL Frontiers

🤩 Python Package for Scraping Amazon Product Reviews ✨

Python course for 2nd year NLP students at NRU HSE, 2018-2019

一款强大的大模型微调数据集生成和管理工具。

YiraBot: Simplifying Web Scraping for All. A user-friendly tool for developers and enthusiasts, offering command-line ease and Python integration. Ideal for research, SEO, and data collection.

GAMECHANGER Policy Analytics Site Crawlers

Python course for 2nd year NLP students at NRU HSE, 2017-2018

Detect bots/crawlers/spiders via user-agent string

A Web Crawler developed in Python.

List of best web crawlers to extract data from the web. Find web crawling tools for different needs.

Web scraper for collecting product and review data from e-commerce sites using Scraping Bee, AWS, Selenium, and Pandas. Focuses on cost-effective solutions, user-friendly interfaces, and efficient data extraction and analysis.

This is an advanced version of the previously released version of web-crawler

Web crawlers based on travel information, crawler data from Ctrip, tuniu, qunar.

This is a simple web crawler created using python

🕷️ Discover and use popular web crawlers across various programming languages to efficiently extract data from the web.

A tutorial on Crawling the Web and Scraping off the Useful bits - without actually doing anything!

一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )