ChatGPT Scraper Demo

A Python script that interacts with ChatGPT using the ScrapingAnt API for browser rendering. It can submit prompts, optionally enable web search mode, and extract responses.

Features

• Browser-based interaction with ChatGPT via ScrapingAnt API
• Optional web search mode toggle
• Configurable response wait time
• HTML response saved for debugging
• Command-line interface

Requirements

• Python 3.7+
• ScrapingAnt API key

Installation

1. Clone this repository or download the files

2. Create a virtual environment (recommended):

   python3 -m venv .venv
   source .venv/bin/activate  # On Windows: .venv\Scripts\activate

3. Install dependencies:

   pip install requests beautifulsoup4

4. Add your ScrapingAnt API key to chatgpt_scraper.py:

   API_KEY = "<YOUR_SCRAPINGANT_API_KEY>"

Obtaining ScrapingAnt API Token

1. Go to ScrapingAnt
2. Click Sign Up or Get Started Free
3. Create an account using email or Google/GitHub OAuth
4. After registration, navigate to your Dashboard
5. Find your API key in the API Key section
6. Copy the API key and paste it into chatgpt_scraper.py

Note: ScrapingAnt offers a free tier with limited API credits. For production usage, consider upgrading to a paid plan.

Usage

Basic usage:

python chatgpt_scraper.py "What is the capital of France?"

Disable web search mode:

python chatgpt_scraper.py "Explain quantum computing" --no-search

Custom wait time (in milliseconds):

python chatgpt_scraper.py "Write a poem about AI" --wait 45000

All options:

python chatgpt_scraper.py --help

Command Line Arguments

Argument	Description	Default
prompt	The prompt to send to ChatGPT	Required
--no-search	Disable web search mode	Enabled
--wait	Wait time for response in ms	35000

Output

The script will:

1. Print the prompt and configuration
2. Save the raw HTML response to chatgpt_response.html
3. Print the extracted ChatGPT response

How It Works

1. Builds a JavaScript snippet to interact with ChatGPT's interface
2. Sends request to ScrapingAnt API with browser rendering enabled
3. JavaScript clicks search button (if enabled), inputs the prompt, and submits
4. Waits for ChatGPT to generate a response
5. Returns HTML which is parsed to extract the response text

Troubleshooting

• No response extracted: Increase --wait time for longer responses
• API errors: Verify your API key is correct and has available credits
• Empty responses: Check chatgpt_response.html for debugging

License

MIT License