Repositories list

• schema-org

  Public
  Extract schema.org objects from HTML documents

  schemaschema-orgjson-ld+ 1structured-data

  PHP

  •

  MIT License

  •1•12•0•0•Updated Nov 21, 2025Nov 21, 2025

• package-template

  Public template
  Template repository for new crwlr packages

  PHP

  •

  MIT License

  •0•1•0•0•Updated Nov 21, 2025Nov 21, 2025

• robots-txt

  Public
  Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping

  web-scrapingrobots-txtweb-crawling+ 4hacktoberfestrobots-exclusion-standardrobots-exclusion-protocolrobots-txt-parser

  PHP

  •

  MIT License

  •2•9•0•0•Updated Nov 21, 2025Nov 21, 2025

• url

  Public
  Swiss Army knife for urls.

  urlphpuri+ 10psr-7urlspunycodeurl-parsingurl-parseruri-parserhacktoberfest+ 3

  PHP

  •

  MIT License

  •5•107•1•0•Updated Nov 21, 2025Nov 21, 2025

• query-string

  Public
  A library for convenient handling of query strings used in HTTP requests.

  phphttpquery+ 7stringquerystringsrequestsmanipulationquerystringhacktoberfestquery-string

  PHP

  •

  MIT License

  •3•20•0•0•Updated Nov 21, 2025Nov 21, 2025

• crawler

  Public
  Library for Rapid (Web) Crawler and Scraper Development

  phpcrawlerscraper+ 8web-crawlerscrapingcrawlingweb-scraperweb-scrapingscraping-websitesweb-crawling+ 1

  PHP

  •

  MIT License

  •13•366•1•1•Updated Aug 5, 2025Aug 5, 2025

• crawler-ext-browser

  Public
  Extension for the crwlr/crawler package containing steps utilizing a headless browser.

  PHP

  •

  MIT License

  •0•0•0•0•Updated Feb 4, 2025Feb 4, 2025

• crwl-extension-utils

  Public
  Utils for extension packages for the crwl.io app.

  PHP

  •0•0•0•0•Updated Dec 8, 2024Dec 8, 2024

• utils

  Public
  Utilities that are needed in multiple crwler packages.

  PHP

  •

  MIT License

  •1•2•0•0•Updated Dec 8, 2024Dec 8, 2024

• html-2-text

  Public
  Convert HTML to formatted plain text.

  PHP

  •

  MIT License

  •0•3•0•0•Updated Nov 6, 2024Nov 6, 2024

• crwl-ext-browser

  Public archive
  Extension configurations for integration of crwlr/crawler-ext-browser into the crwl.io app.

  PHP

  •

  MIT License

  •0•0•0•0•Updated Jul 8, 2024Jul 8, 2024

• laravel-crawler

  Public
  Laravel adapter for the crwlr/crawler package.

  phpcrawlerscraper+ 9laravelweb-crawlerscrapingcrawlinglaravel-packageweb-scrapingweb-crawling+ 2

  PHP

  •

  MIT License

  •0•4•0•0•Updated Sep 29, 2023Sep 29, 2023

• .github

  Public
  0•0•0•0•Updated May 8, 2022May 8, 2022

• graphics

  Public
  Only images to show on github readme pages

  0•0•0•0•Updated May 8, 2022May 8, 2022