Skip to content

Add daily AI News Collector pipeline (OpenAI & Anthropic) #637

@othmane-01

Description

@othmane-01

I built a Python script that collects daily AI news from OpenAI and Anthropic RSS feeds.
It saves the results in JSON format and uploads them into Oxigraph as RDF triples.

Next step: integrate this script into ABI as a Dagster pipeline to automate daily data collection.

Features

  • Fetch RSS news from OpenAI and Anthropic
  • Store data in ai_news.json
  • Upload triples to Oxigraph (http://localhost:7878)
  • Log all executions (news_log.txt)
  • Will replace Windows Task Scheduler with Dagster job

🎯 Goal
Provide an automated daily ingestion pipeline to keep ABI updated with AI model news.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

Status

🏗 In progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions