Cyber security research #636
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 4 commits
September 16, 2025 16:03
🔒 New Features: - Expert cyber security analyst agent with specialized knowledge - Comprehensive ontologies: ThreatLandscape, VulnerabilityManagement, SecurityControls - Advanced ontology reading tools for accessing specialized knowledge - 4 specialized workflows: ThreatAssessment, IncidentResponse, VulnerabilityAssessment, SecurityArchitecture - Integration with major security frameworks (NIST, MITRE ATT&CK, OWASP, ISO 27001) 🛠 Technical Implementation: - Full Agent class implementation (converted from template) - Working create_agent function with OpenAI model integration - Custom ontology reading and searching tools - Makefile integration: make chat-cyber-security-analyst-agent - Config.yaml registration and enablement - Comprehensive documentation with TL;DR 🎯 Capabilities: - Threat intelligence and analysis - Vulnerability assessment and management - Incident response and forensics - Security architecture and controls - Risk management and compliance - Industry framework expertise The agent can now access its specialized knowledge base through ontology tools, making it significantly more powerful for cyber security analysis and consultation.
- Updated directory structure from src/marketplace/modules/domains to src/marketplace/domains - Preserved all cyber-security-analyst work in new location - Updated config.yaml to include cyber-security-analyst module - Resolved merge conflicts with main branch updates
- Restored test1 and test2 directories from backup - Includes the STIX2 JSON files and analysis data - Preserves all work in progress on cyber security analysis
Adds a comprehensive Cyber Security Analyst domain expert to the ABI marketplace. Features: - CyberSecurityAnalystAgent: Expert AI agent for cyber security analysis and consultation - Specialized ontologies: ThreatLandscape, VulnerabilityManagement, SecurityControls, D3FEND - Four specialized workflows: ThreatAssessment, IncidentResponse, VulnerabilityAssessment, SecurityArchitecture - Integration with major security frameworks (NIST, MITRE ATT&CK, OWASP, ISO 27001, D3FEND) - Cyber event analysis pipeline with STIX2 support Technical implementation: - Advanced ontology reading and SPARQL query tools for knowledge base access - Real-world cyber event dataset with 20 major 2025 security incidents - D3FEND defensive technique mapping for threat mitigation recommendations - MISP integration for threat intelligence platform connectivity - Conversational and SPARQL query agents for flexible interaction Files added: - src/marketplace/domains/cyber-security-analyst/agents/CyberSecurityAnalystAgent.py - src/marketplace/domains/cyber-security-analyst/ontologies/ (ThreatLandscape, D3FEND, etc.) - src/marketplace/domains/cyber-security-analyst/workflows/ (4 specialized workflows) - src/marketplace/domains/cyber-security-analyst/pipelines/ (data pipeline infrastructure) - src/marketplace/domains/cyber-security-analyst/events.yaml (20 major cyber events) - src/marketplace/domains/cyber-security-analyst/integrations/MISPIntegration.py Configuration: - Added to config.yaml marketplace modules - Excluded from mypy checks due to existing structural issues in legacy code
added 12 commits
October 7, 2025 15:32
- Implement GraphEnrichmentWorkflow with missing process detection - Add temporal ordering inference using BFO:precedes - Include data quality validation via CCO semantics - Enhance CyberSecurityAnalystAgent with 3 new enrichment tools - Enable kill chain reconstruction and attack timeline analysis - Update system prompt with semantic enrichment capabilities
- Remove 93K+ lines of bloat (workflows, pipelines, apps, integrations) - Create CyberSecurityAgent.py following templatablesparqlquery pattern - Add CyberSecurityQueries.ttl with 7 competency questions as SPARQL tools - Create minimal d3fend-subset.ttl (11 techniques) that actually loads - Keep full d3fend.ttl for reference - Module now: 1 agent, 1 query ontology, 1 data file (events.yaml) - Tools auto-generated from ontology like ABI core
- Remove legacy/ directory with old agents, pipelines, ontologies - Create samples/ directory for data - Move events.yaml to samples/events.yaml - Final structure: agents/, ontologies/, samples/, README, requirements
- Create LoadEventsDataPipeline to transform YAML → RDF triples - Use CSE namespace for cyber security events ontology - Map events to D3FEND attack vectors and defensive techniques - Hook pipeline into module's on_initialized() for auto-loading - Events data now populates Oxigraph on module startup
- Delete requirements.txt (all deps in root pyproject.toml) - Strip __init__.py to only on_initialized() hook - No exports needed - agents auto-discovered by Module.py scanner
- Load 22 competency questions from cqs.yaml - Create list_questions tool to show all available questions - Create answer_question tool to execute SPARQL by number - Map CQ categories to appropriate SPARQL queries - Return 'I don't know' when no data exists - Log SPARQL execution for visibility - Agent responds to 'what questions can you answer?'
- Convert all competency questions from cqs.yaml to TemplatableSparqlQuery - Each CQ mapped to specific SPARQL query against events data - Agent now uses standard get_tools() pattern - Remove custom SPARQL generation logic - Clean separation: TTL defines queries, agent loads them - Follow established ABI pattern for query-driven agents
- Replace incorrect abi.services.chat_model imports - Use langchain_openai.ChatOpenAI for cloud/airgap models - Use langchain_ollama.ChatOllama for local models - Follow pattern from ABI core agents - Support all three AI_MODE values: cloud, local, airgap
- Change abi: prefix to intentMapping: in CyberSecurityQueries.ttl - TemplatableSparqlQuery system expects intentMapping namespace - Remove ontology_file_path param from get_tools() call - Tools are loaded from triplestore via module auto-discovery - TTL files in /ontologies dir are automatically loaded
Testing agent behavior when missing required fields
giacomodecolle
requested changes
Oct 12, 2025
giacomodecolle
left a comment
giacomodecolle
left a comment
There was a problem hiding this comment.
Most SPARQL queries are not answering competency questions. Some other comments on the competency questions as well. The D3FEND snippet has classes which I can't find in the last D3FEND release
|
|
||
| ### Storage Structure | ||
| ``` | ||
| /storage/datastore/cyber/ |
There was a problem hiding this comment.
I can't find this under storage/datastore
| rdfs:label "Defensive Technique" ; | ||
| rdfs:comment "A defensive countermeasure technique defined by D3FEND" . | ||
|
|
||
| d3f:AttackVector a owl:Class ; |
There was a problem hiding this comment.
I can't find this in the current version of d3fend
| rdfs:comment "A method or pathway used by attackers" . | ||
|
|
||
| # D3-SWID: Software Installation Discovery | ||
| d3f:D3-SWID a owl:Class ; |
| rdfs:label "Hardware-based Process Isolation" ; | ||
| rdfs:comment "Using hardware features to isolate processes and prevent unauthorized access" . | ||
|
|
||
| # D3-CSPP: Credential Strength Policy |
| rdfs:label "Multi-Factor Authentication" ; | ||
| rdfs:comment "Requiring multiple forms of authentication to verify user identity" . | ||
|
|
||
| # D3-BDI: Backup and Data Recovery |
| PREFIX cse: <https://abi.cyber-security-events.org/ontology/> | ||
| PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> | ||
|
|
||
| SELECT ?event ?eventName (COUNT(?attackVector) as ?vectorCount) (COUNT(?technique) as ?techniqueCount) WHERE { |
There was a problem hiding this comment.
This I think would need to be tested
| PREFIX cse: <https://abi.cyber-security-events.org/ontology/> | ||
| PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> | ||
|
|
||
| SELECT ?event ?eventName ?date ?severity WHERE { |
| PREFIX cse: <https://abi.cyber-security-events.org/ontology/> | ||
| PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> | ||
|
|
||
| SELECT ?event ?eventName (COUNT(?property) as ?propertyCount) WHERE { |
There was a problem hiding this comment.
This needs to be a CONSTRUCT or INSERT
| PREFIX cse: <https://abi.cyber-security-events.org/ontology/> | ||
| PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> | ||
|
|
||
| SELECT ?event ?eventName ?sourceTitle WHERE { |
| intentMapping:sparqlTemplate """ | ||
| PREFIX cse: <https://abi.cyber-security-events.org/ontology/> | ||
| PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> | ||
|
|
|
TODO:
|
| severity: "critical" #try to remove this and see if it work when I ask the agent to answer the question "" | ||
| category: "supply_chain_attack" | ||
| description: "Sophisticated supply chain attack targeting multiple government agencies through compromised software updates" | ||
| affected_sectors: ["government", "defense", "technology"] |
There was a problem hiding this comment.
Nothing in d3fend to map this into. Let's just create an ICE class called "affected sector" and store these as strings.
|
@jravenel The terms to use to rebuild the slim and then the YAML are:
for relations, we need at least
But if we are able to grab all the sub-relations of d3f:associatedwith it would be best All of the classes and some relations have also been mapped into CCO/BFO already. |
Contributor
Author
|
Ok. Will keep you posted ! Thanks @giacomodecolle |
SPARQL and CQ refactoring
- Generate realistic attack scenarios based on Stuxnet, SolarStorm, HeartBleed, APT29, and Lazarus Group - Create 1,200+ lines of TTL instances covering all competency queries - Include temporal attack chains, artifact relationships, and network correlations - Add detailed analysis document explaining data patterns and query coverage - Enable enterprise-grade cybersecurity analysis and training
jravenel
force-pushed
the
cyber-security
branch
from
06f8335 to
d2b0b31
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Opensourcing cyber security research work with UB