AMDStackGuard

This repository implements a proof-of-concept (PoC) Windows kernel driver as part of an ongoing research effort on hardware-assisted control flow integrity, designed to validate the integrity of user-mode call stacks from ring 0 on AMD64 processors.

The project serves as a fundamental software layer for a broader research initiative on Execution-context consistency validation using hardware-derived signals. The current implementation focuses on secure introspection of user memory pages and detection of simulated return address spoofing anomalies.

Tested on Windows 10 22H2

Research Context

Modern adversarial techniques (such as stack spoofing) manipulate the RSP register to decouple the logical call stack from the actual execution flow, effectively blinding traditional security tools (EDR/Anti-Cheats) that rely on stack traversal.

Research question

Can discrepancies between architecturally visible call stack state and hardware-derived execution traces be used as a reliable indicator of user-mode stack manipulation at runtime?

This research investigates whether execution signals provided by the CPU can be used as a reliable ground truth to validate the integrity of user-mode call stacks, addressing the limitations of existing software-only control flow integrity (CFI) mechanisms, which rely solely on memory-based stack traversal.

Secondary research questions

To what extent can discrepancies between hardware-derived execution flow and memory-resident stack state be used to identify return address manipulation techniques, such as stack spoofing?

What kinds of stack manipulation attacks are detectable in this model, and which ones fall outside its scope?

What is the practical performance overhead of such validation when implemented in a real-time monitoring context?

Research Hypotheses

Hardware-assisted execution sampling can provide a more reliable reference for validating call stack integrity than memory-only approaches, enabling the detection of certain stack manipulation attacks with few false positives and manageable runtime overhead.

Research Claim

AMDStackGuard proposes a hardware-assisted validation model and implements the foundational components required to experimentally evaluate whether user-mode stack spoofing can be detected through execution-context consistency checks such as

1. Memory-resident call stack state (UserRsp)
2. Hardware-derived execution flow indicators (IBS)

Under the stated threat model, a mismatch between these two signals constitutes a strong indicator of stack manipulation that cannot be trivially falsified from user mode.

Research Status

The current prototype validates the memory-resident stack invariant ([RSP] == ExpectedReturn) from kernel mode.

Integration of AMD Instruction-Based Sampling (IBS) for hardware-derived execution trace correlation is planned for the next research phase.

This repository represents Phase 1 of the research roadmap.

Threat model

Attacker capabilities

The attacker is assumed to:

• Execute arbitrary code in user mode within a target process.

• Perform stack manipulation techniques, such as:

  • Overwriting the return address.

  • Stack pointer redirection (RSP) to decouple logical call stacks from the actual execution flow.

  • Attempt to evade user mode security mechanisms and stack-based detection systems.

• The attacker does not have kernel mode execution privileges.

Defender capabilities

The defender:

• Operates a trusted kernel mode driver running at PASSIVE_LEVEL.

• Has read-only introspection access to user mode memory.

• Can collect or correlate execution metadata provided by the hardware (e.g., through performance monitoring functions such as IBS).

• Does not rely on compiler instrumentation, binary rewriting, or application cooperation.

Assumptions

• The kernel and underlying operating system are trusted and uncompromised.

• It is assumed that hardware execution data is more difficult for an adversary in user mode to manipulate than stack structures residing in memory.

• The system is evaluated on commercial AMD64 hardware without the need for specialized firmware modifications.

Out of scope

The following items are explicitly considered out of scope:

• Attackers in kernel mode or adversaries at the hypervisor level.

• Attacks that fully control or spoof hardware performance monitoring tools.

• Non-stack-based control flow hijacking techniques (e.g., JOP without stack corruption).

• Mitigation of speculative execution vulnerabilities unrelated to call stack integrity.

Core Invariant

This research is based on the following assumption:

Under normal execution, the return address stored at the user-mode stack pointer (RSP) must be consistent with the execution flow observed through hardware-assisted execution sampling. While not all discrepancies necessarily imply malicious intent, persistent or structured divergence between memory-resident stack state and hardware-derived execution data constitutes a strong indicator of stack manipulation.

Any persistent divergence between these two representations indicates intentional manipulation of the call stack.

Scope of the Current Prototype

The current implementation serves as a proof-of-concept designed to validate the feasibility of the proposed approach. It intentionally limits:

• Sampling frequency
• Attack coverage
• Performance evaluation depth

These aspects are deferred to subsequent research phases.

Technical Architecture

The solution consists of two components:

1. Kernel driver (.sys):

• Implements safe memory checking using ProbeForRead and structured exception handling (SEH) to prevent blue screens of death when accessing paged user memory
• Exposes an IOCTL interface to validate a specific memory address against an expected value
• Operates at PASSIVE_LEVEL for initial testing stability

2. CLI/user mode simulation:

• Validation client: Retrieves the actual return address and passes the stack pointer (RSP) to the driver for verification.
  • Adversarial simulation: Includes a routine to artificially modify the return address on the stack (spoofing), demonstrating how the driver detects the discrepancy between memory contents and the expected execution flow.

Current Capabilities

• Secure memory introspection: The driver correctly reads memory in user mode without blocking the system, gracefully handling page faults.
• Anomaly detection logic: Implements the comparison logic [RSP] == ExpectedReturn.
• Adversarial test case: Correctly flags a manipulated stack where the return address has been overwritten.

Note: This driver requires test signing mode to be enabled (bcdedit /set testsigning on).

Disclaimer

This code is intended for educational and research purposes only. Its purpose is to demonstrate core programming concepts and memory management techniques.

TODO

• MDL support:
• Transition from ProbeForRead to IoAllocateMdl / MmProbeAndLockPages for robust memory access during high IRQL scenarios (preparation for PMI drivers).
• MSR configuration:
• Implement __writemsr logic to enable IBS execution sampling (IbsOpCtl).
• Interrupt handling:
• Register a callback for performance monitor interrupts (PMI) or examine IBS logs periodically.
• Data correlation:
• Correlate IbsBrTarget (hardware truth) with the value in [UserRsp] (memory truth).
• Perform overhead analysis using benchmarks (e.g., frame rate impact on graphics applications).