Microsoft Sentinel Data Lake Notebooks

Security analysis notebooks for Microsoft Sentinel Data Lake using PySpark and advanced threat hunting techniques.

Notebooks

Notebook	Description
01_Data_Exfiltration_Early_Warning	Compression staging, suspicious uploads, egress spikes, storage audit anomalies
02_Identity_Security_Analysis	Authentication threats, brute force, impossible travel, user behavior
03_Device_Security_Analysis	Endpoint security, credential dumping, lateral movement
04_Advanced_Threat_Hunting	C2 beacon detection, living off the land, data exfiltration, UBA
05_Security_Events_Analysis	Windows security events, authentication, account management
06_ServicePrincipal_SignIn_Analysis	Service principal anomalies, burst detection, risk scoring
07_Notebook_Examples_Scenarios	Guided Sentinel walkthroughs (Entra groups, unusual sign-ins, brute-force)
08_Anomalous_SignIn_Detection	Failed-then-success patterns, impossible travel, threat intel enrichment
09_Shai_Hulud_2_0_Supply_Chain_Hunting	Worm-style lateral movement, credential fan-out, LOLBin abuse

Quick Start

1. Prerequisites: Microsoft Sentinel Data Lake access, VS Code with Sentinel extension, authenticated to Azure
2. Configure: Open any notebook and update the WORKSPACE variable in the configuration cell:

   WORKSPACE = "your-workspace-name"    # 👈 Your Log Analytics workspace name

3. Run: Execute cells in order — each notebook is self-contained

Note: Entra ID asset tables (EntraUsers, EntraGroups, etc.) are read from the "System Tables" data lake tier automatically. Only the Log Analytics workspace name needs to be set.

Troubleshooting

• "Table not found": Check workspace name and verify Data Lake onboarding
• Import errors: Restart kernel, verify Sentinel extension authentication
• Performance issues: Use smaller time windows or larger runtime pool

Runtime Pool Guide

Pool	Use Case
Small	Development, testing (< 1 GB data)
Medium	Regular analysis (1–10 GB data)
Large	Extensive hunting (> 10 GB data)