detection-sigma

name detection-sigma description Generic detection rule creation and management using Sigma, the universal SIEM rule format. Sigma provides vendor-agnostic detection logic for log analysis across multiple SIEM platforms. Use when: (1) Creating detection rules for security monitoring, (2) Converting rules between SIEM platforms (Splunk, Elastic, QRadar, Sentinel), (3) Threat hunting with standardized detection patterns, (4) Building detection-as-code pipelines, (5) Mapping detections to MITRE ATT&CK tactics, (6) Implementing compliance-based monitoring rules. version 0.1.0 maintainer SirAppSec category incident-response tags ["sigma","detection","siem","threat-hunting","mitre-attack","detection-engineering","log-analysis"] frameworks ["MITRE-ATT&CK","NIST","ISO27001"] dependencies {"python":">=3.8","packages":["pysigma","pysigma-backend-splunk","pysigma-backend-elasticsearch","pyyaml"]} references ["https://github.com/SigmaHQ/sigma","https://github.com/SigmaHQ/pySigma","https://sigmahq.io/"] Sigma Detection Engineering Overview Sigma is to log detection what Snort is to network traffic and YARA is to files - a universal signature format for describing security-relevant log events. This skill helps create, validate, and convert Sigma rules for deployment across multiple SIEM platforms, enabling detection-as-code workflows. Core capabilities : Create detection rules using Sigma format Convert rules to 25+ SIEM/EDR backends (Splunk, Elastic, QRadar, Sentinel, etc.) Validate rule syntax and logic Map detections to MITRE ATT&CK framework Build threat hunting queries Implement compliance-based monitoring Quick Start Install Dependencies pip install pysigma pysigma-backend-splunk pysigma-backend-elasticsearch pyyaml Create a Basic Sigma Rule title: Suspicious PowerShell Execution id: 7d6d30b8-5b91-4b90-a71e-4f5a3f5a3c3f status: experimental description: Detects suspicious PowerShell execution with encoded commands references: - https://attack.mitre.org/techniques/T1059/001/ author: Your Name date: YYYY/MM/DD modified: YYYY/MM/DD tags: - attack.execution - attack.t1059.001 logsource: category: process_creation product: windows detection: selection: Image|endswith: '\powershell.exe' CommandLine|contains: - '-enc' - '-EncodedCommand' - 'FromBase64String' condition: selection falsepositives: - Legitimate administrative scripts level: medium Convert Rule to Target SIEM # Convert to Splunk python scripts/sigma_convert.py rule.yml --backend splunk # Convert to Elasticsearch python scripts/sigma_convert.py rule.yml --backend elasticsearch # Convert to Microsoft Sentinel python scripts/sigma_convert.py rule.yml --backend sentinel Core Workflows Workflow 1: Detection Rule Development Progress: [ ] 1. Identify detection requirement from threat intelligence or compliance [ ] 2. Research log sources and field mappings for target environment [ ] 3. Create Sigma rule using standard template [ ] 4. Validate rule syntax: python scripts/sigma_validate.py rule.yml [ ] 5. Test rule against sample logs or historical data [ ] 6. Convert to target SIEM format [ ] 7. Deploy and tune based on false positive rate [ ] 8. Document rule metadata and MITRE ATT&CK mapping Work through each step systematically. Check off completed items. Workflow 2: Threat Hunting Rule Creation For proactive threat hunting based on TTPs: Select MITRE ATT&CK Technique Review threat intelligence for relevant TTPs Identify technique ID (e.g., T1059.001 - PowerShell) See references/mitre-attack-mapping.md for common techniques Identify Log Sources Determine which logs capture the technique Map log source categories (process_creation, network_connection, file_event) Verify log source availability in your environment Define Detection Logic Create selection criteria matching suspicious patterns Add filters to reduce false positives Use field modifiers for robust matching (endswith, contains, re) Validate and Test Run validation: python scripts/sigma_validate.py hunting-rule.yml Test against known-good and known-bad samples Tune detection logic based on results Document and Deploy Add references to threat reports Document false positive scenarios Convert and deploy to production SIEM Workflow 3: Bulk Rule Conversion When migrating between SIEM platforms: # Validate all rules first python scripts/sigma_validate.py --directory rules/ --report validation-report.json # Convert entire rule set python scripts/sigma_convert.py --directory rules/ --backend splunk --output converted/ # Generate deployment report python scripts/sigma_convert.py --directory rules/ --backend splunk --report conversion-report.md Review conversion report for: Successfully converted rules Rules requiring manual adjustment Unsupported field mappings Backend-specific limitations Workflow 4: Compliance-Based Detection For implementing compliance monitoring (PCI-DSS, NIST, ISO 27001): Map Requirements to Detections Identify compliance control requirements Determine required log monitoring See references/compliance-mappings.md Create Detection Rules Use compliance rule templates from assets/compliance-rules/ Tag rules with compliance framework (e.g., tags: [pci-dss.10.2.5]) Set appropriate severity levels Validate Coverage Run: python scripts/compliance_coverage.py --framework pci-dss Review coverage gaps Create additional rules as needed Generate Compliance Report Document detection coverage by control Include sample queries and expected alerts Maintain audit trail for compliance evidence Rule Structure Reference Required Fields title: Human-readable rule name id: UUID (generate with: python -c "import uuid; print(uuid.uuid4())" ) status: stable|test|experimental|deprecated description: Detailed description of what this detects author: Your Name date: YYYY/MM/DD modified: YYYY/MM/DD logsource: category: process_creation|network_connection|file_event|... product: windows|linux|macos|azure|aws|... detection: selection: FieldName: value condition: selection level: informational|low|medium|high|critical Optional Fields references: - https://attack.mitre.org/techniques/T1059/ tags: - attack.execution - attack.t1059.001 falsepositives: - Legitimate use cases fields: - CommandLine - User - ParentImage Detection Conditions # Simple selection detection: selection: Field: value condition: selection # Multiple conditions (AND) detection: selection: Field1: value1 Field2: value2 condition: selection # OR conditions detection: selection1: Field: value1 selection2: Field: value2 condition: selection1 or selection2 # NOT conditions detection: selection: Field: suspicious_value filter: Field: legitimate_value condition: selection and not filter # Complex logic detection: selection: EventID: 4688 suspicious_cmd: CommandLine|contains: - 'powershell' - 'cmd.exe' filter_legitimate: ParentImage|endswith: '\explorer.exe' condition: selection and suspicious_cmd and not filter_legitimate Field Modifiers Common modifiers for flexible matching: |contains - Contains substring (case-insensitive) |endswith - Ends with string |startswith - Starts with string |re - Regular expression match |all - All values must match |base64 - Base64-encoded value matching |base64offset - Base64 with offset variations Example: detection: selection: CommandLine|contains|all: - 'powershell' - '-enc' Image|endswith: '\powershell.exe' Security Considerations Sensitive Data Handling : Sigma rules may reference sensitive field names or patterns. Store rules in version control with appropriate access controls. Avoid including actual sensitive data in example values. Access Control : Detection rules reveal defensive capabilities to adversaries. Implement role-based access for rule repositories. Limit rule modification to authorized detection engineers. Audit Logging : Log all rule deployments, modifications, and deletions. Track who deployed which rules to which systems. Maintain change history for compliance auditing. Compliance : Sigma rules support compliance monitoring (PCI-DSS 10.2, NIST SP 800-53 AU family, ISO 27001 A.12.4). Document rule-to-control mappings for audit evidence. Safe Defaults : Use conservative false positive filtering in production. Start rules at "experimental" status. Test thoroughly in test environment before production deployment. Bundled Resources Scripts scripts/sigma_convert.py - Convert Sigma rules to target SIEM backend formats scripts/sigma_validate.py - Validate Sigma rule syntax and detect common errors scripts/compliance_coverage.py - Analyze detection coverage for compliance frameworks scripts/generate_rule_template.py - Generate Sigma rule template with MITRE ATT&CK tags References references/mitre-attack-mapping.md - Common MITRE ATT&CK techniques and Sigma detection patterns references/log-source-guide.md - Log source categories, products, and field mappings references/compliance-mappings.md - Compliance framework to detection rule mappings references/backend-support.md - Supported SIEM backends and conversion capabilities references/field-modifiers.md - Comprehensive guide to Sigma field modifiers and regex patterns Assets assets/rule-templates/ - Pre-built Sigma rule templates for common attack patterns lateral-movement.yml - Lateral movement detection template privilege-escalation.yml - Privilege escalation detection template persistence.yml - Persistence mechanism detection template credential-access.yml - Credential dumping detection template assets/compliance-rules/ - Compliance-focused rule templates pci-dss-monitoring.yml - PCI-DSS monitoring requirements nist-800-53-audit.yml - NIST 800-53 audit logging requirements iso27001-logging.yml - ISO 27001 logging and monitoring Common Detection Patterns Pattern 1: Process Execution Monitoring Detect suspicious process creation with command-line analysis: logsource: category: process_creation product: windows detection: selection: Image|endswith: - '\powershell.exe' - '\cmd.exe' CommandLine|contains: - 'Invoke-' - 'IEX' - 'FromBase64String' Pattern 2: Network Connection Monitoring Detect suspicious outbound connections: logsource: category: network_connection product: windows detection: selection: Initiated: 'true' DestinationPort: - 4444 - 5555 - 8080 filter: DestinationIp|startswith: - '10.' - '172.16.' - '192.168.' condition: selection and not filter Pattern 3: File Event Monitoring Detect file creation in suspicious locations: logsource: category: file_event product: windows detection: selection: TargetFilename|contains: - '\Windows\Temp\' - '\AppData\Roaming\' TargetFilename|endswith: - '.exe' - '.dll' - '.ps1' Integration Points CI/CD Integration Build detection-as-code pipelines: # .github/workflows/sigma-validation.yml name: Sigma Rule Validation on: [ push , pull_request ] jobs: validate: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Validate Sigma Rules run: | pip install pysigma python scripts/sigma_validate.py --directory rules/ - name: Convert to Production Format run: | python scripts/sigma_convert.py --directory rules/ --backend splunk --output converted/ SIEM Deployment Automated rule deployment: Splunk: Use Splunk REST API or splunk-sdk for savedsearches Elasticsearch: Convert to EQL and deploy via Kibana API Microsoft Sentinel: Convert to KQL and deploy via Azure API QRadar: Convert to AQL and deploy via QRadar API See references/backend-support.md for deployment examples. Threat Intelligence Integration Enrich rules with threat intel: Tag rules with threat actor TTPs Reference threat reports and IOCs Map to MITRE ATT&CK techniques Track rule effectiveness against known threats Troubleshooting Issue: Conversion Fails for Specific Backend Solution : Check backend compatibility and field mappings. Some backends have limitations: Review references/backend-support.md for known limitations Use sigma_convert.py --backend <backend> --debug for detailed error output Check if field names are supported in target backend Consider custom pipeline transformations for unsupported fields Issue: High False Positive Rate Solution : Refine detection logic with additional filters: Review false positive patterns Add exclusion filters for legitimate use cases Use more specific field modifiers (e.g., |endswith vs |contains ) Consider time-based correlation for behavioral detection Test with historical data to validate tuning Issue: Rule Not Triggering in Target SIEM Solution : Verify log source availability and field mappings: Confirm log source is ingested: Check SIEM data sources Verify field names match: Use sigma_convert.py --show-fields to see mapping Test converted query directly in SIEM Check for case sensitivity issues in field values Validate time window and search scope in SIEM MITRE ATT&CK Integration Tag rules with ATT&CK tactics and techniques: tags: - attack.execution # Tactic - attack.t1059.001 # Technique: PowerShell - attack.defense_evasion # Additional tactic - attack.t1027 # Technique: Obfuscated Files Common tactic tags: attack.initial_access attack.execution attack.persistence attack.privilege_escalation attack.defense_evasion attack.credential_access attack.discovery attack.lateral_movement attack.collection attack.exfiltration attack.command_and_control attack.impact For detailed technique mappings, see references/mitre-attack-mapping.md . Best Practices Start with Community Rules : Use SigmaHQ repository (3000+ peer-reviewed rules) as foundation Version Control : Store rules in Git with meaningful commit messages Test Before Deploy : Validate against historical data in test environment Document Tuning : Track false positive patterns and tuning decisions Map to Frameworks : Tag all rules with MITRE ATT&CK and compliance mappings Automate Validation : Use CI/CD to validate rules on every change Monitor Effectiveness : Track rule trigger rates and true positive rates Regular Updates : Review and update rules based on new threat intelligence References Sigma Specification SigmaHQ Rule Repository pySigma Documentation Sigma Converter Web Tool MITRE ATT&CK Framework