AWS CloudTrail Logs Integration

Overview

AWS CloudTrail logs API calls and account activity across your AWS infrastructure. This integration forwards CloudTrail events to Qorrelate for security monitoring, compliance auditing, and operational troubleshooting.

Prerequisites

• CloudTrail trail configured in your AWS account
• S3 bucket or CloudWatch Logs for CloudTrail delivery
• Lambda execution role with appropriate permissions
• Your Qorrelate API endpoint and organization ID

1. Create or Verify CloudTrail Trail

# Create a new trail
aws cloudtrail create-trail \
  --name qorrelate-audit-trail \
  --s3-bucket-name your-cloudtrail-bucket \
  --is-multi-region-trail \
  --enable-log-file-validation

# Start logging
aws cloudtrail start-logging --name qorrelate-audit-trail

2. Create Lambda Function

import boto3
import gzip
import json
import os
import requests
from datetime import datetime

s3 = boto3.client('s3')

def lambda_handler(event, context):
    logs = []
    
    for record in event['Records']:
        bucket = record['s3']['bucket']['name']
        key = record['s3']['object']['key']
        
        response = s3.get_object(Bucket=bucket, Key=key)
        content = gzip.decompress(response['Body'].read()).decode('utf-8')
        cloudtrail_data = json.loads(content)
        
        for event_record in cloudtrail_data.get('Records', []):
            event_name = event_record.get('eventName', '')
            error_code = event_record.get('errorCode')
            
            if error_code:
                severity = "ERROR"
            elif any(x in event_name.lower() for x in ['delete', 'terminate', 'revoke']):
                severity = "WARN"
            else:
                severity = "INFO"
            
            logs.append({
                "timestamp": event_record.get('eventTime'),
                "body": json.dumps(event_record),
                "severity_text": severity,
                "attributes": {
                    "source": "aws-cloudtrail",
                    "event_name": event_name,
                    "event_source": event_record.get('eventSource'),
                    "aws_region": event_record.get('awsRegion'),
                    "user_identity": event_record.get('userIdentity', {}).get('arn'),
                    "source_ip": event_record.get('sourceIPAddress'),
                    "error_code": error_code
                }
            })
    
    if logs:
        headers = {
            "Content-Type": "application/json",
            "Authorization": f"Bearer {os.environ['QORRELATE_API_KEY']}",
            "X-Organization-Id": os.environ["QORRELATE_ORG_ID"]
        }
        
        for i in range(0, len(logs), 100):
            batch = logs[i:i+100]
            requests.post(
                f"{os.environ['QORRELATE_ENDPOINT']}/v1/logs",
                headers=headers,
                json={"logs": batch}
            )
    
    return {'statusCode': 200, 'body': f'Processed {len(logs)} events'}

3. Configure S3 Event Trigger

aws s3api put-bucket-notification-configuration \
  --bucket your-cloudtrail-bucket \
  --notification-configuration '{
    "LambdaFunctionConfigurations": [
      {
        "LambdaFunctionArn": "arn:aws:lambda:region:account:function:cloudtrail-to-qorrelate",
        "Events": ["s3:ObjectCreated:*"],
        "Filter": {
          "Key": {
            "FilterRules": [
              {"Name": "suffix", "Value": ".json.gz"}
            ]
          }
        }
      }
    ]
  }'

Alternative: CloudWatch Logs Subscription

# Enable CloudWatch Logs delivery for CloudTrail
aws cloudtrail update-trail \
  --name qorrelate-audit-trail \
  --cloud-watch-logs-log-group-arn "arn:aws:logs:region:account:log-group:cloudtrail-logs" \
  --cloud-watch-logs-role-arn "arn:aws:iam::account:role/cloudtrail-cloudwatch-role"

# Create subscription filter
aws logs put-subscription-filter \
  --log-group-name cloudtrail-logs \
  --filter-name qorrelate-forwarder \
  --filter-pattern "" \
  --destination-arn "arn:aws:lambda:region:account:function:cloudtrail-to-qorrelate"

Security-Focused Queries

# Find all failed API calls
source:aws-cloudtrail AND error_code:*

# Track IAM changes
source:aws-cloudtrail AND event_source:iam.amazonaws.com

# Monitor root account usage
source:aws-cloudtrail AND user_identity:*root*

# Detect unauthorized access attempts
source:aws-cloudtrail AND error_code:AccessDenied

# Track console logins
source:aws-cloudtrail AND event_name:ConsoleLogin

Verifying the Integration

1. Deploy the Lambda function
2. Configure S3 or CloudWatch Logs trigger
3. Perform an action in AWS (e.g., list S3 buckets)
4. Wait 5-15 minutes for CloudTrail to deliver logs
5. Search in Qorrelate: source:aws-cloudtrail