Exploring Threat Analysis in Amazon GuardDuty with Amazon Q Developer

Introduction

Hello! I'm Otaka from KINTO Technologies' Cloud Security Group.
My daily work involves establishing cloud environment guardrails and monitoring and improvement activities using CSPM and threat detection.

As part of our group's daily operations, we perform detailed log analysis when Amazon GuardDuty detects threats to verify whether they represent normal operations. To streamline this log analysis process, I conducted a verification using Amazon Q Developer.

Responding to Threat Detection with Amazon GuardDuty

At KINTO Technologies, we use Amazon GuardDuty for threat detection in our AWS environment.
When Amazon GuardDuty detects a threat, notifications are sent to a Slack channel. Upon receiving these notifications, the security team performs detailed log analysis. When necessary, we interview the relevant users and record progress in JIRA tickets to manage response status.

Current Amazon GuardDuty threat detection response flow

In this initiative, we aim to improve efficiency in the log analysis task within the threat detection response flow by utilizing Amazon Q Developer.

Amazon GuardDuty threat detection response flow incorporating AI-based log analysis

What is Amazon Q Developer

Amazon Q Developer is a development assistant AI provided by AWS.
It specializes in cloud development using AWS and supports various use cases such as:

• Investigating resources, events, and activities in AWS environments
• Troubleshooting AWS environments
• Development, debugging, and coding assistance
• Document creation

Amazon Q Developer can analyze related logs for threats detected by Amazon GuardDuty even in its default state. For example, when requesting log investigation along with the Findings ID of a threat detected by Amazon GuardDuty, Amazon Q Developer autonomously conducts investigation and analysis using built-in tools as shown below.
(The followings are excerpts of the log investigation process performed by Amazon Q Developer.)

Amazon Q Developer autonomously performing log analysis

The report generated by Amazon Q Developer summarizes the detected threats, including the user who performed the operation and affected resources, and also includes recommended response suggestions.

Analysis report output by Amazon Q Developer in default state

In actual operations, we also check and analyze the authentication status of the user who performed the action and the log of the operation time to determine whether it was a normal activity. In addition to the above, further investigation is required.
Also, while conducting detailed log investigation and analysis through repeated interactions with Amazon Q Developer via chat, various challenges became apparent.

Challenges in Log Analysis

While Amazon Q Developer can analyze threats detected by Amazon GuardDuty even in its default state, there are many challenges in performing practical-level detailed log analysis.

1. AWS API Throttling Limits

When Amazon Q Developer receives error responses to API requests executed using built-in tools or tools provided by MCP servers, it autonomously makes corrections and resends the API request.
For example, if API request syntax is incorrect, it autonomously corrects the command and resends the request. However, if API request retries become too frequent, the throttling limit may be reached, which can prevent the necessary information from being obtained and ultimately lead to inaccurate or unexpected responses.

API requests executed by Amazon Q Developer reach throttling limits, preventing information retrieval.

2. Collected Log Limitations

Amazon Q Developer can search and analyze logs stored in AWS CloudTrail, but cannot analyze logs that haven't been collected in the first place.
AWS CloudTrail automatically collects management event logs (such as IAM user creation or EC2 instance launches) by default, but data event logs (such as file uploads to S3 buckets and Lambda function executions) are not collected unless manually enabled. Therefore, even when Amazon GuardDuty detects suspicious access to S3 buckets or Lambda functions, if data event log collection hasn't been enabled beforehand, searching AWS CloudTrail logs with Amazon Q Developer cannot retrieve information necessary for detailed analysis.
If Amazon Q Developer cannot obtain information necessary for log analysis, it may generate inaccurate responses based on speculation.

Amazon Q Developer attempts to search data events even though data event collection is not enabled in AWS CloudTrail.

Since collecting and storing logs in AWS environments incurs costs, organizations may intentionally not collect or store certain logs based on budget constraints. It's necessary to organize which logs Amazon Q Developer can utilize when analyzing threats detected by Amazon GuardDuty, considering the current log collection status in the AWS environment.

3. Context Window Limitations

The length of text that generative AI can process at once is called the context window, representing the AI's memory.Context windows are counted in units called tokens, and within a certain token count range including past interactions, the AI can generate responses based on previous conversation content.
When extremely long text is input into prompts or long interactions continue with the generative AI, and the total token count exceeds the context window range, Amazon Q Developer forgets previously given instructions, potentially failing to provide expected responses or outputting inaccurate information.

When context window is exceeded, Amazon Q Developer automatically summarizes past information and forgets detailed content

Amazon Q Developer also has context window limitations, currently set at 200K tokens (as of October 16, 2025).
Token counting includes information obtained through API requests and responses. It's necessary to not only avoid unnecessary prompt input but also avoid repeating unnecessary API calls that consume unnecessary tokens.

4. Output Consistency Challenges

Generative AI produces different output formats and included information each time, even when given identical prompts. When instructing Amazon Q Developer to generate log analysis reports, the information included in reports and report structure may differ each time.
In such cases, it becomes difficult to obtain necessary information and consider responses based on reports.

Approaches to Overcome Challenges

As shown above, there are many challenges in performing detailed analysis of threats detected by Amazon GuardDuty using Amazon Q Developer.
To overcome these challenges and make Amazon Q Developer operate as expected, improvements are needed through two approaches:

• Enable Amazon Q Developer to efficiently collect information necessary for detailed log analysis and generate responses based on accurate information.
• Provide Amazon Q Developer with detailed instructions regarding procedures for conducting detailed log analysis and necessary operations, enabling it to execute tasks similar to actual analysis work.

Specifically, improvements were made through MCP integration and prompt adjustments for Amazon Q Developer.

1. MCP Integration

MCP (Model Context Protocol) is a standard protocol defined for generative AI to integrate with external tools and data sources. Amazon Q Developer supports MCP. By specifying the MCP servers to integrate in the configuration files and launching Amazon Q Developer, you can use external tools and reference data.

For detailed log analysis of threats detected by Amazon GuardDuty, the following tools are integrated with Amazon Q Developer.

At our company, we have built a SIEM (Security Information and Event Management) using Splunk Cloud, aggregating various logs such as AWS CloudTrail logs and Entra ID authentication logs. By integrating with the Splunk MCP Server provided by Splunk and providing tools that can execute Splunk Cloud search queries, Amazon Q Developer can perform detailed cross-sectional analysis not limited to logs stored in AWS CloudTrail.

2. Prompt Engineering

Prompt engineering is a technique for eliciting intended responses and improving response accuracy by carefully designing the input provided to generative AI. By providing generative AI with instruction background and additional information, or giving detailed instructions regarding task execution, adjustments are made to generate expected responses.

For detailed log analysis of threats detected by Amazon GuardDuty, response accuracy is improved by providing Amazon Q Developer with the following information through prompts.

In prompt engineering, insufficient information provided to generative AI may not effectively adjust output content. Prompts were adjusted to the appropriate level of detail by specifying conditions and instructions using generative AI.

Improvements and Results from Adjustments

After implementing the above adjustments, the following report was generated when requesting log analysis from Amazon Q Developer again. The instructions input to the prompt were the same text as when requesting analysis from Amazon Q Developer in its default state.

# GuardDuty Finding Detailed Analysis Report

**Finding ID**: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
**Detection Region**: ap-northeast-1  
**AWS Account ID**: 000000000000 
**AWS Account Name**: (Unknown)  
**Analysis Date/Time**: 2025-10-16 17:44:14  

## 1. Executive Summary

### Key Findings
- **Actual User Identification**: User Name (< User Email Address >)
- **Outsourced User**: Identified as outsourced company user by "xxxx-" prefix
- **False Positive Possibility**: High (possibility of normal work by legitimate user)
- **CloudTrail Logs**: Target AWS account logs do not exist in Splunk, detailed authentication chain tracking unavailable

### Emergency Response Necessity Assessment
- **Urgency**: Low
- **Reason**: Activity by legitimate user during normal business hours, Entra ID authentication also normal

### Key Investigation Results Summary
- Finding occurrence time: September 9, 2025 13:20:05 JST (Monday during normal business hours)
- Operation content: GetObject API execution from S3 bucket "< S3-Bucket-Name >"
- User: User Name from outsourced company
- Abnormality: Anomaly detected by GuardDuty machine learning, but legitimate operation during business hours

### Recommended Actions (Prioritized)
1. **Priority: Low** - Confirm work content with relevant user (just in case)
2. **Priority: Low** - Verify S3 access logs in detail (if necessary)
3. **Priority: Medium** - Consider adjusting GuardDuty machine learning baseline

## 2. Threat Overview

**Threat Type**: Exfiltration:S3/AnomalousBehavior  
**Severity**: 8.0 (High)  
**Detection Content**: Anomalous S3 API calls by IAM entity

The "anomalous" S3 GetObject API call detected by GuardDuty indicates deviation from machine learning baseline, but is actually determined to be operation by legitimate user during normal business hours.

## 3. Threat Details

### Detailed Attack Method Analysis
- **API**: S3 GetObject
- **Target Bucket**: < S3-Bucket-Name >
- **Authentication Method**: AuthHeader (legitimate AWS authentication)
- **User Agent**: aws-sdk-rust/1.3.8 (development tool usage)

### Attacker Profiling
Not an actual attacker, but operation by legitimate user "User Name":
- User from outsourced company ("xxx-" prefix)
- Uses AWS SSO for normal work
- Track record of work in development environment (sandbox)

### Attack Timeline Analysis
- **2025-09-09 13:20:05 JST**: S3 GetObject API execution
- **2025-09-09 13:29:43 JST**: Detection by GuardDuty
- **Same date/time**: Confirmed successful AWS SSO login in Entra ID

### Detailed Impact Scope Assessment
- **Affected Resources**: S3 bucket "< S3-Bucket-Name >"
- **Data Leakage Risk**: Low (sandbox bucket, legitimate user)
- **Lateral Movement Risk**: None
- **Persistent Threat**: None

### Comparison with Similar Attack Patterns
Anomaly detection by GuardDuty machine learning, differs from actual attack patterns:
- Use of legitimate authentication chain
- Operation during normal business hours
- Work in development environment

### Security Control Bypass Techniques
No bypass techniques used, all through legitimate authentication/authorization processes:
- SSO login via Identity Center
- Operation with appropriate role permissions
- Use of legitimate AWS SDK

## 4. Principal Information

**User Name**: < Role Name >  
**Actual User**: User Name (< User Email Address >)  
**User Type**: AssumedRole (via Identity Center)  
**Access Key ID**: < Access Key ID >  
**Principal ID**: < Principle ID>  

### Detailed User Attributes
- **Name**: User Name
- **Affiliation**: Outsourced company ("xxx-" prefix)
- **Email Address**: < User Email address>
- **Role**: < Role Name >

## 5. Abnormality Analysis

### Time-based Abnormality Assessment
- **Finding Occurrence Date**: Monday (normal business day)
- **Finding Occurrence Time**: 13:20:05 JST (normal business hours)
- **Abnormality Level**: **Low** - Activity during normal business hours
- **Risk Assessment**: Low risk

### Geographic Abnormality
- **Access Source**: Japan (< ISP Name >)
- **Expected Business Location**: Within Japan
- **Abnormality**: None - Access from expected business location

### Operation Pattern Abnormality
- **Tool Used**: aws-sdk-rust (development tool)
- **Operation Content**: S3 GetObject (data retrieval)
- **Target**: sandbox bucket (development environment)
- **Abnormality**: Low - Normal operation in development work

### Authentication Chain Abnormality
- **Entra ID Authentication**: Normal (error code 0)
- **Identity Center**: Normal SSO session
- **AWS Authentication**: Legitimate AssumedRole
- **Abnormality**: None - All legitimate processes

### Entra ID Authentication Log Analysis Results

#### Executed Splunk Queries and Results

**1. Basic Information Retrieval**:

index=< example > userPrincipalName="< User Email Address >" | table userDisplayName userPrincipalName

**Result**: Confirmed user name "User Name"

**2. 30-Day Behavior Pattern**:

index=< example > userPrincipalName="< User Email Address >" earliest=-30d | stats count, values(location.city), values(deviceDetail.browser), values(appDisplayName) by userPrincipalName

**Result**: 
- Total login count: 59 times
- Access region: < Access Area >
- Browser: Chrome 138.0.0
- Apps used: AWS SSO, Microsoft 365 Support Service, Microsoft Account Controls V2, Microsoft Teams Web Client

**3. Time Pattern Analysis**:

index=< example > userPrincipalName="< User Email Address >" earliest=-7d | eval weekday=strftime(_time, "%A"), hour=strftime(_time, "%H") | stats count by weekday, hour | sort weekday, hour

**Result**: 
- Many accesses at 16:00 hour regardless of weekdays/holidays
- Multiple accesses at midnight 0:00 hour confirmed (Friday, Thursday, Tuesday, Wednesday)
- Access patterns outside normal business hours also exist

**4. Authentication Status**:

index=< example > userPrincipalName="< User Email Address >" | stats count by status.errorCode, status.failureReason | sort -count

**Result**: 
- Successful logins: 28 times (error code 0)
- Authentication failures: Not confirmed
- Authentication success rate: 100%

**5. Authentication Logs Around Finding Occurrence Time**:

index=< example > userPrincipalName="< User Email Address >" (Around Finding occurrence time)

**Result**: 
- Successful AWS SSO login from IPv6 address (< IPv6 Address >)
- Access from within Japan
- Authentication successful (error code 0)

### Identity Verification Results
**Identity Confirmation through Behavior Analysis**: 
- **Match with Normal Patterns**: High degree of match
- **Access Region**: Access from normal < Access Area >
- **Applications Used**: Normal use of AWS SSO
- **Authentication Success Rate**: 100% (no abnormal failures)
- **User authenticity**: **High** - Matches normal behavior patterns

## 6. Risk Assessment

### Overall Risk Assessment: **Low**

#### Rationale
1. **Operation by Legitimate User**: Operation by actual outsourced user
2. **Normal Business Hours**: Normal business hours on Monday at 13:20
3. **Legitimate Authentication Process**: Normal authentication chain via Identity Center
4. **Development Environment**: Development work in sandbox bucket
5. **Identity Verification**: Matches legitimate user's normal patterns in Entra ID logs

#### False Positive Possibility
**High** - GuardDuty's machine learning baseline may not have sufficiently learned this user's normal patterns

#### Security Impact
- **Data Leakage Risk**: Extremely low
- **System Compromise Risk**: None
- **Lateral Spread Risk**: None

## 7. Detailed Investigation

### CloudTrail Log Existence Confirmation Results
**Detailed authentication chain tracking is difficult because CloudTrail logs for target AWS account (000000000000) do not exist in Splunk**

Executed query:
index=< example > sourcetype=aws:cloudtrail account=000000000000 | head 1

Result: 0 logs found

### Candidate User Investigation from Source IP Address
**Candidate User Investigation by Geographic Proximity**:
- Direct search with IP address from Finding details (< IP Address >): 0 results
- Actual authentication logs use IPv6 address (< IPv6 Address >)
- Both accesses from within Japan with geographic consistency

### Points Requiring Additional Investigation
1. **GuardDuty Baseline Adjustment**: Learning normal patterns for outsourced users
2. **S3 Access Logs**: Verify detailed logs at bucket level if necessary
3. **User Work Confirmation**: Confirm work content at relevant time (just in case)

### Unclear Points or Questions
1. Specific learning baseline that GuardDuty determined as "abnormal"
2. Access patterns of other users to same bucket
3. Business necessity for using aws-sdk-rust tool

## 8. Recommended Countermeasures

### Immediate Response (Priority: Low)
1. **User Confirmation**: Confirm work content with User Name (just in case)
2. **Continuous Monitoring**: Monitor for recurrence with similar patterns

### Short-term Response (Within 1 week)
1. **GuardDuty Adjustment**: Promote learning of normal patterns for outsourced users
2. **Access Pattern Analysis**: Establish normal access patterns for sandbox bucket

### Medium to Long-term Response (Within 1 month)
1. **Baseline Optimization**: Adjust GuardDuty machine learning baseline
2. **Monitoring Rule Review**: Establish monitoring rules for outsourced users
3. **Documentation**: Document normal development work patterns

### Reasons for Determining No Response Necessary
- Legitimate operation by legitimate user during normal business hours
- All authentication processes normal
- Security risk extremely low
- High possibility of false positive

**Conclusion**: This Finding is determined to be a false positive, requiring no response as a security incident. Recommend adjusting GuardDuty baseline.

Compared to requesting log analysis of threats detected by Amazon GuardDuty from Amazon Q Developer in its default state, the following improvements were observed.
By adjusting Amazon Q Developer through MCP integration and prompt engineering, the information included in log analysis output and its content could be brought closer to expectations.

• By specifying report format to generate in prompts, necessary information was stably included in output content across multiple verifications.
• Risk assessment of detected threats could be performed using log data on Splunk Cloud, enabling analysis closer to actual practice.

However, further improvements are needed in the following aspects:

• Through multiple verifications, reports containing inaccurate information were sometimes generated.
• Despite clear instructions in prompts, outputs ignoring instructions sometimes occurred.
• When AWS CloudTrail logs were not imported to Splunk Cloud, risk assessment was sometimes performed using only Entra ID logs without searching AWS CloudTrail logs directly using tools provided by CloudTrail MCP Server.

Hallucinations cannot be completely eliminated, and continuous adjustment of Amazon Q Developer is necessary while continuing to verify output content and monitor executed operations.

Conclusion

While log analysis using Amazon Q Developer has many challenges in its default state, by performing MCP integration and prompt adjustments, responses that appear usable for actual log analysis could be generated. However, the generated content cannot be completely trusted, and continued adjustment and improvement regarding response accuracy and actually performed operations appear necessary.

Additionally, since this verification focused on methods of utilizing Amazon Q Developer for log analysis, this article does not address other aspects. From the perspective of securely leveraging generative AI, there remain many challenges to be improved for practical operation, such as enforcing minimum privilege management for MCP servers and handling credential management.

I will continue to explore utilization methods for Amazon Q Developer while working on improving and strengthening security operations.