You will frequently need to quickly sort through large data to promote security findings in order to react to emerging risks. Lately, Amazon Detective unveiled two different aspects to assist you in doing this. A new trade data mechanism makes it easier to use Detective’s data in many applications and automated workflows, and new visualizations in Detective show connections between entities related to numerous Amazon GuardDuty findings.

 <p>By using these new features, you can quickly analyze, correlate, and visualize the large amounts of data generated by sources like <a href="https://aws.amazon.com/vpc/" target="_blank" rel="noopener">Amazon Virtual Private Cloud (Amazon VPC)</a> Flow Logs, <a href="https://aws.amazon.com/cloudtrail/" target="_blank" rel="noopener">AWS CloudTrail</a>, Amazon GuardDuty findings, and <a href="https://aws.amazon.com/eks/" target="_blank" rel="noopener">Amazon Elastic Kubernetes Service (Amazon EKS)</a> audit logs.</p> <p>In this post, we’ll show you how you can use these new features to help reduce the time it takes to assess, investigate, and prioritize a security incident.</p> <h2>A security finding is raised</h2> <p>The workflow starts with GuardDuty. GuardDuty continuously monitors AWS accounts, <a href="https://aws.amazon.com/ec2/" target="_blank" rel="noopener">Amazon Elastic Compute Cloud (Amazon EC2)</a> instances, EKS clusters, and data stored in <a href="https://aws.amazon.com/s3/" target="_blank" rel="noopener">Amazon Simple Storage Service (Amazon S3)</a> for malicious activity without the use of security software or agents.</p> <p>If GuardDuty detects potential malicious activity, such as anomalous behavior, credential exfiltration, or command and control (C2) infrastructure communication, it generates detailed security incidents called <a href="https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_findings.html" target="_blank" rel="noopener">findings</a>.</p> <p>Depending on the severity and complexity of the GuardDuty finding, the resolution might require deep investigation. Consider an example that involves cryptocurrency mining. If you frequently see a cryptocurrency finding on your EC2 instances, you might have a recurring malware issue that has enabled a backdoor. If a threat actor is attempting to compromise your AWS environment, they typically perform a sequence of actions that lead to multiple findings and unusual behavior. When security findings are investigated in isolation, it can lead to a misinterpretation of their significance and difficulty in finding the root cause. When you need more context around a finding, Detective can help.</p> <p>Detective automatically collects log data and events from sources like CloudTrail logs, Amazon VPC Flow Logs, GuardDuty findings, and Amazon EKS audit logs and maintains up to a year of aggregated data for analysis. Detective uses machine learning to create a behavioral graph for these data sources that helps show how security issues have evolved. It highlights what AWS resources might be compromised and flags unusual activity like new API calls, new user agents, and new AWS Regions.</p> <p>The search capabilities work across AWS workloads, providing the information required to show the potential impact of an incident. Detective helps you answer questions like: How did this security incident happen? What AWS resources were affected? How can we prevent this from happening again?</p> <h2>Finding groups help connect the dots of an incident</h2> <p>You can use <a href="https://docs.aws.amazon.com/detective/latest/userguide/groups-about.html" target="_blank" rel="noopener">finding groups</a>, a recent feature of Detective, to help with your investigations. A <em>finding group</em> is a collection of entities related to a single potential security incident that should be investigated together. An <em>entity </em>can be an AWS resource like an EC2 instance, IAM role, or GuardDuty finding, but it can also be an IP address or user agent. For a full list of entities collected, see <a href="https://docs.aws.amazon.com/detective/latest/userguide/detective-search.html" target="_blank" rel="noopener">Searching for a finding or entity</a> in the Detective User Guide.</p> <p>Grouping these entities together helps provide context and a more complete understanding of the threat landscape. This makes it simpler for you to identify relationships between different events and to assess the overall impact of a potential threat.</p> <p>In the cryptocurrency mining example described previously, finding groups could show the relationship between the cryptocurrency mining finding and a C2 finding so that you know the two are related and the AWS resources affected. To learn more about working with Detective finding groups, see <a href="https://aws.amazon.com/blogs/security/how-to-improve-security-incident-investigations-using-amazon-detective-finding-groups/" target="_blank" rel="noopener">How to improve security incident investigations using Amazon Detective finding groups</a>.</p> <p>Figure 1 shows the finding groups overview page on the Detective console, with a list of finding groups filtered by status. The dashboard also shows the severity, title, observed tactics, accounts, entities, and the total number of findings for each finding group. For more information about the attributes of finding groups, see <a href="https://docs.aws.amazon.com/detective/latest/userguide/groups-about.html" target="_blank" rel="noopener">Analyzing finding groups</a>.</p> <div id="attachment_29073" class="wp-caption aligncenter"> <img aria-describedby="caption-attachment-29073" src="https://www.infracom.com.sg/wp-content/uploads/2023/04/img1-1-1024x452-1.png" alt="Figure 1: Finding groups overview" width="760" class="size-large wp-image-29073"> <p id="caption-attachment-29073" class="wp-caption-text">Figure 1: Finding groups overview</p> </div> <p>To see details about the finding group, select the title of the finding group to access the details page, which includes <strong>Details</strong>, <strong>Visualization</strong>, <strong>Involved entities</strong>, and<strong> Involved findings</strong>. On this panel, you can view entities and findings included in a finding group and interact with them. The information presented is the same in the <strong>Visualization </strong>panel, the <strong>Involved entities </strong>panel, and the<strong> Involved findings</strong> panel<strong>. </strong>The different views allow you to view the information in the way that is helpful for you. Figure 2 shows an example of the <strong>Details</strong> and <strong>Visualization</strong> for a specific finding group.</p> <div id="attachment_29074" class="wp-caption aligncenter"> <img aria-describedby="caption-attachment-29074" src="https://www.infracom.com.sg/wp-content/uploads/2023/04/img2-1-1024x917-1.png" alt="Figure 2: Details and Visualization" width="760" class="size-large wp-image-29074"> <p id="caption-attachment-29074" class="wp-caption-text">Figure 2: Details and Visualization</p> </div> <blockquote> <p><strong>Note</strong>: Finding groups with over 100 nodes (findings and entities) do not include a graph visualization.</p> </blockquote> <h2>Visualizations to show you the situation</h2> <p>The new visualizations in Detective provide three layouts that display the same information from finding groups, but allow you to choose and arrange the different entities so that you can focus on the highest priority finding or resources.</p> <p>To determine what each visual element represents, choose the <strong>Legend</strong> in the bottom left corner of the panel. You can change the placement of findings in the <strong>Visualization</strong> panel by selecting a different layout from the <strong>Select layout</strong> dropdown menu. Figure 2 in the preceding section shows the <strong>Force-directed</strong> layout, where the positioning of entities and findings presents an even distribution of links with minimal overlap, while maintaining consistent distance between items.</p> <p>Figure 3 shows the <strong>Visualization </strong>panel with the <strong>Circle</strong> layout, where nodes are displayed in a circular layout. You can use the <strong>Legend</strong> to understand the different categories of <strong>Findings</strong>, <strong>Compute</strong>, <strong>Network</strong>, <strong>Identity</strong>, <strong>Storage</strong>, or <strong>Other</strong>.</p> <p>If you’re unfamiliar with these terms, see <a href="https://docs.aws.amazon.com/detective/latest/adminguide/detective-terms-concepts.html" target="_blank" rel="noopener">Amazon Detective terms and concepts</a> to learn more.</p> <div id="attachment_29075" class="wp-caption aligncenter"> <img aria-describedby="caption-attachment-29075" src="https://www.infracom.com.sg/wp-content/uploads/2023/04/img3-1-1024x594-1.png" alt="Figure 3: Visualization panel with Circle layout and Legend" width="760" class="size-large wp-image-29075"> <p id="caption-attachment-29075" class="wp-caption-text">Figure 3: Visualization panel with Circle layout and Legend</p> </div> <p>Figure 4 shows the <strong>Visualization</strong> panel with the <strong>Grid</strong> layout, where nodes are divided into four different columns: evidence, identity entities, GuardDuty findings, and other entities (compute, network, and storage).</p> <div id="attachment_29076" class="wp-caption aligncenter"> <img aria-describedby="caption-attachment-29076" src="https://www.infracom.com.sg/wp-content/uploads/2023/04/img4-1-1024x582-1.png" alt="Figure 4: Visualization panel with Grid layout" width="760" class="size-large wp-image-29076"> <p id="caption-attachment-29076" class="wp-caption-text">Figure 4: Visualization panel with Grid layout</p> </div> <p>In the <strong>Visualization</strong> panel, you can select one or more (using ctrl/cmd+click) nodes. Selected nodes are listed next to the graph, and you can select each node’s title for more information. Selecting an entity’s title opens a new page that displays detailed information about that entity, whereas selecting a finding or evidence expands the right sidebar to show details on the selected finding or evidence.</p> <p>You can rearrange chosen entities and findings as needed to help improve your understanding of their connections. This can help speed up your assessment of findings. Figure 5 shows the <strong>Visualization </strong>panel with four nodes selected and the sidebar displaying information relevant to the selected finding.</p> <div id="attachment_29077" class="wp-caption aligncenter"> <img aria-describedby="caption-attachment-29077" src="https://www.infracom.com.sg/wp-content/uploads/2023/04/img5-1-1024x426-1.png" alt="Figure 5: Visualization panel with evidence selected" width="760" class="size-large wp-image-29077"> <p id="caption-attachment-29077" class="wp-caption-text">Figure 5: Visualization panel with evidence selected</p> </div> <p>Finding groups and visualizations provide an overview of the entities and resources related to a security activity. Presenting the information in this way highlights the interconnections between various activities. This means that you no longer have to use multiple tools or query different services to collect information or investigate entities and resources. This can help you reduce triage and scoping times and make your investigations faster and more comprehensive.</p> <h2>Increased flexibility for investigation with simpler data access</h2> <p>To expand the scope of your investigation or confirm if a security incident has taken place, you might want to combine data from Detective with your own tools or different services. This is where export data comes into play.</p> <p>Detective has several <strong>Summary</strong> page panels that you can use as a starting point for your investigations because they highlight potentially suspicious activity. The panels include roles and users with the most API call volume, EC2 instances with the most traffic volume, and EKS clusters with the most Kubernetes pods created.</p> <p>With export data, you can now export these panels as common-separated values (CSV) files and import the data into other AWS services or third-party applications, or manipulate the data with spreadsheet programs.</p> <h3>Export from the Detective console Summary screen</h3> <p>In the <a href="https://console.aws.amazon.com/detective/" target="_blank" rel="noopener">Detective console</a>, on the <strong>Summary</strong> page, you will see an <strong>Export</strong> option on several summary panes. This is enabled and available for anyone with access to Detective. Figure 6 shows summary information for the roles and users with the most API call volume.</p> <div id="attachment_29078" class="wp-caption aligncenter"> <img aria-describedby="caption-attachment-29078" src="https://www.infracom.com.sg/wp-content/uploads/2023/04/img6-1-1024x349-1.png" alt="Figure 6: Detective console Summary screen" width="760" class="size-large wp-image-29078"> <p id="caption-attachment-29078" class="wp-caption-text">Figure 6: Detective console Summary screen</p> </div> <p>Choose <strong>Export</strong> to download a CSV file containing the data for the summary information. The file is downloaded to your browser’s default download folder on your local device. When you view the data, it will look something like the spreadsheet in Figure 7:</p> <div id="attachment_29079" class="wp-caption aligncenter"> <img aria-describedby="caption-attachment-29079" src="https://www.infracom.com.sg/wp-content/uploads/2023/04/img7-1-1024x197-1.png" alt="Figure 7: Example CSV data export from Detective console Summary" width="760" class="size-large wp-image-29079"> <p id="caption-attachment-29079" class="wp-caption-text">Figure 7: Example CSV data export from Detective console Summary</p> </div> <h3>Export from the Detective console Search screen</h3> <p>You can also export data using the <strong>Search</strong> capability of Detective. After you apply specific filters to search for findings or entities based on your use case, an <strong>Export</strong> button appears at the top of the search results in Detective. Figure 8 shows an example of filtering for a particular CIDR range. Choose <strong>Export</strong> to download the CSV file containing the filtered data.</p> <div id="attachment_29080" class="wp-caption aligncenter"> <img aria-describedby="caption-attachment-29080" src="https://www.infracom.com.sg/wp-content/uploads/2023/04/img8-1-1024x392-1.png" alt="Figure 8: Detective search results filtering for a CIDR range" width="760" class="size-large wp-image-29080"> <p id="caption-attachment-29080" class="wp-caption-text">Figure 8: Detective search results filtering for a CIDR range</p> </div> <h2>Conclusion</h2> <p>In this blog post, you learned how to use the two new features of Detective to visualize findings and export data. By using these new features, you and your teams can investigate an incident in the way that best fits your workflow. New visualizations show the entities involved in an issue and surface nuanced connections that can be difficult to find when you’re faced with line after line of log data. The new data export feature makes it simpler to integrate the insights discovered in Detective with the tools and automations that your team is already using.</p> <p>These features are automatically enabled for both existing and new customers in <a href="https://docs.aws.amazon.com/general/latest/gr/detective.html" target="_blank" rel="noopener">AWS Regions that support Detective</a>. There is no additional charge for finding groups. If you don’t currently use Detective, you can <a href="https://aws.amazon.com/detective/pricing/" target="_blank" rel="noopener">start a free 30-day trial</a>. For more information on finding groups, see <a href="https://docs.aws.amazon.com/detective/latest/userguide/groups-about.html" target="_blank" rel="noopener">Analyzing finding groups</a> in the Amazon Detective User Guide.</p> <p>If you have feedback about this post, submit comments in the <strong>Comments</strong> section below. You can also start a new thread on <a href="https://repost.aws/tags/TAUrK2r73PTHyirNVS4hKn6w/amazon-detective" target="_blank" rel="noopener">Amazon Detective re:Post</a> or <a href="https://console.aws.amazon.com/support/home" target="_blank" rel="noopener">contact AWS Support</a>.</p> <p><strong>Want more AWS Security news? Follow us on <a title="Twitter" href="https://twitter.com/AWSsecurityinfo" target="_blank" rel="noopener noreferrer">Twitter</a>.</strong></p> <!-- '"` -->