AWS S3

Overview

This source supports ingestion of multiple log sources through AWS S3.

Supported Sources:

  • Cisco Umbrella
  • AWS GuardDuty
  • DNS Filter
  • AWS CloudTrail

Setup Instructions

Configuring a cloud source in Scope is a two-step process.

  • Generating AWS S3 credentials and gathering the required information in the AWS Management Console. Please refer to Section 1 – AWS S3 Setup

  • Setting up the AWS S3 cloud source in the Scope application. Please refer to Section 2 - Scope Setup

AWS S3 Setup

To get started, you’ll need to generate and gather the following information from the AWS Management Console –

  1. Access Key
  2. Secret Access Key
  3. Regions
  4. Bucket Name
  5. Prefix (for individual data provider)

Step 1: Create an IAM User

Note: If an IAM User is already available with the required permissions, skip this step and proceed to Step 2*.

  • Sign in to the AWS Management Console.

  • Navigate to Identity and Access Management (type IAM in the search bar -> select IAM from the dropdown).

  • Click on Users in the left navigation pane.

  • Click on Create user.

  • In the Specify user details section, enter a name in the User name field and click on Next.

  • In the Set Permissions section, select Attach policies directly and in the Permissions policies, search and select the permission AmazonS3ReadOnlyAccess for the user, then click on Next.

  • In the Review and create section, review the User details and Permissions summary, then click on the Create User button.

  • The IAM user will be created.

Step 2: Generate Access Key & Secret Access Key

  • (If Step 1 is skipped) Sign in to the AWS Management Console. Navigate to Identity & Access Management (by searching IAM and selecting IAM in the search bar) and select Users from the side panel.

  • Click on the created / required IAM user to generate the access key.

  • Select the Security Credentials tab.

  • Click on Create Access Key.

  • In the Access key best practices & alternatives section, click on Third-party service, select the Confirmation checkbox, and click Next.

  • (Optional) In the Select description tag section, provide a description.

  • Click on the Create access key button.

  • In the Retrieve access keys section, the Access key and Secret access key are generated. Copy the Access key and Secret access key, or click on the Download .csv file button to download the keys.

    Note: The access keys generated here can be viewed or downloaded only at this time. The keys cannot be recovered later. Store them in a safe location immediately. If the keys are lost, new access keys can be generated by repeating Step 2 and disabling the old key.

The generated Access Key and Secret Access Key are to be configured in Scope Setup: Step 1 for initiating the AWS S3 log ingestion.

Step 3: Determine AWS Region

  • Sign in to the AWS Management Console.

  • Click on the Profile Name and navigate to Account.

  • Identify either the enabled AWS Regions or the preferred AWS Regions from which AWS S3 data provider logs are to be ingested.

The selected Region(s) are to be configured in Scope Setup: Step 1 when registering the AWS S3 cloud source.

Step 4: Determine the Required AWS S3 Bucket

  • Sign in to the AWS Management Console.

  • In the Home Page search box, search for S3 and click on S3 from the search list.

  • In the S3 General Purpose Bucket section, identify the required bucket from which events are to be ingested.

The Bucket name is to be configured in Scope Setup: Step 1 when registering the AWS S3 cloud source.

Step 5: Configure Supported Data Providers to Export Logs to AWS S3

Refer to the source-specific documentation below for detailed configuration steps for each supported data provider.

Scope Setup

Step 1: AWS S3 Cloud Source Registration in the Scope Application

Once the credentials and required information are gathered, they must be configured in the Scope application to establish the connection and enable data ingestion from the AWS S3 environment.

In the Scope application, to register an AWS S3 cloud source, navigate to the cloud source registration page

  • Log into the Scope application

  • Select the required Organization from the Organization dropdown

  • Navigate to the side menu -> Administration

  • Navigate to the Cloud sources tab

  • Click on the +Add Source button

  • In the Add Source pop-up, Select the AWS S3 source from the Source dropdown and configure the other parameters across the two tabs as described below.

Input Method Config Tab

Provide the following parameters in the Input Method Config tab –

  • Site: The user defined name for the AWS S3 cloud source.

  • Bucket Name: The bucket name determined in Step 4.

  • Access Key: The Access Key generated in Step 2.

  • Secret Access Key: The Secret Access Key generated in Step 2.

  • Region: Select the Region(s) from which AWS S3 events are ingested (determined in Step 3).

  • Polling Interval: The polling interval for making periodic API calls to the AWS S3 SDK. The user can select the time interval from the dropdown.

  • Contact Email: The email address of the person who registers the AWS S3 cloud source in Scope.

Data Provider Tab

Configure the required data providers in the Data Provider tab. Each data provider has its own accordion section –

Cisco Umbrella

  • Select the Cisco Umbrella checkbox.

  • Prefix: Provide the required prefix.

  • Ingest From: The start date from which the logs should be ingested from the AWS S3 account.

    Note: This option is available only at the time of registering the data provider. The user will not be able to edit it later.

  • Log Type: Select the log types from the dropdown. By default, all supported log types are selected.

AWS GuardDuty

  • Select the AWS GuardDuty checkbox.

  • Prefix: Provide the required prefix.

  • Ingest From: The start date from which the logs should be ingested from the AWS S3 account.

    Note: This option is available only at the time of registering the data provider. The user will not be able to edit it later.

DNS Filter

  • Select the DNS Filter checkbox.

  • Prefix: Provide the required prefix.

  • Ingest From: The start date from which the logs should be ingested from the AWS S3 account.

    Note: This option is available only at the time of registering the data provider. The user will not be able to edit it later.

AWS CloudTrail

  • Select the AWS CloudTrail checkbox.

  • Prefix: Provide the required prefix.

  • Ingest From: The start date from which the logs should be ingested from the AWS S3 account.

    Note: This option is available only at the time of registering the data provider. The user will not be able to edit it later.

  • Log Type: Select the log types from the dropdown. By default, all supported log types are selected.

Once the required connection parameters are entered, the AWS S3 registration is complete in Scope and is ready for ingestion of AWS S3 logs.