• Splunk-Export-Audit
  • Table Of Contents
  • Introduction
  • Components
  • Setup a Splunk Trial for Testing
  • Design Goals
  • Quickstart For Setup On OCI Side
    • Create Compartments and Groups
    • Create a Dynamic Group
    • Create Tenancy IAM policy - Splunk Export Group
    • Create Tenancy IAM policy - Splunk Export Dynamic Group
    • Create Compartment Level IAM Policy
    • Create a VCN, Subnet & Security Lists
    • Configure Cloud Shell
    • The Cloud Shell
    • Create a Function Application
    • Getting Started with Fn Deployment
    • Step-8
    • Deploy the Functions
    • Create an API Gateway
    • Create API Gateway Deployment Endpoints
    • Create Notification Channels
    • Create Streaming
    • Set the Environment Variables for Each Function
  • Invoke and Test !
  • Health Checks for Scheduled Trigger
    • Target Settings

For Integrated SecOps, the SIEM plays an important component and Splunk is a very popular SIEM solution. The setup described below helps in near-realtime, zero-touch audit-log export to the Splunk Http event Collector for Indexing and Analysis.The same architecture can be used to export to almost any SIEM as it leverages the HTTP Event Collector.

A Scalable and Low Cost Splunk event exporter to publish OCI Audit Logs to Splunk.

• The OCI Audit API is queried for audit events every 2 minutes for all regions and all compartments relevant to the tenancy.
• The OCI Functions trigger a series of queries and publish events to the Splunk HTTP Event Collector End point.
• OCI Health Checks are used to trigger a GET Request on to a public HTTPS Endpoint every 5-min / 1-min / 30s
• The OCI API Gateway is used to Front-End the Functions. to allow for the fn invokation process through HTTP(S) Requests rather than using the OCI Fn Invocation mechanism or the oci-curl
• OCI Streaming as a scalable source of persistence that stores state, audit events, most up-to-date compartment & region list in the tenancy.
• OCI Notifications are used to notify downstream functions for triggering along with information on stream-cursor, offset, partition information to the next function.
• The Splunk HTTP event Collector is a simplified mechanism that splunk provides to publish events

• Splunk provides a 15 day Cloud trial and the capability to store about 5GB worth of event data that you forward/export to Splunk. Here's the link to sign-up Splunk Sign Up
• Select Splunk-Cloud, provide your data and Login to Splunk.
• To setup the HTTP Event Collector which we leverage, the solution refer to link Setup HTTP event collector
• Points to note

Self Perpetuating
Event driven 
Scalable 
Low-Cost
Zero maintenance

This quickstart assumes you have working understanding of basic principles of OCI around IAM | Networking and you know how to get around the OCI Console.

• Burger-Menu --> Identity --> Compartments | Users | Groups

1. Create a Compartment splunk-export-compartment
2. Create a Group splunk-export-users
3. Add Required User to group splunk-export-users
4. Create a Dynamic Group splunk-export-dg
5. Write appropriate IAM Policies at the tenancy level and compartment level.

• Burger-Menu --> Identity --> Dynamic Groups
• Create a Dynamic Group splunk-export-dg Instances that meet the criteria defined by any of these rules will be included in the group.

ANY {instance.compartment.id = [splunk-export-compartment OCID]}
ANY {resource.type = 'ApiGateway', resource.compartment.id =[splunk-export-compartment OCID]}
ANY {resource.type = 'fnfunc', resource.compartment.id = [splunk-export-compartment OCID]}

• Burger-Menu --> Identity --> Policies
• Create an IAM Policy splunk-export-tenancy-policy with the following policy statements in the root compartment

Allow group splunk-export-users to manage repos in tenancy
Allow group splunk-export-users to read audit-events in tenancy
Allow group splunk-export-users to read tenancies in tenancy
Allow group splunk-export-users to read compartments in tenancy
Allow service FaaS to read repos in tenancy

• Burger-Menu --> Identity --> Policies
• Create an IAM Policy splunk-export-dg-tenancy-policy with the following policy statements in the root compartment

Allow dynamic-group splunk-export-dg to read audit-events in tenancy
Allow dynamic-group splunk-export-dg to read tenancies in tenancy
Allow dynamic-group splunk-export-dg to read compartments in tenancy

• Burger-Menu --> Identity --> Policies Create an IAM Policy splunk-export-compartment-dg-policy inside the compartment splunk-export-compartment

Allow service FaaS to use all-resources in compartment splunk-export-compartment
Allow dynamic-group splunk-export-dg to use ons-topics in compartment splunk-export-compartment
Allow dynamic-group splunk-export-dg to use stream-pull in compartment splunk-export-compartment
Allow dynamic-group splunk-export-dg to use stream-push in compartment splunk-export-compartment
Allow dynamic-group splunk-export-dg to use virtual-network-family in compartment splunk-export-compartment

• Burger-Menu --> Networking --> Virtual Cloud Networks
• Use VCN Quick Start to Create a VCN splunk-export-vcn with Internet.
• Connectivity Go to Security List and Create a Stateful Ingress Rule in the Default Security list to allow Ingress Traffic in TCP 443
• Go to Default Security List and verify if a Stateful Egress Rule is available in the Default Security List to allow egress traffic in all ports and all protocols

Setup Cloud Shell in your tenancy - Link

• Oracle Cloud Infrastructure Cloud (OCI) Shell is a web browser-based terminal accessible from the Oracle Cloud Console.
• Cloud Shell provides access to a Linux shell, with a pre-authenticated Oracle Cloud Infrastructure CLI, a pre-authenticated Functions, Ansible installation, and other useful tools for following Oracle Cloud Infrastructure service tutorials and labs.

• Burger-Menu --> Developer Services --> Functions
• Create a Function Application splunk-export-app in the compartment splunk-export-compartment while selecting splunk-export-vcn and the Public Subnet

If you are not an IAM Policy Expert, just create these policies as shown in the Function Pre-requisites

• Setup Papertrail / OCI Logging Service to debug Function executions if required. Setup PaperTrail , check them out.

Click on the Getting started Icon after Function Application Creation is done.

Follow the steps on the screen for simplified Fn Deployment. While you have the option of local Fn Development environment, I'd recommend using the cloud shell if you simply want to deploy Functions.

Follow the steps until Step-7

Instead of creating a new fn we are deploying an existing function. So clone

Clone the Repo in the cloud shell git clone https://github.com/vamsiramakrishnan/splunk-export-audit.git

Each folder within the repo represents a function , go to each folder and deploy the function using the the fn --verbose deploy

cd splunk-export-audit
cd list-regions
fn --verbose deploy splunk-export-app list-regions

cd fetch-audit-events
fn --verbose deploy splunk-export-app fetch-audit-events

cd publish-to-splunk
fn --verbose deploy splunk-export-app publish-to-splunk

The Deploy Automatically Triggers an Fn Build and Fn Push to the Container registry repo setup for the functions.

• Burger-Menu --> Developer Services --> API Gateway
• Create an API Gateway splunk-export-apigw in the compartment splunk-export-compartmentwhile selecting splunk-export-vcn and the Public Subnet

Map the endpoint as follows

Note:The API Gateway is setup in this example with HTTPS without an Auth Mechanism , but this can be setup with an authorizer Function , that works with a simple Token mechanism

• Burger-Menu --> Application Integration --> Notifications
• Create two notification channels splunk-fetch-audit-event splunk-publish-to-splunk. Create subscriptions to Trigger the Functions

• Burger-Menu --> Analytics --> Streaming

Note: Using a Single Partition and Single Streaming endpoint for Audit, Can scale based on requirements

These environment variables help call other functions. One after the other.

Invoke Once and the loop will stay active as long as the tenancy does continuously pushing events to Splunk .

curl --location --request GET '[apigateway-url].us-phoenix-1.oci.customer-oci.com/regions/listregions'

If all is well in papertrail/ oci-function-logs/metrics, proceed.

Create a Health Check named splunk-export-health-check with the following settings

Get the API-Gateway URL where the list-regions Fn is deployed. Example https://<Random-Alphanumeric-String>.apigateway.us-phoenix-1.oci.customer-oci.com/regions/listregions Copy the String <Random-Alphanumeric-String>.apigateway.us-phoenix-1.oci.customer-oci.com and ignore the rest

Note:The API Gateway is setup in this example with HTTPS without an Auth Mechanism , but this can be setup with an authorizer Function , that works with a simple Token mechanism.If Auth is setup , the token can be specified in the Header of the Health Check.

This project is open source. Please submit your contributions by forking this repository and submitting a pull request! Oracle appreciates any contributions that are made by the open source community.

Copyright (c) 2024 Oracle and/or its affiliates.

Licensed under the Universal Permissive License (UPL), Version 1.0.

See LICENSE for more details.

ORACLE AND ITS AFFILIATES DO NOT PROVIDE ANY WARRANTY WHATSOEVER, EXPRESS OR IMPLIED, FOR ANY SOFTWARE, MATERIAL OR CONTENT OF ANY KIND CONTAINED OR PRODUCED WITHIN THIS REPOSITORY, AND IN PARTICULAR SPECIFICALLY DISCLAIM ANY AND ALL IMPLIED WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, AND FITNESS FOR A PARTICULAR PURPOSE. FURTHERMORE, ORACLE AND ITS AFFILIATES DO NOT REPRESENT THAT ANY CUSTOMARY SECURITY REVIEW HAS BEEN PERFORMED WITH RESPECT TO ANY SOFTWARE, MATERIAL OR CONTENT CONTAINED OR PRODUCED WITHIN THIS REPOSITORY. IN ADDITION, AND WITHOUT LIMITING THE FOREGOING, THIRD PARTIES MAY HAVE POSTED SOFTWARE, MATERIAL OR CONTENT TO THIS REPOSITORY WITHOUT ANY REVIEW. USE AT YOUR OWN RISK.