Getting Insights with OCI Audit Log with Logging Analytics (via Service Connector)

In this blog we will discuss the mechanics for forwarding OCI Audit Logs to the Logging Analytics service from the Oracle Cloud Observability and Management platform, and discuss how this pattern can be extended to other log sources.

Logging Analytics at present has a number of different mechanisms for log ingestion. The primary method is through the use of management agents, running on your application hosts, however we can hardly install an agent on the cloud control plane, so we need to use an alternative mechanism. For this scenario, we are going to take advantage of one of the native OCI integrations, using a ‘Service Connector’ which is an OCI component designed for moving data from one service to another. Logging Analytics can be used as a target for Service Connectors, and so this dramatically simplifies the integration by leveraging native OCI capability.

At a high level, the required tasks to get the cloud control plane Audit Logs to Logging Analytics involve:
1. Setup Prerequisites
2. Create Log Group in Logging Analytics
3. Configure Service Connector to route OCI Audit logs to Logging Analytics
4. Review OCI Audit Logs in Logging Analytics
5. Access OCI Audit Dashboard

1 – GENERAL PREREQUISITES

For details on the prerequisites for Logging Analytics, please refer to the documentation here.

1.1 Setup the IAM Policies for Services and Groups.

The rest of these steps assume the existence of a compartment and valid groups and policies configured for Logging Analytics. These are covered in the prerequisites document above.

Security Note: Policies can be set for 'in tenancy' or 'in compartment' - which provides broader or narrower access. In some scenarios you may have a security group which is monitoring audit logs across all resources, in others you may have separate groups per-project.
Make sure to consider this and set the access for your log admins group as appropriate.

2 – CREATE LOG GROUP

Log Groups represent logical groupings of logs, and can be used to define access privileges, such that only Audit Administrators might be able to review Audit logs or similar. If you followed the general prerequisites section, your user likely has access to every log group, but this could be refined if required.

2.1 In OCI Console, navigate to:

OBSERVABILITY & MANAGEMENT > LOGGING ANALYTICS > ADMINISTRATION > LOG GROUPS > CREATE LOG GROUP

2.2 Complete required inputs and Click on Create

For example:
Compartment = demo_sandbox
Name = logging-analytics-demo-LogGroup

2.3 Review Log Group

3 – CREATE SERVICE CONNECTOR

Service Connectors are an OCI mechanism for connecting services together. Since OCI services often have standard formats and mechanisms for ingesting data, as well as known outputs, Service Connectors provide a pre-configured mechanism for connecting those outputs and inputs together. In this example we are specifying inputs of OCI Audit logs and the output will be targeting is the Logging Analytics service

3.1 In OCI Console, navigate to:

OBSERVABILITY & MANAGEMENT > SERVICE CONNECTOR HUB > CREATE SERVICE CONNECTOR

3.2 Complete required inputs and then click on Create

For example:
Connector Name = oci_audit_to_logging_analytics
Description = Send OCI Audit Logs to Logging Analytics

Configure Source Connector:
+ Source = Logging
+ Target = Logging Analytics

Configure Source Connection :
+ Compartment = demo_sandbox
+ Log Group = _Audit

Configure Target Connection :
+ Compartment = demo_sandbox
+ Log Group = logging-analytics-demo-LogGroup

Note: Since this is a self contained demo, we are just taking the Audit events from this compartment and forwarding them to Logging Analytics. If you were implementing this on a larger scale across compartments, you can click '+ Another Log' and include the Audit data from other compartments.

Once created, the service connector will automatically push logs from the Audit service to Logging Analytics.

STEP 4 – REVIEW OCI AUDIT LOGS IN LOG EXPLORER

Now that the logs are flowing from OCI Audit and into Logging Analytics, we can start to explore and analyse the data.

4.1 In OCI Console, navigate to:

OBSERVABILITY & MANAGEMENT > LOGGING ANALYTICS > LOG EXPLORER

4.2 Set the Log Group Compartment:

For example:
Log Group Compartment = demo_sandbox

You should see records related to ‘OCI Audit Logs’ (our specified Log Source) have been created.

4.3 For a slightly more useful view, change Visualizations to Records with Histogram

5.4 Review OCI Audit Logs

5 – ACCESS OCI AUDIT DASHBOARD

While the log explorer allows for pretty rich analytics and exploration, typical operations monitoring will require a dashboard which reflects meaningful statistics and metrics, and so provides a view of the system at a glance. OCI provides an out of the box dashboard called ‘OCI Audit Analysis’ which can help you get immediate insights.

5.1 To see the OCI Audit Analysis dashboard navigate to:
OBSERVABILITY & MANAGEMENT > LOGGING ANALYTICS > DASHBOARD > OCI AUDIT ANALYSIS

You should now see the OCI AUDIT DASHBOARD

Once this has been set up, you have an ‘at-a-glance’ view of activities in your tenancy, with the ability to drill down into individual actions or groups of actions. While in this example only a single compartment is being monitored, the same mechanism can easily be expanded to bring events from the entire tenancy into the same dashboard. The Service Connector mechanism makes the entire process very simple and straightforward, and a range of other OCI services can easily be connected to provide application logging from the control plane.