Analytics Support for API Platform

[O-sura]

Problem

The API Platform currently lacks a unified, policy-driven analytics pipeline to reliably capture and publish API usage insights without affecting request latency.

Solution

• Introduce an Analytics Policy to decide what to be tracked and extract essential metadata.
• Use Envoy’s ext_proc and ALS mechanisms to collect request/response data asynchronously.
• Implement an ALS server to aggregate and filter analytics logs based on policy decisions.
• Add a Publisher component to batch, enrich, and push analytics events to external platforms (e.g., Moesif).
• Ensure the entire pipeline remains low-latency, extensible, and fully integrated into the API Platform architecture.

High-Level Architecture

User Flow

[malinthaprasan]

1. Do we have a list of attributes we capture for the event?

we also need to think about how do we model this in Moesif (Gateway to Moesif side mapping - cc: @HiranyaKavishani @DivyaPremanantha)

• How we map gateways vs moesif side
• How we retrieve the keys (though gateway registration?)
• How we pass the keys to the policy engine. (through the Policy configuration (admin level)? )

[O-sura]

This doc contains event types and the parameters contains in an event sent to Moesif from the API Manager and there are dashboards already available for populating these data as well. Hence as for the initial implementation, we'll also be capturing and sending the same set of attributes

And for that Gateway to Moesif side mapping part, as discussed offline, we will proceed with implementing the basic analytics publishing flow for now and will take these up in a separate discussion once the foundational implementation is in place.