SIEM Integration: Microsoft Sentinel

This guide brings the GrantFlow audit feed into Microsoft Sentinel. A small poller on a Linux machine pulls new events every minute and writes them to a daily file; the Azure Monitor Agent sends those files to a custom table in your Sentinel workspace. Create a feed token first; see SIEM Integration.

A packaged Sentinel solution is planned

A GrantFlow data connector for Microsoft Sentinel is planned. It will poll the feed directly, with no machine of your own. Until it ships, the setup below is the supported way to connect. It uses the same feed, so you can move to the connector later.

Before you start

You need:

• A Linux machine to run the poller: an Azure VM, or any server connected with Azure Arc. It needs outbound HTTPS to the GrantFlow API and to Azure Monitor.
• Microsoft Sentinel on a Log Analytics workspace, and the rights to create a table and a data collection rule in it (Microsoft Sentinel Contributor and Monitoring Contributor, or higher).
• A feed token and the endpoint from Audit & Compliance → SIEM Integration.

1. Prepare the machine

On the Linux machine:

1. Copy the reference poller to /opt/grantflow/grantflow_feed.py.

2. Create the directories and store the token where only root can read it:

   sudo install -d -m 700 /etc/grantflow /var/lib/grantflow
   sudo install -d -m 700 /var/log/grantflow
   sudo sh -c 'umask 077; cat > /etc/grantflow/feed-token'   # paste the token, then Ctrl+D

The audit files hold user names, IP addresses and justifications, so the log directory stays closed to other local accounts. You open it to the Azure Monitor Agent in step 4, once the agent is installed. Do not start the poller yet: the agent only collects lines written after its data collection rule is deployed, so events polled before then would never reach Sentinel.

2. Create the table

In your Log Analytics workspace, create a custom table named GrantFlowAudit_CL with exactly two columns:

Column	Type
TimeGenerated	datetime
RawData	string

Each GrantFlow event lands in RawData as one JSON line. The parser in step 5 turns it into columns.

3. Collect the file with the Azure Monitor Agent

1. In Microsoft Sentinel, open Configuration → Data connectors and install Custom Logs via AMA from the Content hub if it is not listed.
2. Open the connector page and select Create data collection rule.
3. Basics: name the rule, for example dcr-grantflow-audit.
4. Resources: select the Linux machine from step 1. The agent is installed if it is missing.
5. Collect:
   • Select device type: leave Custom new table.
   • Table name: GrantFlowAudit_CL.
   • File pattern: /var/log/grantflow/audit-*.json.
   • Transform: leave source.
6. Review + create.

Once the poller runs, events reach the table a few minutes after it writes them.

4. Start polling

Start the poller only now, with the table, the data collection rule and the agent in place.

1. Let the agent read the log directory, and nobody else. The agent runs as the azuremonitoragent account, which the data collection rule created in step 3. Check that it exists with id azuremonitoragent; if your agent runs under another account, use that account's group instead.

   sudo chgrp azuremonitoragent /var/log/grantflow
   sudo chmod 2750 /var/log/grantflow

   The setgid bit (2) gives every new file in the directory the same group, so each day's file is readable by the agent without being world-readable.

2. Run the poller every minute from /etc/cron.d/grantflow-feed. Replace the endpoint with the one from GrantFlow:

   * * * * * root umask 027; GRANTFLOW_FEED_URL="https://api.grantflow.cloud/api/v1/audit-feed/events" GRANTFLOW_FEED_TOKEN="$(cat /etc/grantflow/feed-token)" GRANTFLOW_CURSOR_FILE=/var/lib/grantflow/feed.cursor /usr/bin/flock -n /var/lib/grantflow/feed.lock sh -c 'python3 /opt/grantflow/grantflow_feed.py >> /var/log/grantflow/audit-$(date +\%Y\%m\%d).json'
   17 3 * * * root find /var/log/grantflow -name 'audit-*.json' -mtime +7 -delete

   flock holds a lock for the whole polling cycle, so a slow run cannot overlap the next one, and umask 027 keeps each file readable by root and the agent's group only. The \% signs are escaped because cron treats a bare % as a line break.

The poller writes one file per day and never renames or truncates one, so there is no rotation to coordinate: a file stays where the agent reads it until the second line removes it after seven days. Avoid copytruncate rotation for this feed. Events appended between the copy and the truncation are lost from both files, although the poller has already saved their cursor.

After a minute or two, the token's card in GrantFlow shows Pulling. The first run starts at "now"; to import history once, see Polling.

5. Add a parser function

Save this query as a workspace function named GrantFlowAudit (Logs → Save → Save as function):

GrantFlowAudit_CL
| extend e = parse_json(RawData)
| project
    TimeGenerated = coalesce(todatetime(e.timestamp), todatetime(e.recordedAt)),
    RecordedAt = todatetime(e.recordedAt),
    EventId = tostring(e.id),
    Action = tostring(e.action),
    Category = tostring(e.category),
    Outcome = tostring(e.outcome),
    ActorId = tostring(e.actor.id),
    ActorType = tostring(e.actor.type),
    ActorUpn = tostring(e.actor.upn),
    ActorName = tostring(e.actor.displayName),
    ResourceType = tostring(e.resource.type),
    ResourceId = tostring(e.resource.id),
    Channel = tostring(e.channel),
    ClientIp = tostring(e.client.ip),
    ClientCountry = tostring(e.client.geo.countryCode),
    AuthStrength = tostring(e.auth.strength),
    AuthMethods = e.auth.methods,
    CorrelationId = tostring(e.correlationId),
    Details = e.details,
    GrantFlowTenantId = tostring(e.tenantId)
| summarize arg_max(RecordedAt, *) by EventId

The last line removes the occasional duplicate: the feed delivers at least once. The full list of fields is in the event reference.

Example queries

Approved activations in the last day:

GrantFlowAudit
| where TimeGenerated > ago(1d)
| where Action == "activation.approve" and Outcome == "success"
| project TimeGenerated, ActorUpn, ResourceId, Details

Changes made from a weak sign-in:

GrantFlowAudit
| where Category == "change"
| where AuthStrength in ("single_factor", "mfa")
| project TimeGenerated, Action, ActorUpn, AuthStrength, AuthMethods, ClientIp

Signs of a stolen feed token, a good candidate for an analytics rule:

GrantFlowAudit
| where Action in ("audit_feed_token.new_origin", "audit_feed_token.history_requested", "audit_feed_token.auth_failed")
| project TimeGenerated, Action, Outcome, TokenId = ActorId, Details

Troubleshooting

Symptom	What to check
The card stays Never used	Run the cron command by hand on the machine and read its output.
HTTP 401 from the poller	The token is wrong, revoked or expired. Create a new one and replace /etc/grantflow/feed-token.
HTTP 422 cursor_rejected	Delete /var/lib/grantflow/feed.cursor, run once with GRANTFLOW_START_TIME set to just before the last event you have, then remove it. The parser removes the duplicates.
The file grows but the table stays empty	Check that the data collection rule is associated with the machine, that the file pattern is /var/log/grantflow/audit-*.json, and that /var/log/grantflow and its files belong to the agent's group. The agent only picks up lines written after the rule was deployed.