Webhooks

Configure secure HTTP endpoints that trigger workflows in response to external events.

What are Webhooks?

Webhooks enable external systems to trigger M3 Forge workflows by sending HTTP POST requests to unique URLs. Each webhook trigger generates a dedicated endpoint that:

• Accepts JSON payloads
• Validates authentication credentials
• Triggers the configured workflow
• Passes the payload as workflow input

This pattern enables integrations with:

• CI/CD systems (GitHub Actions, GitLab CI, Jenkins)
• Payment processors (Stripe, PayPal)
• Communication platforms (Slack, Discord, email services)
• Monitoring tools (Datadog, PagerDuty)
• Custom applications (your own services)

Creating a Webhook

https://your-instance/api/webhooks/wh_abc123xyz

curl -X POST https://your-instance/api/webhooks/wh_abc123xyz \
  -H "Authorization: Bearer YOUR_SECRET" \
  -H "Content-Type: application/json" \
  -d '{"test": true}'

Authentication Methods

None (Public Webhook)

No authentication required. Anyone with the URL can trigger the workflow.

{
  "auth_type": "none"
}

Request:

curl -X POST https://your-instance/api/webhooks/wh_abc123 \
  -H "Content-Type: application/json" \
  -d '{"key": "value"}'

Bearer Token

Shared secret in the Authorization header.

{
  "auth_type": "bearer_token",
  "secret": "your-secret-token"
}

Request:

curl -X POST https://your-instance/api/webhooks/wh_abc123 \
  -H "Authorization: Bearer your-secret-token" \
  -H "Content-Type: application/json" \
  -d '{"key": "value"}'

Secret generation:

• Auto-generate via UI (32-character random string)
• Provide custom secret (minimum 16 characters)
• Store in password manager or secrets vault

Best practices:

• Use cryptographically random secrets (not dictionary words)
• Rotate secrets periodically (every 90 days)
• Never commit secrets to version control
• Use different secrets for dev/staging/production

HMAC SHA-256

Cryptographic signature computed from payload and secret.

{
  "auth_type": "hmac_sha256",
  "secret": "your-hmac-secret"
}

Request:

PAYLOAD='{"key":"value"}'
SECRET="your-hmac-secret"
 
# Compute signature
SIGNATURE=$(echo -n "$PAYLOAD" | \
  openssl dgst -sha256 -hmac "$SECRET" | \
  awk '{print $2}')
 
# Send request
curl -X POST https://your-instance/api/webhooks/wh_abc123 \
  -H "X-Webhook-Signature: sha256=$SIGNATURE" \
  -H "Content-Type: application/json" \
  -d "$PAYLOAD"

import hmac
import hashlib
import json
import requests
 
payload = {"key": "value"}
secret = b"your-hmac-secret"
payload_bytes = json.dumps(payload).encode('utf-8')
 
# Compute HMAC signature
signature = hmac.new(secret, payload_bytes, hashlib.sha256).hexdigest()
 
# Send request
response = requests.post(
    'https://your-instance/api/webhooks/wh_abc123',
    headers={
        'X-Webhook-Signature': f'sha256={signature}',
        'Content-Type': 'application/json',
    },
    json=payload
)

import crypto from 'crypto';
import fetch from 'node-fetch';
 
const payload = { key: 'value' };
const secret = 'your-hmac-secret';
const payloadString = JSON.stringify(payload);
 
// Compute HMAC signature
const signature = crypto
  .createHmac('sha256', secret)
  .update(payloadString)
  .digest('hex');
 
// Send request
const response = await fetch(
  'https://your-instance/api/webhooks/wh_abc123',
  {
    method: 'POST',
    headers: {
      'X-Webhook-Signature': `sha256=${signature}`,
      'Content-Type': 'application/json',
    },
    body: payloadString,
  }
);

Advantages over bearer token:

• Payload integrity - Detects tampering (modified payload invalidates signature)
• Replay protection - Include timestamp in payload, reject old requests
• Secret rotation - Can verify with old and new secrets during rotation

Security properties:

1. Signature computed from exact payload bytes (order matters)
2. Any modification to payload invalidates signature
3. Secret never transmitted (only signature sent)
4. Signature cannot be forged without secret

IP Allowlisting

Restrict webhook sources to known IP addresses.

{
  "auth_type": "bearer_token",
  "secret": "your-secret",
  "allowed_ips": [
    "203.0.113.0/24",
    "198.51.100.42"
  ]
}

CIDR notation:

• 203.0.113.0/24 - Range from 203.0.113.0 to 203.0.113.255
• 198.51.100.42/32 - Single IP (same as 198.51.100.42)
• 0.0.0.0/0 - Allow all IPs (defeats the purpose)

Finding source IPs:

• GitHub: GitHub webhook IPs  (check hooks field)
• Stripe: Stripe webhook IPs 
• Custom services: Check server logs for request source IP

Payload Format

Webhook body becomes workflow input data.

Request:

{
  "event": "user.signup",
  "user": {
    "id": 12345,
    "email": "user@example.com"
  },
  "timestamp": "2024-03-19T10:30:00Z"
}

Workflow receives:

{
  "data": {
    "event": "user.signup",
    "user": {
      "id": 12345,
      "email": "user@example.com"
    },
    "timestamp": "2024-03-19T10:30:00Z"
  },
  "metadata": {
    "trigger_type": "webhook",
    "webhook_id": "wh_abc123xyz",
    "source_ip": "203.0.113.42",
    "received_at": "2024-03-19T10:30:01Z",
    "headers": {
      "user-agent": "GitHub-Hookshot/abc123",
      "x-github-event": "push"
    }
  }
}

Access payload fields in nodes via JSONPath:

• $.data.event → "user.signup"
• $.data.user.email → "user@example.com"
• $.metadata.source_ip → "203.0.113.42"

Content Types

Supported Content-Type headers:

Most webhook sources send application/json.

Size Limits

• Maximum payload size: 10 MB
• Request timeout: 30 seconds
• Rate limit: 100 requests per minute per webhook ID

Exceeding limits returns HTTP 413 (Payload Too Large) or 429 (Too Many Requests).

Response Codes

Webhook requests receive immediate HTTP responses:

Success response:

{
  "status": "success",
  "run_id": "run_abc123xyz",
  "workflow_id": "extract-pipeline",
  "message": "Workflow triggered successfully"
}

Error response:

{
  "status": "error",
  "error": "Invalid authentication credentials",
  "code": "WEBHOOK_AUTH_FAILED"
}

Retry Behavior

Webhook sources typically retry failed requests:

Best practices:

• Return 200 OK as quickly as possible (within 5 seconds)
• Workflow execution is async (don’t wait for completion)
• Log all webhook requests for debugging
• Monitor retry counts to detect issues

Common Integrations

GitHub Push Events

Trigger workflow on code commits.

GitHub webhook configuration:

1. Go to repository Settings → Webhooks → Add webhook
2. Payload URL: https://your-instance/api/webhooks/wh_abc123
3. Content type: application/json
4. Secret: Your bearer token or HMAC secret
5. Events: Select “Just the push event”

Payload example:

{
  "ref": "refs/heads/main",
  "repository": {
    "name": "my-repo",
    "full_name": "user/my-repo"
  },
  "commits": [
    {
      "id": "abc123",
      "message": "Update README",
      "author": {
        "name": "Jane Doe",
        "email": "jane@example.com"
      }
    }
  ]
}

Workflow access:

• Branch: $.data.ref
• Repo: $.data.repository.name
• Commit message: $.data.commits[0].message

Stripe Payment Events

Trigger workflow on successful payments.

Stripe webhook configuration:

1. Dashboard → Developers → Webhooks → Add endpoint
2. Endpoint URL: https://your-instance/api/webhooks/wh_abc123
3. Events to send: payment_intent.succeeded
4. Webhook signing secret: Use for HMAC verification

Payload example:

{
  "type": "payment_intent.succeeded",
  "data": {
    "object": {
      "id": "pi_abc123",
      "amount": 5000,
      "currency": "usd",
      "customer": "cus_xyz789"
    }
  }
}

Workflow access:

• Amount: $.data.data.object.amount
• Customer: $.data.data.object.customer

Slack Slash Commands

Trigger workflow from Slack messages.

Slack app configuration:

1. Create Slack app at api.slack.com/apps
2. Features → Slash Commands → Create New Command
3. Request URL: https://your-instance/api/webhooks/wh_abc123
4. Command: /process-document

Payload example:

{
  "token": "verification-token",
  "command": "/process-document",
  "text": "https://example.com/doc.pdf",
  "user_id": "U12345678",
  "channel_id": "C87654321"
}

Workflow access:

• Document URL: $.data.text
• User: $.data.user_id

Response:

Slack expects a response within 3 seconds. Return 200 OK immediately, then send results via Slack API later.

Debugging Failed Webhooks

View Webhook Logs

Webhook requests are logged with:

• Request timestamp
• Source IP
• Payload (truncated if > 1KB)
• Response code
• Error message (if failed)

Access logs in:

1. Automation page → Select webhook sensor
2. View tick timeline
3. Click failed tick for details

Common Issues

401 Unauthorized:

• Verify Authorization header is included
• Check bearer token matches configured secret
• For HMAC, ensure signature computed correctly

403 Forbidden:

• Source IP not in allowlist
• Check sender IP: curl https://api.ipify.org
• Add IP to allowed_ips array

400 Bad Request:

• Payload is not valid JSON
• Test with: echo $PAYLOAD | jq . to validate JSON
• Check Content-Type header is application/json

Workflow not triggering:

• Webhook returns 200 OK but no run created
• Check sensor is active (not paused/disabled)
• Verify target_workflow_id exists
• Review sensor tick logs for errors

Testing Tools

Manual test:

# Simple test
curl -X POST https://your-instance/api/webhooks/wh_abc123 \
  -H "Authorization: Bearer SECRET" \
  -H "Content-Type: application/json" \
  -d '{"test": true}'
 
# With verbose output
curl -v -X POST https://your-instance/api/webhooks/wh_abc123 \
  -H "Authorization: Bearer SECRET" \
  -H "Content-Type: application/json" \
  -d @payload.json

Webhook testing services:

• webhook.site  - Inspect requests
• ngrok  - Local development tunneling
• Postman  - API testing

M3 Forge test mode:

Click “Test Webhook” in sensor detail view to trigger manually without external request.

Security Best Practices

Authentication

• Always use authentication in production (bearer token or HMAC)
• Rotate secrets every 90 days
• Use HTTPS only (reject HTTP requests)
• Validate payload schema in workflow

Network Security

• IP allowlist when source IPs are known and stable
• Rate limiting to prevent abuse (built-in: 100 req/min)
• Firewall rules to restrict access to webhook endpoints

Payload Validation

Add validation in workflow START node:

{
  "type": "code",
  "config": {
    "language": "python",
    "code": "assert 'user' in data, 'Missing user field'\nassert data['user']['id'] > 0, 'Invalid user ID'"
  }
}

Fail fast on malformed payloads.

Monitoring

• Alert on failures (repeated 401/403 errors)
• Track unusual patterns (sudden spike in requests)
• Log all requests for audit trail
• Monitor latency (slow responses may indicate attack)

Performance Optimization

Async Processing

Workflows execute asynchronously. Webhook returns 200 OK immediately, before workflow completes.

Timeline:

1. Webhook request arrives (0ms)
2. Authentication validated (5ms)
3. Workflow queued (10ms)
4. 200 OK returned to sender (10ms)
5. Workflow executes (seconds to minutes later)

This ensures webhook sources don’t timeout waiting for workflow completion.

Payload Size

Large payloads (> 1 MB) increase processing time:

• Prefer references - Send URLs, not full content
• Compress data - Use gzip encoding
• Paginate - Send multiple small webhooks instead of one large

Idempotency

Webhook sources may retry failed requests, causing duplicate processing.

Make workflows idempotent:

# Check if already processed
existing = db.query(f"SELECT * FROM jobs WHERE webhook_id = '{webhook_id}'")
if existing:
    return {"status": "already_processed"}
 
# Process and record
process_data(data)
db.insert(f"INSERT INTO jobs (webhook_id, ...) VALUES ('{webhook_id}', ...)")

Use webhook ID or request ID as deduplication key.

Next Steps

• Configure trigger types for other automation patterns
• Set up scheduling for time-based execution
• Monitor webhook health in the Automation dashboard
• Integrate webhooks into workflows for real-time processing