DEV Community: Mahmoud Sayed

Apigee Logger Shared Flow - Implementation Guide

Mahmoud Sayed — Mon, 08 Sep 2025 20:45:33 +0000

Apigee Logger Shared Flow - Implementation Guide

Overview

This comprehensive logging solution provides structured logging for Apigee API proxies with support for both ELK (Elasticsearch, Logstash, Kibana) and Apache Spark. The solution includes advanced data masking capabilities to protect sensitive information.

Features

Core Functionality

Dual Destination Logging: Simultaneously logs to ELK and Spark platforms
Request/Response Logging: Captures complete API transaction details
Sensitive Data Masking: Automatically masks passwords, tokens, keys, PII data
Performance Metrics: Tracks latency and processing times
Error Handling: Comprehensive error capture and logging
Asynchronous Processing: Non-blocking log transmission

Data Masking Capabilities

Email Masking: john.doe@example.com → jo***@example.com
Phone Masking: 555-123-4567 → 555***4567
Credit Card Masking: 1234-5678-9012-3456 → 1234****3456
SSN Masking: 123-45-6789 → XXX-XX-6789
Token/Key Masking: Full masking for security tokens and API keys
Password Masking: Complete masking of password fields

Installation Steps

1. Deploy Shared Flow Components

Create the following directory structure in your Apigee project:

shared-flows/
└── logger-shared-flow/
    ├── sharedflowbundle/
    │   ├── logger-shared-flow.xml
    │   ├── policies/
    │   │   ├── Extract-Request-Data.xml
    │   │   ├── Mask-Sensitive-Request-Data.xml
    │   │   ├── Build-Request-Log-Payload.xml
    │   │   ├── Send-Request-Log-ELK.xml
    │   │   ├── Send-Request-Log-Spark.xml
    │   │   ├── Extract-Response-Data.xml
    │   │   ├── Mask-Sensitive-Response-Data.xml
    │   │   ├── Build-Response-Log-Payload.xml
    │   │   ├── Send-Response-Log-ELK.xml
    │   │   └── Send-Response-Log-Spark.xml
    │   └── resources/
    │       └── jsc/
    │           ├── mask-sensitive-data.js
    │           └── mask-sensitive-response-data.js

2. Configure Environment Variables

Create KVM (Key-Value Map) entries for your environment:

# Using Apigee Management API
curl -X POST \
  https://api.enterprise.apigee.com/v1/organizations/{org}/environments/{env}/keyvaluemaps \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "logger-config",
    "encrypted": false
  }'

# Add configuration entries
curl -X POST \
  https://api.enterprise.apigee.com/v1/organizations/{org}/environments/{env}/keyvaluemaps/logger-config/entries \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "elk.logging.enabled",
    "value": "true"
  }'

3. Deploy the Shared Flow

# Deploy using Apigee CLI
apigee sharedflows deploy -o {org} -e {env} -n logger-shared-flow -f ./shared-flows/logger-shared-flow

Configuration

ELK Configuration Parameters

Parameter	Description	Example Value
`elk.logging.enabled`	Enable/disable ELK logging	`true`
`elk.endpoint.url`	Elasticsearch endpoint URL	`https://elasticsearch.company.com:9200`
`elk.index.name`	Index pattern for logs	`apigee-logs-{YYYY.MM.DD}`
`elk.auth.token`	Bearer token for authentication	`eyJhbGciOiJIUzI1NiIs...`

Spark Configuration Parameters

Parameter	Description	Example Value
`spark.logging.enabled`	Enable/disable Spark logging	`true`
`spark.endpoint.url`	Spark cluster endpoint	`https://spark.company.com`
`spark.topic.name`	Kafka topic for logs	`apigee-api-logs`
`spark.partition.key`	Partitioning strategy	`default`
`spark.auth.credentials`	Base64 encoded credentials	`dXNlcjpwYXNzd29yZA==`

Usage in API Proxies

Request Logging (PreFlow)

Add this FlowCallout policy in your API proxy's PreFlow Request:

<FlowCallout async="false" continueOnError="true" enabled="true" name="Log-API-Request">
    <DisplayName>Log API Request</DisplayName>
    <SharedFlowBundle>logger-shared-flow</SharedFlowBundle>
</FlowCallout>

Response Logging (PostFlow)

Add this FlowCallout policy in your API proxy's PostFlow Response:

<FlowCallout async="false" continueOnError="true" enabled="true" name="Log-API-Response">
    <DisplayName>Log API Response</DisplayName>
    <SharedFlowBundle>logger-shared-flow</SharedFlowBundle>
</FlowCallout>

Complete API Proxy Integration Example

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ProxyEndpoint name="default">
    <Description/>
    <FaultRules/>
    <PreFlow name="PreFlow">
        <Request>
            <Step>
                <Name>Log-API-Request</Name>
                <Condition>request.verb != "OPTIONS"</Condition>
            </Step>
        </Request>
        <Response/>
    </PreFlow>
    <PostFlow name="PostFlow">
        <Request/>
        <Response>
            <Step>
                <Name>Log-API-Response</Name>
                <Condition>request.verb != "OPTIONS"</Condition>
            </Step>
        </Response>
    </PostFlow>
    <Flows/>
    <HTTPProxyConnection>
        <BasePath>/api/v1/users</BasePath>
        <Properties/>
        <VirtualHost>secure</VirtualHost>
    </HTTPProxyConnection>
    <RouteRule name="default">
        <TargetEndpoint>default</TargetEndpoint>
    </RouteRule>
</ProxyEndpoint>

Log Structure

Request Log Schema

{
  "eventType": "api_request",
  "timestamp": "2025-01-01T12:00:00.000Z",
  "correlationId": "12345678-1234-1234-1234-123456789012",
  "environment": "prod",
  "organization": "company",
  "apiProxy": "users-api",
  "request": {
    "method": "POST",
    "uri": "/api/v1/users",
    "headers": {
      "authorization": "Be***er token",
      "xApiKey": "ak_****5678"
    },
    "payload": {
      "username": "jo***oe",
      "email": "jo***@example.com",
      "password": "********"
    }
  },
  "client": {
    "ip": "192.168.1.100",
    "userAgent": "Mozilla/5.0..."
  },
  "masking": {
    "status": "completed",
    "timestamp": "2025-01-01T12:00:00.000Z"
  }
}

Response Log Schema

{
  "eventType": "api_response",
  "timestamp": "2025-01-01T12:00:01.500Z",
  "correlationId": "12345678-1234-1234-1234-123456789012",
  "response": {
    "statusCode": 201,
    "statusMessage": "Created",
    "payload": {
      "user_id": "user123",
      "access_token": "ey***abc",
      "refresh_token": "rt***xyz"
    }
  },
  "performance": {
    "totalLatency": 1500,
    "targetLatency": 800
  },
  "error": {
    "isError": false
  }
}

Security Considerations

Sensitive Data Protection

All passwords are completely masked
API keys and tokens show only prefix/suffix
Email addresses mask the local part
Phone numbers mask middle digits
Credit cards show only first 4 and last 4 digits
SSN shows only last 4 digits

Best Practices

Regular Token Rotation: Rotate ELK and Spark authentication tokens regularly
Network Security: Ensure encrypted connections (HTTPS/TLS) to logging endpoints
Access Control: Implement proper IAM policies for log access
Data Retention: Configure appropriate log retention policies
Monitoring: Set up alerts for logging failures

Troubleshooting

Common Issues

Logging Disabled: Check KVM configuration for enabled flags
Authentication Failures: Verify bearer tokens and credentials
Network Timeouts: Adjust timeout settings in ServiceCallout policies
Masking Errors: Check JavaScript policy execution logs

Debug Mode

Enable debug logging by adding this variable to your KVM:

debug.logging.enabled = true

Performance Monitoring

Monitor these metrics:

Log transmission latency
Failed log deliveries
Masking operation time
Memory usage during processing

Advanced Configuration

Custom Masking Rules

To add custom masking rules, modify the JavaScript resources:

// Add custom sensitive field
var customSensitiveFields = ['custom_field', 'proprietary_data'];
sensitiveFields = sensitiveFields.concat(customSensitiveFields);

// Add custom masking pattern
maskingPatterns['custom_field'] = function(value) {
    // Custom masking logic
    return customMaskFunction(value);
};

Environment-Specific Settings

Create separate KVM configurations for different environments:

# Development
logger-config-dev

# Testing  
logger-config-test

# Production
logger-config-prod

Integration with Monitoring Tools

The logs can be integrated with various monitoring and alerting tools:

ELK Alerts: Create Elasticsearch Watcher alerts for error conditions
Spark Monitoring: Use Spark Streaming for real-time analytics
Grafana Dashboards: Create visualization dashboards for log metrics
Splunk Integration: Forward logs to Splunk for enterprise monitoring

This comprehensive logging solution provides enterprise-grade logging capabilities while maintaining security and performance standards.

I'll create a comprehensive Apigee shared flow for logging to ELK and Spark with data masking capabilities. This will include request/response logging and sensitive data protection.

Apigee Logger Shared Flow - Main Flow

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<SharedFlow revision="1" name="logger-shared-flow">
    <DisplayName>ELK Spark Logger Shared Flow</DisplayName>
    <Description>Comprehensive logging shared flow for ELK and Spark with data masking</Description>

    <Step>
        <Name>Extract-Request-Data</Name>
        <Condition>(request.verb != "OPTIONS")</Condition>
    </Step>

    <Step>
        <Name>Mask-Sensitive-Request-Data</Name>
        <Condition>(request.verb != "OPTIONS")</Condition>
    </Step>

    <Step>
        <Name>Build-Request-Log-Payload</Name>
        <Condition>(request.verb != "OPTIONS")</Condition>
    </Step>

    <Step>
        <Name>Send-Request-Log-ELK</Name>
        <Condition>(request.verb != "OPTIONS") and (elk.logging.enabled = true)</Condition>
    </Step>

    <Step>
        <Name>Send-Request-Log-Spark</Name>
        <Condition>(request.verb != "OPTIONS") and (spark.logging.enabled = true)</Condition>
    </Step>

    <Step>
        <Name>Extract-Response-Data</Name>
        <Condition>(request.verb != "OPTIONS")</Condition>
    </Step>

    <Step>
        <Name>Mask-Sensitive-Response-Data</Name>
        <Condition>(request.verb != "OPTIONS")</Condition>
    </Step>

    <Step>
        <Name>Build-Response-Log-Payload</Name>
        <Condition>(request.verb != "OPTIONS")</Condition>
    </Step>

    <Step>
        <Name>Send-Response-Log-ELK</Name>
        <Condition>(request.verb != "OPTIONS") and (elk.logging.enabled = true)</Condition>
    </Step>

    <Step>
        <Name>Send-Response-Log-Spark</Name>
        <Condition>(request.verb != "OPTIONS") and (spark.logging.enabled = true)</Condition>
    </Step>
</SharedFlow>

Extract Request Data Policy

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ExtractVariables async="false" continueOnError="true" enabled="true" name="Extract-Request-Data">
    <DisplayName>Extract Request Data</DisplayName>
    <Properties/>

    <!-- Extract basic request information -->
    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>

    <!-- Request Headers -->
    <Header name="Content-Type">
        <Pattern ignoreCase="true">{request.header.content-type}</Pattern>
    </Header>

    <Header name="Authorization">
        <Pattern ignoreCase="true">{request.header.authorization}</Pattern>
    </Header>

    <Header name="User-Agent">
        <Pattern ignoreCase="true">{request.header.user-agent}</Pattern>
    </Header>

    <Header name="X-API-Key">
        <Pattern ignoreCase="true">{request.header.x-api-key}</Pattern>
    </Header>

    <!-- Query Parameters -->
    <QueryParam name="username">
        <Pattern ignoreCase="true">{request.queryparam.username}</Pattern>
    </QueryParam>

    <QueryParam name="password">
        <Pattern ignoreCase="true">{request.queryparam.password}</Pattern>
    </QueryParam>

    <QueryParam name="token">
        <Pattern ignoreCase="true">{request.queryparam.token}</Pattern>
    </QueryParam>

    <QueryParam name="api_key">
        <Pattern ignoreCase="true">{request.queryparam.api_key}</Pattern>
    </QueryParam>

    <!-- URI Path Parameters -->
    <URIPath>
        <Pattern ignoreCase="false">/v{version}/*/users/{user_id}/*</Pattern>
    </URIPath>

    <!-- JSON Path Extraction for Request Body -->
    <JSONPayload>
        <Variable name="req.username" type="string">
            <JSONPath>$.username</JSONPath>
        </Variable>
        <Variable name="req.password" type="string">
            <JSONPath>$.password</JSONPath>
        </Variable>
        <Variable name="req.email" type="string">
            <JSONPath>$.email</JSONPath>
        </Variable>
        <Variable name="req.phone" type="string">
            <JSONPath>$.phone</JSONPath>
        </Variable>
        <Variable name="req.ssn" type="string">
            <JSONPath>$.ssn</JSONPath>
        </Variable>
        <Variable name="req.credit_card" type="string">
            <JSONPath>$.credit_card</JSONPath>
        </Variable>
        <Variable name="req.api_key" type="string">
            <JSONPath>$.api_key</JSONPath>
        </Variable>
        <Variable name="req.access_token" type="string">
            <JSONPath>$.access_token</JSONPath>
        </Variable>
        <Variable name="req.refresh_token" type="string">
            <JSONPath>$.refresh_token</JSONPath>
        </Variable>
        <Variable name="req.client_secret" type="string">
            <JSONPath>$.client_secret</JSONPath>
        </Variable>
    </JSONPayload>

    <Source>request</Source>
</ExtractVariables>

Mask Sensitive Request Data Policy

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Javascript async="false" continueOnError="true" enabled="true" name="Mask-Sensitive-Request-Data">
    <DisplayName>Mask Sensitive Request Data</DisplayName>
    <Properties/>
    <ResourceURL>jsc://mask-sensitive-data.js</ResourceURL>
</Javascript>

var sensitiveFields = [
    'password', 'pwd', 'pass', 'secret', 'token', 'key', 'auth', 'authorization',
    'username', 'user', 'email', 'phone', 'ssn', 'social', 'credit_card', 'cc',
    'access_token', 'refresh_token', 'client_secret', 'api_key', 'x-api-key'
];

var maskingPatterns = {
    'email': function(value) {
        if (!value) return value;
        var parts = value.split('@');
        if (parts.length === 2) {
            var localPart = parts[0];
            var maskedLocal = localPart.length > 2 ? 
                localPart.substring(0, 2) + '*'.repeat(localPart.length - 2) : 
                '*'.repeat(localPart.length);
            return maskedLocal + '@' + parts[1];
        }
        return maskDefault(value);
    },
    'phone': function(value) {
        if (!value) return value;
        var cleaned = value.replace(/\D/g, '');
        if (cleaned.length >= 10) {
            return cleaned.substring(0, 3) + '*'.repeat(cleaned.length - 6) + cleaned.substring(cleaned.length - 3);
        }
        return maskDefault(value);
    },
    'credit_card': function(value) {
        if (!value) return value;
        var cleaned = value.replace(/\D/g, '');
        if (cleaned.length >= 12) {
            return cleaned.substring(0, 4) + '*'.repeat(cleaned.length - 8) + cleaned.substring(cleaned.length - 4);
        }
        return maskDefault(value);
    },
    'ssn': function(value) {
        if (!value) return value;
        var cleaned = value.replace(/\D/g, '');
        if (cleaned.length === 9) {
            return 'XXX-XX-' + cleaned.substring(5);
        }
        return maskDefault(value);
    }
};

function maskDefault(value) {
    if (!value) return value;
    if (value.length <= 4) {
        return '*'.repeat(value.length);
    }
    return value.substring(0, 2) + '*'.repeat(value.length - 4) + value.substring(value.length - 2);
}

function maskSensitiveValue(key, value) {
    if (!value) return value;

    var lowerKey = key.toLowerCase();

    // Check for specific masking patterns
    for (var pattern in maskingPatterns) {
        if (lowerKey.includes(pattern)) {
            return maskingPatterns[pattern](value);
        }
    }

    // Check if it's a sensitive field
    for (var i = 0; i < sensitiveFields.length; i++) {
        if (lowerKey.includes(sensitiveFields[i])) {
            return maskDefault(value);
        }
    }

    return value;
}

function maskHeaders() {
    var headers = ['authorization', 'x-api-key', 'x-auth-token'];

    for (var i = 0; i < headers.length; i++) {
        var headerName = headers[i];
        var headerValue = context.getVariable('request.header.' + headerName);
        if (headerValue) {
            var maskedValue = maskSensitiveValue(headerName, headerValue);
            context.setVariable('masked.request.header.' + headerName, maskedValue);
        }
    }
}

function maskQueryParams() {
    var queryParams = ['username', 'password', 'token', 'api_key'];

    for (var i = 0; i < queryParams.length; i++) {
        var paramName = queryParams[i];
        var paramValue = context.getVariable('request.queryparam.' + paramName);
        if (paramValue) {
            var maskedValue = maskSensitiveValue(paramName, paramValue);
            context.setVariable('masked.request.queryparam.' + paramName, maskedValue);
        }
    }
}

function maskJsonFields() {
    var jsonFields = [
        'req.username', 'req.password', 'req.email', 'req.phone', 'req.ssn',
        'req.credit_card', 'req.api_key', 'req.access_token', 'req.refresh_token',
        'req.client_secret'
    ];

    for (var i = 0; i < jsonFields.length; i++) {
        var fieldName = jsonFields[i];
        var fieldValue = context.getVariable(fieldName);
        if (fieldValue) {
            var key = fieldName.replace('req.', '');
            var maskedValue = maskSensitiveValue(key, fieldValue);
            context.setVariable('masked.' + fieldName, maskedValue);
        }
    }
}

function maskRequestPayload() {
    try {
        var requestPayload = context.getVariable('request.content');
        if (requestPayload && context.getVariable('request.header.content-type') && 
            context.getVariable('request.header.content-type').indexOf('application/json') > -1) {

            var payload = JSON.parse(requestPayload);
            var maskedPayload = maskObjectRecursively(payload);
            context.setVariable('masked.request.payload', JSON.stringify(maskedPayload, null, 2));
        } else {
            context.setVariable('masked.request.payload', requestPayload || '');
        }
    } catch (e) {
        context.setVariable('masked.request.payload', 'Error parsing request payload: ' + e.message);
    }
}

function maskObjectRecursively(obj) {
    if (obj === null || obj === undefined) return obj;

    if (typeof obj !== 'object') return obj;

    if (Array.isArray(obj)) {
        return obj.map(function(item) {
            return maskObjectRecursively(item);
        });
    }

    var maskedObj = {};
    for (var key in obj) {
        if (obj.hasOwnProperty(key)) {
            var value = obj[key];
            if (typeof value === 'object' && value !== null) {
                maskedObj[key] = maskObjectRecursively(value);
            } else if (typeof value === 'string') {
                maskedObj[key] = maskSensitiveValue(key, value);
            } else {
                maskedObj[key] = value;
            }
        }
    }
    return maskedObj;
}

// Execute masking
try {
    maskHeaders();
    maskQueryParams();
    maskJsonFields();
    maskRequestPayload();

    context.setVariable('masking.request.status', 'completed');
    context.setVariable('masking.request.timestamp', new Date().toISOString());
} catch (error) {
    context.setVariable('masking.request.status', 'error');
    context.setVariable('masking.request.error', error.message);
}

Build Request Log Payload Policy

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage async="false" continueOnError="true" enabled="true" name="Build-Request-Log-Payload">
    <DisplayName>Build Request Log Payload</DisplayName>
    <Properties/>

    <AssignTo createNew="true" transport="http" type="request">logRequest</AssignTo>

    <Set>
        <Headers>
            <Header name="Content-Type">application/json</Header>
        </Headers>
    </Set>

    <AssignVariable>
        <Name>log.event.type</Name>
        <Value>api_request</Value>
    </AssignVariable>

    <AssignVariable>
        <Name>log.timestamp</Name>
        <Value>{system.timestamp}</Value>
    </AssignVariable>

    <AssignVariable>
        <Name>log.correlation.id</Name>
        <Value>{messageid}</Value>
    </AssignVariable>

    <Payload contentType="application/json" variablePrefix="@" variableSuffix="#">
{
  "eventType": "api_request",
  "timestamp": "@system.timestamp#",
  "correlationId": "@messageid#",
  "environment": "@environment.name#",
  "organization": "@organization.name#",
  "apiProxy": "@apiproxy.name#",
  "apiProxyRevision": "@apiproxy.revision#",
  "request": {
    "method": "@request.verb#",
    "uri": "@request.uri#",
    "path": "@request.path#",
    "queryString": "@request.querystring#",
    "protocol": "@request.scheme#",
    "headers": {
      "contentType": "@masked.request.header.content-type#",
      "userAgent": "@request.header.user-agent#",
      "authorization": "@masked.request.header.authorization#",
      "xApiKey": "@masked.request.header.x-api-key#",
      "host": "@request.header.host#",
      "accept": "@request.header.accept#",
      "acceptEncoding": "@request.header.accept-encoding#"
    },
    "queryParams": {
      "username": "@masked.request.queryparam.username#",
      "password": "@masked.request.queryparam.password#",
      "token": "@masked.request.queryparam.token#",
      "apiKey": "@masked.request.queryparam.api_key#"
    },
    "pathParams": {
      "version": "@version#",
      "userId": "@user_id#"
    },
    "payload": @masked.request.payload#,
    "size": "@request.header.content-length#"
  },
  "client": {
    "ip": "@client.ip#",
    "host": "@client.host#",
    "port": "@client.port#",
    "userAgent": "@request.header.user-agent#"
  },
  "security": {
    "authenticated": "@is.authenticated#",
    "userId": "@user.id#",
    "clientId": "@client_id#",
    "scopes": "@oauth.scope#"
  },
  "flow": {
    "name": "logger-shared-flow",
    "step": "request-logging",
    "executionTime": "@flow.execution.time#"
  },
  "masking": {
    "status": "@masking.request.status#",
    "timestamp": "@masking.request.timestamp#",
    "fieldsCount": "@masking.fields.count#"
  },
  "metadata": {
    "source": "apigee-gateway",
    "version": "1.0",
    "logLevel": "INFO",
    "component": "request-logger"
  }
}
    </Payload>
</AssignMessage>

Send Request Log to ELK Policy

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ServiceCallout async="true" continueOnError="true" enabled="true" name="Send-Request-Log-ELK">
    <DisplayName>Send Request Log to ELK</DisplayName>
    <Properties/>

    <Request clearPayload="false" variable="logRequest">
        <Set>
            <Headers>
                <Header name="Content-Type">application/json</Header>
                <Header name="Authorization">Bearer {elk.auth.token}</Header>
                <Header name="X-Correlation-ID">{messageid}</Header>
            </Headers>
        </Set>
    </Request>

    <Response>elkRequestResponse</Response>

    <HTTPTargetConnection>
        <Properties>
            <Property name="keepalive">true</Property>
            <Property name="connect.timeout.millis">5000</Property>
            <Property name="io.timeout.millis">30000</Property>
        </Properties>
        <URL>{elk.endpoint.url}/{elk.index.name}/_doc</URL>
    </HTTPTargetConnection>
</ServiceCallout>

Send Request Log to Spark Policy

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ServiceCallout async="true" continueOnError="true" enabled="true" name="Send-Request-Log-Spark">
    <DisplayName>Send Request Log to Spark</DisplayName>
    <Properties/>

    <Request clearPayload="false" variable="logRequest">
        <Set>
            <Headers>
                <Header name="Content-Type">application/json</Header>
                <Header name="Authorization">Basic {spark.auth.credentials}</Header>
                <Header name="X-Correlation-ID">{messageid}</Header>
                <Header name="X-Spark-Topic">{spark.topic.name}</Header>
            </Headers>
        </Set>
    </Request>

    <Response>sparkRequestResponse</Response>

    <HTTPTargetConnection>
        <Properties>
            <Property name="keepalive">true</Property>
            <Property name="connect.timeout.millis">3000</Property>
            <Property name="io.timeout.millis">15000</Property>
        </Properties>
        <URL>{spark.endpoint.url}/api/v1/logs/stream</URL>
    </HTTPTargetConnection>
</ServiceCallout>

Extract Response Data Policy

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ExtractVariables async="false" continueOnError="true" enabled="true" name="Extract-Response-Data">
    <DisplayName>Extract Response Data</DisplayName>
    <Properties/>

    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>

    <!-- Response Headers -->
    <Header name="Content-Type">
        <Pattern ignoreCase="true">{response.header.content-type}</Pattern>
    </Header>

    <Header name="Authorization">
        <Pattern ignoreCase="true">{response.header.authorization}</Pattern>
    </Header>

    <Header name="Set-Cookie">
        <Pattern ignoreCase="true">{response.header.set-cookie}</Pattern>
    </Header>

    <Header name="X-Auth-Token">
        <Pattern ignoreCase="true">{response.header.x-auth-token}</Pattern>
    </Header>

    <!-- JSON Path Extraction for Response Body -->
    <JSONPayload>
        <Variable name="resp.access_token" type="string">
            <JSONPath>$.access_token</JSONPath>
        </Variable>
        <Variable name="resp.refresh_token" type="string">
            <JSONPath>$.refresh_token</JSONPath>
        </Variable>
        <Variable name="resp.api_key" type="string">
            <JSONPath>$.api_key</JSONPath>
        </Variable>
        <Variable name="resp.client_secret" type="string">
            <JSONPath>$.client_secret</JSONPath>
        </Variable>
        <Variable name="resp.password" type="string">
            <JSONPath>$.password</JSONPath>
        </Variable>
        <Variable name="resp.username" type="string">
            <JSONPath>$.username</JSONPath>
        </Variable>
        <Variable name="resp.email" type="string">
            <JSONPath>$.email</JSONPath>
        </Variable>
        <Variable name="resp.phone" type="string">
            <JSONPath>$.phone</JSONPath>
        </Variable>
        <Variable name="resp.ssn" type="string">
            <JSONPath>$.ssn</JSONPath>
        </Variable>
        <Variable name="resp.credit_card" type="string">
            <JSONPath>$.credit_card</JSONPath>
        </Variable>
        <Variable name="resp.session_id" type="string">
            <JSONPath>$.session_id</JSONPath>
        </Variable>
        <Variable name="resp.user_id" type="string">
            <JSONPath>$.user_id</JSONPath>
        </Variable>
        <Variable name="resp.status" type="string">
            <JSONPath>$.status</JSONPath>
        </Variable>
        <Variable name="resp.message" type="string">
            <JSONPath>$.message</JSONPath>
        </Variable>
        <Variable name="resp.error_code" type="string">
            <JSONPath>$.error.code</JSONPath>
        </Variable>
        <Variable name="resp.error_message" type="string">
            <JSONPath>$.error.message</JSONPath>
        </Variable>
        <Variable name="resp.expires_in" type="integer">
            <JSONPath>$.expires_in</JSONPath>
        </Variable>
        <Variable name="resp.token_type" type="string">
            <JSONPath>$.token_type</JSONPath>
        </Variable>
    </JSONPayload>

    <Source>response</Source>
</ExtractVariables>

Mask Sensitive Response Data Policy

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Javascript async="false" continueOnError="true" enabled="true" name="Mask-Sensitive-Response-Data">
    <DisplayName>Mask Sensitive Response Data</DisplayName>
    <Properties/>
    <ResourceURL>jsc://mask-sensitive-response-data.js</ResourceURL>
</Javascript>

// Reuse the masking functions from request masking
var sensitiveFields = [
    'password', 'pwd', 'pass', 'secret', 'token', 'key', 'auth', 'authorization',
    'username', 'user', 'email', 'phone', 'ssn', 'social', 'credit_card', 'cc',
    'access_token', 'refresh_token', 'client_secret', 'api_key', 'x-api-key',
    'session_id', 'sessionid', 'cookie'
];

var maskingPatterns = {
    'email': function(value) {
        if (!value) return value;
        var parts = value.split('@');
        if (parts.length === 2) {
            var localPart = parts[0];
            var maskedLocal = localPart.length > 2 ? 
                localPart.substring(0, 2) + '*'.repeat(localPart.length - 2) : 
                '*'.repeat(localPart.length);
            return maskedLocal + '@' + parts[1];
        }
        return maskDefault(value);
    },
    'phone': function(value) {
        if (!value) return value;
        var cleaned = value.replace(/\D/g, '');
        if (cleaned.length >= 10) {
            return cleaned.substring(0, 3) + '*'.repeat(cleaned.length - 6) + cleaned.substring(cleaned.length - 3);
        }
        return maskDefault(value);
    },
    'credit_card': function(value) {
        if (!value) return value;
        var cleaned = value.replace(/\D/g, '');
        if (cleaned.length >= 12) {
            return cleaned.substring(0, 4) + '*'.repeat(cleaned.length - 8) + cleaned.substring(cleaned.length - 4);
        }
        return maskDefault(value);
    },
    'ssn': function(value) {
        if (!value) return value;
        var cleaned = value.replace(/\D/g, '');
        if (cleaned.length === 9) {
            return 'XXX-XX-' + cleaned.substring(5);
        }
        return maskDefault(value);
    },
    'session': function(value) {
        if (!value) return value;
        if (value.length > 8) {
            return value.substring(0, 4) + '*'.repeat(value.length - 8) + value.substring(value.length - 4);
        }
        return maskDefault(value);
    }
};

function maskDefault(value) {
    if (!value) return value;
    if (value.length <= 4) {
        return '*'.repeat(value.length);
    }
    return value.substring(0, 2) + '*'.repeat(value.length - 4) + value.substring(value.length - 2);
}

function maskSensitiveValue(key, value) {
    if (!value) return value;

    var lowerKey = key.toLowerCase();

    // Check for specific masking patterns
    for (var pattern in maskingPatterns) {
        if (lowerKey.includes(pattern)) {
            return maskingPatterns[pattern](value);
        }
    }

    // Check if it's a sensitive field
    for (var i = 0; i < sensitiveFields.length; i++) {
        if (lowerKey.includes(sensitiveFields[i])) {
            return maskDefault(value);
        }
    }

    return value;
}

function maskResponseHeaders() {
    var headers = ['authorization', 'set-cookie', 'x-auth-token'];

    for (var i = 0; i < headers.length; i++) {
        var headerName = headers[i];
        var headerValue = context.getVariable('response.header.' + headerName);
        if (headerValue) {
            var maskedValue = maskSensitiveValue(headerName, headerValue);
            context.setVariable('masked.response.header.' + headerName, maskedValue);
        }
    }
}

function maskResponseJsonFields() {
    var jsonFields = [
        'resp.access_token', 'resp.refresh_token', 'resp.api_key', 'resp.client_secret',
        'resp.password', 'resp.username', 'resp.email', 'resp.phone', 'resp.ssn',
        'resp.credit_card', 'resp.session_id'
    ];

    for (var i = 0; i < jsonFields.length; i++) {
        var fieldName = jsonFields[i];
        var fieldValue = context.getVariable(fieldName);
        if (fieldValue) {
            var key = fieldName.replace('resp.', '');
            var maskedValue = maskSensitiveValue(key, fieldValue);
            context.setVariable('masked.' + fieldName, maskedValue);
        }
    }
}

function maskResponsePayload() {
    try {
        var responsePayload = context.getVariable('response.content');
        if (responsePayload && context.getVariable('response.header.content-type') && 
            context.getVariable('response.header.content-type').indexOf('application/json') > -1) {

            var payload = JSON.parse(responsePayload);
            var maskedPayload = maskObjectRecursively(payload);
            context.setVariable('masked.response.payload', JSON.stringify(maskedPayload, null, 2));
        } else {
            context.setVariable('masked.response.payload', responsePayload || '');
        }
    } catch (e) {
        context.setVariable('masked.response.payload', 'Error parsing response payload: ' + e.message);
    }
}

function maskObjectRecursively(obj) {
    if (obj === null || obj === undefined) return obj;

    if (typeof obj !== 'object') return obj;

    if (Array.isArray(obj)) {
        return obj.map(function(item) {
            return maskObjectRecursively(item);
        });
    }

    var maskedObj = {};
    for (var key in obj) {
        if (obj.hasOwnProperty(key)) {
            var value = obj[key];
            if (typeof value === 'object' && value !== null) {
                maskedObj[key] = maskObjectRecursively(value);
            } else if (typeof value === 'string') {
                maskedObj[key] = maskSensitiveValue(key, value);
            } else {
                maskedObj[key] = value;
            }
        }
    }
    return maskedObj;
}

// Execute masking
try {
    maskResponseHeaders();
    maskResponseJsonFields();
    maskResponsePayload();

    context.setVariable('masking.response.status', 'completed');
    context.setVariable('masking.response.timestamp', new Date().toISOString());
} catch (error) {
    context.setVariable('masking.response.status', 'error');
    context.setVariable('masking.response.error', error.message);
}

Build Response Log Payload Policy

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage async="false" continueOnError="true" enabled="true" name="Build-Response-Log-Payload">
    <DisplayName>Build Response Log Payload</DisplayName>
    <Properties/>

    <AssignTo createNew="true" transport="http" type="request">logResponse</AssignTo>

    <Set>
        <Headers>
            <Header name="Content-Type">application/json</Header>
        </Headers>
    </Set>

    <AssignVariable>
        <n>log.response.event.type</n>
        <Value>api_response</Value>
    </AssignVariable>

    <AssignVariable>
        <n>log.response.timestamp</n>
        <Value>{system.timestamp}</Value>
    </AssignVariable>

    <AssignVariable>
        <n>log.response.latency</n>
        <Value>{client.received.end.timestamp - client.received.start.timestamp}</Value>
    </AssignVariable>

    <Payload contentType="application/json" variablePrefix="@" variableSuffix="#">
{
  "eventType": "api_response",
  "timestamp": "@system.timestamp#",
  "correlationId": "@messageid#",
  "environment": "@environment.name#",
  "organization": "@organization.name#",
  "apiProxy": "@apiproxy.name#",
  "apiProxyRevision": "@apiproxy.revision#",
  "response": {
    "statusCode": @response.status.code#,
    "statusMessage": "@response.reason.phrase#",
    "headers": {
      "contentType": "@response.header.content-type#",
      "contentLength": "@response.header.content-length#",
      "server": "@response.header.server#",
      "cacheControl": "@response.header.cache-control#",
      "authorization": "@masked.response.header.authorization#",
      "setCookie": "@masked.response.header.set-cookie#",
      "xAuthToken": "@masked.response.header.x-auth-token#"
    },
    "payload": @masked.response.payload#,
    "size": "@response.header.content-length#"
  },
  "performance": {
    "totalLatency": "@client.received.end.timestamp - client.received.start.timestamp#",
    "targetLatency": "@target.received.end.timestamp - target.sent.start.timestamp#",
    "requestProcessingTime": "@request.processing.time#",
    "responseProcessingTime": "@response.processing.time#"
  },
  "backend": {
    "url": "@target.url#",
    "host": "@target.host#",
    "port": "@target.port#",
    "ssl": "@target.ssl#",
    "statusCode": "@target.response.status.code#"
  },
  "client": {
    "ip": "@client.ip#",
    "host": "@client.host#",
    "port": "@client.port#"
  },
  "security": {
    "authenticated": "@is.authenticated#",
    "userId": "@user.id#",
    "clientId": "@client_id#",
    "scopes": "@oauth.scope#"
  },
  "flow": {
    "name": "logger-shared-flow",
    "step": "response-logging",
    "executionTime": "@flow.execution.time#"
  },
  "masking": {
    "status": "@masking.response.status#",
    "timestamp": "@masking.response.timestamp#",
    "fieldsCount": "@masking.fields.count#"
  },
  "error": {
    "isError": @is.error#,
    "code": "@error.code#",
    "message": "@error.message#",
    "source": "@error.source#"
  },
  "analytics": {
    "apiProduct": "@apiproduct.name#",
    "developer": "@developer.email#",
    "app": "@client_id#"
  },
  "metadata": {
    "source": "apigee-gateway",
    "version": "1.0",
    "logLevel": "INFO",
    "component": "response-logger"
  }
}
    </Payload>
</AssignMessage>

Send Response Log to ELK Policy

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ServiceCallout async="true" continueOnError="true" enabled="true" name="Send-Response-Log-ELK">
    <DisplayName>Send Response Log to ELK</DisplayName>
    <Properties/>

    <Request clearPayload="false" variable="logResponse">
        <Set>
            <Headers>
                <Header name="Content-Type">application/json</Header>
                <Header name="Authorization">Bearer {elk.auth.token}</Header>
                <Header name="X-Correlation-ID">{messageid}</Header>
                <Header name="X-ELK-Index">{elk.index.name}</Header>
            </Headers>
        </Set>
    </Request>

    <Response>elkResponseResponse</Response>

    <HTTPTargetConnection>
        <Properties>
            <Property name="keepalive">true</Property>
            <Property name="connect.timeout.millis">5000</Property>
            <Property name="io.timeout.millis">30000</Property>
            <Property name="retry.count">2</Property>
            <Property name="retry.interval">1000</Property>
        </Properties>
        <URL>{elk.endpoint.url}/{elk.index.name}/_doc</URL>
    </HTTPTargetConnection>
</ServiceCallout>

Send Response Log to Spark Policy

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ServiceCallout async="true" continueOnError="true" enabled="true" name="Send-Response-Log-Spark">
    <DisplayName>Send Response Log to Spark</DisplayName>
    <Properties/>

    <Request clearPayload="false" variable="logResponse">
        <Set>
            <Headers>
                <Header name="Content-Type">application/json</Header>
                <Header name="Authorization">Basic {spark.auth.credentials}</Header>
                <Header name="X-Correlation-ID">{messageid}</Header>
                <Header name="X-Spark-Topic">{spark.topic.name}</Header>
                <Header name="X-Spark-Partition">{spark.partition.key}</Header>
            </Headers>
        </Set>
    </Request>

    <Response>sparkResponseResponse</Response>

    <HTTPTargetConnection>
        <Properties>
            <Property name="keepalive">true</Property>
            <Property name="connect.timeout.millis">3000</Property>
            <Property name="io.timeout.millis">15000</Property>
            <Property name="retry.count">1</Property>
            <Property name="retry.interval">500</Property>
        </Properties>
        <URL>{spark.endpoint.url}/api/v1/logs/stream</URL>
    </HTTPTargetConnection>
</ServiceCallout>

Configuration Files and Environment Variables

<!-- KVM Configuration for Environment Variables -->
<!-- Create KVM entries for the following variables: -->

<!-- ELK Configuration -->
<KVM name="logger-config">
    <Entry name="elk.logging.enabled">true</Entry>
    <Entry name="elk.endpoint.url">https://your-elasticsearch-cluster.com:9200</Entry>
    <Entry name="elk.index.name">apigee-logs-{YYYY.MM.DD}</Entry>
    <Entry name="elk.auth.token">your-elk-bearer-token</Entry>

    <!-- Spark Configuration -->
    <Entry name="spark.logging.enabled">true</Entry>
    <Entry name="spark.endpoint.url">https://your-spark-cluster.com</Entry>
    <Entry name="spark.topic.name">apigee-api-logs</Entry>
    <Entry name="spark.partition.key">default</Entry>
    <Entry name="spark.auth.credentials">base64-encoded-credentials</Entry>
</KVM>

<!-- Shared Flow Deployment Descriptor -->
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<SharedFlowBundle revision="1" name="logger-shared-flow">
    <ConfigurationVersion majorVersion="4" minorVersion="0"/>
    <CreatedAt>2025-01-01T00:00:00.000Z</CreatedAt>
    <CreatedBy>api-developer@company.com</CreatedBy>
    <Description>ELK Spark Logger Shared Flow with Data Masking</Description>
    <DisplayName>Logger Shared Flow</DisplayName>
    <LastModifiedAt>2025-01-01T00:00:00.000Z</LastModifiedAt>
    <LastModifiedBy>api-developer@company.com</LastModifiedBy>
    <ManifestVersion majorVersion="1" minorVersion="0"/>
    <Policies>
        <Policy>Extract-Request-Data</Policy>
        <Policy>Mask-Sensitive-Request-Data</Policy>
        <Policy>Build-Request-Log-Payload</Policy>
        <Policy>Send-Request-Log-ELK</Policy>
        <Policy>Send-Request-Log-Spark</Policy>
        <Policy>Extract-Response-Data</Policy>
        <Policy>Mask-Sensitive-Response-Data</Policy>
        <Policy>Build-Response-Log-Payload</Policy>
        <Policy>Send-Response-Log-ELK</Policy>
        <Policy>Send-Response-Log-Spark</Policy>
    </Policies>
    <Resources>
        <Resource>jsc://mask-sensitive-data.js</Resource>
        <Resource>jsc://mask-sensitive-response-data.js</Resource>
    </Resources>
</SharedFlowBundle>

<!-- How to use this shared flow in your API Proxy -->
<!-- 
1. Deploy the shared flow to your Apigee environment
2. Add FlowCallout policies in your API proxy to call the shared flow

Example FlowCallout for Request logging:
<FlowCallout async="false" continueOnError="true" enabled="true" name="Log-Request">
    <DisplayName>Log Request</DisplayName>
    <SharedFlowBundle>logger-shared-flow</SharedFlowBundle>
</FlowCallout>

Add this in the PreFlow Request section of your API proxy.

Example FlowCallout for Response logging:
<FlowCallout async="false" continueOnError="true" enabled="true" name="Log-Response">
    <DisplayName>Log Response</DisplayName>
    <SharedFlowBundle>logger-shared-flow</SharedFlowBundle>
</FlowCallout>

Add this in the PostFlow Response section of your API proxy.
-->

<!-- Environment-specific configuration -->
<!-- 
For different environments (dev, test, prod), create separate KVM entries:

DEV Environment:
- elk.endpoint.url: https://dev-elasticsearch.company.com:9200
- spark.endpoint.url: https://dev-spark.company.com

TEST Environment:
- elk.endpoint.url: https://test-elasticsearch.company.com:9200
- spark.endpoint.url: https://test-spark.company.com

PROD Environment:
- elk.endpoint.url: https://prod-elasticsearch.company.com:9200
- spark.endpoint.url: https://prod-spark.company.com
-->

Essential Apigee Shared Flows and Policies

Mahmoud Sayed — Mon, 08 Sep 2025 19:58:06 +0000

Essential Apigee Shared Flows and Policies

Core Shared Flows You Should Build

1. Security Enforcement Shared Flow

Purpose: Centralized security validation including API key validation, OAuth token verification, and IP restrictions.

Implementation:

<!-- Security-Enforcement-SF.xml -->
<SharedFlow name="Security-Enforcement-SF">
  <Step>
    <Name>IP-Whitelist-Policy</Name>
  </Step>
  <Step>
    <Name>API-Key-Validation</Name>
  </Step>
  <Step>
    <Name>OAuth-Token-Validation</Name>
    <Condition>request.header.authorization != null</Condition>
  </Step>
  <Step>
    <Name>Rate-Limiting-Policy</Name>
  </Step>
  <Step>
    <Name>Quota-Policy</Name>
  </Step>
</SharedFlow>

Key Policies:

IP Whitelist Policy:

<!-- IP-Whitelist-Policy.xml -->
<AccessControl name="IP-Whitelist-Policy">
  <IPRules noRuleMatchAction="DENY">
    <MatchRule action="ALLOW">
      <SourceAddress mask="24">192.168.1.0</SourceAddress>
    </MatchRule>
    <MatchRule action="ALLOW">
      <SourceAddress mask="16">10.0.0.0</SourceAddress>
    </MatchRule>
  </IPRules>
</AccessControl>

API Key Validation:

<!-- API-Key-Validation.xml -->
<VerifyAPIKey name="API-Key-Validation">
  <APIKey ref="request.queryparam.apikey"/>
</VerifyAPIKey>

OAuth Token Validation:

<!-- OAuth-Token-Validation.xml -->
<OAuthV2 name="OAuth-Token-Validation">
  <Operation>VerifyAccessToken</Operation>
  <AccessToken ref="request.header.authorization"/>
</OAuthV2>

2. Request Validation Shared Flow

Purpose: Validate request headers, parameters, and payload structure.

Implementation:

<!-- Request-Validation-SF.xml -->
<SharedFlow name="Request-Validation-SF">
  <Step>
    <Name>Content-Type-Validation</Name>
  </Step>
  <Step>
    <Name>Request-Size-Check</Name>
  </Step>
  <Step>
    <Name>JSON-Schema-Validation</Name>
    <Condition>request.header.content-type = "application/json"</Condition>
  </Step>
  <Step>
    <Name>Required-Headers-Check</Name>
  </Step>
  <Step>
    <Name>Request-Sanitization</Name>
  </Step>
</SharedFlow>

Key Policies:

Content Type Validation:

<!-- Content-Type-Validation.xml -->
<RaiseFault name="Content-Type-Validation">
  <Condition>request.header.content-type != "application/json" AND request.verb != "GET"</Condition>
  <FaultResponse>
    <Set>
      <StatusCode>400</StatusCode>
      <ReasonPhrase>Bad Request</ReasonPhrase>
    </Set>
    <Set>
      <Payload contentType="application/json">
        {
          "error": {
            "code": "INVALID_CONTENT_TYPE",
            "message": "Content-Type must be application/json"
          }
        }
      </Payload>
    </Set>
  </FaultResponse>
</RaiseFault>

Request Size Check:

<!-- Request-Size-Check.xml -->
<RaiseFault name="Request-Size-Check">
  <Condition>request.header.content-length > 1048576</Condition>
  <FaultResponse>
    <Set>
      <StatusCode>413</StatusCode>
      <ReasonPhrase>Payload Too Large</ReasonPhrase>
    </Set>
    <Set>
      <Payload contentType="application/json">
        {
          "error": {
            "code": "PAYLOAD_TOO_LARGE",
            "message": "Request payload exceeds 1MB limit"
          }
        }
      </Payload>
    </Set>
  </FaultResponse>
</RaiseFault>

JSON Schema Validation:

<!-- JSON-Schema-Validation.xml -->
<JSONThreatProtection name="JSON-Schema-Validation">
  <ArrayElementCount>100</ArrayElementCount>
  <ContainerDepth>10</ContainerDepth>
  <ObjectEntryCount>50</ObjectEntryCount>
  <ObjectEntryNameLength>100</ObjectEntryNameLength>
  <StringValueLength>500</StringValueLength>
</JSONThreatProtection>

3. Error Handling Shared Flow

Purpose: Standardize error responses across all APIs.

Implementation:

<!-- Error-Handling-SF.xml -->
<SharedFlow name="Error-Handling-SF">
  <Step>
    <Name>Extract-Error-Details</Name>
  </Step>
  <Step>
    <Name>Log-Error</Name>
  </Step>
  <Step>
    <Name>Format-Error-Response</Name>
  </Step>
  <Step>
    <Name>Set-CORS-Headers</Name>
  </Step>
</SharedFlow>

Key Policies:

Extract Error Details:

<!-- Extract-Error-Details.xml -->
<ExtractVariables name="Extract-Error-Details">
  <Source>message</Source>
  <VariablePrefix>error</VariablePrefix>
  <JSONPayload>
    <Variable name="code">
      <JSONPath>$.fault.detail.errorcode</JSONPath>
    </Variable>
    <Variable name="message">
      <JSONPath>$.fault.faultstring</JSONPath>
    </Variable>
  </JSONPayload>
</ExtractVariables>

Log Error:

<!-- Log-Error.xml -->
<MessageLogging name="Log-Error">
  <Syslog>
    <Message>API Error - Code: {error.code}, Message: {error.message}, API: {apiproxy.name}, Client: {client_ip}</Message>
    <Host>logs.company.com</Host>
    <Port>514</Port>
    <Protocol>TCP</Protocol>
  </Syslog>
</MessageLogging>

Format Error Response:

<!-- Format-Error-Response.xml -->
<AssignMessage name="Format-Error-Response">
  <Set>
    <Headers>
      <Header name="Content-Type">application/json</Header>
    </Headers>
    <Payload contentType="application/json">
      {
        "error": {
          "code": "{error.code}",
          "message": "{error.message}",
          "timestamp": "{system.timestamp}",
          "path": "{request.uri}",
          "requestId": "{messageid}"
        }
      }
    </Payload>
  </Set>
</AssignMessage>

4. Response Enhancement Shared Flow

Purpose: Add consistent response headers, CORS support, and response transformation.

Implementation:

<!-- Response-Enhancement-SF.xml -->
<SharedFlow name="Response-Enhancement-SF">
  <Step>
    <Name>Add-Response-Headers</Name>
  </Step>
  <Step>
    <Name>CORS-Policy</Name>
  </Step>
  <Step>
    <Name>Remove-Sensitive-Headers</Name>
  </Step>
  <Step>
    <Name>Response-Cache-Control</Name>
  </Step>
</SharedFlow>

Key Policies:

Add Response Headers:

<!-- Add-Response-Headers.xml -->
<AssignMessage name="Add-Response-Headers">
  <Set>
    <Headers>
      <Header name="X-API-Version">{apiproxy.revision}</Header>
      <Header name="X-Request-ID">{messageid}</Header>
      <Header name="X-Response-Time">{client.received.end.timestamp - client.received.start.timestamp}ms</Header>
      <Header name="X-Rate-Limit-Remaining">{ratelimit.Quota-Policy.remaining.count}</Header>
    </Headers>
  </Set>
</AssignMessage>

CORS Policy:

<!-- CORS-Policy.xml -->
<AssignMessage name="CORS-Policy">
  <Set>
    <Headers>
      <Header name="Access-Control-Allow-Origin">*</Header>
      <Header name="Access-Control-Allow-Methods">GET, POST, PUT, DELETE, OPTIONS</Header>
      <Header name="Access-Control-Allow-Headers">Content-Type, Authorization, X-API-Key</Header>
      <Header name="Access-Control-Max-Age">3628800</Header>
    </Headers>
  </Set>
  <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
</AssignMessage>

5. Logging and Analytics Shared Flow

Purpose: Centralized logging for monitoring, debugging, and analytics.

Implementation:

<!-- Logging-Analytics-SF.xml -->
<SharedFlow name="Logging-Analytics-SF">
  <Step>
    <Name>Extract-Request-Data</Name>
  </Step>
  <Step>
    <Name>Log-Request-Response</Name>
  </Step>
  <Step>
    <Name>Custom-Analytics</Name>
  </Step>
  <Step>
    <Name>Performance-Metrics</Name>
  </Step>
</SharedFlow>

Key Policies:

Extract Request Data:

<!-- Extract-Request-Data.xml -->
<ExtractVariables name="Extract-Request-Data">
  <Source>request</Source>
  <Header name="user-agent">
    <Pattern ignoreCase="true">{user_agent}</Pattern>
  </Header>
  <Header name="x-forwarded-for">
    <Pattern ignoreCase="true">{client_ip}</Pattern>
  </Header>
  <QueryParam name="version">
    <Pattern ignoreCase="true">{api_version}</Pattern>
  </QueryParam>
</ExtractVariables>

Log Request Response:

<!-- Log-Request-Response.xml -->
<MessageLogging name="Log-Request-Response">
  <Syslog>
    <Message>
      {
        "timestamp": "{system.timestamp}",
        "apiproxy": "{apiproxy.name}",
        "method": "{request.verb}",
        "uri": "{request.uri}",
        "client_ip": "{client_ip}",
        "user_agent": "{user_agent}",
        "response_code": "{response.status.code}",
        "response_time": "{client.received.end.timestamp - client.received.start.timestamp}",
        "request_id": "{messageid}"
      }
    </Message>
    <Host>logs.company.com</Host>
    <Port>514</Port>
    <Protocol>TCP</Protocol>
  </Syslog>
</MessageLogging>

6. Traffic Management Shared Flow

Purpose: Handle rate limiting, quotas, and traffic shaping.

Implementation:

<!-- Traffic-Management-SF.xml -->
<SharedFlow name="Traffic-Management-SF">
  <Step>
    <Name>Spike-Arrest</Name>
  </Step>
  <Step>
    <Name>Rate-Limit-Policy</Name>
  </Step>
  <Step>
    <Name>Quota-Policy</Name>
  </Step>
  <Step>
    <Name>Concurrent-Rate-Limit</Name>
  </Step>
</SharedFlow>

Key Policies:

Spike Arrest:

<!-- Spike-Arrest.xml -->
<SpikeArrest name="Spike-Arrest">
  <Rate>100pm</Rate>
  <Identifier ref="client_ip"/>
</SpikeArrest>

Rate Limit Policy:

<!-- Rate-Limit-Policy.xml -->
<Quota name="Rate-Limit-Policy">
  <Allow count="1000"/>
  <Interval>1</Interval>
  <TimeUnit>hour</TimeUnit>
  <Identifier ref="verifyapikey.API-Key-Validation.client_id"/>
</Quota>

Advanced Shared Flows

7. Circuit Breaker Shared Flow

Purpose: Implement circuit breaker pattern for backend service resilience.

<!-- Circuit-Breaker-SF.xml -->
<SharedFlow name="Circuit-Breaker-SF">
  <Step>
    <Name>Check-Circuit-State</Name>
  </Step>
  <Step>
    <Name>Record-Success</Name>
    <Condition>response.status.code < 400</Condition>
  </Step>
  <Step>
    <Name>Record-Failure</Name>
    <Condition>response.status.code >= 500</Condition>
  </Step>
  <Step>
    <Name>Update-Circuit-State</Name>
  </Step>
</SharedFlow>

8. Request Transformation Shared Flow

Purpose: Handle common request/response transformations.

<!-- Request-Transformation-SF.xml -->
<SharedFlow name="Request-Transformation-SF">
  <Step>
    <Name>JSON-to-XML</Name>
    <Condition>request.header.accept = "application/xml"</Condition>
  </Step>
  <Step>
    <Name>Add-Default-Parameters</Name>
  </Step>
  <Step>
    <Name>Normalize-Headers</Name>
  </Step>
</SharedFlow>

How to Build and Deploy Shared Flows

Step 1: Create Shared Flow Structure

# Create directory structure
mkdir shared-flows/Security-Enforcement-SF
cd shared-flows/Security-Enforcement-SF

# Create required directories
mkdir policies
mkdir sharedflowbundle
mkdir sharedflowbundle/policies

Step 2: Create Shared Flow Bundle

<!-- sharedflowbundle/Security-Enforcement-SF.xml -->
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<SharedFlowBundle revision="1" name="Security-Enforcement-SF">
    <Description>Centralized security enforcement</Description>
    <SharedFlows>
        <SharedFlow name="default">
            <Step>
                <Name>API-Key-Validation</Name>
            </Step>
            <Step>
                <Name>Rate-Limiting-Policy</Name>
            </Step>
        </SharedFlow>
    </SharedFlows>
    <Policies>
        <Policy>API-Key-Validation</Policy>
        <Policy>Rate-Limiting-Policy</Policy>
    </Policies>
</SharedFlowBundle>

Step 3: Using Apigee CLI (Recommended)

# Install Apigee CLI
npm install -g apigeecli

# Authenticate
apigeecli token gen -a

# Deploy shared flow
apigeecli sharedflows create \
  -n Security-Enforcement-SF \
  -p ./sharedflowbundle \
  -o your-org

# Deploy to environment
apigeecli sharedflows deploy \
  -n Security-Enforcement-SF \
  -o your-org \
  -e test

Step 4: Using in API Proxy

<!-- In your API Proxy flow -->
<Flow name="PreFlow">
  <Request>
    <Step>
      <Name>Security-Flow-Callout</Name>
    </Step>
  </Request>
</Flow>

<!-- Flow Callout Policy -->
<FlowCallout name="Security-Flow-Callout">
  <SharedFlowBundle>Security-Enforcement-SF</SharedFlowBundle>
</FlowCallout>

Best Practices for Shared Flows

1. Design Principles

Single Responsibility: Each shared flow should have one clear purpose
Parameterization: Use flow variables for configuration
Error Handling: Always include proper error handling
Documentation: Maintain clear documentation for each shared flow

2. Variable Management

<!-- Use consistent variable naming -->
<AssignMessage name="Set-Shared-Flow-Variables">
  <Set>
    <Variable name="sf.security.enabled">true</Variable>
    <Variable name="sf.ratelimit.count">1000</Variable>
    <Variable name="sf.ratelimit.timeunit">hour</Variable>
  </Set>
</AssignMessage>

3. Conditional Execution

<!-- Make shared flows conditional -->
<Step>
  <Name>Security-Enforcement</Name>
  <Condition>sf.security.enabled = true</Condition>
</Step>

4. Version Management

Use semantic versioning (v1.0.0, v1.1.0)
Maintain backward compatibility
Document breaking changes
Use environment-specific configurations

5. Testing Strategy

// Example test for shared flow
describe('Security-Enforcement-SF', function() {
  it('should validate API key', function() {
    // Test API key validation
  });

  it('should enforce rate limits', function() {
    // Test rate limiting
  });
});

Deployment Pipeline Example

CI/CD Pipeline (GitHub Actions)

# .github/workflows/deploy-shared-flows.yml
name: Deploy Shared Flows
on:
  push:
    branches: [main]
    paths: ['shared-flows/**']

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Setup Node.js
        uses: actions/setup-node@v2
        with:
          node-version: '16'

      - name: Install Apigee CLI
        run: npm install -g apigeecli

      - name: Deploy to Test
        run: |
          apigeecli sharedflows create -o ${{ secrets.APIGEE_ORG }} -n Security-Enforcement-SF -p ./shared-flows/Security-Enforcement-SF
          apigeecli sharedflows deploy -o ${{ secrets.APIGEE_ORG }} -e test -n Security-Enforcement-SF

Monitoring and Maintenance

1. Health Checks

Monitor shared flow usage across proxies
Track error rates and performance impact
Set up alerts for shared flow failures

2. Performance Monitoring

<!-- Add performance tracking -->
<AssignMessage name="Track-Shared-Flow-Performance">
  <Set>
    <Variable name="sf.start.time">{system.timestamp}</Variable>
  </Set>
</AssignMessage>

<!-- At the end of shared flow -->
<AssignMessage name="Calculate-SF-Performance">
  <Set>
    <Variable name="sf.execution.time">{system.timestamp - sf.start.time}</Variable>
  </Set>
</AssignMessage>

3. Documentation Template

# Shared Flow: Security-Enforcement-SF

## Purpose
Centralized security validation for all API proxies.

## Features
- API key validation
- OAuth token verification
- Rate limiting
- IP whitelisting

## Usage

xml

Security-Enforcement-SF


## Configuration Variables
- `sf.security.enabled`: Enable/disable security checks
- `sf.ratelimit.count`: Rate limit threshold

## Dependencies
- OAuth provider configuration
- API product configuration

These shared flows will significantly accelerate your API development and ensure consistency across your API ecosystem. Start with the core flows (Security, Error Handling, Logging) and gradually build more specialized ones based on your specific needs.

Apigee API Product Design

Mahmoud Sayed — Sun, 07 Sep 2025 14:27:19 +0000

API Product Design in Apigee is a crucial strategic step that goes far beyond just bundling proxies. It's about designing a product line for your APIs, focusing on the consumer's needs, business value, and monetization.

Here’s a breakdown of the key aspects, considerations, and examples.

What is API Product Design?

It's the process of defining how your API capabilities are packaged, priced, presented, and governed for different audiences. It answers questions like:

Who is this product for?
What capabilities do they get?
How much does it cost?
What are the usage limits?
How is it differentiated from other products?

Key Dimensions of API Product Design

You can think of designing an API product across four main dimensions:

1. Functional Scope (The "What")

This defines the actual API resources (proxies, endpoints, and operations) included in the product.

By Capability: Grouping related functionality.
- Example: A Weather-Data-Product includes /forecast, /current, and /alerts endpoints.
By Data Source: Grouping access to different datasets.
- Example: A Financial-Data-Product includes stock-prices, forex-rates, and crypto-feed proxies.
By Granularity: Offering different levels of data detail.
- Example: A User-Data-Basic product returns standard user fields, while a User-Data-Premium product includes sensitive data like email and phone number (with proper security!).

Design Choice: Do we create one giant product with everything, or many small, focused products? (Hint: Usually, many small products are better for targeting different audiences).

2. Audience & Access (The "Who")

This defines who can see and use the product. Apigee manages this through API Product visibility and API Key verification.

Public: Anyone on the developer portal can see and request it. (e.g., Free-Tier).
Private: Only visible to specific individuals, groups, or teams. You must explicitly invite a developer. (e.g., Partner-Access, Internal-Apps).
Internal: A product not on the portal at all, only accessible by directly creating an App and Key in the Apigee admin UI. Used for internal service-to-service communication.

Design Choice: Is this product for external third-party developers, specific partners, or our own internal mobile apps?

3. Commercial & Usage Model (The "How Much")

This is where you define the business model and operational limits.

Quotas & Rate Limits: The most common way to differentiate tiers.
- Free Tier: 100 requests/day
- Bronze Tier: 1,000 requests/day
- Silver Tier: 10,000 requests/day
- Gold Tier: 100,000 requests/day + 10 requests/second
Monetization:
- Freemium: Free tier with paid upgrades (very common).
- Pay-as-you-go: Price per X number of API calls.
- Tiered Subscription: Fixed monthly price for a quota package.
- Revenue Share: For partners driving transactions via your API.
Service Level Agreement (SLA): Higher-tier products can be associated with stricter SLAs (e.g., 99.95% uptime), though this is often managed operationally outside of Apigee.

Design Choice: How do we make money from this? How do we prevent abuse and ensure fair use?

4. Operational & Security Policy (The "Rules")

This defines the policies enforced on the API product.

Security:
- OAuth Scopes: A premium product might require a specific scope (e.g., scope=premium_data) in the OAuth token.
- IP Whitelisting: A product for a trusted partner might have a policy that only allows calls from their known IP addresses.
Traffic Management:
- Spike Arrest: Protect your backend from traffic spikes (e.g., 15 requests per second).
- Response Caching: A product offering static reference data might have a cache applied to improve performance.

Design Choice: What extra policies (beyond quotas) are needed to protect the backend for this specific audience?

Practical Design Examples

Let's design a product line for a hypothetical CompanyX API platform that offers Weather, News, and Currency data.

Example 1: The Onboarding Product

Product Name: CompanyX-Explorer
Audience: Public. New, unverified developers trying out the APIs.
Functional Scope: Read-only access to basic endpoints.
- weather-api:/v1/current/{city}
- news-api:/v1/headlines
- currency-api:/v1/convert?from=USD&to=EUR (limited)
Commercial Model: Freemium. Free, no revenue.
Policies & Limits:
- Quota: 100 requests/day, 5 requests/minute.
- Spike Arrest: 5 requests/second.
- Caching: None.
Purpose: Remove friction for sign-ups, let developers experiment and see value.

Example 2: The Targeted B2B Product

Product Name: CompanyX-Weather-Enterprise
Audience: Private. Specific enterprise customers who have signed a contract.
Functional Scope: Full access to all weather capabilities.
- weather-api:/v1/current/{city}
- weather-api:/v1/forecast/{city}
- weather-api:/v1/historical/{city} (premium endpoint)
- weather-api:/v1/alerts (premium endpoint)
Commercial Model: Tiered Subscription. $999/month.
Policies & Limits:
- Quota: 500,000 requests/month, 50 requests/second.
- Spike Arrest: 50 requests/second.
- Security: May require a valid OAuth token with the historical_data scope.
Purpose: A high-value, revenue-generating product for serious business customers.

Example 3: The Internal Product

Product Name: internal-currency-data-consumer
Audience: Internal. The company's own mobile app team.
Functional Scope: Full access to currency data for the app.
- currency-api:/v1/convert
- currency-api:/v1/history
Commercial Model: Not applicable. Internal cost.
Policies & Limits:
- Quota: Very high or none. Trusted internal consumer.
- Security: Might use a simpler authentication method or be whitelisted.
Purpose: Enable internal applications to consume APIs without going through the public developer portal.

The Design Workflow

Identify Personas: Who are your developers? (e.g., hobbyists, startups, enterprises).
Map Capabilities to Needs: What does each persona need? What are they willing to pay for?
Bundle & Price: Create product bundles that meet those needs and assign a commercial model.
Define Policies: Add the necessary quotas, security, and traffic rules to enforce the product design.
Publish & Iterate: Publish the products to the portal, gather feedback, and adapt your product lineup.

In essence, API Product Design in Apigee is where business strategy meets technical implementation. It transforms your APIs from technical endpoints into valuable, marketable products.

Apigee API Products, Developer, Apps and API Keys

Mahmoud Sayed — Sun, 07 Sep 2025 13:57:55 +0000

The Big Picture: The Apigee Security Model

Apigee uses a API Key as the primary credential for API access. This key is not created in isolation; it's the result of a relationship between a Developer, their App, and the API Products that app is authorized to consume.

Here’s the flow of how they work together:

You (the API provider) publish an API Product.
A Developer registers in your developer portal.
The Developer creates an App and selects which API Products it needs access to.
Apigee generates a unique API Key for that App.
The Developer includes this key in the requests their App makes.
Apigee checks if the key is valid and if it provides access to the requested API Product.

1. API Product

An API Product is a bundle of API resources (proxies) that you, as the API provider, offer to developers. It's the main commercial and legal unit of consumption. It defines:

Which Proxies/Endpoints: Which API proxies (and their specific paths) are included.
Quotas: How many requests can be made per minute, hour, day, etc.
Access Policies: Who can access it (e.g., internal vs. external developers).

Think of it like a **"cable TV package". You don't buy individual channels; you buy a package (e.g., "Sports Lite" or "Movie Lover Bundle").**

Example:
You work at CompanyX and have built several APIs. You create two products:

CompanyX-Free-Tier
- Proxies Included: weather-api, news-api
- Quota: 100 calls per day, 5 calls per minute
- Environments: test (for trying it out)
CompanyX-Premium-Weather
- Proxies Included: weather-api (includes all endpoints, even the premium ones like historical data)
- Quota: 10,000 calls per day, 100 calls per minute
- Environments: test, prod

2. Developer

A Developer represents a user or a company that consumes your APIs. They are typically registered through your developer portal.

They are the entity that owns Apps.
They have a profile with contact information (email, name).

Think of them as the **"account holder" for the cable TV subscription.**

Example:

Developer Name: Jane Smith
Email: jane.smith@example.com
Company: Cool Mobile Apps Inc.

3. App

An App is a specific project or application created by a Developer that needs to call your APIs. A single Developer can have multiple Apps (e.g., an iOS app and an Android app).

The App is the entity that is granted access to API Products.
When an App is approved for an API Product, Apigee generates credentials (API Keys) for it.

Think of it as the **"set-top box" in a specific room. The account holder (Developer) can have multiple set-top boxes (Apps), each with its own remote (API Key), all under the same subscription plan (API Products).**

Example:
Developer Jane Smith creates two apps:

App Name: Weather-Widget-iOS
- Purpose: Her company's iOS weather application.
App Name: Weather-Widget-Android
- Purpose: Her company's Android weather application.

4. API Key

The API Key is a unique string (like a password) that is generated for a specific App. It is the secret token that must be presented in every API request to Apigee (usually in the x-apikey header or as a query parameter).

Apigee validates this key to:
1. Authenticate: Is this a key I issued? Is it active?
2. Authorize: Does the App that owns this key have permission to access the specific API Product that contains the requested API proxy?

Think of it as the **"unique signal" or "authorization card" for that specific set-top box (App). Without it, you can't get the channels (APIs) you subscribed to.**

Example:

For the App Weather-Widget-iOS, Apigee generates:
- API Key: Rz4uP91KQ2m5cLb3Fv6Hs8dJqA0xWnEy

This key is configured in the app's code. Every API call includes it:

curl -H "x-apikey: Rz4uP91KQ2m5cLb3Fv6Hs8dJqA0xWnEy" \
  "https://companyx.apigee.net/v1/weather/forecast?city=London"

How It All Fits Together: A Practical Example

Let's walk through the entire lifecycle.

Step 1: API Provider (You) creates API Products.

You create CompanyX-Free-Tier and CompanyX-Premium-Weather.

Step 2: Developer (Jane) signs up.

Jane signs up on dev.companyx.com and her account is created.

Step 3: Developer (Jane) creates an App.

Jane logs in, goes to her dashboard, and clicks "New App".
She names it Weather-Widget-iOS.
She checks the box to request access to the CompanyX-Free-Tier product.

Step 4: API Provider (You) approves the request.

(This can also be automatic). You approve her app's access to the free tier.

Step 5: Apigee generates Credentials.

In Jane's developer portal dashboard, she can now see her App Weather-Widget-iOS.
On its details page, a new API Key (e.g., Rz4uP91KQ2m5cLb3Fv6Hs8dJqA0xWnEy) is displayed. This key is automatically approved for the CompanyX-Free-Tier product.

Step 6: The App makes a call.

Jane's iOS developer codes the app to include the API key in all requests.
The app calls: https://companyx.apigee.net/v1/weather/forecast?city=London

Step 7: Apigee validates the call.
When the request hits Apigee, it performs these checks almost instantly:

"Is the key Rz4uP91KQ2m5cLb3Fv6Hs8dJqA0xWnEy valid and active?" -> Yes, it belongs to the App Weather-Widget-iOS.
"What API Products is this App approved for?" -> CompanyX-Free-Tier.
"Does the CompanyX-Free-Tier product include the proxy and path being called (/v1/weather/forecast)?" -> Yes, it includes the weather-api proxy.
"Has the App exceeded its quota (100/day)?" -> No, it's only made 50 calls today.
✅ Validation passed! The request is forwarded to the backend weather service.

If Jane had tried to call a premium endpoint like /v1/weather/historical, step 3 would have failed because her CompanyX-Free-Tier product does not include that endpoint. Apigee would block the request and return a 403 Forbidden error.

This model provides a powerful, flexible, and auditable way to manage and monetize API access.

Here are flowcharts that visualize the relationships between API Products, Developers, Apps, and API Keys in Apigee, from both the provider's and consumer's perspectives.

Flowchart 1: Provider's Administrative View (How Things Are Built)

This chart shows how you, the API provider, assemble and offer an API Product.

flowchart TD
    A[API Provider] --> B[Create API Proxies]

    subgraph Proxies [Backend Services]
        B1[weather-api]
        B2[news-api]
        B3[payment-api]
    end

    B --> Proxies

    Proxies --> C[Bundle into API Products]

    subgraph Products [API Products - The 'Packages']
        P1[Free-Tier]<br>Includes: B1, B2<br>Quota: 100/day
        P2[Premium-Weather]<br>Includes: B1 all endpoints<br>Quota: 10,000/day
    end

    C --> Products
    Products --> D[Publish to Developer Portal]
    D --> E[Developers Discover & Subscribe]

Flowchart 2: Consumer's Journey (How Access is Granted)

This chart shows the process a developer follows to get and use an API key.

flowchart TD
    A[Developer] --> B[Signs up on Developer Portal]
    B --> C[Creates an 'App' e.g. Weather-Widget-iOS]
    C --> D[Requests access to an API Product e.g. Free-Tier]
    D --> E{Provider Approval<br>Auto or Manual}
    E -- Approved --> F
    E -- Rejected --> G[Access Denied]
    F[Apigee Generates Unique API Key for that App] --> H[Developer embeds Key in App Code]
    H --> I[App makes API call with Key]
    I --> J{Apigee Verification}

    subgraph J [Verification Steps]
        J1[Key Valid?]
        J2[Product Approved?]
        J3[URL Path Included?]
        J4[Quota Available?]
    end

    J -- All Checks Pass --> K[Request forwarded to Backend]
    J -- Any Check Fails --> L[Error e.g. 403 Forbidden]

Flowchart 3: Runtime Request Flow (What Happens on Every API Call)

This chart details the specific verification steps Apigee performs for every incoming request.

flowchart LR
    A[Incoming API Request<br>with API Key] --> B{Verify API Key}
    B -- Invalid/Revoked --> C[Block Request: 401 Unauthorized]
    B -- Valid --> D[Look Up App & Developer Details]
    D --> E{Check API Product Access}
    E -- Product not approved<br>for this URL --> F[Block Request: 403 Forbidden]
    E -- Product approved --> G{Check Quota & Rate Limits}
    G -- Quota Exceeded --> H[Block Request: 429 Too Many Requests]
    G -- Quota Available --> I[Allow Request to Backend]
    I --> J[Log Analytics Data<br>e.g. for App, Developer, Product]

Key Relationships Visualized

The core relationship between the entities can be summarized in this hierarchy:

API Product (The "Package")
    ^
    | contains
    |
API Proxies (The "Channels")
    ^
    | is accessed via
    |
App (The "Set-Top Box")
    ^
    | owns
    |
Developer (The "Account Holder")
    ^
    | uses
    |
API Key (The "Signal"/Password)

This shows that an API Product bundles Proxies. An App (owned by a Developer) is approved for a product. The API Key is the credential for that specific App, granting it the rights of the product it's approved for.

Apigee Target Servers VS. Target Endpoints

Mahmoud Sayed — Wed, 03 Sep 2025 15:32:30 +0000

Let's break down the difference between Target Servers and Target Endpoints, complete with examples.

The Core Difference: Abstraction vs. Implementation

Think of it this way:

Target Server: A named, reusable configuration object that defines where your backend service is (host, port, TLS settings). It's about the destination.
Target Endpoint: An XML configuration block inside your API Proxy that defines how to call the backend. It's about the operation (the request/response). It often uses a Target Server to know where to send the request.

You can achieve the same result without Target Servers by defining the backend URL directly in the Target Endpoint. Target Servers add a layer of abstraction that provides significant benefits.

Target Endpoint (The "How")

A Target Endpoint is a fundamental building block of an Apigee API proxy. It's defined in an XML file (e.g., default.xml) within the proxy bundle and represents the outbound connection to your backend service.

Key Characteristics:

Proxy-Specific: Defined within a single API proxy.
Defines the HTTP Interaction: It contains the configuration for the actual request and response flow to the backend, including:
- Pre-processing logic (with <PreFlow>)
- Post-processing logic (with <PostFlow>)
- Fault rules for backend errors
- The crucial <HTTPTargetConnection> element which points to the actual backend.

Example 1: Target Endpoint WITHOUT a Target Server (Direct URL)

This is often called a "pass-through" or directly configured target.

<!-- File: /targets/default.xml -->
<TargetEndpoint name="default">
  <PreFlow name="PreFlow">
    <!-- You can add policies here, like adding an API key to the backend request -->
  </PreFlow>
  <HttpTargetConnection>
    <!-- The backend URL is hardcoded right here -->
    <URL>https://api.my-internal-service.com/v1/orders</URL>
  </HttpTargetConnection>
  <PostFlow name="PostFlow">
    <!-- You can add policies here to process the backend response -->
  </PostFlow>
</TargetEndpoint>

Example 2: Target Endpoint WITH a Target Server (Using Abstraction)

This endpoint doesn't know the exact URL; it knows the name of a Target Server.

<!-- File: /targets/default.xml -->
<TargetEndpoint name="default">
  <PreFlow name="PreFlow"/>
  <HttpTargetConnection>
    <!-- It references a Target Server by name -->
    <!-- The environment (e.g., 'test', 'prod') is determined at runtime -->
    <LoadBalancer>
      <Server name="target-server-orders"/>
    </LoadBalancer>
    <!-- You can still add a path to the base URL defined in the Target Server -->
    <Path>/v1/orders</Path>
  </HttpTargetConnection>
  <PostFlow name="PostFlow"/>
</TargetEndpoint>

Target Server (The "Where")

A Target Server is an environment-specific entity that you create and manage outside of any individual API proxy. It's a reusable definition of a backend host.

Key Characteristics:

Environment-Specific & Reusable: Created once in an environment (e.g., test, prod) and can be used by many different API proxies.
Defines Connection Details: It holds the low-level network information:
- Host: The backend server's hostname or IP.
- Port: The port number (e.g., 443 for HTTPS, 80 for HTTP).
- SSLInfo: Configuration for TLS/SSL (e.g., whether to ignore validation errors for self-signed certs in dev).
Enables Load Balancing: You can define multiple Target Servers with the same name to form a load balancing group.

How to Create a Target Server:
You create it via the Apigee UI (Admin -> Environments -> Target Servers) or the Management API.

Example: Creating a Target Server for Dev

Name: target-server-orders
Host: orders-service.dev.mycompany.net
Port: 443
SSL Info: Enabled (This is the default)

Example: Creating a Target Server for Prod (with Load Balancing)

You would create two or more Target Servers with the exact same name but different hosts.

Server 1:
- Name: target-server-orders-prod-lb
- Host: orders-service-prod-1.mycompany.com
- Port: 443
Server 2:
- Name: target-server-orders-prod-lb 
- Host: orders-service-prod-2.mycompany.com
- Port: 443

Apigee will automatically load balance requests between these two hosts.

Side-by-Side Comparison

Feature	Target Endpoint	Target Server
Scope	API Proxy (defined inside a proxy)	Environment (defined outside proxies, reusable)
Purpose	Define the how (request/response flow, policies).	Define the where (host, port, TLS settings).
Configuration	XML file in the proxy bundle.	Admin UI or Management API.
Content	PreFlow, PostFlow, Fault Rules, `<HTTPTargetConnection>`.	Host, Port, SSL Info.
Load Balancing	Configures the strategy (e.g., round-robin) in the XML.	Provides the members (the actual servers) of the pool.
Example Use	Adding headers, transforming the request, handling errors.	Staging vs. Production backend URLs.

When to Use Which?

Use a Target Endpoint (with a direct URL) when:
- You are prototyping quickly.
- The backend URL is simple and unlikely to change.
- The proxy is tightly coupled to a single, specific backend.
Use a Target Endpoint WITH a Target Server when:
- Different Backends for Different Environments: You have a different backend URL for your test, stage, and prod environments. This is the most common and recommended reason.
- Reusability: The same backend host is used by multiple API proxies (e.g., a shared authentication service).
- Load Balancing: You need to distribute traffic across multiple instances of your backend.
- Operational Decoupling: You want to change the backend host/port (e.g., during a migration) without redeploying any API proxies. You just update the Target Server configuration.

Summary

The Target Endpoint is the "what to do" when calling the backend. The Target Server is the "where to do it." By using them together, you create a clean separation of concerns, making your API proxies more portable, reusable, and easier to manage across different environments.

Apigee Organization Components

Mahmoud Sayed — Thu, 13 Feb 2025 21:25:34 +0000

Apigee Organization Components

Apigee Organization Components (With Images & Charts)

Apigee, Google's API Management platform, is structured into different organizational components that help manage APIs efficiently. Below is an overview of these components with explanations and visual representations.

1. Apigee Organization (Org)

An Organization (Org) is the highest-level entity in Apigee and represents a company or business that owns and manages APIs.

📌 Key Features:

Contains environments, APIs, users, and policies.
Defines security and governance policies.
Linked to a Google Cloud Project.

📊 Visualization:

+----------------------------+
|       Apigee Org           |
|----------------------------|
| - API Proxies              |
| - Environments             |
| - Developers               |
| - Developer Apps           |
| - API Products             |
+----------------------------+

2. Environments & Environment Groups

Environments allow you to separate different stages of API development (e.g., Development, Staging, Production).

📌 Key Features:

Each environment can have its own configurations.
APIs are deployed in different environments for better control.

📊 Visualization:

+------------------------------+
|        Apigee Org            |
|------------------------------|
| Environments                 |
| - Dev (Development)          |
| - Test (Testing)             |
| - Prod (Production)          |
+------------------------------+

🔗 Environment Groups

Environment Groups allow multiple environments to be grouped and exposed under a single domain.
Example: You can have api.mycompany.com serving requests from different environments dynamically.

📊 Environment Group Example:

+------------------------------------------+
| Environment Group: api.mycompany.com     |
|------------------------------------------|
| - Dev (api-dev.mycompany.com)           |
| - Test (api-test.mycompany.com)         |
| - Prod (api.mycompany.com)              |
+------------------------------------------+

3. API Proxies

An API Proxy is an abstraction layer that enables developers to expose backend services in a controlled manner.

📌 Key Features:

Acts as a gateway to manage API traffic.
Helps enforce security, rate limiting, and analytics.
Supports transformation via policies (e.g., XML to JSON).

📊 How API Proxies Work:

Client Request → Apigee API Proxy → Backend Service

📊 Diagram:

+---------------------------+
|        API Proxy          |
|---------------------------|
| - Request Processing      |
| - Policies (Security, Caching) |
| - Response Handling       |
+---------------------------+

4. API Products

An API Product is a bundle of API Proxies that are exposed to developers under a single plan.

📌 Key Features:

Helps group APIs based on functionality.
Used to apply rate limiting and access control.
Allows monetization of APIs.

📊 Example API Product Structure:

+---------------------------+
|       API Product         |
|---------------------------|
| - API Proxy 1 (User Data) |
| - API Proxy 2 (Payments)  |
| - Rate Limit: 1000 calls/day |
+---------------------------+

5. Developer & Developer Apps

Apigee allows external developers to register and obtain API keys.

📌 Key Features:

Developers: Individuals or companies using the APIs.
Developer Apps: Applications created by developers that consume API Products.
API Keys: Each app gets a unique key to access APIs.

📊 Diagram:

+--------------------------+
|       Developer          |
|--------------------------|
| - Name: John Doe        |
| - Apps: MyMobileApp     |
+--------------------------+

        |
        v

+--------------------------+
|    Developer App        |
|--------------------------|
| - API Key: XYZ123       |
| - API Product: Payments |
+--------------------------+

6. Policies & Flows

Apigee uses Policies to enforce security, transformation, and traffic management.

📌 Types of Policies:

Security Policies: OAuth, API Key validation.
Traffic Management: Rate limiting, caching.
Transformation: XML to JSON conversion.
Logging & Analytics: Debugging and monitoring.

📊 Example API Flow with Policies:

Client Request → Security Policy → Traffic Control Policy → Backend Service

📊 Diagram:

+---------------------------+
|       API Proxy          |
|---------------------------|
| 1. Verify API Key Policy |
| 2. Rate Limiting Policy  |
| 3. Transform JSON Policy |
+---------------------------+

Final Apigee Organization Architecture

Here's an overview of the entire Apigee structure:

📊 Full Architecture:

+------------------------------------------------------+
|                    Apigee Org                        |
|------------------------------------------------------|
| - Environments (Dev, Test, Prod)                    |
| - Environment Groups (api.mycompany.com)            |
| - API Proxies (User API, Payments API)              |
| - API Products (Subscription Plan, Free Plan)       |
| - Developers & Apps (App1, App2, API Keys)          |
| - Policies (Security, Rate Limiting, Caching)       |
+------------------------------------------------------+

Conclusion

Apigee is structured to efficiently manage APIs, enforce security, and provide scalability. The main components are:

Organization: The top-level container.
Environments & Groups: Staging areas for APIs.
API Proxies: Entry points for requests.
API Products: Bundles of APIs with access control.
Developers & Apps: Users consuming APIs.
Policies: Traffic control, security, and transformation. 🚀

Acquia Certification: What You Need to Know

Mahmoud Sayed — Wed, 07 Jun 2023 19:15:32 +0000

Introduction

Acquia certification is one way to demonstrate your expertise in Drupal development and increase your chances of landing a job. In this blog post, we will discuss everything you need to know about Acquia certification and why it's worth pursuing.

Types of Acquia Certification

Acquia Certified Drupal Developer
Acquia Certified Drupal Site Builder
Acquia Certified Acquia Cloud Pro
Acquia Certified Campaign Studio Marketing Pro

How to Get Acquia Certified

To become Acquia certified, you must pass the relevant exam. Before taking the exam, you should prepare by studying the exam objectives and taking practice tests.
Make account on ACQUIA academy following this link
https://community.acquiaacademy.com/learn
and select which type of certificate and learning path you need.
https://docs.acquia.com/certification/study-guides/d10-site-builder/

Resources for prepare to exam (Drupal Site Builder)

Druplizeme https://drupalize.me/guide/build-drupal-sites
ACQUIA Academy https://community.acquiaacademy.com/learn
OS Training https://www.youtube.com/watch?v=-DYSucV1_9w&list=PLtaXuX0nEZk9MKY_ClWcPkGtOEGyLTyCO

Drupal Learning Resources

Mahmoud Sayed — Sun, 12 Mar 2023 19:19:14 +0000

This resources links for learning Drupal

Site Building
- Drupalizme (Recommanded) https://drupalize.me/guide/build-drupal-sites
- ACQUIA https://community.acquiaacademy.com/learn/lp/85/Drupal%2520Site%2520Builder%2520Certification%2520Learning%2520Plan
- OSTrining
  https://ostraining.com/courses/the-beginners-guide-to-drupal-8/
  
  https://www.youtube.com/playlist?list=PLtaXuX0nEZk9MKY_ClWcPkGtOEGyLTyCO
- Webwash
  https://www.udemy.com/course/drupal-8-site-building/
  
  https://www.webwash.net/course-list/
Development

Remove /web from URL in Drupal

Mahmoud Sayed — Tue, 20 Sep 2022 17:30:16 +0000

By default, when you create your Drupal project using composer it will create project under /web directory.

When you’re using shared hosting for your Drupal website which points your domain to /public_html folder and you’re not allow to change that pointing directory

Now whenever your targeted audience will have to visit your website, they must open it like this http://example.com/web. and which is not a good user experience to add /web at the end of your URL to view home page of your website.

So, how can we serve our website from /public_html/web folder but no need to append /web in URL by your targeted audience?

Here is the summary, which I learned:

1- Move all files and folders from web to root folder
2- Open composer.json remove /web and add ./

"drupal-scaffold": {
 "locations": {
    "web-root": "./"
 }
},

3- Remove /web from composer.json in installer-paths

"installer-paths": {
  "core": [
    "type:drupal-core"
   ],
  "libraries/{$name}": [
   "type:drupal-library"
  ],
   "modules/contrib/{$name}": [
     "type:drupal-module"
   ],
   "profiles/contrib/{$name}": [
     "type:drupal-profile"
    ],
    "themes/contrib/{$name}": [
     "type:drupal-theme"
     ],
     "drush/Commands/contrib/{$name}": [
      "type:drupal-drush"
     ],
     "modules/custom/{$name}": [
       "type:drupal-custom-module"
     ],
     "profiles/custom/{$name}": [
       "type:drupal-custom-profile"
    ],
    "themes/custom/{$name}": [
       "type:drupal-custom-theme"
      ]
  },

4- Open file autoload.php replace code with:

return require __DIR__ . '/vendor/autoload.php';

5- Remove vendor and core folders and run this command:

$composer install
$drush cr

If there is a better way than this, please let me know in the comments. Thanks

Drupal Seed Data

Mahmoud Sayed — Sun, 14 Aug 2022 22:47:09 +0000

Seed Data
You use seeding to provide initial values for lookup lists,
for demo purposes, proof of concepts and of course for development.

To modules\custom\my_module\src we add a directory called SeedData. We add a file
called SeedDataGenerator.php to modules\custom\offer\src\SeedData.
The class will for now just create a dummy user.

<?php

namespace Drupal\my_module\SeedData;

use Drupal\user\Entity\User;
use Drush\Drush;

/**
 * SeedDataGenerator.
 *
 * @package Drupal\my_module
 */
class SeedDataGenerator {

  /**
   * Function to create a seed data.
   *
   * @param string $entity
   *   The type of entity that needs to be created.
   *
   * @return null|int
   *   The number of entities created.
   */
  public function generate(string $entity){
    $count = 0;
    switch ($entity) {
      case 'user':
        $count = $this->seedUser();
        break;
    }
    return $count;
  }

  /**
   * @return int
   *   The number of users created.
   * @throws \Drupal\Core\Entity\EntityStorageException
   */
  private function seedUser() {
    $count = 0;

    $user = User::create();
    $user->setUsername('testUser');
    $user->setPassword('P@ssw0rd');
    $user->activate();
    $user->enforceIsNew();
    Drush::output()->writeln('<comment>Creating user test</comment>' );
    if ($user->save()) {
      $count++;
    }
    return $count;
  }
}

Proceed with adding a drush command that will trigger the class:
Add custom/my_module/src/Commands/SeedGeneratorCommand.php and configure
further. The final file looks like this:

<?php

namespace Drupal\my_module\Commands;

use Drupal\my_module\SeedData\SeedDataGenerator;
use Drush\Commands\DrushCommands;
use Drush\Drush;

/**
 * Class SeedGeneratorCommand.
 *
 * @package Drupal\my_module\Commands
 */
class SeedGeneratorCommand extends DrushCommands {

  /**
   * Runs the mymoduleCreateSeeds command.
   *
   * @command mymodule-create-seeds
   * @aliases mymodulecs
   * @usage drush mymodule-create-seeds
   * Display 'Seed data created'
   */
  public function mymoduleCreateSeeds():void {
    $seed = new SeedDataGenerator();
    $count = $seed->generate('user');
    Drush::output()->writeln('<info>'. $count . ' user(s) created</info>' );
  }

}

One more thing. Add a file custom/my_module/my_module.services.yml and add:

services:
  offer.commands:
    class: Drupal\offer\Commands\SeedGeneratorCommand
    tags:
      - { name: drush.command }

That’s it! Clear cache and see if our system has registered our command

$drush offer-create-seeds

References:

Learning drupal 9 as a framework

Drupal Interview questions

Mahmoud Sayed — Tue, 19 Jul 2022 21:58:29 +0000

- What is the entity in Drupal?
In Drupal, entity is a general concept that represents a noun (person, place, or thing). Out of the box, there are a number of different types of entities in Drupal, _each meant to represent a specific type of data. _
Such as user and Content

Entity has two types Configuration and Content
Content such as [User, Node, Taxonomy]

Bundle is a subtype of content entity such as [Article, Basic page, Term]

- Difference between Field Types, Field Formatters and Field Widgets?
Field Types this defines the type of value that will be stored in database [integer, string...].
Field Widgets this using in edit value and configured from form display.
Field Formatter this defines how value will be rendered and displayed configured from manage display.

- What is rendered array?
The core structure of Drupal's Render API is the render array, which is a hierarchical associative array containing data to be rendered and properties describing how the data should be rendered.

- Cache in Drupal
A cache stores frequently requested pages or the parts of the pages, and these pages can be shown to users with fewer resources and with a faster speed than usual.

Drupal 8 enables two modules: Internal Page Cache and Internal Dynamic Page Cache. Internal Page Cache caches pages for anonymous users. Internal Dynamic Page Cache caches contents of the page except for the personalized pieces, so they can be used for anonymous and authorized users. Each object of the page contains metadata, and this piece of metadata tells the Internal Dynamic Page Cache module if it must cache the page or not.

- Create custom token
Create custom token

- Create custom Drush command
Drush command

*- Can we optimize any Drupal website? If yes, then how? *
Ans: Yes, we can do it. The steps are given below:

Enable page caching
Compress CSS files
JS Aggregation
Moving media files and static files to CDN
Theme optimization

- How hooks working in Drupal?
Hooks are how modules can interact with the core code of Drupal.
Hooks allow modules to alter and extend the behavior of Drupal core, or another module. They are one of the various ways that code components in Drupal can communicate with one another. Using hooks, a module developer can change how core, or another module works -- without changing the existing code.

*- How Drupal handles the page request: Bootstrap Process? *
LINK
What Drupal does:

Separates the internal path from the full URL.
Bootstraps and initialize the database, sessions etc.
Maps the path to a callback function.
Modules can hook into the process and extend functionality and alter the content.
The Theme System generates the HTML and styles it.
Drupal returns a fully formed HTML page to the browser
The browser renders the HTML page for the user

How to use multiple databases in Drupal 8
Article

Drupal Event Subscriber
Article

Why are we using module config ignore?
Ignore config files you don't need to upload it.

Rest API in Drupal

Apigee With Drupal

Apigee Edge :
The Apigee Edge module lets you connect your Drupal developer portal site with Apigee Edge, Apigee X, and Apigee hybrid organizations.

Apigee API Catalog:
Apigee’s API Catalog module lets you document your APIs in your Drupal developer portal using OpenAPI specifications and Apigee SmartDocs. Publishing API documentation to your portals makes it easy for your developers to learn, test, and evaluate your APIs.

Apigee Monetization:
The Apigee Monetization module lets you integrate a Drupal developer portal with a monetized organization in Apigee Edge, hybrid, or Apigee X.

Apigee Developer Portal Kickstart:
The Apigee Developer Portal Kickstart distribution enables you to quickly evaluate and create a new Apigee developer portal using Drupal.

References

https://www.adaface.com/blog/drupal-interview-questions

Drupal - API Resources for Custom login

Mahmoud Sayed — Mon, 18 Jul 2022 22:29:22 +0000

Drupal 9 rest custom login resource, return session data to build cookie in frontend.
missing csrf (can be obtained at /session/token).

Create rest plugin using Drush command.
$ drush generate plugin-rest-resource
Or using an alias
$ drush gen rest-resource

This is a POST resource, so run $ drush cr
Using Rest UI enable resource and add a permission for anonymous role.

<?php

namespace Drupal\custom_rest_api\Plugin\rest\resource;

use Drupal\Core\Session\AccountProxyInterface;
use Drupal\rest\ModifiedResourceResponse;
use Drupal\rest\Plugin\ResourceBase;
use Psr\Log\LoggerInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\Core\Session\SessionManagerInterface;
use Drupal\Core\Extension\ModuleHandlerInterface;
use Drupal\Core\Password\PasswordInterface;
use Symfony\Component\HttpKernel\Exception\BadRequestHttpException;

/**
 * Represents Custom login resource records as resources.
 *
 * @RestResource (
 *   id = "custom_rest_api_custom_login_resource",
 *   label = @Translation("Custom login resource"),
 *   uri_paths = {
 *     "create" = "/api/custom/login"
 *   }
 * )
 *
 * @DCG
 * This plugin exposes database records as REST resources. In order to enable it
 * import the resource configuration into active configuration storage. You may
 * find an example of such configuration in the following file:
 * core/modules/rest/config/optional/rest.resource.entity.node.yml.
 * Alternatively you can make use of REST UI module.
 * @see https://www.drupal.org/project/restui
 * For accessing Drupal entities through REST interface use
 * \Drupal\rest\Plugin\rest\resource\EntityResource plugin.
 */
class CustomLoginResource extends ResourceBase {
  /**
   * A current user instance.
   *
   * @var \Drupal\Core\Session\AccountProxyInterface
   */
  protected $currentUser;

  protected $sessionManager;

  protected $moduleHandler;

  protected $password;

  /**
   * Constructs a new CustomLoginResource object.
   *
   * @param array $configuration
   *   A configuration array containing information about the plugin instance.
   * @param string $plugin_id
   *   The plugin_id for the plugin instance.
   * @param mixed $plugin_definition
   *   The plugin implementation definition.
   * @param array $serializer_formats
   *   The available serialization formats.
   * @param \Psr\Log\LoggerInterface $logger
   *   A logger instance.
   * @param \Drupal\Core\Session\AccountProxyInterface $current_user
   *   A current user instance.
   */
  public function __construct(
    array $configuration,
          $plugin_id,
          $plugin_definition,
    array $serializer_formats,
    LoggerInterface $logger,
    AccountProxyInterface $current_user,
    SessionManagerInterface $session_manager,
    ModuleHandlerInterface $module_handler,
    PasswordInterface $password) {
    parent::__construct($configuration, $plugin_id, $plugin_definition, $serializer_formats, $logger);

    $this->currentUser = $current_user;
    $this->sessionManager = $session_manager;
    $this->moduleHandler = $module_handler;
    $this->password = $password;
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition) {
    return new static(
      $configuration,
      $plugin_id,
      $plugin_definition,
      $container->getParameter('serializer.formats'),
      $container->get('logger.factory')->get('exp_fs'),
      $container->get('current_user'),
      $container->get('session_manager'),
      $container->get('module_handler'),
      $container->get('password')
    );
  }

  /**
   * Responds to POST requests.
   *
   * @return \Drupal\rest\ModifiedResourceResponse
   *   The HTTP response object.
   *
   * @throws \Symfony\Component\HttpKernel\Exception\HttpException
   *   Throws exception expected.
   */
  public function post($data) {
    $this->validate($data);
    $pass_check = FALSE;
    $name = $data['name'];
    $pass = $data['pass'];

    $account = user_load_by_name(trim($name));
    if ($account) {
      $pass_check = $this->password->check(trim($pass), $account->getPassword());
    }
    else {
      $body = [
        'error' => 'Wrong username and/or password.',
      ];
    }

    if ($pass_check == FALSE) {
      $body = [
        'error' => 'Wrong username and/or password..',
      ];
    }
    else {
      $session = \Drupal::service('session');
      $session->migrate();
      $session->set('uid', $account->id());
      $this->moduleHandler->invokeAll('user_login', [$account]);
      user_login_finalize($account);

      $sess_name = $this->sessionManager->getName();
      $sess_id = $this->sessionManager->getId();

      $body = [
        'sess_name' => $sess_name,
        'sess_id' => $sess_id,
        'current_user' => [
          'name' => $account->getAccountName(),
          'uid' => $account->id(),
          'roles' => $account->getRoles(),
        ],
      ];
    }

    return new ModifiedResourceResponse($body, 200);
  }

  /**
   * Validates incoming record.
   *
   * @param mixed $record
   *   Data to validate.
   *
   * @throws \Symfony\Component\HttpKernel\Exception\BadRequestHttpException
   */
  protected function validate($record) {
    if (!is_array($record) || count($record) == 0) {
      throw new BadRequestHttpException(t('No record content received'));
    }

    if (empty($record['name'])) {
      throw new BadRequestHttpException(t('name id is required'));
    }
    if (empty($record['pass'])) {
      throw new BadRequestHttpException(t('Password date is required'));
    }
  }

}