API Gateway as the Access Layer:

Building an AI gateway with Amazon API Gateway provides enterprise governance:

Key Capabilities:

• Authorization - JWT validation, IAM, Cognito integration
• Rate Limiting - Usage plans and API keys
• Quota Management - Per-tenant limits
• Request Throttling - Protect backend from overload
• WAF Integration - Protection against attacks
• Canary Releases - Gradual rollout of changes

Architecture:

Client → API Gateway → Lambda → Bedrock
              ↓
         WAF, Auth, Throttling

Benefits:

• Centralized access control
• Cost allocation by tenant/API key
• Consistent logging and monitoring
• Decouples clients from Bedrock directly

API Gateway Options for GenAI:

REST APIs:

• Full feature set (throttling, caching, WAF)
• Higher latency than HTTP APIs
• Best for: Enterprise features, complex authorization

HTTP APIs:

• Lower latency, lower cost
• Limited features vs REST
• Best for: Performance-critical applications

WebSocket APIs:

• Bidirectional communication
• Persistent connections
• Best for: Streaming responses, chat applications

Private APIs:

• VPC-only access
• No internet exposure
• Best for: Internal applications, compliance

Amazon EventBridge for GenAI Workflows:

Use Cases:

• Trigger GenAI processing on file uploads
• React to model inference completions
• Cross-account event routing
• Audit and compliance logging

Event Patterns:

{
  "source": ["aws.s3"],
  "detail-type": ["Object Created"],
  "detail": {
    "bucket": {"name": ["documents-bucket"]},
    "object": {"key": [{"prefix": "input/"}]}
  }
}

Architecture:

S3 Upload → EventBridge → Step Functions → Bedrock
                  ↓
              Archive (audit)

Benefits:

• Loose coupling between components
• Easy to add new consumers
• Built-in archive and replay
• Cross-account routing

Common Event-Driven GenAI Patterns:

1. Document Processing Pipeline:

S3 → EventBridge → Step Functions
                        ↓
                   Textract → Bedrock → S3 Output

2. Real-Time Classification:

Kinesis → Lambda → Bedrock → DynamoDB
              ↓
         EventBridge (alerts)

3. Batch Inference Completion:

Bedrock Batch → EventBridge → Lambda → SNS
    (completed)              (notify)

4. Cross-Account AI Services:

Account A: Event → EventBridge (central bus)
                        ↓
Account B: → Step Functions → Bedrock

Integrating Bedrock into Microservices:

Service Mesh Pattern:

• Each microservice calls Bedrock independently
• Shared SDK configuration
• Distributed tracing with X-Ray

API Gateway Pattern:

• Central AI gateway service
• Other services call gateway
• Centralized governance

Sidecar Pattern:

• AI proxy container alongside service
• Handles model calls and caching
• Consistent across services

Best Practices:

• Isolate AI logic in dedicated service
• Use async patterns where possible
• Implement circuit breakers
• Cache responses when appropriate

Connecting GenAI to Legacy Systems:

Common Challenges:

• SOAP/XML APIs vs REST/JSON
• Synchronous blocking calls
• Limited scalability
• Security constraints

Integration Strategies:

1. API Facade:

Legacy → API Gateway → Lambda (transform) → Bedrock

• Transform SOAP to REST
• Handle authentication differences
• Abstract legacy complexity

2. Event Bridge:

Legacy → SQS → Lambda → Bedrock → SQS → Legacy

• Asynchronous decoupling
• Legacy polls for results
• No changes to legacy system

3. Database Integration:

Legacy → DB → Lambda (trigger) → Bedrock → DB

• DynamoDB Streams or RDS proxy
• Legacy reads results from DB
• Minimal integration surface

Robust Error Handling for GenAI:

Common Bedrock Errors: | Error | Cause | Handling | |-------|-------|----------| | ThrottlingException | Rate limit exceeded | Exponential backoff | | ValidationException | Invalid input/parameters | Validate before sending | | ModelTimeoutException | Model took too long | Retry or use smaller model | | AccessDeniedException | IAM permission issue | Fix IAM policy | | ServiceUnavailableException | Bedrock unavailable | Retry with backoff |

Retry Strategy:

import time
from botocore.config import Config

config = Config(
    retries={
        'max_attempts': 3,
        'mode': 'adaptive'  # Exponential backoff
    }
)
bedrock = boto3.client('bedrock-runtime', config=config)

Implementing Fallbacks:

1. Model Fallback:

Claude 3.5 Sonnet (primary)
    ↓ (failure)
Claude 3 Haiku (fallback)
    ↓ (failure)
Cached response or error

2. Regional Fallback:

us-east-1 (primary)
    ↓ (unavailable)
us-west-2 (secondary)

3. Graceful Degradation:

Full GenAI response (normal)
    ↓ (overloaded)
Cached similar response
    ↓ (no cache)
Static fallback message

Implementation with Step Functions:

• Use Catch blocks for error handling
• Retry with exponential backoff
• Choice state for fallback routing
• Error notification via SNS