Key Rotation Settings - AI Orchestrator SDK

Core Settings

These are the fundamental settings that control basic key rotation behavior.

enabled

Optional Default: true

boolean

What it does: Master on/off switch for the entire key rotation feature.

keyRotation: {
  enabled: false  // Completely disables rotation
}

usageLimit

Required

number

What it does: Number of successful API requests before automatically rotating to the next key.

⚠️ Only successful requests count. Failed requests don't increment the counter.

keyRotation: {
  usageLimit: 100  // Rotate after 100 successful requests
}

additionalKeys

Optional Default: []

string[]

What it does: Additional API keys to rotate through (beyond the main apiKey).

📝 Rotation order: key-1 → key-2 → key-3 → key-4 → key-1 (cycles back)

apiKey: 'key-1',  // Main key (always included)
keyRotation: {
  usageLimit: 100,
  additionalKeys: ['key-2', 'key-3', 'key-4']  // 4 total keys
}

Time-Based Settings

Configure time windows and cooldown periods for more sophisticated rotation strategies.

usageWindowMs

Optional Default: undefined

number

What it does: Time window for usage counting (in milliseconds). After this time expires, the usage count for a key resets to 0.

💡 Perfect for providers with hourly/daily quotas!

keyRotation: {
  usageLimit: 1000,
  usageWindowMs: 60 * 60 * 1000  // 1 hour = 3,600,000 ms
}

// Behavior: Each key can make 1000 requests per hour.
// After 1 hour, it can make another 1000.

⚠️ Without this setting: Once a key hits the limit, it won't reset until you cycle through all keys.

cooldownMs

Optional Default: undefined

number

What it does: Waiting period (in milliseconds) before reusing an exhausted key when all keys are depleted.

keyRotation: {
  usageLimit: 100,
  cooldownMs: 5 * 60 * 1000  // 5 minutes = 300,000 ms
}

// Behavior:
// 1. All 3 keys reach their limit
// 2. Manager waits 5 minutes
// 3. Resets usage and allows requests again

Retry & Resilience Settings

Control retry behavior when fetching new keys from renewal endpoints.

retryAttempts

Optional Default: 3

number

What it does: Number of retry attempts when fetching a new key from the renewal endpoint.

📌 Only used when renewal is configured.

keyRotation: {
  usageLimit: 100,
  retryAttempts: 5,  // Try 5 times before giving up
  renewal: { /* ... */ }
}

retryDelayMs

Optional Default: 500

number

What it does: Delay (in milliseconds) between retry attempts when renewal fails.

keyRotation: {
  usageLimit: 100,
  retryAttempts: 3,
  retryDelayMs: 2000,  // Wait 2 seconds between each retry
  renewal: { /* ... */ }
}

Key Renewal Configuration

Dynamically fetch new API keys from an HTTP endpoint when all keys are exhausted.

renewal

Optional Default: undefined

KeyRenewalConfig

What it does: Configuration object for fetching new API keys from an HTTP endpoint when all keys are exhausted.

keyRotation: {
  usageLimit: 100,
  additionalKeys: ['key-2'],
  renewal: {
    endpoint: 'https://your-backend.com/api/get-key',
    method: 'POST',
    headers: {
      'Authorization': 'Bearer your-service-token'
    },
    body: {
      service: 'openai',
      environment: 'production'
    },
    keyPath: 'data.apiKey',  // Extract from: { data: { apiKey: "..." } }
    timeoutMs: 10000  // 10 second timeout
  }
}

Renewal Sub-Properties

endpoint

Required

string

URL to fetch new API keys from.

endpoint: 'https://api.yourcompany.com/keys/generate'

method

Optional Default: 'GET'

'GET' | 'POST'

HTTP method for the renewal request.

method: 'POST'

headers

Optional

Record<string, string>

Custom headers for authentication.

headers: {
  'Authorization': 'Bearer token123',
  'X-Custom-Header': 'value'
}

body

Optional

any

Request body (for POST requests).

body: {
  provider: 'openai',
  purpose: 'production'
}

keyPath

Optional Default: 'apiKey'

string

JSON path to extract the key from response (supports nested paths and arrays).

// For response: { data: { apiKey: "sk-..." } }
keyPath: 'data.apiKey'

// For response: { keys: [{ value: "sk-..." }] }
keyPath: 'keys[0].value'

timeoutMs

Optional

number

Request timeout in milliseconds.

timeoutMs: 15000 // 15 second timeout

Practical Examples

Example 1: Simple Round-Robin Rotation

Most common use case - distribute load evenly across multiple keys.

keyRotation: {
  usageLimit: 100,
  additionalKeys: ['key-2', 'key-3']
}

// Flow:
// - First 100 requests use key-1
// - Next 100 requests use key-2
// - Next 100 requests use key-3
// - Then back to key-1

Example 2: Time-Windowed Usage Limits

Perfect for hourly or daily API quotas.

keyRotation: {
  usageLimit: 1000,
  usageWindowMs: 60 * 60 * 1000,  // 1 hour
  additionalKeys: ['key-2', 'key-3']
}

// Behavior:
// - Each key can make 1000 requests per hour
// - After 1 hour, usage count resets
// - Allows sustained high throughput

Example 3: With Cooldown Period

Wait before reusing exhausted keys.

keyRotation: {
  usageLimit: 50,
  cooldownMs: 10 * 60 * 1000,  // 10 minutes
  additionalKeys: ['key-2', 'key-3', 'key-4', 'key-5']
}

// Behavior:
// - Each key handles 50 requests
// - If all 5 keys exhausted, wait 10 minutes
// - Then reuse with reset counters

Example 4: Dynamic Key Renewal

Fetch new keys from your backend when needed.

keyRotation: {
  usageLimit: 500,
  additionalKeys: ['key-2'],
  retryAttempts: 5,
  retryDelayMs: 1000,
  renewal: {
    endpoint: 'https://api.yourcompany.com/keys/generate',
    method: 'POST',
    headers: {
      'X-API-Token': 'your-internal-token'
    },
    keyPath: 'key',
    timeoutMs: 15000
  }
}

// Behavior:
// - Start with 2 keys
// - When both exhausted, fetch new key from endpoint
// - Retry up to 5 times with 1 second delay
// - Add new key to pool and continue rotating

Example 5: Complete Configuration

All settings working together.

const orchestrator = new AIOrchestrator({
  providers: [
    {
      name: 'openai',
      apiKey: 'sk-key-1',
      models: [
        { id: 'gpt-4o-mini', type: 'text', cost: 0.15, quality: 'high' }
      ],
      keyRotation: {
        enabled: true,
        usageLimit: 1000,
        usageWindowMs: 3600000,    // 1 hour
        cooldownMs: 300000,        // 5 minutes
        retryAttempts: 3,
        retryDelayMs: 2000,
        additionalKeys: [
          'sk-key-2', 'sk-key-3', 'sk-key-4', 'sk-key-5'
        ],
        renewal: {
          endpoint: 'https://api.internal.com/keys',
          method: 'GET',
          headers: { 'Authorization': 'Bearer token' },
          keyPath: 'apiKey',
          timeoutMs: 10000
        }
      }
    }
  ]
});