Agent and API Provider Management

Overview | Quick Start | How It Works | Configuration Options | Use Cases | Best Practices | FAQs | Troubleshooting

Availability

• Version: Available since v0.2

Overview

Agent and API Provider Management enables you to deploy, configure, and monitor the Tyk Governance agents that discover and synchronize APIs from various providers.

Understanding Agents and API Providers

Agents are lightweight services that act as secure bridges between your API providers and the Tyk Governance Hub. They:

• Run in your environment (on-premises, cloud, or hybrid)
• Connect to one or more API providers to discover APIs
• Securely transmit API data to the Governance Hub
• Support high availability deployments
• Can be monitored through the “Agents” section in the UI

API Providers are the systems where your APIs are defined and managed, such as:

• Tyk Dashboard
• AWS API Gateway

Each provider is configured within an agent and can be monitored through the “API Providers” section in the UI, where you can see:

• How many APIs are discovered from each provider
• The connection status of each provider
• The last synchronization time
• Provider-specific details

Key Benefits

• Centralized Agent Visibility: Monitor all agents from a single dashboard
• Scheduled Synchronization: Configure automatic API discovery at regular intervals
• High Availability Deployment: Deploy multiple agent replicas with automatic leader election
• Detailed Status Monitoring: Track agent health, connection status, and synchronization history
• Secure Communication: All agent-to-hub communication is encrypted and authenticated

Dependencies

• Requires Tyk Governance and agents v0.2 or higher
• Agents support Kubernetes-based high availability, which requires Kubernetes cluster access

Quick Start

In this tutorial, we’ll set up a new agent, configure it to connect to an API provider, and monitor its status.

Prerequisites

• Access to Tyk Governance Hub
• Access to an API provider (Tyk Dashboard or AWS API Gateway)
• A server or container environment to run the agent

Step-by-Step

1. Create a New Agent

   • From the Agents page, click the New agent button in the top-right corner.

   

   • In the New agent form, enter:
     • Name: A descriptive name for the agent (required)
     • Description: Details about the agent’s purpose or location (required)

   

   • Click Create agent to save the new agent.

   

   • Alternatively, use the API to create a new agent:

   # Replace with your actual Governance URL and API key
   GOVERNANCE_URL="https://your-governance-instance.tyk.io"
   API_KEY="your-access-key"
   AGENT_NAME="My API Provider Agent"
   
   # Create agent
   curl -s -X POST "${GOVERNANCE_URL}/api/agents/" \
       -H "X-API-Key: ${API_KEY}" \
       -H "Content-Type: application/json" \
       -d '{
           "name": "'"${AGENT_NAME}"'"
       }'
   

   Note the agent ID returned in the response:

   {"id":"a51d9bd0-bafe-4749-8285-e18641b151f2","name":"My API Provider Agent","last_heartbeat":"0001-01-01T00:00:00Z","status":"INACTIVE","providers":null}
   

2. Generate an Agent Token

   • In the Agent information section, find the Access token field.

   • If no token exists, click Generate new access token.

   • For existing tokens:

     • Click the eye icon to show/hide the token
     • Use the copy icon to copy the token to your clipboard
     • Use this token in your agent configuration file

   

   

   • Alternatively, generate an authentication token for the agent using the API:

   # Replace with the agent ID from the previous step
   AGENT_ID="a51d9bd0-bafe-4749-8285-e18641b151f2"
   
   # Generate token
   curl -X POST "${GOVERNANCE_URL}/api/auth/token/" \
       -H "X-API-Key: ${API_KEY}" \
       -H "Content-Type: application/json" \
       -d '{
           "agent_id": "'"${AGENT_ID}"'"
       }'
   

   Save the token from the response:

   {
       "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."
   }
   

3. Create Agent Configuration File

   Create a file named agent-config.yaml with the following content:

   # Tyk Governance Agent Configuration
   governanceDashboard:
       server:
           # The gRPC endpoint URL of the Tyk Governance service
           url: "your-governance-instance.tyk.io:50051"
   
       auth:
           # Authentication token for this agent
           token: "your-agent-token"
   
   # API Provider Configurations
   instances:
       # Example Tyk Provider Configuration
       - name: "tyk-provider"
          type: "tyk"
          config:
            host: "http://your-tyk-dashboard:3000"
            auth: "your-tyk-dashboard-api-key"
   
       # Example AWS API Gateway Configuration
       - name: "aws-provider"
          type: "aws"
          config:
           accessKeyId: "your-aws-access-key-id"
           accessKeySecret: "your-aws-access-key-secret"
           region: "us-east-1"
   

4. Deploy the Agent

   Run the agent using Docker or as Kubernetes Deployment. See Installation for details.

5. Monitor Agent Status

   • Check the agent status in the Governance Hub. A healthy agent will appear as “Healthy”:

   

   • Check the agent status in the Governance Hub using API:

   curl "${GOVERNANCE_URL}/api/agents/${AGENT_ID}" \
       -H "X-API-Key: ${API_KEY}"
   

   A healthy agent will show:

   {
       "id": "a51d9bd0-bafe-4749-8285-e18641b151f2",
       "name": "My API Provider Agent",
       "last_heartbeat": "2025-07-16T12:34:56Z",
       "status": "ACTIVE",
       "providers": [
           {
               "name": "tyk-provider",
               "type": "tyk"
           },
           {
               "name": "aws-provider",
               "type": "aws"
           }
       ]
   }
   

6. Trigger Initial Synchronization

   Trigger a synchronization to verify the agent is working correctly.

Validation

• The agent should appear as “Healthy” or have status as “ACTIVE” in the agent list
• The agent’s last heartbeat timestamp should be recent
• After synchronization, APIs from your providers should appear in the API Repository

How It Works

Agent Management works through a distributed architecture where agents deployed in your environments connect to the central Governance Hub using secure gRPC streams.

Agent Architecture

    flowchart LR
    subgraph "User Environment"
        Agent[Governance Agent] --> Provider1[Tyk Dashboard]
        Agent --> Provider2[AWS API Gateway]
        Agent --> Provider3[Other Providers]
    end
    
    subgraph "Governance Hub"
        Hub[Governance Service] --- APIRepo[API Repository]
    end
    
    Agent <-->|gRPC Streams| Hub
  

The agent establishes two persistent gRPC streams with the Governance Hub:

1. Health Stream: Sends regular heartbeats to indicate the agent is alive and functioning
2. Sync Stream: Used for API synchronization operations

When multiple agent replicas are deployed with leader election enabled, they use Kubernetes leader election to ensure only one instance actively performs synchronization, while others stand by as hot backups.

Synchronization Process

Synchronization can be triggered in three ways:

1. Manual Trigger: Through the Governance Hub UI or API

   

2. Scheduled Sync: At regular intervals configured in the agent. See Understanding Scheduled Synchronization.

3. Initial Connection: When an agent first connects to the Governance Hub

Understanding Scheduled Synchronization

There are two complementary ways to configure scheduled synchronization:

1. Agent Configuration File (Agent-side scheduling)

   Configure the syncSchedule section in the agent’s configuration file:

   # Agent configuration with scheduled sync
   syncSchedule:
     enabled: true
     intervalMinutes: 60  # Sync every hour
   

   With this configuration:

   • The agent will independently initiate synchronization every intervalMinutes
   • This happens regardless of any schedules configured on the Governance Hub
   • It’s a “pull” model where the agent decides when to sync
   • Minimum interval is 5 minutes

2. Governance Hub API (Hub-side scheduling)

   Use the /api/agents/{id}/sync-jobs endpoint to configure provider-specific schedules:

   # Schedule sync for a specific provider
   curl -X POST "${GOVERNANCE_URL}/api/agents/${AGENT_ID}/sync-jobs" \
       -H "X-API-Key: ${API_KEY}" \
       -H "Content-Type: application/json" \
       -d '{
           "repeat_every": "12h",
           "provider": "tyk-provider",
           "start_from": "2023-06-01T12:00:00Z"
       }'
   

   With this approach:

   • The Governance Hub’s scheduler component manages the schedule
   • The Hub initiates synchronization by sending requests to the agent
   • It’s a “push” model where the Hub tells the agent when to sync
   • You can set different schedules for different providers
   • Supports Go duration format (e.g., “1h”, “12h”, “7d”)

Best Practice: You can use both mechanisms together for maximum flexibility and reliability:

• Set a conservative baseline schedule in the agent config (e.g., daily)
• Use the sync-jobs API for more frequent or specific provider syncs

During synchronization:

1. The agent receives a sync request from the hub (or initiates it based on its schedule)
2. The agent queries each configured API provider for APIs
3. The agent processes and normalizes the API definitions
4. The agent streams the API definitions to the hub
5. The hub processes and stores the API definitions
6. The hub reconciles the API inventory, marking missing APIs as deleted

Deployment Scenarios

Multi-Provider API Discovery

Deploy agents to connect to different API providers across your organization, creating a comprehensive API inventory that spans platforms.

# Agent configuration with multiple providers
instances:
  - name: "tyk-dashboard"
    type: "tyk"
    config:
      host: "http://tyk-dashboard:3000"
      auth: "your-tyk-api-key"
  
  - name: "aws-us-east"
    type: "aws"
    config:
      accessKeyId: "your-aws-access-key"
      accessKeySecret: "your-aws-secret-key"
      region: "us-east-1"
  
  - name: "aws-eu-west"
    type: "aws"
    config:
      accessKeyId: "your-aws-access-key"
      accessKeySecret: "your-aws-secret-key"
      region: "eu-west-1"

High Availability Agent Deployment

Deploy multiple agent replicas in Kubernetes to ensure continuous API discovery even if some instances fail.

# Kubernetes deployment with leader election
apiVersion: apps/v1
kind: Deployment
metadata:
  name: governance-agent
spec:
  replicas: 3  # Multiple replicas for redundancy
  template:
    spec:
      containers:
      - name: agent
        env:
        - name: POD_NAME # When leader election is enabled, POD_NAME environment variables must be set
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE # When leader election is enabled, POD_NAMESPACE environment variables must be set
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: TYK_AGENT_LEADERELECTION_ENABLED
          value: "true"

Scheduled API Synchronization

Configure agents to automatically synchronize APIs at regular intervals to maintain an up-to-date inventory.

# Agent configuration with scheduled sync
syncSchedule:
  enabled: true
  intervalMinutes: 120  # Sync every 2 hours

Configuration Options

Agent Configuration File

The agent is configured using a YAML configuration file with the following sections:

Core Configuration

# Governance Dashboard Connection
governanceDashboard:
  server:
    url: "your-governance-instance.tyk.io:50051"
  auth:
    token: "your-agent-token"

# Log level (debug, info, warn, error)
logLevel: info

# Health probe configuration
healthProbe:
  server:
    port: 5959

API Provider Configuration

# API Provider Configurations
instances:
  # Tyk Provider
  - name: "tyk-provider"
    type: "tyk"
    config:
      host: "http://dashboard:3000"
      auth: "your-tyk-api-key"
  
  # AWS API Gateway Provider
  - name: "aws-provider"
    type: "aws"
    config:
      accessKeyId: "your-aws-access-key-id"
      accessKeySecret: "your-aws-access-key-secret"
      region: "us-east-1"
      # Optional session token for temporary credentials
      sessionToken: "your-aws-session-token"

Scheduled Synchronization (New in v0.2)

# Sync schedule configuration
syncSchedule:
  # Enable or disable scheduled synchronization
  enabled: true
  # Interval in minutes between syncs (minimum 5)
  intervalMinutes: 60

gRPC Connection (New in v0.2)

rpc:
  # Keepalive configures the keepalive settings for the gRPC connection.
  keepalive:
    # Enabled controls whether keepalive is enabled.
    enabled: true
    # Time is the duration after which if there are no activities, ping will be sent.
    time: 30s
    # Timeout is the duration the client waits for a response to a keepalive ping.
    timeout: 20s
    # PermitWithoutStream if true allows sending pings even without active streams.
    permitWithoutStream: true

High Availability Configuration (New in v0.2)

leaderElection:
  # Enable or disable leader election
  enabled: true
  # Name of the Kubernetes lease object used for leader election
  leaseName: "governance-agent-lock"
  # Namespace where the lease object will be created
  # If not specified, the agent's namespace will be used
  leaseNamespace: ""
  # Duration that non-leader candidates will wait before attempting to acquire leadership
  leaseDuration: "15s"
  # Duration that the acting leader will retry refreshing leadership before giving up
  renewDeadline: "10s"
  # Duration the leader elector clients should wait between leadership acquisition attempts
  retryPeriod: "2s"

Environment Variables

The agent supports configuration through environment variables:

Scheduled Synchronization Variables (New in v0.2)

gRPC Connection Variables (New in v0.2)

Leader Election Variables (New in v0.2)

Required Environment Variables for Leader Election

When leader election is enabled, the following environment variables must also be set:

• POD_NAME: The name of the pod (used as the identity for leader election)
• POD_NAMESPACE: The namespace of the pod (used for creating the lease object)

These are typically set automatically when deploying with Kubernetes using the downward API:

env:
  - name: POD_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name
  - name: POD_NAMESPACE
    valueFrom:
      fieldRef:
        fieldPath: metadata.namespace

Use Cases

Multi-Region API Discovery

For organizations with APIs deployed across multiple geographic regions, deploy region-specific agents to efficiently discover and monitor APIs while respecting network boundaries.

Implementation:

1. Deploy agents in each region where APIs are hosted
2. Configure each agent with the appropriate regional API providers
3. Use descriptive names to identify regional agents
4. Monitor all agents from the central Governance Hub

Benefits:

• Reduced latency for API discovery operations
• Respect for network boundaries and security zones
• Improved reliability with region-specific agents
• Clear organization of APIs by region

Secure Environment Bridging

For organizations with strict network segmentation, use agents to securely bridge between isolated environments without compromising security boundaries.

Implementation:

1. Deploy agents in each network segment
2. Configure outbound-only connections from agents to the Governance Hub
3. Use separate agents for production and non-production environments
4. Implement appropriate network security controls around agent traffic

Benefits:

• Maintain network isolation while enabling governance
• No inbound connections required to sensitive environments
• Granular control over which APIs are discovered
• Separation of concerns between environments

Automated API Lifecycle Tracking

Use scheduled synchronization to automatically track the lifecycle of APIs, including when they’re created, updated, or deleted from source providers.

Implementation:

1. Configure agents with scheduled synchronization
2. Set appropriate sync intervals based on change frequency
3. Use the API Repository to monitor API status
4. Create reports showing API lifecycle events

Benefits:

• Automatic detection of API changes
• Historical record of API lifecycle events
• Reduced manual tracking effort
• Improved visibility into API landscape evolution

Best Practices and Recommendations

• Use descriptive agent names that indicate their purpose and scope
• Deploy agents close to API providers to minimize latency and network issues
• Configure appropriate sync intervals based on how frequently your APIs change
• Use leader election for high availability when deploying multiple agent replicas
• Monitor agent health regularly to ensure continuous API discovery
• Rotate API provider credentials periodically for better security
• Use the principle of least privilege when configuring API provider credentials
• Start with manual syncs before enabling scheduled synchronization
• Implement network security controls around agent traffic
• Maintain agent versions to ensure compatibility with the Governance Hub

FAQs

Troubleshooting