Using the AI Gateway to support APIs for AI applications
API Connect provides a UI wizard to create AI-aware APIs and products, plus integration with AI services to forward requests and manage responses.
AI Gateway functionality allows you to access AI Gateway policies directly on the DataPower API Gateway. You can create REST proxies that use the AI Gateway to control access to AI models, providing a seamless way to manage AI access within your API infrastructure while ensuring enhanced security and governance of your AI models.
The AI Gateway makes it easy for enterprises to manage access to API endpoints used by AI applications. The AI Gateway simplifies the integration of AI into new and existing OpenAPI 3.0 APIs in API Connect to access a set of operations exposed by an AI service.
- Use cases
- There are two cases where you should use the AI Gateway:
- As a reverse proxy for an existing API
In this case, the API contract with the AI service is unchanged.
- As a development tool for new APIs that use an AI service.
You can create APIs directly in API Connect and define policies in the API workflow to manage access to AI service.
- As a reverse proxy for an existing API
- Benefits
- With the API Connect
AI Gateway, you can
centrally manage the use of AI through policy enforcement, data encryption, masking of sensitive
data, access control, audit trails and more, in support of your compliance obligations.
Using the AI Gateway to manage your API access to AI service lets you add controls to the API execution through the following features:
- An interface to each supported AI service
API Connect provides a policy that enables your API to send requests to a specified AI service.
- Response caching
The response to an API call to an AI service is cached, which improves response time for API calls and provides cost optimization for the API provider. When you create an API in API Connect, you can specify the duration for that API's response caching.
- Rate limiting
The AI Gateway enforces defined rate limits on APIs and API Plans (which manage access to individual APIs). You can configure rate limits based on the number of requests or generated tokens allowed within a particular time interval.
- Tokenization
Tokens are used as a unit of cost for LLM APIs. With the AI Gateway, rate limits can use the number of tokens generated by a request as a means of limiting usage and thus controlling costs. The token limit determines how many tokens are allowed to pass thru the gateway within a specified period of time.
- Predefined policies and logic structures
The AI Gateway can leverage all of the existing policies and logic constructs provided by API Connect (for example, invoke, redact, and validate) in the API execution flow.
- API analytics
API Connect provides an analytics service that provides insights into AI Gateway usage. You can use the analytics data on the AI usage dashboard to manage costs and track performance. For more information, see Accessing analytics in the API Manager UI.
- An interface to each supported AI service
The AI Gateway supports the following AI services: