Migrating from OpenAI API to Inferway
Migrate existing applications from OpenAI API to Inferway by updating base_url and model with standard OpenAI SDKs.
Two settings to change
Migrating an existing codebase from OpenAI API to Inferway requires changing only two settings: base_url and model.
Point base_url to https://api.inferway.ai/v1 and set model to an available model ID from the Inferway catalog (/models). Pass your Inferway API key in the standard Authorization Bearer header or client apiKey setting.
Python SDK example
Using the standard openai Python SDK (version 1.0+):
import os from openai import OpenAI client = OpenAI( base_url='https://api.inferway.ai/v1', api_key=os.environ['INFERWAY_API_KEY'], ) stream = client.chat.completions.create( model='your-model-id', messages=[{'role': 'user', 'content': 'Hello!'}], stream=True, ) for chunk in stream: if chunk.choices and chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end='', flush=True) print()
Node SDK example
Using the standard openai Node SDK:
import OpenAI from 'openai'; const client = new OpenAI({ baseURL: 'https://api.inferway.ai/v1', apiKey: process.env.INFERWAY_API_KEY, }); const stream = await client.chat.completions.create({ model: 'your-model-id', messages: [{ role: 'user', content: 'Hello!' }], stream: true, }); for await (const chunk of stream) { const content = chunk.choices[0]?.delta?.content; if (content) process.stdout.write(content); } process.stdout.write('\n');
Parameter compatibility and differences
Supported parameters on /v1/chat/completions: model, messages (system, user, assistant, and tool roles with text and inline image blocks), stream (SSE delta streaming), temperature, top_p, max_tokens (a non-streaming request is held to 4,096 output tokens in total, max_tokens × n. With an API key a larger or unset max_tokens is not refused: the gateway serves it at 4,096 ÷ n tokens per candidate and names the applied cap in the X-Inferway-Max-Tokens-Applied and X-Inferway-Max-Tokens-Reason response headers, so a long answer can end early with finish_reason length. A guest request without a key that names a total above 4,096 gets a 400 invalid_request. Set stream: true for longer output), stop (string or array of stop sequences), tools and tool_choice (function calling), response_format, and stream_options.
Endpoints and parameters not supported by the gateway: legacy completions (/v1/completions) and embeddings. Top-level priority is stripped.