Why this matters now

If you’re building a Laravel app that calls AI models, you probably have something like this somewhere:

$response = Http::withToken(config('services.openai.key'))
    ->post('https://api.openai.com/v1/chat/completions', [...]);

That works until OpenAI has an outage, your key gets rate-limited, or you realize you’re paying $30/M output tokens for a task that Luna could handle for $6. A single-provider integration is tech debt you will pay for later — in downtime, cost, or both.

This tutorial builds a multi-provider AI routing layer in Laravel that:

  • Routes each task to the cheapest adequate model
  • Falls back automatically when a provider fails
  • Lets you add new providers without touching business logic

It’s the implementation counterpart to the multi-provider gateway concept and the cost comparison guide.

Architecture overview

Task Request
    │
    ▼
┌─────────────────────────────┐
│      Router                 │
│  - decides which provider   │
│    based on task profile    │
└─────────────────────────────┘
    │
    ▼
┌─────────────────────────────┐
│  Provider Interface         │
│  ┌─────────┐ ┌──────────┐  │
│  │ OpenAI  │ │ Anthropic │  │
│  │ (Luna)  │ │ (Sonnet)  │  │
│  └─────────┘ └──────────┘  │
│  ┌─────────┐ ┌──────────┐  │
│  │ Google  │ │ Mistral  │  │
│  │ (Gemini)│ │ (Medium) │  │
│  └─────────┘ └──────────┘  │
└─────────────────────────────┘
    │
    ▼
Fallback chain on failure

Step 1: The provider contract

Start with an interface that every provider driver implements:

<?php

namespace App\Services\Ai\Contracts;

use App\Services\Ai\Data\AiRequest;
use App\Services\Ai\Data\AiResponse;

interface AiProviderContract
{
    public function send(AiRequest $request): AiResponse;

    public function isAvailable(): bool;

    public function name(): string;

    public function costPerTask(AiRequest $request): float;
}

And the value objects:

<?php

namespace App\Services\Ai\Data;

class AiRequest
{
    public function __construct(
        public readonly string $systemPrompt,
        public readonly string $userPrompt,
        public readonly string $taskType,      // 'chat', 'coding', 'extraction'
        public readonly int    $maxTokens = 2048,
        public readonly float  $temperature = 0.7,
        public readonly array  $extra = [],
    ) {}
}

class AiResponse
{
    public function __construct(
        public readonly string  $content,
        public readonly string  $provider,
        public readonly string  $model,
        public readonly int     $inputTokens,
        public readonly int     $outputTokens,
        public readonly float   $cost,
        public readonly bool    $success = true,
        public readonly ?string $error = null,
    ) {}
}

Step 2: Provider implementations

Each provider gets its own driver class. Here’s the Anthropic driver as an example:

<?php

namespace App\Services\Ai\Drivers;

use App\Services\Ai\Contracts\AiProviderContract;
use App\Services\Ai\Data\AiRequest;
use App\Services\Ai\Data\AiResponse;
use Illuminate\Support\Facades\Http;

class AnthropicDriver implements AiProviderContract
{
    public function __construct(
        private readonly string $apiKey,
        private readonly string $model,     // 'claude-sonnet-5' or 'claude-opus-4-8'
        private readonly float  $inputPrice,  // per 1M tokens
        private readonly float  $outputPrice,
    ) {}

    public function send(AiRequest $request): AiResponse
    {
        $start = microtime(true);

        $response = Http::withToken($this->apiKey)
            ->withHeader('anthropic-version', '2026-01-01')
            ->post('https://api.anthropic.com/v1/messages', [
                'model' => $this->model,
                'max_tokens' => $request->maxTokens,
                'temperature' => $request->temperature,
                'system' => $request->systemPrompt,
                'messages' => [
                    ['role' => 'user', 'content' => $request->userPrompt],
                ],
            ]);

        if ($response->failed()) {
            return new AiResponse(
                content: '',
                provider: $this->name(),
                model: $this->model,
                inputTokens: 0,
                outputTokens: 0,
                cost: 0,
                success: false,
                error: $response->body(),
            );
        }

        $data = $response->json();
        $inputTokens = $data['usage']['input_tokens'] ?? 0;
        $outputTokens = $data['usage']['output_tokens'] ?? 0;

        return new AiResponse(
            content: $data['content'][0]['text'] ?? '',
            provider: $this->name(),
            model: $this->model,
            inputTokens: $inputTokens,
            outputTokens: $outputTokens,
            cost: ($inputTokens / 1_000_000 * $this->inputPrice)
                + ($outputTokens / 1_000_000 * $this->outputPrice),
            success: true,
        );
    }

    public function isAvailable(): bool
    {
        return filled($this->apiKey);
    }

    public function name(): string
    {
        return 'anthropic';
    }

    public function costPerTask(AiRequest $request): float
    {
        // Estimate based on typical input/output ratio for the task type
        $estimatedInput = match ($request->taskType) {
            'coding' => 8000,
            'extraction' => 4000,
            default => 2000,
        };
        $estimatedOutput = $request->maxTokens;

        return ($estimatedInput / 1_000_000 * $this->inputPrice)
            + ($estimatedOutput / 1_000_000 * $this->outputPrice);
    }
}

The OpenAI and Gemini drivers follow the same pattern — same contract, different endpoints and authentication. Each driver receives its model name and pricing at construction time, which makes them configurable from a single config file.

Step 3: Configuration

Create config/ai-providers.php:

<?php

return [
    'default' => env('AI_DEFAULT_PROVIDER', 'openai'),

    'providers' => [
        'openai' => [
            'driver' => 'openai',
            'api_key' => env('OPENAI_API_KEY'),
            'models' => [
                'luna' => [
                    'model' => 'gpt-5.6-luna',
                    'input_price' => 1.00,
                    'output_price' => 6.00,
                    'tasks' => ['chat', 'extraction', 'classification'],
                ],
                'terra' => [
                    'model' => 'gpt-5.6-terra',
                    'input_price' => 2.50,
                    'output_price' => 15.00,
                    'tasks' => ['coding', 'reasoning'],
                ],
            ],
        ],

        'anthropic' => [
            'driver' => 'anthropic',
            'api_key' => env('ANTHROPIC_API_KEY'),
            'models' => [
                'sonnet-5' => [
                    'model' => 'claude-sonnet-5',
                    'input_price' => 3.00,
                    'output_price' => 15.00,
                    'tasks' => ['coding', 'reasoning', 'chat'],
                ],
                'opus-4-8' => [
                    'model' => 'claude-opus-4-8',
                    'input_price' => 5.00,
                    'output_price' => 25.00,
                    'tasks' => ['hard_problems', 'agentic'],
                ],
            ],
        ],

        'google' => [
            'driver' => 'google',
            'api_key' => env('GOOGLE_API_KEY'),
            'models' => [
                'gemini-pro' => [
                    'model' => 'gemini-3.1-pro',
                    'input_price' => 2.00,
                    'output_price' => 12.00,
                    'tasks' => ['chat', 'reasoning', 'long_context'],
                ],
            ],
        ],
    ],

    'routing' => [
        'strategy' => env('AI_ROUTING_STRATEGY', 'cost'), // 'cost', 'priority', 'random'

        // Task profiles map task types to preferred models
        'profiles' => [
            'chat' => ['openai:luna', 'anthropic:sonnet-5', 'google:gemini-pro'],
            'coding' => ['anthropic:sonnet-5', 'openai:terra', 'google:gemini-pro'],
            'extraction' => ['openai:luna', 'google:gemini-pro'],
            'reasoning' => ['anthropic:sonnet-5', 'openai:terra'],
            'hard_problems' => ['anthropic:opus-4-8', 'openai:terra'],
            'long_context' => ['google:gemini-pro'],
        ],

        // Models to skip if budget exceeds this per task
        'max_cost_per_task' => [
            'chat' => 0.05,
            'extraction' => 0.03,
            'coding' => 0.50,
        ],

        'fallback' => [
            'enabled' => true,
            'max_attempts' => 3,
        ],
    ],
];

This config encapsulates the entire routing strategy. Changing which model handles which task type is a config change, not a code change.

Step 4: The router

The router orchestrates everything:

<?php

namespace App\Services\Ai;

use App\Services\Ai\Contracts\AiProviderContract;
use App\Services\Ai\Data\AiRequest;
use App\Services\Ai\Data\AiResponse;
use Illuminate\Support\Facades\Log;

class AiRouter
{
    /** @var array<string, AiProviderContract[]> */
    private array $instances = [];

    public function __construct(
        private readonly array $config,
    ) {
        $this->bootProviders();
    }

    public function send(AiRequest $request): AiResponse
    {
        $profile = $this->config['routing']['profiles'][$request->taskType] ?? [];
        $maxCost = $this->config['routing']['max_cost_per_task'][$request->taskType] ?? PHP_FLOAT_MAX;

        $attempts = 0;
        $maxAttempts = $this->config['routing']['fallback']['max_attempts'];

        foreach ($profile as $modelKey) {
            if ($attempts >= $maxAttempts) {
                break;
            }

            [$providerName, $model] = explode(':', $modelKey);

            if (!isset($this->instances[$providerName][$model])) {
                continue;
            }

            $driver = $this->instances[$providerName][$model];

            // Skip if this model's estimated cost exceeds the task budget
            if ($driver->costPerTask($request) > $maxCost) {
                continue;
            }

            $attempts++;
            $response = $driver->send($request);

            if ($response->success) {
                Log::info('AI request succeeded', [
                    'provider' => $response->provider,
                    'model' => $response->model,
                    'cost' => $response->cost,
                    'task_type' => $request->taskType,
                    'attempt' => $attempts,
                ]);

                return $response;
            }

            Log::warning('AI request failed, attempting fallback', [
                'provider' => $response->provider,
                'model' => $response->model,
                'error' => $response->error,
                'task_type' => $request->taskType,
                'attempt' => $attempts,
            ]);
        }

        return new AiResponse(
            content: '',
            provider: 'none',
            model: 'none',
            inputTokens: 0,
            outputTokens: 0,
            cost: 0,
            success: false,
            error: 'All providers failed',
        );
    }

    private function bootProviders(): void
    {
        foreach ($this->config['providers'] as $name => $providerConfig) {
            $driverClass = match ($providerConfig['driver']) {
                'openai' => Drivers\OpenAiDriver::class,
                'anthropic' => Drivers\AnthropicDriver::class,
                'google' => Drivers\GoogleDriver::class,
                default => throw new \InvalidArgumentException("Unknown driver: {$providerConfig['driver']}"),
            };

            foreach ($providerConfig['models'] as $modelKey => $modelConfig) {
                $this->instances[$name][$modelKey] = new $driverClass(
                    apiKey: $providerConfig['api_key'],
                    model: $modelConfig['model'],
                    inputPrice: $modelConfig['input_price'],
                    outputPrice: $modelConfig['output_price'],
                );
            }
        }
    }
}

Step 5: Using it in your app

Register the router as a singleton in a service provider:

<?php

namespace App\Providers;

use App\Services\Ai\AiRouter;
use Illuminate\Support\ServiceProvider;

class AiServiceProvider extends ServiceProvider
{
    public function register(): void
    {
        $this->app->singleton(AiRouter::class, function () {
            return new AiRouter(config('ai-providers'));
        });
    }
}

Then use it anywhere in your Laravel app:

use App\Services\Ai\AiRouter;
use App\Services\Ai\Data\AiRequest;

class DocumentController extends Controller
{
    public function __construct(
        private readonly AiRouter $ai,
    ) {}

    public function summarize(Document $document): JsonResponse
    {
        $request = new AiRequest(
            systemPrompt: 'Summarize the following document in 3 bullet points.',
            userPrompt: $document->content,
            taskType: 'extraction',
            maxTokens: 500,
        );

        $response = $this->ai->send($request);

        if (!$response->success) {
            return response()->json([
                'error' => 'AI service unavailable',
            ], 503);
        }

        return response()->json([
            'summary' => $response->content,
            'cost' => $response->cost,
            'provider' => $response->provider,
        ]);
    }
}

Step 6: Queue it for production

For anything beyond a simple chat wrapper, dispatch AI tasks to a queue so failures trigger retries without blocking the user:

<?php

namespace App\Jobs;

use App\Services\Ai\AiRouter;
use App\Services\Ai\Data\AiRequest;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;

class ProcessAiTask implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable;

    public int $tries = 1; // Router handles its own retries
    public int $timeout = 120;

    public function __construct(
        private readonly AiRequest $request,
        private readonly string $callbackClass,
        private readonly array $callbackParams = [],
    ) {}

    public function handle(AiRouter $router): void
    {
        $response = $router->send($this->request);

        if (!$response->success) {
            $this->fail(new \RuntimeException(
                "AI processing failed after all fallbacks: {$response->error}"
            ));
            return;
        }

        // Fire the callback with the result
        $callback = app($this->callbackClass);
        $callback->handle($response, ...$this->callbackParams);
    }
}

Production considerations

Rate limiting: Each provider has different rate limits. Add a rate-limiter middleware per driver that queues requests when approaching limits.

Cost tracking: Log every AiResponse to a database table. Monthly cost reports by provider, model, and task type pay for themselves many times over by revealing where you’re overpaying.

Health checks: Run a periodic check against each provider’s status endpoint and mark drivers as unavailable proactively rather than discovering downtime through failed user requests.

Caching: For deterministic tasks (extraction, classification), cache responses keyed by prompt hash. This cuts costs significantly on repeated workloads.

The three-line summary

  1. Define a AiProviderContract interface and implement a driver per provider
  2. Configure routing profiles that map task types to preferred models with cost budgets
  3. The router iterates the profile, skips over-budget models, and falls back on failure

For a complete working example, check the GitHub repo (example code in the /examples directory).

About the author: Charles Jasthyn De La Cueva is an Admin Officer at PSU’s Quality Assurance Office and the builder behind ParSU-EDMS / QAOSYS. He writes about AI tools, infrastructure, and practical agent deployment.