Bedrock Sonnet v2 and Vertex Gemini 2.0 flash: Cannot read properties of undefined (reading 'toolCalls')

[kstraszewski]

Description

When i try to use anthropic.claude-3-5-sonnet-20241022-v2:0 on amazon-bedrock, and gemini-2.0-flash-exp on Vertex im getting this error:
[500] Cannot read properties of undefined (reading 'toolCalls')

Code example

  const response = await streamText({
    model,
    system: 'You are a helpful assistant. You always try to help the user with their questions.',
    messages: messages as CoreMessage[],
    onFinish: (result) => {
      console.log(result)
    },
  })

AI provider

"ai": "4.0.27", "@ai-sdk/amazon-bedrock": "1.0.6", "@ai-sdk/anthropic": "1.0.6", "@ai-sdk/google": "1.0.12", "@ai-sdk/google-vertex": "2.0.13", "@ai-sdk/openai": "1.0.13", "@ai-sdk/vue": "1.0.7",

Additional context

No response

[Nishchit14]

I am facing the same issue with xai's grok-beta

[trevorpfiz]

I am seeing this with the Google Generative AI Provider and gemini-2.0-flash-exp as well

[lgrammel]

Can you provide more details on the messages & tool that you pass in? I have tried to reproduce the issue on latest dev; the following works for me though:

import { vertex } from '@ai-sdk/google-vertex';
import { CoreMessage, streamText, tool } from 'ai';
import 'dotenv/config';
import * as readline from 'node:readline/promises';
import { z } from 'zod';

const terminal = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

const messages: CoreMessage[] = [];

async function main() {
  while (true) {
    const userInput = await terminal.question('You: ');

    messages.push({ role: 'user', content: userInput });

    const result = streamText({
      model: vertex('gemini-2.0-flash-exp'),
      tools: {
        weather: tool({
          description: 'Get the weather in a location',
          parameters: z.object({
            location: z
              .string()
              .describe('The location to get the weather for'),
          }),
          execute: async ({ location }) => ({
            location,
            temperature: 72 + Math.floor(Math.random() * 21) - 10,
          }),
        }),
      },
      maxSteps: 5,
      messages,
    });

    process.stdout.write('\nAssistant: ');
    for await (const delta of result.textStream) {
      process.stdout.write(delta);
    }
    process.stdout.write('\n\n');

    messages.push(...(await result.response).messages);
  }
}

main().catch(console.error);

[lgrammel]

Also, can you send me the lines around line 4639 from the file mentioned in the stack trace for flush @Nishchit14 -- if I look on npm the lines don't match up:

https://www.npmjs.com/package/ai/v/4.0.27?activeTab=code

[lgrammel]

I isolated a potential cause. When an error happens that causes the stream to be aborted / no steps to be recorded, this will currently trigger the error that you are seeing. Can you also see if there are any errors in the fullStream response property?

[Nishchit14]

@lgrammel I agree, The issue happens when the error occurred from ai provider's api request.

If it is handy then can you please share the fullStream property doc link?

[lgrammel]

@Nishchit14 I think I have a potential fix, but the fix indicates that you also face an another error. Here are the docs: https://sdk.vercel.ai/docs/reference/ai-sdk-core/stream-text#returns (see fullStream in there)

[kstraszewski]

Can you provide more details on the messages & tool that you pass in? I have tried to reproduce the issue on latest dev; the following works for me though:

import { vertex } from '@ai-sdk/google-vertex';
import { CoreMessage, streamText, tool } from 'ai';
import 'dotenv/config';
import * as readline from 'node:readline/promises';
import { z } from 'zod';

const terminal = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

const messages: CoreMessage[] = [];

async function main() {
  while (true) {
    const userInput = await terminal.question('You: ');

    messages.push({ role: 'user', content: userInput });

    const result = streamText({
      model: vertex('gemini-2.0-flash-exp'),
      tools: {
        weather: tool({
          description: 'Get the weather in a location',
          parameters: z.object({
            location: z
              .string()
              .describe('The location to get the weather for'),
          }),
          execute: async ({ location }) => ({
            location,
            temperature: 72 + Math.floor(Math.random() * 21) - 10,
          }),
        }),
      },
      maxSteps: 5,
      messages,
    });

    process.stdout.write('\nAssistant: ');
    for await (const delta of result.textStream) {
      process.stdout.write(delta);
    }
    process.stdout.write('\n\n');

    messages.push(...(await result.response).messages);
  }
}

main().catch(console.error);

Im sending super simple messages:

[ { role: 'user', content: 'test' } ]

When i use:
anthropic.claude-3-5-haiku-20241022-v1:0
anthropic.claude-3-5-sonnet-20240620-v1:0
gemini-1.5-flash-002
gemini-1.5-pro-002

Everything works great, but when i change the string into gemini-2.0-flash-exp or anthropic.claude-3-5-sonnet-20241022-v2:0 (for vertex or bedrock), im getting this error.

On frontend side

const {error} = useChat()
// error returns "An error occurred"

on backend side im getting
[500] Cannot read properties of undefined (reading 'toolCalls')

[lgrammel]

I'm releasing 4.0.30 with a fix. The error masks another error that will be available in fullStream or if you enable error message forwarding / log it: https://sdk.vercel.ai/docs/ai-sdk-ui/chatbot#error-messages