Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: Knowledge base citations/attribution stripped from agent response when invoked through orchestrator #192

Open
CaelanAlonge opened this issue Jan 3, 2025 · 1 comment
Labels
bug Something isn't working triage

Comments

@CaelanAlonge
Copy link

Expected Behaviour

  • Agents invoked through multiagent orchestrator retain the attributions (IE citing their sources) when used with a knowledge base. This is configured through the following built in prompt.

If you reference information from a search result within your answer, you must include a citation to source where the information was found. Each result has a corresponding source ID that you should reference.\nNote that <sources> may contain multiple <source> if you include information from multiple results in your answer.\nDo NOT directly quote the <search_results> in your answer. Your job is to answer the user's question as concisely as possible.\nYou must output your answer in the following format. Pay attention and follow the formatting and spacing exactly:\n<answer>\n<answer_part>\n<text>\nfirst answer text\n</text>\n<sources>\n<source>source ID</source>\n</sources>\n</answer_part>\n<answer_part>\n<text>\nsecond answer text\n</text>\n<sources>\n<source>source ID</source>\n</sources>\n</answer_part>\n</answer>"

Current Behaviour

  • I can see in the response from bedrock in modelinvocation logs in cloudwatch that the agent is indeed getting the response in that format with citations included, however the final Agent Response text content is stripped of all that information and contains the agent response without the citations.

Code snippet

# Route the query to the orchestrator
    response = await orchestrator.route_request(
        user_query,
        user_id=str(user_id),
        session_id=str(topic_id)
    )

Possible Solution

No response

Steps to Reproduce

  • invoke either a BedrockLLMAgent or AmazonBedrockAgent through multi-agent-orchestrator route_request
  • either enable trace or watch model invocation logs in cloudwatch
  • trace/logs will show agent response with source citations
  • final response from orchestrator will be the raw text WITHOUT source citations.
@CaelanAlonge CaelanAlonge added the bug Something isn't working label Jan 3, 2025
@github-actions github-actions bot added the triage label Jan 3, 2025
@brnaba-aws brnaba-aws moved this from Todo to In review in multi-agent-orchestrator board Jan 6, 2025
@brnaba-aws
Copy link
Contributor

See #193

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
Development

No branches or pull requests

2 participants