[Frontend] Support tool calling and reasoning parser

[WangErXiao]

Remove the mutual exclusion restriction between tool calling and reasoning parser, as models like QwQ-32B can support both functionalities simultaneously. Additionally, having tool calling parse only from the content rather than the reasoning_content can improve the accuracy and performance of tool calling.

This PR resolves the issue #14490

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gaocegege]

Thanks for the PR. I will help review it. FYI I had a refactor for the reasoning support #14428, not sure if you can use is_reasoning_end instead of a new func here.

[WangErXiao]

Thanks for the PR. I will help review it. FYI I had a refactor for the reasoning support #14428, not sure if you can use is_reasoning_end instead of a new func here.

Thanks very much. I'll take a look at #14428. If you add is_reasoning_end, I will update the PR accordingly.

[WangErXiao]

Thanks for the PR. I will help review it. FYI I had a refactor for the reasoning support #14428, not sure if you can use is_reasoning_end instead of a new func here.

I have reviewed PR #14428, and I think we can reuse the is_reasoning_end function. After your review, I will proceed with the refactoring.

[gaocegege]

The streaming logic looks good to me, but does it also support full generation?

Besides this, we could also add some example code and tests

[gaocegege]

And we should:

• Remove It is not compatible with [tool_calling](https://docs.vllm.ai/en/latest/features/tool_calling.html). in reasoning outputs docs
• Add a new column in models section in reasoning outputs docs to show which model supports tool calling

[WangErXiao]

The streaming logic looks good to me, but does it also support full generation?

Besides this, we could also add some example code and tests

Yes, It support full generation, I will add some example code and tests.

[WangErXiao]

And we should:

• Remove It is not compatible with [tool_calling](https://docs.vllm.ai/en/latest/features/tool_calling.html). in reasoning outputs docs
• Add a new column in models section in reasoning outputs docs to show which model supports tool calling

OK

[gaocegege]

I think we can merge this before the refactor, I could rebase in that PR.

[WangErXiao]

I think this pr can be ready to merge. @gaocegege @russellb

[szpnygo]

Will this request be included in version 0.7.4? When will it be released?

[gaocegege]

Will this request be included in version 0.7.4? When will it be released?

@szpnygo I hope so.

[aarnphm]

can you change the title to "[Frontend] Support tool calling and reasoning parser"

[WangErXiao]

can you change the title to "[Frontend] Support tool calling and reasoning parser"

OK, done

[WangErXiao]

cc @DarkLight1337 @simon-mo @mgoin

[gaocegege]

@robertgshaw2-redhat @mgoin Could you please take a look? It blocks some other PRs about the reasoning parser.

[WangErXiao]

cc @simon-mo

[fwensen]

when streaming request model the output tool message will be incompleted

[WangErXiao]

when streaming request model the output tool message will be incompleted

#15177

It's maybe tool parser problem.
You can change

                # Coalesce any additional queued outputs
                while not q.empty():
                    next_out = q.get_nowait()
                    if sampling_params.output_kind == RequestOutputKind.DELTA:
                        out.add(next_out)
                    else:
                        out = next_out

to

                # Coalesce any additional queued outputs
               if not q.empty():
                    next_out = q.get_nowait()
                    if sampling_params.output_kind == RequestOutputKind.DELTA:
                        out.add(next_out)
                    else:
                        out = next_out