Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 16 additions & 11 deletions docs/source/ko/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -369,16 +369,16 @@
title: (๋ฒˆ์—ญ์ค‘) Optimize inference using `torch.compile()`
title: (๋ฒˆ์—ญ์ค‘) ์„ฑ๋Šฅ ๋ฐ ํ™•์žฅ์„ฑ
- sections:
- local: contributing
title: ๐Ÿค— Transformers์— ๊ธฐ์—ฌํ•˜๋Š” ๋ฐฉ๋ฒ•
- local: add_new_model
title: ๐Ÿค— Transformers์— ์ƒˆ๋กœ์šด ๋ชจ๋ธ์„ ์ถ”๊ฐ€ํ•˜๋Š” ๋ฐฉ๋ฒ•
- local: add_new_pipeline
title: ์–ด๋–ป๊ฒŒ ๐Ÿค— Transformers์— ํŒŒ์ดํ”„๋ผ์ธ์„ ์ถ”๊ฐ€ํ•˜๋‚˜์š”?
- local: testing
title: ํ…Œ์ŠคํŠธ
- local: pr_checks
title: Pull Request์— ๋Œ€ํ•œ ๊ฒ€์‚ฌ
- local: contributing
title: ๐Ÿค— Transformers์— ๊ธฐ์—ฌํ•˜๋Š” ๋ฐฉ๋ฒ•
- local: add_new_model
title: ๐Ÿค— Transformers์— ์ƒˆ๋กœ์šด ๋ชจ๋ธ์„ ์ถ”๊ฐ€ํ•˜๋Š” ๋ฐฉ๋ฒ•
- local: add_new_pipeline
title: ์–ด๋–ป๊ฒŒ ๐Ÿค— Transformers์— ํŒŒ์ดํ”„๋ผ์ธ์„ ์ถ”๊ฐ€ํ•˜๋‚˜์š”?
- local: testing
title: ํ…Œ์ŠคํŠธ
- local: pr_checks
title: Pull Request์— ๋Œ€ํ•œ ๊ฒ€์‚ฌ
title: ๋ฆฌ์†Œ์Šค
- isExpanded: false
sections:
Expand Down Expand Up @@ -1243,4 +1243,9 @@
- local: in_translation
title: (๋ฒˆ์—ญ์ค‘)Environment Variables
title: Reference
title: API
title: API
- title: ์ถ”๋ก 
isExpanded: false
sections:
- local: jan
title: 1์›”
32 changes: 32 additions & 0 deletions docs/source/ko/jan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Jan: ๋กœ์ปฌ LLM ์ œ๊ณต์ž๋กœ serving API ์‚ฌ์šฉํ•˜๊ธฐ [[jan-using-the-serving-api-as-a-local-llm-provider]]

์ด ์˜ˆ์ œ๋Š” `transformers serve`๋ฅผ [Jan](https://jan.ai/) ์•ฑ์˜ ๋กœ์ปฌ LLM ์ œ๊ณต์ž๋กœ ์‚ฌ์šฉํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค. Jan์€ ์™„์ „ํžˆ ๊ธฐ๊ธฐ์—์„œ ์‹คํ–‰๋˜๋Š” ChatGPT ๋Œ€์ฒด ๊ทธ๋ž˜ํ”ฝ ์ธํ„ฐํŽ˜์ด์Šค์ž…๋‹ˆ๋‹ค. `transformers serve`์— ๋Œ€ํ•œ ์š”์ฒญ์€ ๋กœ์ปฌ ์•ฑ์—์„œ ์ง์ ‘ ์˜ค๋Š”๋ฐ - ์ด ์„น์…˜์€ Jan์— ์ดˆ์ ์„ ๋งž์ถ”๊ณ  ์žˆ์ง€๋งŒ, ๋กœ์ปฌ ์š”์ฒญ์„ ์ˆ˜ํ–‰ํ•˜๋Š” ๋‹ค๋ฅธ ์•ฑ์—๋„ ์ผ๋ถ€ ์ง€์นจ์„ ํ™•์žฅํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

## ๋กœ์ปฌ์—์„œ ๋ชจ๋ธ ์‹คํ–‰ํ•˜๊ธฐ [[running-models-locally]]

`transformers serve`๋ฅผ Jan๊ณผ ์—ฐ๊ฒฐํ•˜๋ ค๋ฉด ์ƒˆ ๋ชจ๋ธ ์ œ๊ณต์ž๋ฅผ ์„ค์ •ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค("Settings" > "Model Providers"). "Add Provider"๋ฅผ ํด๋ฆญํ•˜๊ณ  ์ƒˆ ์ด๋ฆ„์„ ์„ค์ •ํ•˜์„ธ์š”. ์ƒˆ ๋ชจ๋ธ ์ œ๊ณต์ž ํŽ˜์ด์ง€์—์„œ ๋‹ค์Œ ํŒจํ„ด์œผ๋กœ "Base URL"๋งŒ ์„ค์ •ํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค:

```shell
http://[host]:[port]/v1
```

์—ฌ๊ธฐ์„œ `host`์™€ `port`๋Š” `transformers serve` CLI ๋งค๊ฐœ๋ณ€์ˆ˜์ž…๋‹ˆ๋‹ค(๊ธฐ๋ณธ๊ฐ’์€ `localhost:8000`). ์ด ์„ค์ •์„ ์™„๋ฃŒํ•œ ํ›„ "Refresh"๋ฅผ ๋ˆ„๋ฅด๋ฉด "Models" ์„น์…˜์—์„œ ์ผ๋ถ€ ๋ชจ๋ธ์„ ๋ณผ ์ˆ˜ ์žˆ์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. "API key" ํ…์ŠคํŠธ ํ•„๋“œ์— ํ…์ŠคํŠธ๋ฅผ ์ถ”๊ฐ€ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค - ์ด ๋ฐ์ดํ„ฐ๋Š” ์‹ค์ œ๋กœ ์‚ฌ์šฉ๋˜์ง€ ์•Š์ง€๋งŒ ํ•„๋“œ๋Š” ๋น„์›Œ๋‘˜ ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค. ์‚ฌ์šฉ์ž ์ •์˜ ๋ชจ๋ธ ์ œ๊ณต์ž ํŽ˜์ด์ง€๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์ด ๋ณด์—ฌ์•ผ ํ•ฉ๋‹ˆ๋‹ค:

<h3 align="center">
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/transformers_serve_jan_model_providers.png"/>
</h3>

์ด์ œ ์ฑ„ํŒ…ํ•  ์ค€๋น„๊ฐ€ ๋˜์—ˆ์Šต๋‹ˆ๋‹ค!

> [!TIP]
> `transformers serve`๋ฅผ ํ†ตํ•ด ๋ชจ๋“  `transformers`-ํ˜ธํ™˜ ๋ชจ๋ธ์„ Jan์— ์ถ”๊ฐ€ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ƒ์„ฑํ•œ ์‚ฌ์šฉ์ž ์ •์˜ ๋ชจ๋ธ ์ œ๊ณต์ž์—์„œ "Models" ์„น์…˜์˜ "+" ๋ฒ„ํŠผ์„ ํด๋ฆญํ•˜๊ณ  Hub ์ €์žฅ์†Œ ์ด๋ฆ„(์˜ˆ: `Qwen/Qwen3-4B`)์„ ์ถ”๊ฐ€ํ•˜์„ธ์š”.

## ๋ณ„๋„ ๊ธฐ๊ธฐ์—์„œ ๋ชจ๋ธ ์‹คํ–‰ํ•˜๊ธฐ [[running-models-on-a-separate-machine]]

์ด ์˜ˆ์ œ๋ฅผ ๋งˆ๋ฌด๋ฆฌํ•˜๊ธฐ ์œ„ํ•ด ์ข€ ๋” ๊ณ ๊ธ‰ ์‚ฌ์šฉ ์‚ฌ๋ก€๋ฅผ ์‚ดํŽด๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค. ๋ชจ๋ธ์„ ์ œ๊ณตํ•  ๊ฐ•๋ ฅํ•œ ๊ธฐ๊ธฐ๊ฐ€ ์žˆ์ง€๋งŒ ๋‹ค๋ฅธ ์žฅ์น˜์—์„œ Jan์„ ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ์„ ์„ ํ˜ธํ•œ๋‹ค๋ฉด ํฌํŠธ ํฌ์›Œ๋”ฉ์„ ์ถ”๊ฐ€ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. Jan ๊ธฐ๊ธฐ์—์„œ ์„œ๋ฒ„๋กœ `ssh` ์ ‘์†์ด ๊ฐ€๋Šฅํ•˜๋‹ค๋ฉด, Jan ๊ธฐ๊ธฐ์˜ ํ„ฐ๋ฏธ๋„์— ๋‹ค์Œ์„ ์ž…๋ ฅํ•˜์—ฌ ์ˆ˜ํ–‰ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค

```
ssh -N -f -L 8000:localhost:8000 your_server_account@your_server_IP -p port_to_ssh_into_your_server
```

ํฌํŠธ ํฌ์›Œ๋”ฉ์€ Jan์—๋งŒ ๊ตญํ•œ๋˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค: ๋‹ค๋ฅธ ๊ธฐ๊ธฐ์—์„œ ์‹คํ–‰ ์ค‘์ธ `transformers serve`๋ฅผ ์„ ํƒํ•œ ์•ฑ๊ณผ ์—ฐ๊ฒฐํ•˜๋Š” ๋ฐ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.