๐Ÿง ํƒ์ • ์ˆ˜์‚ฌ: ์ฒซ ๋ฒˆ์งธ ์‚ฌ๋ก€

๊ฐœ์š”

์ด๋ฒˆ ํผ์ฆ์—์„œ๋Š” ํฌ๋ž˜์‹œ๊ฐ€ ๋ฐœ์ƒํ•˜๋Š” GPU ํ”„๋กœ๊ทธ๋žจ์ด ์ฃผ์–ด์ง‘๋‹ˆ๋‹ค. ์†Œ์Šค ์ฝ”๋“œ๋ฅผ ๋ณด์ง€ ์•Š๊ณ  (cuda-gdb) ๋””๋ฒ„๊น… ๋„๊ตฌ๋งŒ์œผ๋กœ ๋ฌธ์ œ๋ฅผ ์ฐพ์•„๋‚ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ๋””๋ฒ„๊น… ์Šคํ‚ฌ์„ ๋ฐœํœ˜ํ•ด ๋ฏธ์Šคํ„ฐ๋ฆฌ๋ฅผ ํ’€์–ด๋ณด์„ธ์š”!

์‚ฌ์ „ ์ค€๋น„: Mojo GPU ๋””๋ฒ„๊น…์˜ ํ•ต์‹ฌ์„ ๋จผ์ € ์™„๋ฃŒํ•ด์„œ CUDA-GDB ์„ค์ •๊ณผ ๊ธฐ๋ณธ ๋””๋ฒ„๊น… ๋ช…๋ น์–ด๋ฅผ ์ตํ˜€๋‘์„ธ์š”. ์•„๋ž˜ ๋ช…๋ น์„ ์‹คํ–‰ํ–ˆ๋Š”์ง€ ํ™•์ธํ•˜์„ธ์š”:

pixi run -e nvidia setup-cuda-gdb

์ด ๋ช…๋ น์€ ์‹œ์Šคํ…œ์˜ CUDA ์„ค์น˜๋ฅผ ์ž๋™์œผ๋กœ ๊ฐ์ง€ํ•˜๊ณ  GPU ๋””๋ฒ„๊น…์— ํ•„์š”ํ•œ ๋งํฌ๋ฅผ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค.

ํ•ต์‹ฌ ๊ฐœ๋…

์ด๋ฒˆ ๋””๋ฒ„๊น… ์ฑŒ๋ฆฐ์ง€์—์„œ ๋ฐฐ์šธ ๋‚ด์šฉ:

  • ์ฒด๊ณ„์ ์ธ ๋””๋ฒ„๊น…: ์˜ค๋ฅ˜ ๋ฉ”์‹œ์ง€๋ฅผ ๋‹จ์„œ ์‚ผ์•„ ๊ทผ๋ณธ ์›์ธ ์ฐพ๊ธฐ
  • ์˜ค๋ฅ˜ ๋ถ„์„: ํฌ๋ž˜์‹œ ๋ฉ”์‹œ์ง€์™€ ์Šคํƒ ์ถ”์ (stack trace) ํ•ด์„ํ•˜๊ธฐ
  • ๊ฐ€์„ค ์ˆ˜๋ฆฝ: ๋ฌธ์ œ์— ๋Œ€ํ•œ ํ•ฉ๋ฆฌ์ ์ธ ์ถ”์ธก ์„ธ์šฐ๊ธฐ
  • ๋””๋ฒ„๊น… ์›Œํฌํ”Œ๋กœ์šฐ: ๋‹จ๊ณ„๋ณ„ ์กฐ์‚ฌ ๊ณผ์ • ์ตํžˆ๊ธฐ

์ฝ”๋“œ ์‹คํ–‰

๋จผ์ € ์ „์ฒด ์ฝ”๋“œ๋ฅผ ๋ณด์ง€ ์•Š๊ณ  ์ปค๋„๋งŒ ์‚ดํŽด๋ด…์‹œ๋‹ค:

fn add_10(
    output: UnsafePointer[Scalar[dtype], MutAnyOrigin],
    a: UnsafePointer[Scalar[dtype], MutAnyOrigin],
):
    i = thread_idx.x
    output[i] = a[i] + 10.0


๋ฒ„๊ทธ๋ฅผ ์ง์ ‘ ๊ฒฝํ—˜ํ•˜๋ ค๋ฉด ํ„ฐ๋ฏธ๋„์—์„œ ๋‹ค์Œ ๋ช…๋ น์„ ์‹คํ–‰ํ•˜์„ธ์š” (pixi ์ „์šฉ):

pixi run -e nvidia p09 --first-case

ํ”„๋กœ๊ทธ๋žจ์ด ํฌ๋ž˜์‹œํ•˜๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์€ ์ถœ๋ ฅ์ด ๋‚˜ํƒ€๋‚ฉ๋‹ˆ๋‹ค:

First Case: Try to identify what's wrong without looking at the code!

stack trace was not collected. Enable stack trace collection with environment variable `MOJO_ENABLE_STACK_TRACE_ON_ERROR`
Unhandled exception caught during execution: At open-source/max/mojo/stdlib/stdlib/gpu/host/device_context.mojo:2082:17: CUDA call failed: CUDA_ERROR_INVALID_IMAGE (device kernel image is invalid)
To get more accurate error information, set MODULAR_DEVICE_CONTEXT_SYNC_MODE=true.
/home/ubuntu/workspace/mojo-gpu-puzzles/.pixi/envs/nvidia/bin/mojo: error: execution exited with a non-zero result: 1

๊ณผ์ œ: ํƒ์ • ์ˆ˜์‚ฌ

๋„์ „: ์ฝ”๋“œ๋ฅผ ๋ณด์ง€ ์•Š์€ ์ƒํƒœ์—์„œ, ์ด ํฌ๋ž˜์‹œ๋ฅผ ์กฐ์‚ฌํ•˜๊ธฐ ์œ„ํ•œ ๋””๋ฒ„๊น… ์ „๋žต์€ ๋ฌด์—‡์ผ๊นŒ์š”?

๋‹ค์Œ ๋ช…๋ น์œผ๋กœ ์‹œ์ž‘ํ•ด ๋ณด์„ธ์š”:

pixi run -e nvidia mojo debug --cuda-gdb --break-on-launch problems/p09/p09.mojo --first-case
ํŒ
  1. ํฌ๋ž˜์‹œ ๋ฉ”์‹œ์ง€๋ฅผ ๊ผผ๊ผผํžˆ ์ฝ๊ธฐ - CUDA_ERROR_ILLEGAL_ADDRESS๋Š” GPU๊ฐ€ ์ž˜๋ชป๋œ ๋ฉ”๋ชจ๋ฆฌ์— ์ ‘๊ทผํ•˜๋ ค ํ–ˆ๋‹ค๋Š” ๋œป์ž…๋‹ˆ๋‹ค
  2. ๋ธŒ๋ ˆ์ดํฌํฌ์ธํŠธ ์ •๋ณด ํ™•์ธ - CUDA-GDB๊ฐ€ ๋ฉˆ์ถœ ๋•Œ ํ‘œ์‹œ๋˜๋Š” ํ•จ์ˆ˜ ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ์‚ดํŽด๋ณด์„ธ์š”
  3. ๋ชจ๋“  ํฌ์ธํ„ฐ๋ฅผ ์ฒด๊ณ„์ ์œผ๋กœ ๊ฒ€์‚ฌ - print๋กœ ๊ฐ ํฌ์ธํ„ฐ ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ํ™•์ธํ•˜์„ธ์š”
  4. ์ˆ˜์ƒํ•œ ์ฃผ์†Œ ์ฐพ๊ธฐ - ์œ ํšจํ•œ GPU ์ฃผ์†Œ๋Š” ๋ณดํ†ต ํฐ 16์ง„์ˆ˜์ž…๋‹ˆ๋‹ค (0x0์€ ๋ฌด์—‡์„ ์˜๋ฏธํ• ๊นŒ์š”?)
  5. ๋ฉ”๋ชจ๋ฆฌ ์ ‘๊ทผ ํ…Œ์ŠคํŠธ - ๊ฐ ํฌ์ธํ„ฐ๋กœ ๋ฐ์ดํ„ฐ์— ์ ‘๊ทผํ•ด์„œ ์–ด๋А ๊ฒƒ์ด ์‹คํŒจํ•˜๋Š”์ง€ ํ™•์ธํ•˜์„ธ์š”
  6. ์ฒด๊ณ„์ ์œผ๋กœ ์ ‘๊ทผ - ํƒ์ •์ฒ˜๋Ÿผ ์ฆ๊ฑฐ๋ฅผ ๋”ฐ๋ผ๊ฐ€๋ฉฐ ์ฆ์ƒ์—์„œ ๊ทผ๋ณธ ์›์ธ๊นŒ์ง€ ์ถ”์ ํ•˜์„ธ์š”
  7. ์œ ํšจํ•œ ํŒจํ„ด๊ณผ ๊ทธ๋ ‡์ง€ ์•Š์€ ํŒจํ„ด ๋น„๊ต - ํ•œ ํฌ์ธํ„ฐ๊ฐ€ ์ž‘๋™ํ•˜๊ณ  ๋‹ค๋ฅธ ๊ฑด ์•ˆ ๋œ๋‹ค๋ฉด, ๋ฌธ์ œ๊ฐ€ ์žˆ๋Š” ์ชฝ์— ์ง‘์ค‘ํ•˜์„ธ์š”
๐Ÿ’ก ์กฐ์‚ฌ ๊ณผ์ •๊ณผ ํ•ด๊ฒฐ์ฑ…

CUDA-GDB๋กœ ๋‹จ๊ณ„๋ณ„ ์กฐ์‚ฌ

๋””๋ฒ„๊ฑฐ ์‹คํ–‰

pixi run -e nvidia mojo debug --cuda-gdb --break-on-launch problems/p09/p09.mojo --first-case

๋ธŒ๋ ˆ์ดํฌํฌ์ธํŠธ ์ •๋ณด ํ™•์ธ

CUDA-GDB๊ฐ€ ๋ฉˆ์ถ”๋ฉด ๋ฐ”๋กœ ์œ ์šฉํ•œ ๋‹จ์„œ๊ฐ€ ๋‚˜ํƒ€๋‚ฉ๋‹ˆ๋‹ค:

(cuda-gdb) run
CUDA thread hit breakpoint, p09_add_10_... (output=0x302000000, a=0x0)
    at /home/ubuntu/workspace/mojo-gpu-puzzles/problems/p09/p09.mojo:31
31          i = thread_idx.x

๐Ÿ” ์ฒซ ๋ฒˆ์งธ ๋‹จ์„œ: ํ•จ์ˆ˜ ์‹œ๊ทธ๋‹ˆ์ฒ˜์— (output=0x302000000, a=0x0)์ด ๋ณด์ž…๋‹ˆ๋‹ค

  • output์€ ์œ ํšจํ•œ GPU ๋ฉ”๋ชจ๋ฆฌ ์ฃผ์†Œ๋ฅผ ๊ฐ€์ง€๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค
  • a๋Š” 0x0 - null ํฌ์ธํ„ฐ์ž…๋‹ˆ๋‹ค!

์ฒด๊ณ„์ ์ธ ๋ณ€์ˆ˜ ๊ฒ€์‚ฌ

(cuda-gdb) next
32          output[i] = a[i] + 10.0
(cuda-gdb) print i
$1 = 0
(cuda-gdb) print output
$2 = (!pop.scalar<f32> * @register) 0x302000000
(cuda-gdb) print a
$3 = (!pop.scalar<f32> * @register) 0x0

์ฆ๊ฑฐ ์ˆ˜์ง‘:

  • โœ… ์Šค๋ ˆ๋“œ ์ธ๋ฑ์Šค i=0์€ ์œ ํšจํ•ฉ๋‹ˆ๋‹ค
  • โœ… ๊ฒฐ๊ณผ ํฌ์ธํ„ฐ 0x302000000์€ ์˜ฌ๋ฐ”๋ฅธ GPU ์ฃผ์†Œ์ž…๋‹ˆ๋‹ค
  • โŒ ์ž…๋ ฅ ํฌ์ธํ„ฐ 0x0์€ null์ž…๋‹ˆ๋‹ค

๋ฌธ์ œ ํ™•์ธ

(cuda-gdb) print a[i]
Cannot access memory at address 0x0

๊ฒฐ์ •์  ์ฆ๊ฑฐ: null ์ฃผ์†Œ์˜ ๋ฉ”๋ชจ๋ฆฌ์— ์ ‘๊ทผํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค - ๋ฐ”๋กœ ์ด๊ฒƒ์ด ํฌ๋ž˜์‹œ์˜ ์›์ธ์ž…๋‹ˆ๋‹ค!

๊ทผ๋ณธ ์›์ธ ๋ถ„์„

๋ฌธ์ œ์ : ์ด์ œ --first-crash์˜ ์ฝ”๋“œ๋ฅผ ๋ณด๋ฉด, ํ˜ธ์ŠคํŠธ ์ฝ”๋“œ๊ฐ€ GPU ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ์ œ๋Œ€๋กœ ํ• ๋‹นํ•˜์ง€ ์•Š๊ณ  null ํฌ์ธํ„ฐ๋ฅผ ๋งŒ๋“ค๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค:

 input_buf = ctx.enqueue_create_buffer[dtype](0)  # 0๊ฐœ์˜ ์š”์†Œ๋ฅผ ๊ฐ€์ง„ `DeviceBuffer`๋ฅผ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค. ์š”์†Œ๊ฐ€ 0๊ฐœ์ด๋ฏ€๋กœ ๋ฉ”๋ชจ๋ฆฌ๊ฐ€ ํ• ๋‹น๋˜์ง€ ์•Š์•„ NULL ํฌ์ธํ„ฐ๊ฐ€ ๋ฉ๋‹ˆ๋‹ค!

์™œ ํฌ๋ž˜์‹œ๊ฐ€ ๋ฐœ์ƒํ•˜๋Š”๊ฐ€:

  1. ctx.enqueue_create_buffer[dtype](0)์€ 0๊ฐœ ์š”์†Œ๋ฅผ ๊ฐ€์ง„ DeviceBuffer๋ฅผ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค.
  2. ํ• ๋‹นํ•  ์š”์†Œ๊ฐ€ ์—†์œผ๋‹ˆ null ํฌ์ธํ„ฐ๋ฅผ ๋ฐ˜ํ™˜ํ•ฉ๋‹ˆ๋‹ค.
  3. ์ด null ํฌ์ธํ„ฐ๊ฐ€ GPU ์ปค๋„๋กœ ์ „๋‹ฌ๋ฉ๋‹ˆ๋‹ค.
  4. ์ปค๋„์ด a[i]์— ์ ‘๊ทผํ•˜๋ ค ํ•  ๋•Œ null์„ ์—ญ์ฐธ์กฐ โ†’ CUDA_ERROR_ILLEGAL_ADDRESS

์ˆ˜์ • ๋ฐฉ๋ฒ•

Null ํฌ์ธํ„ฐ ์ƒ์„ฑ์„ ์ ์ ˆํ•œ ๋ฒ„ํผ ํ• ๋‹น์œผ๋กœ ๊ต์ฒดํ•ฉ๋‹ˆ๋‹ค:

# ์ž˜๋ชป๋œ ๋ฐฉ๋ฒ•: Null ํฌ์ธํ„ฐ ์ƒ์„ฑ
input_buf = ctx.enqueue_create_buffer[dtype](0)

# ์˜ฌ๋ฐ”๋ฅธ ๋ฐฉ๋ฒ•: ์•ˆ์ „ํ•œ ์ฒ˜๋ฆฌ๋ฅผ ์œ„ํ•ด ์‹ค์ œ GPU ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ํ• ๋‹นํ•˜๊ณ  ์ดˆ๊ธฐํ™”
input_buf = ctx.enqueue_create_buffer[dtype](SIZE)
input_buf.enqueue_fill(0)

ํ•ต์‹ฌ ๋””๋ฒ„๊น… ๊ตํ›ˆ

ํŒจํ„ด ์ธ์‹:

  • 0x0 ์ฃผ์†Œ๋Š” ํ•ญ์ƒ null ํฌ์ธํ„ฐ์ž…๋‹ˆ๋‹ค
  • ์œ ํšจํ•œ GPU ์ฃผ์†Œ๋Š” ํฐ 16์ง„์ˆ˜์ž…๋‹ˆ๋‹ค (์˜ˆ: 0x302000000)

๋””๋ฒ„๊น… ์ „๋žต:

  1. ํฌ๋ž˜์‹œ ๋ฉ”์‹œ์ง€ ์ฝ๊ธฐ - ๋Œ€์ฒด๋กœ ๋ฌธ์ œ ์œ ํ˜•์— ๋Œ€ํ•œ ํžŒํŠธ๋ฅผ ์ค๋‹ˆ๋‹ค
  2. ํ•จ์ˆ˜ ํŒŒ๋ผ๋ฏธํ„ฐ ํ™•์ธ - CUDA-GDB๊ฐ€ ๋ธŒ๋ ˆ์ดํฌํฌ์ธํŠธ ์ง„์ž… ์‹œ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค
  3. ๋ชจ๋“  ํฌ์ธํ„ฐ ๊ฒ€์‚ฌ - ์ฃผ์†Œ๋ฅผ ๋น„๊ตํ•ด์„œ null์ด๋‚˜ ์ž˜๋ชป๋œ ๊ฒƒ์„ ์ฐพ์Šต๋‹ˆ๋‹ค
  4. ๋ฉ”๋ชจ๋ฆฌ ์ ‘๊ทผ ํ…Œ์ŠคํŠธ - ์ˆ˜์ƒํ•œ ํฌ์ธํ„ฐ๋ฅผ ์—ญ์ฐธ์กฐํ•ด ๋ด…๋‹ˆ๋‹ค
  5. ํ• ๋‹น ์ง€์ ๊นŒ์ง€ ์ถ”์  - ๋ฌธ์ œ์˜ ํฌ์ธํ„ฐ๊ฐ€ ์–ด๋””์„œ ์ƒ์„ฑ๋˜์—ˆ๋Š”์ง€ ์ฐพ์Šต๋‹ˆ๋‹ค

๐Ÿ’ก ํ•ต์‹ฌ ํ†ต์ฐฐ: ์ด๋Ÿฐ ์œ ํ˜•์˜ null ํฌ์ธํ„ฐ ๋ฒ„๊ทธ๋Š” GPU ํ”„๋กœ๊ทธ๋ž˜๋ฐ์—์„œ ๋งค์šฐ ํ”ํ•ฉ๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์„œ ๋ฐฐ์šด ์ฒด๊ณ„์ ์ธ CUDA-GDB ์กฐ์‚ฌ ๋ฐฉ๋ฒ•์€ ๋‹ค๋ฅธ ๋งŽ์€ GPU ๋ฉ”๋ชจ๋ฆฌ ๋ฌธ์ œ, ๊ฒฝ์Ÿ ์ƒํƒœ, ์ปค๋„ ํฌ๋ž˜์‹œ๋ฅผ ๋””๋ฒ„๊น…ํ•  ๋•Œ๋„ ๊ทธ๋Œ€๋กœ ์ ์šฉ๋ฉ๋‹ˆ๋‹ค.

๋‹ค์Œ ๋‹จ๊ณ„: ํฌ๋ž˜์‹œ์—์„œ ์กฐ์šฉํ•œ ๋ฒ„๊ทธ๋กœ

ํฌ๋ž˜์‹œ ๋””๋ฒ„๊น…์„ ์ตํ˜”์Šต๋‹ˆ๋‹ค! ์ด์ œ ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค:

  • ์˜ค๋ฅ˜ ๋ฉ”์‹œ์ง€๋ฅผ ๋‹จ์„œ๋กœ GPU ํฌ๋ž˜์‹œ๋ฅผ ์ฒด๊ณ„์ ์œผ๋กœ ์กฐ์‚ฌ
  • ํฌ์ธํ„ฐ ์ฃผ์†Œ ๊ฒ€์‚ฌ๋ฅผ ํ†ตํ•ด null ํฌ์ธํ„ฐ ๋ฒ„๊ทธ ์‹๋ณ„
  • ๋ฉ”๋ชจ๋ฆฌ ๊ด€๋ จ ๋””๋ฒ„๊น…์— CUDA-GDB๋ฅผ ํšจ๊ณผ์ ์œผ๋กœ ์‚ฌ์šฉ

๋‹ค์Œ ๋„์ „: ํƒ์ • ์ˆ˜์‚ฌ: ๋‘ ๋ฒˆ์งธ ์‚ฌ๋ก€

๊ทธ๋Ÿฐ๋ฐ ํ”„๋กœ๊ทธ๋žจ์ด ํฌ๋ž˜์‹œํ•˜์ง€ ์•Š๋Š”๋‹ค๋ฉด์š”? ์™„๋ฒฝํ•˜๊ฒŒ ์‹คํ–‰๋˜์ง€๋งŒ ์ž˜๋ชป๋œ ๊ฒฐ๊ณผ๊ฐ€ ๋‚˜์˜จ๋‹ค๋ฉด?

๋‘ ๋ฒˆ์งธ ์‚ฌ๋ก€๋Š” ์ „ํ˜€ ๋‹ค๋ฅธ ์œ ํ˜•์˜ ๋””๋ฒ„๊น… ๋„์ „์ž…๋‹ˆ๋‹ค:

  • ๊ธธ์žก์ด๊ฐ€ ๋˜์–ด์ค„ ํฌ๋ž˜์‹œ ๋ฉ”์‹œ์ง€๊ฐ€ ์—†์Šต๋‹ˆ๋‹ค
  • ์กฐ์‚ฌํ•  ๋šœ๋ ทํ•œ ํฌ์ธํ„ฐ ๋ฌธ์ œ๋„ ์—†์Šต๋‹ˆ๋‹ค
  • ๋ฌธ์ œ๋ฅผ ๊ฐ€๋ฆฌํ‚ค๋Š” ์Šคํƒ ์ถ”์ ๋„ ์—†์Šต๋‹ˆ๋‹ค
  • ์ฒด๊ณ„์ ์ธ ์กฐ์‚ฌ๊ฐ€ ํ•„์š”ํ•œ ์ž˜๋ชป๋œ ๊ฒฐ๊ณผ๋งŒ ์žˆ์Šต๋‹ˆ๋‹ค

์ƒˆ๋กญ๊ฒŒ ์ตํžˆ๊ฒŒ ๋  ์Šคํ‚ฌ:

  • ๋กœ์ง ๋ฒ„๊ทธ ํƒ์ง€ - ํฌ๋ž˜์‹œ ์—†์ด ์•Œ๊ณ ๋ฆฌ์ฆ˜ ์˜ค๋ฅ˜ ์ฐพ๊ธฐ
  • ํŒจํ„ด ๋ถ„์„ - ์ž˜๋ชป๋œ ์ถœ๋ ฅ์—์„œ ๊ทผ๋ณธ ์›์ธ๊นŒ์ง€ ๊ฑฐ์Šฌ๋Ÿฌ ์˜ฌ๋ผ๊ฐ€๊ธฐ
  • ์‹คํ–‰ ํ๋ฆ„ ๋””๋ฒ„๊น… - ์ตœ์ ํ™” ๋•Œ๋ฌธ์— ๋ณ€์ˆ˜ ๊ฒ€์‚ฌ๊ฐ€ ์•ˆ ๋  ๋•Œ ๋Œ€์ฒ˜ํ•˜๊ธฐ

์—ฌ๊ธฐ์„œ ๋ฐฐ์šด ์ฒด๊ณ„์ ์ธ ์กฐ์‚ฌ ๋ฐฉ๋ฒ• - ๋‹จ์„œ ์ฝ๊ธฐ, ๊ฐ€์„ค ์„ธ์šฐ๊ธฐ, ์ฒด๊ณ„์ ์œผ๋กœ ํ…Œ์ŠคํŠธํ•˜๊ธฐ - ์€ ์•ž์œผ๋กœ ๋งˆ์ฃผํ•  ๋” ๋ฏธ๋ฌ˜ํ•œ ๋กœ์ง ์˜ค๋ฅ˜๋ฅผ ๋””๋ฒ„๊น…ํ•˜๋Š” ๊ธฐ์ดˆ๊ฐ€ ๋ฉ๋‹ˆ๋‹ค.