[Conference Notes] NV TW LLM Developer Day 2024

title: [Conference心得] NV TW LLM Developer Day 2024
published: false
date: 2024-01-11 00:00:00 UTC
tags: 
canonical_url: https://www.evanlin.com/conf-nv-llm-devday/
---

![image-20240111122733576](https://www.evanlin.com/images/2022/image-20240111122733576.png)

# Event Information

[NVIDIA LLM Developer Day](https://www.nvidia.com/zh-tw/events/llm-developer-day/) ([Playback](https://www.nvidia.com/en-us/on-demand/))

- 09:30 - 11:00 Taiwan Time - Session 1: Fast Track to Developing Large Language Models (LLMs): [slide](https://on24static.akamaized.net/event/44/33/88/7/rt/1/documents/resourceList1704771459472/llmdevdaysession1taiwanwilliam1704771459472.pdf)

- 11:10 - 12:50 Taiwan Time - Session 2a: Tailoring Your Own Large Language Model Applications: [slides](https://on24static.akamaized.net/event/44/33/90/1/rt/1/documents/resourceList1704769052626/llmdevdaysession2taiwan11704769052626.pdf)

- 15:30 Taiwan Time - Session 2b: Large Language Models and Generative AI in Biosciences

- 17:10 Taiwan Time - Session 3: Running Your Own Large Language Model

# Session 1: Fast Track to Developing Large Language Models (LLMs)

## Case (1) - Customer Service System for a Music Company

![image-20240111095051777](https://www.evanlin.com/images/2022/image-20240111095051777.png) ![image-20240111095054776](https://www.evanlin.com/images/2022/image-20240111095054776.png)

- The first case: Using LLM for the customer service system of a musical instrument company.
- Receive customer emails, customer tags
- Find possible RAG responses based on the questions. (Displayed below), to help customer service respond.

**Thoughts:**

- It's a bit strange to use OpenAI instead of LLAMA for NV DevDay.
- The OpenAI version inside is < 1.0, but the Openai package was already updated to 1.0 on 11/07. (Because I was caught off guard at the time)

### 1.1 Interpreting the email prompt

![image-20240111095509845](https://www.evanlin.com/images/2022/image-20240111095509845.png)

- List the good and bad points in separate items.
- Categorize urgency.
- Determine the tone of the email
- Structured output

### 1.2 Analysis Prompt

![image-20240111095935738](https://www.evanlin.com/images/2022/image-20240111095935738.png)

- Data processing prompt

![image-20240111100529570](https://www.evanlin.com/images/2022/image-20240111100529570.png)

- Formatting output prompt

![image-20240111100620053](https://www.evanlin.com/images/2022/image-20240111100620053.png)

- Define user data flow through [COLANG](https://www.colanginstitute.org/).

### Unreleased product Nemotron-3

![image-20240111102836188](https://www.evanlin.com/images/2022/image-20240111102836188.png)

- NeVA (NeMo Vision and Language Assistant)

#### Other:

GTC 2024 - Physical GTC 03/18.

# More References:

- [COLANG](https://www.colanginstitute.org/)