> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pawa-ai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Top-p (Nucleus Sampling)

> A comprehensive guide to understanding and using top-p (nucleus sampling) in LM requests, including practical examples, best practices, and advanced tips.

# Top-p (Nucleus Sampling)

Top-p, also known as **nucleus sampling**, is a powerful parameter used in language model (LM) requests that controls **the diversity of the generated text**. Along with the temperature parameter, top-p plays a central role in determining how **creative, coherent, or safe** the model’s output will be.

Unlike temperature, which rescales the probability distribution of all tokens, top-p **limits the sampling to the most probable subset of tokens** whose cumulative probability mass reaches a certain threshold `p`. This makes top-p an essential tool for balancing **creativity and control**.

***

### What Top-p Is

Top-p (nucleus sampling) works as follows:

1. The model generates a **probability distribution** over all possible next tokens.
2. Tokens are sorted from **highest to lowest probability**.
3. The model selects the **smallest set of tokens** whose cumulative probability is ≥ `p`.
4. The next token is sampled **only from this set**.

For example:

* Suppose the top token probabilities are `[0.4, 0.3, 0.2, 0.05, 0.05]`.
* If `top_p = 0.8`, the model will only consider `[0.4, 0.3, 0.2]` (cumulative probability = 0.9 ≥ 0.8).
* Tokens outside this subset (0.05, 0.05) are **ignored**, reducing the risk of unlikely or irrelevant outputs.

> Think of top-p as a **dynamic cutoff** — it only considers the "nucleus" of high-probability options.

***

### How Top-p Affects Model Behavior

Top-p influences how deterministic or creative the model output is, in ways that complement the **temperature parameter**:

| Top-p Value    | Behavior               | Example Use Cases                                  | Notes                                                                                                                 |
| -------------- | ---------------------- | -------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| **0.0 – 0.1**  | Extremely conservative | Data extraction, structured tasks                  | Only the very top token(s) are considered. Outputs are highly deterministic, almost identical every time.             |
| **0.2 – 0.5**  | Low diversity          | Summarization, customer support                    | Mostly deterministic but allows slight variation. Prevents repetition and adds mild naturalness.                      |
| **0.6 – 0.8**  | Balanced               | General-purpose text, Q\&A, light creative writing | Includes enough options for variety without producing irrelevant or incoherent content.                               |
| **0.9 – 0.95** | High diversity         | Creative content, brainstorming, storytelling      | Generates varied outputs with some risk of deviation from the prompt.                                                 |
| **>0.95**      | Very high diversity    | Experimental or artistic applications              | Almost all tokens are considered; output may be highly unpredictable. Rarely used in production for structured tasks. |

***

### Comparison: Top-p vs Temperature

| Parameter       | Effect                                                      | Key Difference                                                          |
| --------------- | ----------------------------------------------------------- | ----------------------------------------------------------------------- |
| **Temperature** | Rescales probability distribution globally                  | Controls randomness in selection but considers all tokens               |
| **Top-p**       | Limits sampling to the "nucleus" of high-probability tokens | Provides a **dynamic cutoff**, ignoring low-probability tokens entirely |

> **Tip:** Top-p and temperature can be **combined** for fine-grained control. Example: `temperature=0.7` and `top_p=0.9` → encourages creativity but avoids wild outputs.

***

### Choosing Top-p for Your Use Case

### Low Top-p (0.0 – 0.5)

* Use for tasks that require **reliability and determinism**.
* Examples: filling forms, extracting structured data, generating code, summarizing official documents.

### Medium Top-p (0.6 – 0.8)

* Balanced approach for general content creation.
* Allows **natural variation** without producing off-topic responses.
* Examples: blog summaries, FAQs, instructional text, standard Q\&A.

### High Top-p (0.9 – 0.95)

* For **creative or generative tasks** where variety is desired.
* Examples: poetry, storytelling, marketing copy, idea generation.

### Extreme Top-p (>0.95)

* Experimental scenarios only.
* May produce incoherent or unpredictable outputs.
* Rarely recommended for production.

***

### Practical Examples

### Example 1: Same Prompt, Different Top-p

Prompt: *“Write a tagline for an AI productivity app.”*

* **Top-p = 0.3**
  > “Work smarter, achieve more.”
  * Short, safe, deterministic.

* **Top-p = 0.7**
  > “Boost focus, conquer your tasks, and get more done every day.”
  * Balanced, slightly creative.

* **Top-p = 0.95**
  > “Turn ideas into action, ride the wave of productivity, and make every second count!”
  * Highly creative, risk of slightly over-the-top phrasing.

***