LLM Interview Questions and Answers: Large Language Model Prep for Freshers and Experienced

An LLM generates text one token at a time by predicting a probability distribution over its vocabulary for the next token, picking one, appending it to the input and repeating, a process called autoregressive generation. The model has no plan for the full sentence in advance; each step depends only on the tokens so far. Generation stops at an end-of-sequence token or a length limit. For the bigger picture, see what an LLM is.

A decoder-only transformer is a stack of transformer blocks with causal (masked) self-attention, so each token can only attend to earlier tokens. Encoder-only models like BERT see the whole input both ways and suit classification, while encoder-decoder models like T5 suit translation. Most chat LLMs use decoder-only because one simple next-token objective scales well on huge text corpora and handles every task as text generation. The trade-off: no bidirectional view of the input.

Self-attention lets each token build its representation by mixing information from other tokens, weighted by relevance. Each token is projected into a query (what I am looking for), a key (what I contain) and a value (what I pass on). Scores come from the dot product of a query with every key, scaled and passed through softmax, and the output is the weighted sum of values. Multi-head attention runs several of these in parallel to capture different relationships. More in attention mechanism in LLMs.

Attention on its own treats the input as a set, so without position information "dog bites man" and "man bites dog" look the same. Positional encoding adds word-order information. The original transformer used fixed sinusoidal encodings, some models learn position embeddings, and many recent open models such as the Llama family use rotary position embeddings (RoPE), which encode relative position inside the attention calculation. RoPE also makes it easier to extend context length later.

Tokenization splits text into the units a model reads, usually subwords. BPE starts from characters or bytes and repeatedly merges the most frequent adjacent pair into a new token until the vocabulary reaches a target size. Common words become single tokens while rare words split into pieces, so the model never meets a truly unknown word. The catch: tokenizers trained mostly on English often split Hindi or Tamil into many more tokens, which raises cost and uses up context faster.

It is the number of parameters, the learned weights of the network: 7 billion or 70 billion. Parameter count drives memory needs. In 16-bit precision each parameter takes 2 bytes, so a 7B model needs roughly 14 GB just for weights, before the KV cache and activations. More parameters usually mean better capability but slower, costlier inference.

A base model is the raw result of pretraining: it continues text in the style of its training data, so asking it a question may produce more questions instead of an answer. An instruction-tuned (or chat) model has been further trained on instruction and response examples, often plus preference tuning, so it follows requests and holds a conversation. Base models are useful starting points for your own fine-tuning; instruction models are what you deploy for assistants.

LLMs hallucinate because they are trained to produce plausible next tokens, not verified facts, and nothing in plain next-token prediction rewards saying "I don't know". Rare facts are weakly stored in the weights, so the model fills gaps with fluent guesses. Sampling randomness, missing context and training data errors add to it. Grounding with retrieval, abstention instructions and output checks reduce it but never fully remove it. See AI hallucination.

Pretraining teaches a model language and world knowledge by predicting the next token over a very large text corpus; it is the most expensive stage. Supervised fine-tuning then trains on curated prompt and ideal-response pairs so the model learns to follow instructions. RLHF adds human preference: people rank outputs, a reward model learns those rankings, and the LLM is optimised towards higher reward. In the InstructGPT paper, labellers preferred outputs from a 1.3B tuned model over the 175B GPT-3.

DPO (Direct Preference Optimization) trains the model directly on pairs of preferred and rejected responses using a simple classification-style loss, with no separate reward model and no reinforcement learning loop. It still keeps a frozen reference model so the tuned model does not drift too far. Teams like it because it is simpler and more stable to run than PPO-based RLHF. The trade-off is that it learns only from the fixed preference pairs you give it, so data quality matters even more.

LoRA freezes the base model and trains small low-rank matrices added to selected weight layers, so only a tiny fraction of parameters is updated. QLoRA goes further: it loads the frozen base model in 4-bit precision and trains LoRA adapters on top, which cuts GPU memory sharply. The QLoRA paper showed a 65B model fine-tuned on a single 48 GB GPU. The trade-off is slower training steps and a small possible quality loss from quantization. See fine-tuning explained.

Fine-tune when you need to change how the model behaves, not what it knows: a fixed output format, a domain style, a narrow classification task, or a smaller model that matches a bigger one on one job. Use prompting first because it is cheapest, and RAG when the model needs fresh or private facts. A good sign you need fine-tuning is a prompt that has grown to pages of rules and examples and still fails. For knowledge-heavy work, see our RAG interview questions.

Catastrophic forgetting is when fine-tuning on a narrow task makes the model worse at things it used to do well, such as general reasoning or another language. It happens because updates for the new data overwrite useful weights. You limit it with parameter-efficient methods like LoRA, lower learning rates and fewer epochs, and by mixing some general instruction data into the training set. Always evaluate on a general benchmark before and after, not only on your task.

Greedy decoding always picks the single most likely next token, which is deterministic but can get repetitive. Top-k sampling keeps only the k most likely tokens and samples among them. Top-p (nucleus) sampling keeps the smallest set of tokens whose probabilities add up to p, say 0.9, so the candidate pool grows or shrinks with the model's confidence. Temperature reshapes the distribution before any of these. For code and extraction, use low randomness; for creative writing, allow more.

Beam search keeps the best few partial sequences at each step and finally returns the one with the highest overall probability. It works well for tasks with one correct output, such as translation or speech recognition. For open-ended chat, the highest-probability text tends to be generic and repetitive, and beam search multiplies compute by the beam width. That is why chat systems usually use sampling with temperature and top-p instead.

The KV cache stores the key and value vectors of all previous tokens at every layer, so each new token only computes attention for itself instead of reprocessing the whole sequence. Without it, generation would be far slower. The cost is memory: the cache grows with sequence length, number of layers and batch size, and for long contexts it can exceed the model weights. That is why serving tricks like grouped-query attention and paged memory management focus on shrinking or organising the KV cache.

Quantization stores model weights (and sometimes activations) in fewer bits, for example 8-bit or 4-bit integers instead of 16-bit floats. A 7B model drops from roughly 14 GB to around 4 GB at 4-bit, so it fits on a single consumer GPU and runs cheaper. You can lose some quality, usually small at 8-bit and more noticeable at aggressive 4-bit or lower, especially on maths, code and less common languages. Always re-run your evaluation set after quantizing.

Use an inference server built for LLMs, such as vLLM or similar open-source options, rather than a plain web server around the model. Key techniques are continuous batching (adding new requests to the batch as others finish), paged KV cache memory, quantization and streaming tokens to users. Add autoscaling and rate limits. Measure time to first token and tokens per second separately, because users feel the first and your GPU bill feels the second.

Perplexity measures how surprised a model is by real text: it is the exponential of the average negative log-likelihood per token, and lower is better. It is useful for comparing pretraining runs on the same tokenizer and data. It is not enough because it says nothing about whether answers are correct, safe, helpful or well formatted, and it cannot be compared across different tokenizers. A model can have low perplexity and still give wrong answers to your users.

Public benchmarks such as MMLU or coding tests give a rough ranking, but they can be contaminated by training data and rarely match your task. So I build a custom test set of 50 to 200 real examples with expected answers or a grading rubric. Score with exact checks where possible (JSON validity, correct label), LLM-as-judge for open answers, and human review on a sample. Re-run it on every model or prompt change.

LLM-as-a-judge uses a strong model to grade another model's outputs against a rubric, which scales evaluation far beyond what humans can review. Known pitfalls include position bias (preferring the first answer in a pair), verbosity bias (preferring longer answers) and self-preference (favouring outputs from its own model family). Reduce them by swapping answer order, using specific rubrics with examples, asking for a score with a short reason, and checking the judge against human labels before trusting it.

Standard self-attention cost grows quadratically with sequence length, and the KV cache grows linearly, so long inputs are slow and memory-hungry. Models also need to be trained or adapted for long positions, often by scaling RoPE. Even when a model accepts very long input, research such as the "Lost in the Middle" paper found accuracy drops when key information sits in the middle of the context. So a long context window does not replace good retrieval.

Speculative decoding speeds up generation by letting a small, fast draft model propose several tokens, which the large model then verifies in a single forward pass. Accepted tokens are kept, and at the first rejection the large model supplies its own token and drafting starts again. With the standard acceptance rule, the output distribution matches the large model, so quality stays the same. The speed-up depends on how often the draft model guesses right, which is highest for predictable text such as code.

First check for a distribution gap: are real user inputs longer, messier, in Hinglish, or about topics missing from training data? Next, check for leakage between training and validation sets, which inflates scores. Confirm the production prompt template exactly matches the training format, since a changed system prompt or chat template often breaks fine-tuned models. Finally, sample 100 production failures, label them, and add them to both the training and evaluation sets.

Stream tokens so the first words appear quickly, then work on time to first token: shorter system prompts, prompt caching where the provider supports it, and fewer retrieved chunks. Reduce output length with explicit limits. Route simple queries to a smaller model and keep the large one for hard cases. On self-hosted setups, add quantization, continuous batching and a server region close to your users. Measure p95 latency, not the average, because the slowest users complain first.

Expect higher token counts for Indic scripts with many tokenizers, which raises cost and fills the context window faster. Quality usually drops in lower-resource languages, and users mix scripts, writing Hindi in Roman letters. Evaluation is harder because few public benchmarks match your domain. I would build a small test set per language with native speakers, compare a few models on it, and consider models or tokenizers trained with more Indian-language data.

Treat it as a grounding problem, not a prompting problem. Feed the actual policy text through retrieval, instruct the model to quote numbers only from the provided context, and add an "I could not find this" fallback. Lower the temperature for factual answers. Then add a post-check that compares every number in the answer with the source text and blocks or flags mismatches.

Full fine-tuning will not fit, because weights, gradients and optimizer states together need many times the 14 GB the 16-bit weights alone take. Use QLoRA: load the base model in 4-bit, train only LoRA adapters, and turn on gradient checkpointing to trade compute for activation memory. Keep sequence length and batch size small and use gradient accumulation to reach an effective batch size. Merge or keep adapters separate at the end depending on how you serve the model.