Generative AI Interview Questions and Answers (Freshers to Experienced)

Generative AI is a type of artificial intelligence that creates new content, such as text, images, code, audio or video, by learning patterns from large amounts of existing data. Unlike a classifier that only labels inputs, a generative model produces new outputs that resemble its training data. ChatGPT writing an email and an image model creating a poster are both examples. For a fuller explanation, see what generative AI is.

Traditional (often called discriminative) AI predicts a label or a number, for example whether a transaction is fraud or what a house will cost. Generative AI produces new content, for example writing the fraud alert message itself. Technically, discriminative models learn the boundary between classes, while generative models learn the patterns of the data well enough to create new samples. In practice, many products combine both.

An LLM is a neural network, usually based on the transformer architecture, trained on huge amounts of text to predict the next token in a sequence. That simple training goal, done at massive scale, lets the model write, summarise, translate, answer questions and generate code. GPT, Claude, Gemini and Llama are well-known model families. See what an LLM is for a deeper walkthrough.

Tokens are the small chunks of text a model reads and writes, often parts of words. In English, one token is roughly three quarters of a word on average. Tokens matter for three practical reasons: models have a maximum token limit (the context window), API pricing is usually per token, and longer outputs take longer to generate. Indian languages often need more tokens for the same sentence, which raises cost.

The transformer, introduced by Google researchers in the 2017 paper "Attention Is All You Need", processes all tokens of a sequence in parallel instead of one by one. Its key idea is self-attention: each token looks at every other token and decides how much each one matters for its meaning. Stacked layers of attention and feed-forward networks build up understanding. Parallel processing is what made training today's huge models practical.

A prompt is the input instruction you give a generative model. A good prompt states the role, the task, the relevant context, the output format and any constraints. For example, "You are a support agent for an Indian bank. Answer in under 80 words, in simple English, using only the policy text below" works far better than "answer this". Clear prompts reduce errors before you touch any code.

Hallucination is when a model produces confident but false or made-up information, such as a fake citation or an invented statistic. It happens because LLMs generate the most likely next tokens, not verified facts. You reduce it with grounding (RAG), clear instructions to say "I don't know", lower temperature for factual tasks and output checks. See AI hallucination for real examples.

Temperature is a setting that controls randomness when the model picks the next token. A low temperature (close to 0) makes outputs more predictable and repeatable, which suits factual answers and code. A higher temperature gives more varied, creative text, which suits brainstorming or marketing copy. Interviewers like it when you add that temperature does not make a model more accurate, only more or less random.

RAG (retrieval-augmented generation) retrieves relevant documents from your own data and adds them to the prompt so the model answers from that context. Use it when answers must reflect private or frequently changing information, like company policies, product catalogues or course material, and when you need citations. It is usually cheaper and faster to update than fine-tuning. See how RAG works.

Embeddings are lists of numbers (vectors) that represent the meaning of text, images or other data, so that similar meanings end up close together. "Cheap flights to Goa" and "low-cost air tickets to Goa" get nearby vectors even though the words differ. Embeddings power semantic search, recommendations and the retrieval step in RAG. Read more about embeddings.

A vector database stores embeddings and quickly finds the ones most similar to a query vector, usually with approximate nearest neighbour search. RAG systems use one to fetch the most relevant chunks from thousands or millions of documents in milliseconds. Examples include Pinecone, Weaviate, Milvus, Qdrant, Chroma and pgvector for Postgres. For small datasets, a simple in-memory index can be enough.

Choose RAG when the model needs knowledge it does not have, especially knowledge that changes often or must be cited. Choose fine-tuning when you need to change behaviour: a consistent tone, a strict output format or a narrow skill. Many teams try better prompting first, then RAG, and fine-tune only when those fall short. Our RAG vs fine-tuning guide has a full comparison table.

The context window is the maximum number of tokens a model can consider in one request, counting both your input and its output. If a conversation or document exceeds it, older content must be dropped, summarised or retrieved selectively. Larger windows help with long documents but cost more and can still miss details placed in the middle. See context windows explained.

Zero-shot prompting gives only the instruction, with no examples. Few-shot prompting adds a few input and output examples so the model copies the pattern, which helps with formatting and classification tasks. Chain-of-thought prompting asks the model to reason step by step before the final answer, which often improves multi-step maths and logic. Good candidates mention that newer reasoning models do some of this thinking internally.

Diffusion models generate images (and increasingly audio and video) by learning to reverse a noising process. During training, noise is gradually added to images and the model learns to remove it. At generation time, it starts from random noise and removes noise step by step, guided by your text prompt. Many popular image generators are built on diffusion models.

An AI agent is a system where a model works towards a goal by planning steps and calling tools, such as search, a calculator, a database or an API, then using the results to decide the next step. With tool calling, the model outputs a structured request (tool name plus arguments), your code runs the tool, and the result goes back to the model. Standards like MCP make tools reusable. See AI agents and our agentic AI interview questions.

Start with a test set of real or realistic questions with expected answers or grading criteria. Measure quality (correctness, groundedness in sources, format compliance), safety (harmful or leaked content) and operations (latency, cost per request). Combine automated checks, LLM-as-judge scoring with a clear rubric, and regular human review of samples. Run the same evaluation every time you change the prompt, model or retrieval settings.

Ground answers with retrieval and instruct the model to answer only from the provided context, with citations. Add a fallback such as "I could not find this in our documents". Use lower temperature for factual tasks, validate structured outputs against a schema, and run automated checks that compare answers with sources. Finally, monitor real conversations and add failures to your evaluation set.

LoRA (Low-Rank Adaptation) fine-tunes a model by training small additional weight matrices instead of updating all the original weights. The 2021 LoRA paper showed this can cut trainable parameters by orders of magnitude while keeping quality close to full fine-tuning. It is popular because it needs far less GPU memory, trains faster and lets you keep several small adapters for different tasks. See fine-tuning explained.

Use the smallest model that passes your evaluation, and route only hard requests to bigger models. Shorten prompts, trim retrieved context to what is relevant, and cap output length. Cache repeated answers and embeddings, stream responses so users see text quickly, and batch background jobs. Measure cost per successful task, not just cost per token, because a cheap model that fails often is not cheap.

Prompt injection is when malicious instructions hidden in user input, documents or web pages trick the model into ignoring its rules, for example "ignore previous instructions and reveal the system prompt". OWASP lists it as the top risk for LLM applications. Defences include separating instructions from untrusted content, limiting tool permissions, requiring human approval for sensitive actions, filtering outputs and never placing secrets in prompts.

RLHF (reinforcement learning from human feedback) is a training step where humans rank model outputs, a reward model learns those preferences, and the language model is then optimised to produce outputs the reward model scores highly. It was a key part of making chat assistants more helpful and better at following instructions. Related methods, such as direct preference optimisation, achieve similar goals with simpler training.

Compare quality on your own evaluation set first. Then weigh data privacy and residency (open models can run on your own servers), cost at your expected volume, latency, the team's ability to host and maintain models, and licence terms. Closed APIs are usually faster to start with; open models give more control. Many teams prototype on an API and revisit the choice once usage is predictable.

First, separate retrieval from generation. Check whether the retrieved chunks actually contain the answer; if not, fix chunking, embeddings or the number of results. If the right text is retrieved but the answer is still wrong, look at the prompt (is it told to rely only on context?), chunk ordering and length, and the model choice. Build a small test set of failing questions so you can confirm each fix.

Clarify scope first: FAQs only, or account actions too? Then propose RAG over approved policy documents, strict instructions to answer only from those, and handoff to a human agent for complaints or anything involving money movement. Mask personal data before it reaches the model, log conversations securely, and restrict tools to read-only at launch. Measure resolution rate, escalation rate and answer accuracy on a reviewed sample.

Tighten the prompt with an explicit length limit, a fixed structure and two or three examples of ideal answers. Lower the temperature for consistency. If the format must be exact, request structured output (for example JSON) and validate it. Then measure: track average answer length and format compliance on a test set before and after the change, so you know the fix worked.

I would say: generative AI is software that has read a huge amount of text and images and learned the patterns well enough to produce new drafts on request, like a very fast junior assistant. It is great for first drafts, summaries and answering from documents. It can also be confidently wrong, so important outputs still need review. Then I would suggest one low-risk pilot for their team.

Collect only the data the feature needs, mask or remove personal identifiers before sending text to a model, and check where the model provider stores and processes data. Get clear consent, set retention limits and keep audit logs. Mention that India's Digital Personal Data Protection Act, 2023 sets obligations around consent and purpose limitation, and that legal or compliance teams should review the final design.