AI and ML Interview Questions for Experienced Professionals, With Answers

I use ML only when the pattern is too complex for clear rules, enough good data exists, and some error is acceptable. I first ask what decision the output drives, what a wrong prediction costs, and what a simple rule or heuristic would achieve. If rules get 90% of the value, ship them and collect data. ML adds ongoing costs in monitoring, retraining and explanation, so the gain must justify that maintenance.

Choose based on the input type, the required accuracy, latency, cost per prediction, data privacy and team skills. Structured tabular problems like churn or credit risk usually suit gradient-boosted trees: fast, cheap and explainable. Open-ended language tasks suit an LLM API to start. Fine-tuning a smaller model makes sense when volume is high, the task is narrow and you have quality labelled data. Always compare candidates on the same evaluation set.

I go in order: requirements, data, model, serving, monitoring. Clarify the business metric, latency budget, scale and constraints first. Then define data sources, labels, features and how they are computed consistently in training and serving. Start with a simple baseline, then improve. Decide between batch and real-time inference, plan a safe rollout, and set up monitoring for inputs, predictions and business outcomes, plus a retraining trigger. Close by naming the biggest risks.

Training-serving skew is when the features or preprocessing a model sees in production differ from what it saw in training, so live performance drops without any obvious error. Common causes are separate code paths for offline and online features, time zone or unit differences, and missing values handled differently. Prevent it by sharing one feature transformation code path or a feature store, logging features at serving time, and comparing their distributions with the training data.

Treat data like code: validate it automatically before every training run. I check schema, missing values, duplicates, ranges, label noise and class balance, and compare distributions with previous runs. I also review a random sample manually, because automated checks miss mislabelled examples. For labels from vendors or annotators, measure agreement between labellers. Better data usually improves a model more than tuning, which is why I budget real time for this stage.

Data leakage is when training data includes information that would not be available at prediction time, which inflates offline scores. Classic examples are random splits on time-series data, features computed after the event, and the same customer appearing in train and test. The warning sign is a result that looks too good. I catch it by checking feature timestamps, using time-based or group-based splits, and reviewing the most important features for anything suspicious.

Start from the business outcome and the cost of each error type, then pick the metric that tracks it. For fraud, recall at a fixed false-positive rate may matter more than AUC. For ranking, use NDCG or precision at k rather than accuracy. Validate the link: check whether offline gains actually moved the online metric in past A/B tests. When they disagree, trust the online result and revisit the offline metric.

A mature MLOps setup automates the path from data to a monitored model in production, with reproducibility at each step. That means versioned data and code, automated training pipelines, an experiment tracker, a model registry, automated tests and validation before release, CI/CD for models, safe rollout strategies, monitoring with alerts, and a retraining process. Maturity is less about tools and more about how quickly and safely you can ship and roll back a model.

Data drift is a change in input distributions; concept drift is a change in the relationship between inputs and the target. I monitor input features with statistical measures such as population stability index or KS tests, track prediction distributions, and compare against ground truth once labels arrive, since labels are often delayed. When drift is real and hurts performance, I retrain on recent data, adjust features, or recalibrate thresholds. Not every drift alert needs action.

Roll out gradually and keep a fast rollback. I typically run the new model in shadow mode first, scoring live traffic without affecting users, to compare outputs and latency. Then a canary release to a small slice of traffic, followed by an A/B test on the business metric. Feature flags and a model registry make rollback a configuration change instead of a redeploy. Define success and stop criteria before the rollout starts.

Monitor four layers: system health, data, model quality and business impact. System health covers latency, errors and throughput. Data covers missing values, schema changes and drift. Model quality covers prediction distribution and accuracy once labels arrive. Business impact covers the metric the model was built to move, such as approval rate or conversions. For LLM features, add token cost, refusal rate, user feedback and sampled quality reviews.

LLMOps applies MLOps practices to applications built on large language models, where you often do not train the model yourself. The moving parts shift from training pipelines to prompts, retrieval, model versions from providers, and tool integrations. You version prompts, run evaluation sets on every change, trace multi-step calls, track token cost and latency, and add guardrails for safety. Evaluation is harder because outputs are open-ended text rather than a single label.

Start with a versioned test set of realistic inputs, drawn from real user queries where possible, with expected answers or grading rubrics. Score each run on correctness, groundedness, format and safety using automated checks, LLM-as-judge with a clear rubric, and periodic human review to keep the judge honest. Run the suite in CI whenever prompts, models or retrieval settings change, and add every production failure to the set.

Version everything that can change an output, so any prediction can be traced and reproduced. Code goes in Git; data snapshots or dataset hashes go in a data versioning tool or storage with immutable paths; trained models go in a model registry with metadata about training data and metrics. For LLM apps, prompts and retrieval configs are versioned like code, and each response is logged with the prompt and model version used.

Design it as two pipelines: ingestion and query. Ingestion cleans documents, chunks them sensibly, creates embeddings and stores them with metadata and access permissions in a vector database. The query path often uses hybrid search (keyword plus vector), a reranker, and a prompt that asks for cited answers from the context only. Evaluate retrieval and generation separately, and enforce document permissions at retrieval time.

Use the smallest model that passes your evaluation, and route only hard requests to larger models. Trim prompts and retrieved context, cap output length, and cache repeated responses and embeddings; many providers also offer prompt caching for repeated prefixes. Stream responses to cut perceived latency and batch offline jobs. If self-hosting, quantisation and efficient serving engines help. Track cost per successful task, not cost per token.

Fine-tune when you need to change behaviour, not add knowledge: a consistent style, a strict output format, a narrow classification task, or a smaller model that matches a larger one on a specific job. Use RAG when answers depend on private or changing information. Most teams try prompting first, then RAG, and fine-tune only with good labelled data and an evaluation proving the gain. See RAG vs fine-tuning.

Assume any untrusted text, including user input, retrieved documents and web pages, may contain instructions. Separate system instructions from untrusted content, give tools the least privilege needed, require human approval for sensitive actions, and filter outputs for secrets and personal data. Never place credentials in prompts. OWASP's Top 10 for LLM applications lists prompt injection as a leading risk, so red-team the app regularly with injection attempts.

Let an AI agent act autonomously only when actions are low-risk, reversible and well tested. Anything that moves money, changes customer records, sends external messages or deletes data should need human approval, at least until you have strong evidence of reliability. I set budgets on steps and spend, log every tool call, and widen autonomy gradually as evaluation results justify it.

Responsible AI governance means clear rules for which AI uses are allowed, who approves them, and how they are tested and monitored. In practice: a risk tier for each use case, bias and safety testing before launch, model documentation such as model cards, privacy review, human oversight for high-impact decisions, and an incident process. Frameworks like the NIST AI Risk Management Framework help structure this. In India, align data handling with the DPDP Act, 2023, with legal review.

Choose batch inference when predictions can be computed ahead of time, and real-time when they depend on fresh context at the moment of the request. Nightly product recommendations or churn scores work well in batch: cheaper, simpler and easier to monitor. Fraud checks during a payment or search ranking need real-time inference with strict latency budgets. Many systems mix both, precomputing heavy features in batch and combining them in real time.

Structure it as context, your role, key decisions, a setback and the result. State the business problem and constraints, then what you personally owned versus the team. Pick two decisions and explain the alternatives you rejected and why. Describe one thing that went wrong and how you fixed it. End with measurable impact, such as latency cut, cost saved or a metric moved, and what you would do differently today.

Pick a real failure, own your part, and focus on what changed afterwards. A strong structure: what happened, how it was detected, the root cause, the immediate fix and the lasting process change. For example, a demand forecast that broke during a festival season because training data lacked similar periods, fixed with holiday features and a drift alert. Interviewers want evidence that you learn, not proof that you never fail.

Translate limitations into business terms: what the model gets wrong, how often, and what it costs. Instead of "precision is 0.8", say "about 1 in 5 flagged orders will be genuine customers, so we need a quick review step". Show examples of failures, give a recommended safeguard, and agree on who decides when the model is overridden. Clear expectations early prevent loss of trust later.

First, confirm the drop is real and not a logging or dashboard issue. Then check what changed: model version, features, data pipelines, the UI or traffic mix. Segment the drop by platform, region and user type to localise it. Compare feature distributions before and after release for skew. If the new model is the cause, roll back quickly, then reproduce offline. Finally, add a guardrail metric so the next regression is caught earlier.

Agree on a narrow, valuable scope rather than refusing. I would propose a RAG assistant for the top FAQ categories, answering only from approved documents with citations and handing off to humans for account-specific or sensitive issues. Week one: data and evaluation set. Weeks two and three: build and test. Week four: limited pilot with monitoring. I would state clearly what will not be ready and the risks of skipping evaluation.

Find where the tokens go before cutting anything. Break cost down by feature, endpoint, prompt version and customer, and look for a recent change such as longer prompts, retries in a loop, a new feature or abuse. Quick wins are capping output length, caching, trimming context and routing simple requests to a smaller model. Add per-feature cost dashboards and budget alerts, and re-run evaluation to confirm quality held after each change.

Treat it as a serious issue and investigate before defending the model. Check whether the gap persists after controlling for legitimate risk factors, and whether a feature such as pincode acts as a proxy for a protected attribute. Measure error rates by group, not just approval rates. Fixes may include removing proxy features, reweighting data or adjusting thresholds, with legal and risk teams involved, documented decisions and ongoing fairness monitoring.