AI Engineer Interview Questions and Answers for Freshers

An AI engineer builds and ships software that uses AI models, while a data scientist mainly analyses data and builds models to answer business questions. An AI engineer's day involves calling model APIs, building retrieval pipelines, writing backend code, testing outputs and deploying services. A data scientist spends more time on data cleaning, experiments and statistics. An ML engineer sits in between, often training and serving custom models. In small startups, one person may do all three.

Python dominates AI because almost every important library, from NumPy and pandas to PyTorch, scikit-learn and the official LLM SDKs, is built for it first. Heavy computation runs in optimised C, C++ or CUDA underneath, so Python's own speed matters less than people think. The trade-off: for high-throughput services, teams sometimes use Go, Java or Node.js for the surrounding backend.

A list is an ordered, mutable collection, a tuple is an ordered, immutable collection, and a dictionary stores key and value pairs for fast lookup by key. Use a list for items you will append or change, like chat messages in a conversation. Use a tuple for fixed records, like a (latitude, longitude) pair. Tuples can be dictionary keys because they are hashable; lists cannot.

A REST API lets one program talk to another over HTTP using standard methods such as GET to read data and POST to send data. When you call one, your code sends a request with a URL, headers (often including an API key) and usually a JSON body. The server processes it and returns a status code, like 200 for success or 429 for too many requests, plus a JSON response.

Supervised learning trains on labelled examples, where each input has a known correct output, while unsupervised learning finds patterns in data without labels. Predicting whether a loan will default from past labelled loans is supervised. Grouping customers into segments by spending behaviour is unsupervised clustering. LLMs add a third idea, self-supervised learning, where labels come from the data itself, such as predicting the next word. See AI vs ML vs deep learning for the bigger picture.

Overfitting is when a model memorises the training data, including noise, so it performs well in training but poorly on new data. Underfitting is when the model is too simple to capture the real pattern, so it performs poorly everywhere. You spot overfitting when training accuracy is high but validation accuracy is much lower. Fix overfitting with more data, simpler models or regularisation; fix underfitting with better features or a more expressive model.

We split data so we can measure how a model performs on examples it has never seen. The training set teaches the model, the validation set helps tune choices like hyperparameters or prompts, and the test set gives a final, honest score. If you tune on the test set, your reported accuracy becomes too optimistic. The same idea applies to LLM apps: keep a fixed set of evaluation questions that you never use while writing the prompt.

It depends on which mistake costs more. Precision is the share of predicted positives that are correct; recall is the share of actual positives the model finds. For a UPI fraud alert system, missing real fraud is expensive, so recall matters more, even if some genuine payments get flagged. For a spam filter, wrongly hiding an important email hurts, so precision matters more. The F1 score balances both when you need one number.

You install the provider's SDK, load the API key from an environment variable, and send a list of messages with a model name. A minimal example with the OpenAI Python SDK looks like this:

from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY from the environment
response = client.chat.completions.create(
    model="your-model-name",
    messages=[
        {"role": "system", "content": "You are a helpful tutor."},
        {"role": "user", "content": "Explain recursion in two lines."},
    ],
    temperature=0,
)
print(response.choices[0].message.content)

Other providers follow the same pattern. SDKs change often, so mention that you check the official docs for current method names.

Never hard-code API keys in your source code; load them from environment variables or a secrets manager. Locally, keep them in a .env file that is listed in .gitignore. In deployment, set them in the hosting platform's secret settings. If a key ever lands on GitHub, rotate it immediately, because bots scan public repositories for leaked keys.

Structured outputs means asking a model to return data in a fixed format, usually JSON that matches a schema, instead of free text. Your code can then read fields like name, amount and due_date directly. Many providers now offer a JSON or schema mode that constrains the output. You should still validate the result, for example with Pydantic, and retry or fall back if validation fails.

Wrap every call in error handling, set a timeout, and retry transient failures with exponential backoff. A 429 status means you hit a rate limit, so wait and retry with increasing delays plus a little randomness. Server errors in the 5xx range are often worth one or two retries; a 400 error means your request is wrong, so retrying will not help. Log each failure and show users a friendly fallback message.

Embeddings are vectors of numbers that capture the meaning of text, so similar meanings sit close together. Cosine similarity measures the angle between two vectors: values near 1 mean very similar meaning, values near 0 mean unrelated. In practice, you embed all document chunks once, embed each user question, and return the chunks with the highest similarity. "How do I reset my password" and "forgot login credentials" score high despite sharing few words. See what embeddings are.

A basic RAG pipeline has two phases: indexing and answering. Indexing: load documents, split them into chunks, create an embedding for each chunk and store them in a vector database. Answering: embed the user's question, retrieve the most similar chunks, insert them into the prompt with an instruction to answer only from that context, and return the answer with sources. RAG lets a model answer from private or recent data without retraining. Read how RAG works for diagrams.

There is no universal chunk size, so you start with a sensible default and tune it on an evaluation set. Small chunks give precise matches but may cut off the context needed to answer. Large chunks keep context but add noise and cost more tokens. A common starting point is a few hundred tokens with some overlap, split on natural boundaries like headings or paragraphs. Then test retrieval quality on 20 to 50 real questions and adjust.

You control cost by reducing tokens and choosing the smallest model that meets your quality bar. Shorten system prompts, send only the most relevant retrieved chunks, cap the maximum output length and cache answers to repeated questions. Route simple tasks, like classification, to a cheaper model and keep the larger model for hard ones. Remember that Indian languages often use more tokens per sentence than English, which raises cost for multilingual apps.

Daily git work uses status, add, commit, pull, push, branch and switch or checkout. A merge conflict happens when two branches change the same lines of a file and git cannot decide which version to keep. You open the file, choose or combine the changes between the conflict markers, test the code, then commit the resolution. Good habits: small commits, clear messages and no secrets in the repo.

Wrap your logic in a function, define the input with a Pydantic model, and expose it through a route:

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class Question(BaseModel):
    text: str

@app.post("/ask")
def ask(q: Question):
    answer = answer_question(q.text)  # your RAG or LLM logic
    return {"answer": answer}

Run it locally with uvicorn main:app --reload and open the automatic docs at /docs. FastAPI validates the request body for you and returns a clear error when the input is wrong.

Docker packages your code, Python version, libraries and system dependencies into a container that runs the same way on any machine. It removes the classic "works on my laptop" problem, which is common in AI projects with many pinned library versions. A beginner Dockerfile starts from a Python base image, copies requirements.txt, installs dependencies, copies the code and sets the start command. Keep secrets out of the image and pass them as environment variables at runtime.

An image is a read-only template that contains your application and its environment, while a container is a running instance of that image. You build an image once from a Dockerfile, then start one or many containers from it. A simple analogy: the image is the class, the container is the object.

Separate deterministic code from model calls and test each differently. For unit tests, mock the LLM client so tests run fast, cost nothing and return a fixed response; then check that your parsing, validation and error handling work. For quality, keep a small evaluation set of real inputs with expected properties, such as valid JSON or the correct category, and run it whenever you change the prompt or model. Exact string matching rarely works because outputs vary.

Use async endpoints when the work is mostly waiting on I/O, such as network calls to an LLM API or a database. An LLM call can take several seconds; with async and an async client, the server can handle other requests while it waits instead of blocking. If you use a blocking library inside an async function, you lose that benefit and can freeze the event loop. For CPU-heavy work, such as running a local model, use a worker process instead.

Log every request with a request ID, latency, model name, token usage, status and any errors, but never log passwords, API keys or raw personal data. Track a few metrics: error rate, average and slow-percentile latency, daily cost and user feedback such as thumbs up or down. Review a masked sample of real conversations weekly and add failures to your evaluation set.

Use a simple four-part structure: problem, your design, one challenge you solved, and the result with a number. For example: "I built a RAG assistant for our college placement rulebook. I used FastAPI, an embeddings model and Chroma. Answers were wrong for table-heavy pages, so I changed chunking to keep tables intact. Correct answers on my 40-question test set rose from 24 to 33." Be honest about what you built yourself versus what came from a tutorial.

First check the temperature and sampling settings, because a high temperature adds randomness; set it near 0 for factual tasks. Next, check whether retrieval returns different chunks each time, for example because of ties in similarity or documents being re-indexed. Then check whether the prompt or conversation history changes between calls. Even at temperature 0, outputs are not always perfectly identical across runs, so for strict needs, use structured outputs and validation. See AI hallucination for related failure modes.

Clarify the problem and the input limits before writing any code, then talk through a simple solution first. Write clean code with meaningful names, test it on a normal case and an edge case such as an empty list, and then discuss improvements and time complexity. Expect arrays, strings, hashing and practical tasks like parsing JSON. A correct simple solution beats an unfinished clever one.

I would start by asking what decision the tool supports, because fully automatic rejection is risky. A safer design extracts structured fields (skills, experience, education) from each resume into JSON, matches them against clear job criteria, and shows recruiters a ranked list with reasons, while humans make the final call. Remove names, photos, gender and college names from what the model scores to reduce bias. Handle the resumes as personal data, and test the tool on a reviewed sample before launch.

Your app should fail gracefully instead of crashing. Set timeouts and limited retries, then show a clear message such as "The assistant is busy, please try again in a minute". For important features, add a fallback: a backup model from another provider, cached answers for common questions, or a simple keyword search over your documents. Monitoring and alerts should tell you about the outage before your users do.