NLP Engineer Interview Prep Guide

Self-attention computes pairwise relationships between all tokens in parallel, giving O(1) path length for any dependency versus O(n) for RNNs. Explain the Query, Key, Value computation, scaled dot-product attention, and multi-head attention for capturing different relationship types. Discuss the computational tradeoff: O(n^2) attention complexity versus sequential computation limitations of RNNs. Mention recent efficiency improvements like sparse attention and linear attention.

Discuss the full pipeline: text preprocessing, feature extraction using pre-trained embeddings, multi-label classification model, confidence thresholds for automated routing versus human review, handling new categories over time, and feedback loops for model improvement. Address practical concerns: latency requirements, handling multiple languages, dealing with adversarial or ambiguous inputs, and monitoring for model drift. Compare fine-tuned classifiers versus LLM-based approaches with cost analysis.

Discuss parameter-efficient fine-tuning methods: LoRA, QLoRA, prefix tuning, and adapter layers. Cover learning rate scheduling (small learning rates to preserve pre-trained knowledge), evaluation on both the target task and a hold-out set of general tasks, and data quality requirements for fine-tuning datasets. Compare full fine-tuning versus PEFT approaches in terms of compute cost, storage, and performance. Mention continual learning techniques if the model needs to be updated over time.

Walk through the initial approach and why it fell short, the debugging and error analysis process, the iterations you made (data augmentation, model architecture changes, loss function modifications, evaluation metric adjustments), and the final performance achieved. Show that you approach ML development scientifically: hypothesize, experiment, measure, and iterate. Quantify improvements at each stage.

Discuss the limitations of n-gram overlap metrics: they correlate poorly with human judgment for open-ended generation. Cover modern evaluation approaches: human evaluation protocols, LLM-as-judge frameworks, task-specific metrics (factual consistency, toxicity, coherence), embedding-based similarity scores, and adversarial evaluation for safety. Mention the importance of evaluation dataset design and inter-annotator agreement for human evaluations.

Analyze the retrieval pipeline: check embedding quality for the domain, evaluate chunking strategy (chunk size and overlap), test different retrieval methods (dense vs sparse vs hybrid), examine the reranking stage, and verify the LLM prompt uses retrieved context effectively. Create a retrieval evaluation dataset with known relevant documents. Consider domain-specific embedding fine-tuning, query expansion, and metadata filtering to improve precision. Discuss the tradeoff between recall and precision at different stages.

Encoder-only (BERT): bidirectional context, best for classification, NER, and embedding tasks. Decoder-only (GPT): autoregressive generation, best for text generation and in-context learning. Encoder-decoder (T5): best for sequence-to-sequence tasks like translation and summarization. Discuss why decoder-only models have become dominant for general-purpose AI and how task framing (casting classification as generation) enables this.

Cover optimization techniques: model distillation, quantization (INT8, FP16), pruning, batching strategies, caching for repeated queries, and choosing the right serving infrastructure (TensorRT, ONNX Runtime, vLLM for LLMs). Discuss the latency-quality tradeoff and how you measured the impact of optimization on model quality. Include specific latency numbers and throughput improvements you achieved.