Strategizing Your Preparation for Machine Learning Interviews | by Kartik Singhal

Table of Contents

Decoding Job Roles and identify focus areas

Early in my career, I faced multiple rejections during my interview preparations. Each failure taught me valuable lessons which eventually landed me my dream job as Machine Learning (ML) Engineer at Meta. The key to success wasn’t talent or luck, but consistent learning and targeted preparation.

In many articles about ML interviews, you’ll find descriptions of the rounds and typical questions, but I have found that they often fall short in guiding for strategic preparation. This doesn’t mean they are not helpful, but it can sometimes lead to encountering unexpected questions.

Understanding the broad spectrum of ML roles — by job responsibility and specialization — can significantly refine your interview strategy, increase your confidence, and help minimize uncertainties. These distinctions are often not apparent from the job titles alone, so identifying them will give you valuable insights, enabling you to approach your next ML interview with precision. Let’s explore this spectrum:

Image by Author: A sample of job titles across ML role spectrum. Unfortunately, each company defines these titles differently, so reviewing job descriptions is crucial.

ML roles can vary widely based on their primary technical responsibilities and area of specialization.

1) Technical Responsibility:

Data Analysis / Modeling:

Skills: Data analysis, feature engineering, model development and training, statistical analysis, experiment design.

ML Services and Infrastructure:

Skills: Training and Inference services, scalability, model deployment, API integration.

2) Area of Specialization:

Generalist:

Skills: Work on a variety of problem spaces, employ a broad range of ML techniques, and adapt to different requirements of the team.

Specialist:

Skills: Deep expertise in the chosen domain (such as Natural Language Processing (NLP), Computer Vision (CV), or industry-specific areas like self-driving cars and robotics), advanced knowledge of domain-specific tools.

Decoding Job Descriptions

Now that you understand the spectrum of ML roles, you can better identify the true responsibilities of a role from its job description. I once interviewed for a position at a self-driving car company where the focus was heavily on computer vision and sensor fusion. Despite my strong background in general ML algorithms, I was unprepared for specific questions about convolutional neural networks (CNNs) and kernel functions. This experience taught me the importance of understanding the hidden requirements within job descriptions.

Understanding job requirements is crucial for two main reasons:

It helps you eliminate roles that do not fit your goals.
It provides clues about the specific domain the job relates to.

Below are some examples of how I identify keywords in job descriptions to map roles to the ML spectrum.

Image by Author: Example job descriptions extracted from publicly available Google Job Search Tool with highlighted keywords to identify the core ML responsibilities.

Note: that Job descriptions often lack details, so always seek out more information from recruiters if it’s unclear.

Now that you have figured out requirements from the job description, you can start thinking about your preparation strategy. But before that, lets look at the most common interview rounds in ML.

Before diving in preparation strategy for the different roles, let’s quickly understand the 4 different types of ML rounds.

ML Fundamentals/Breadth: This round assesses an engineer’s understanding of ML fundamentals across various topics. It’s often a rapid-fire session where interviewers may jump between different topics or might ask general questions within an area.
ML Case Study / Depth: This round focuses on specialized topics and detailed case studies, from your past projects and/or specific domain knowledge. This round is particularly interesting and the most open-ended of all interview types and generally aimed for above entry-level jobs with some experience.
ML System Design: This round resembles a typical software engineering system design interview, applying similar principles. You are given a product space (e.g., designing a YouTube recommendation ML system) and asked to define the problem, outline the design process, and communicate your thoughts including tradeoffs. Interviewers look for your problem-solving approach, thought process, and high-level design skills.
ML Coding: This round is uncommon but more frequently seen in startups. The strategy is simple: merge your ML coding preparation with the fundamentals in ML breadth, practicing coding for basic models.

You can find more details and resources to prepare for these rounds in this article (article1).

Since you are now familiar with the interview rounds, let’s discuss the steps to develop the preparation strategy tailored for your specific roles.

Ensure you have a solid grasp of the basics (example list of topics) and you can start preparing this even before applying for interviews. This foundation is crucial no matter which ML role or level you’re targeting.

Image by Author: My experience of the most crucial interview rounds based on core ML responsibilities and level. Note that this doesn’t imply that other rounds aren’t important, but it provides an idea on where to focus your preparation.

Identify your goals and job roles where you are planning to apply and prepare specifically for it.

Data/Modeling Roles:

Pay attention to company/job-specific fundamentals

Every company and job is unique, and so are their requirements and the types of questions you might face in interviews. I learned this firsthand when I interviewed for a Junior Machine Learning Engineer position at a real estate company. They asked me about regression trees and the splitting criteria for continuous variables — a topic I hadn’t prepared for since my background was in NLP and classification problems. In hindsight, it was clear that, given the company’s focus on house sales and price predictions, regression problems were common.

Tip: Identify whether the role is generalist or specialist:

— Generalist Roles: Often require knowledge of ML fundamentals and Deep Learning, including multi-layer perceptrons, backpropagation, CNNs, RNNs, and LSTMs.

— Specialist Roles: For example, in NLP, familiarize yourself with techniques like word2vec. In one NLP interview, I was specifically asked about the fundamental theory of word2vec.

Understanding the specific demands of the job and team will help you tailor your preparation and improve your chances of success.

2. Prepare for domain specific knowledge

As roles become more specialized, the focus shifts heavily toward domain-specific knowledge and depth. Note that as you aim for more senior-level positions, the expectation for specialization becomes more pronounced.

Some common areas of specialization to think about:

Ranking/Recommendations: Essential for Search (eg: Google, Amazon) and Discovery (eg: Facebook, Instagram, Netflix). These roles generally offer the most opportunities. I recommend every ML scientist/engineer to understand ranking and recommendation systems, due to number of jobs available.
Ads: Understand ads-specific challenges like calibration and bidding. Ad systems need to balance maximizing revenue with maintaining user experience. Companies like Google and Pinterest invest heavily to optimize for factors like click-through rates and conversion rates, ensuring ads are both effective for advertisers and relevant to users.
Natural Language Processing (NLP): Understand transformers, attention mechanisms, and LLMs. NLP powers features like Google’s search autocomplete, Amazon’s Alexa, and Apple’s Siri. More recently its become highly in-demand due to emergence of ChatGPT and LLMs.
Computer Vision: Understand CNNs, RNNs, LSTMs, Image representation in features, Object detection and Classification. Computer vision is used in various applications from facial recognition to autonomous driving.

3. Research Company Blogs and Papers

Many companies have ML blogs that provide insights into their work, Some popular blogs I follow:

Google AI, Pinterest Engineering, Meta AI, Netflix Research, Amazon Science, AWS ML, Microsoft Research, Snapchat Engineering, Uber Engineering, Doordash Eng blog.
Articles related to the team/domain you’re interviewing for or you are interested in help provide insights into their challenges and potential interview questions. Also discussing these topics can spark valuable conversations with your interviewer.
Tip: Engaging in a meaningful conversation with your interviewer, especially towards the end, can sometimes (but not always) leave a positive impression and compensate for an otherwise moderate interview. It demonstrates your ability to research and understanding of their problem space.

ML Services and Infrastructure Roles:

Pay attention to company/job-specific tech stack

For ML Services and Infrastructure roles, the ML system design round becomes crucial. These interviews typically focus on the tech stacks and systems relevant to the team or company you are interviewing with.

Examples:

Streaming Services (e.g., Netflix): Study video recommendation systems, streaming data processing, and content delivery networks (CDNs).
Search/Recommendations Roles (e.g., Google, Amazon, Doordash, Instagram): Focus on User content feed recommendations and common questions like “Recommend restaurants on a food delivery app” or “Design a user feed”. Tip: Designing a component of recommendation systems is one of the most frequently asked questions in ML System design rounds.
Ads (e.g., Pinterest, Snapchat, Facebook, Youtube): Understand Ad ranking and related challenges like multi-stage ranking, real-time bidding, and user segmentation.

Recommend this Course for overall ML System design preparation: Educative.io’s ML System design

2. Understand domain-specific tradeoffs

Just like traditional software engineering, ML services and infrastructure come with their own set of tradeoffs. Demonstrating your understanding of these tradeoffs during interviews can highlight your clear thinking and problem-solving skills.

Example:

During an interview, I was asked to discuss the pros and cons of real-time inference versus batch inference. These tradeoffs typically fall into two main dimensions:

Latency: Low latency use-cases, such as user recommendation feeds and search, require real-time inference to provide immediate results.
Cost: High-cost use-cases often opt for batch inference to minimize server expenses, as seen in systems like spam detection, image analysis, and financial reporting, which process large amounts of data.

Understanding when to use each approach is crucial. For example, real-time inference is essential for applications needing immediate responses, like recommendation systems and search. In contrast, batch inference is more suitable for tasks that can be processed periodically, such as spam detection or financial analysis, where cost efficiency is a priority.

Tailored preparation is important for ML interviews because it:

Helps you think about your career goals and your area of interests
Addresses specific role and company demands
Helps understand domain-specific nuances
Increases confidence about the problem space and chances of success

By focusing on these areas, you can navigate your next ML interview with confidence.

As you navigate the journey of preparing for ML interviews, it’s essential to track your progress and learnings. Keep a journal or use digital tools to document:

The previous interview questions
The papers/blogs you’ve studied
Key bullet points from your research

Consistent tracking not only helps you stay organized but also boosts your confidence as you see your knowledge and skills grow. It took me time to realize its value, but now I consistently maintain a Google Doc for this purpose.

Remember, ML research advances rapidly, and new breakthroughs can change interview questions so keeping track is key.

Good luck with your interview preparation, and as always keep learning !

link