Contents
What skills and knowledge an ML engineer needs, we understand together with a machine learning specialist at Gazprom-Media Data and the author of the Data Science course modules at SkillFactory Yulia Mochalova
Recommendations on YouTube, Google translate, a chatbot in a banking app — we interact with artificial intelligence almost every day. For the algorithm to be useful for business and the end consumer, it must first be taught to learn. This is what an ML engineer does.
Who is an ML engineer
An ML engineer or Machine Learning Engineer is a specialist who creates and trains algorithms for working with big data. An ML engineer works in the field of Data Science alongside a data scientist, data analyst, and data engineer.
The tasks of an ML engineer are purely practical. It teaches the computer to find relationships in data and make decisions based on them. It is ML engineers who should be thanked for “smart feeds” in social networks, recommendation algorithms on music streaming, which select content for our interests. Translation services like Google Translate and helper bots like Oleg Tinkoff Bank and Alice Yandex are also partly the merit of ML engineers.
What does an ML engineer do
ML engineer trains artificial intelligence to solve business problems using big data. It creates and deploys an ML Model, an algorithm that describes how the computer will learn, what data to use, what commands to execute, and in what order.
To create a model suitable for business tasks, an ML engineer conducts experiments: trains the model on data, gives it tasks, checks how effective it is and meets the results set by the business (ML metrics). For example, if the goal of the algorithm is to recognize plants from a photo, the key metric will be the number of correct answers: when the algorithm accurately recognizes the rubber ficus and distinguishes it from other types of ficus by external features.
When the experiments come to success, the ML engineer deploys the algorithm to the cloud, mobile app, or chatbot on the existing system. The ML engineer is also responsible for the operation of the finished model: he must ensure that the algorithm works, does not fall or break, and adapts to data changes.
Yulia Mochalova, machine learning specialist at Gazprom-Media Data:
“The profession of an ML engineer includes both analytical and engineering activities. In a broad sense, the task of an ML engineer is to create a complete data life cycle from processing features from different sources and building a model, to rolling it out into production and setting up an automated periodic launch. For example, an ML engineer can write a pipeline based on data about the behavior of a network user in order to predict his interest in buying a particular product.
My typical work day consists of tasks to develop new models or improve existing ones. In addition, there are 1-2 meetings with the team per day and a discussion of current problems and successes. I also take the time to learn new tools. The last one studied is a new version of the Apache Spark distributed computing framework.”
Profession trends
ML engineers can find work in companies that extend the functionality of their products by teaching algorithms. Such services are not tied only to machine learning, but they need it for development. For example, Pinterest and YouTube use machine learning to build recommendation systems. This helps them show the user customized content. This approach is common in retail, banking and finance, healthcare, marketing and science.
The second type of companies that need ML engineers cannot exist without machine learning. The main functionality of their product is built on ML. An example of such a service is the Google Translate translator, which is constantly accumulating a database and improving. In this segment, an ML engineer can work on SaaS systems and mobile applications, creating algorithms for robots and autopilot systems.
The growth of data and the need to use it for the benefit of business has led to a sharp surge in demand for machine learning engineers. At the end of 2021, ML engineers took second place in the ranking of specialists who are lacking in the global IT industry. Attractive conditions are always created for in-demand specialists, including salaries. According to research service Glassdoor, the average salary of an ML engineer is $123 per year. This is a little over $10 per month.
In our country, the median salary of specialists is ₽165, juniors earn from ₽80, middles — from ₽200, and seniors — up to ₽330.
Julia Mochalova:
“One of the advantages of the profession: you can work in almost any field. You can apply algorithms and set up automated data analysis processes in any company where there is data and a desire to benefit from it (profit). Today you can work in retail and then go into bioinformatics.”
What Skills Does an ML Engineer Need?
An ML engineer is a specialist who is at the intersection of big data analysis and development. He must write code, develop applications, and hear what the business wants from him. To balance these areas, he needs the following set of skills:
Hard skills
- Mathematical base: understanding of vectors and matrices, derivatives, probability theory and statistics.
- Knowledge of Python. It is on it that models for machine learning are most often written.
- Ability to work with frameworks Tensorflow, PyTorch.
- Algorithmic thinking: understand how different types of algorithms and data structures work. This will help you write your own algorithms.
- Work skills with Flask, Docker, Pip, PyTest. These are services for creating and deploying web applications. They will be needed when ready-made ML models need to be converted into a product for testing, and later for operation.
Soft skills
- Analytical thinking, logic. You can’t do without them when working with algorithms.
- Communication skills. An ML engineer most often works in a team, so it is important to be able to communicate with colleagues. Communication skills will also come in handy when communicating with business: when you need to get a task, understand what metrics you should show to management in the end, present your work.
- Proactivity. In machine learning, it is important not only to create a model, but also to make it efficient: so that it learns as quickly as possible, reaches a new level with minimal resources. Searching for solutions on your own, optimizing the performance of the model is part of the expertise of an ML engineer.
Julia Mochalova:
“An ML engineer often has to learn and solve non-standard tasks in his work. For example, to get data from a new source, this may require certain libraries and tools. For example, I recently studied the features of reading data from Kafka. You may also be faced with the need to implement and apply the unpopular algorithm from the article. In the first case, oddly enough, the ability to google and sometimes the experience of colleagues helps. In the second, a mathematical base comes in handy, allowing you to understand formulas and apply them to data.
The main skill for an ML engineer is the ability to program in one of the popular languages, now it is Python. An excellent addition will be experience in backend development, knowledge of technologies for distributed data processing (MapReduce, Spark / PySpark) and, of course, confident knowledge of SQL. Important soft skills: teamwork, creativity and flexibility in approaching tasks, as they can be different even within the same project.”
How to become an ML engineer
Machine Learning Engineer is a profession that is difficult to learn from scratch. A good engineer needs at least a solid base of mathematics, so most often people come to ML from related specialties: data engineer, data scientist, data analyst.
The set of tools and skills of an ML engineer is very wide, and it takes a lot of time and motivation to master them. You can understand how interesting this work can be for you in free basic courses, for example, on Stepik or Mooc.
You can immerse yourself in working with frameworks on the TensorFlow YouTube channel, and get basic knowledge in higher mathematics for programmers from John Kron (video in English). You can follow current projects, courses and events in professional communities such as ods.ai and Kaggle.
Julia Mochalova:
“In terms of vertical growth, an ML engineer can become a team lead or grow to a product manager. The prospect of horizontal development is a data engineer. If machine learning gets boring, you can always do the organization of data storage and processing in its purest form, working with storages and setting up ETL processes.”