Accept like data: how businesses learn to profit from big data

By analyzing big data, companies learn to uncover hidden patterns, improving their business performance. The direction is fashionable, but not everyone can benefit from big data due to the lack of a culture of working with them

“The more common a person’s name is, the more likely they are to pay on time. The more floors your house has, the more statistically you are a better borrower. The sign of the zodiac has almost no effect on the likelihood of a refund, but the psychotype does significantly, ”says Stanislav Duzhinsky, an analyst at Home Credit Bank, about unexpected patterns in the behavior of borrowers. He does not undertake to explain many of these patterns – they were revealed by artificial intelligence, which processed thousands of customer profiles.

This is the power of big data analytics: by analyzing a huge amount of unstructured data, the program can discover many correlations that the wisest human analyst does not even know about. Any company has a huge amount of unstructured data (big data) – about employees, customers, partners, competitors, which can be used for business benefit: improve the effect of promotions, achieve sales growth, reduce staff turnover, etc.

The first to work with big data were large technology and telecommunications companies, financial institutions and retail, comments Rafail Miftakhov, director of the Deloitte Technology Integration Group, CIS. Now there is interest in such solutions in many industries. What have companies achieved? And does big data analysis always lead to valuable conclusions?

Banks use big data algorithms primarily to improve customer experience and optimize costs, as well as to manage risk and combat fraud. “In recent years, a real revolution has taken place in the field of big data analysis,” says Duzhinsky. “The use of machine learning allows us to predict the probability of loan default much more accurately – delinquency in our bank is only 3,9%.” For comparison, as of January 1, 2019, the share of loans with overdue payments over 90 days on loans issued to individuals was, according to the Central Bank, 5%.

Even microfinance organizations are puzzled by the study of big data. “Providing financial services without analyzing big data today is like doing math without numbers,” says Andrey Ponomarev, CEO of Webbankir, an online lending platform. “We issue money online without seeing either the client or his passport, and unlike traditional lending, we must not only assess the solvency of a person, but also identify his personality.”

Now the company’s database stores information on more than 500 thousand customers. Each new application is analyzed with this data in about 800 parameters. The program takes into account not only gender, age, marital status and credit history, but also the device from which a person entered the platform, how he behaved on the site. For example, it may be alarming that a potential borrower did not use a loan calculator or did not inquire about the terms of a loan. “With the exception of a few stop factors – say, we do not issue loans to persons under 19 years old – none of these parameters in itself is a reason for refusing or agreeing to issue a loan,” Ponomarev explains. It is the combination of factors that matters. In 95% of cases, the decision is made automatically, without the participation of specialists from the underwriting department.

Big data analysis allows us to derive interesting patterns, Ponomarev shares. For example, iPhone users turned out to be more disciplined borrowers than owners of Android devices – the former receive approval of applications 1,7 times more often. “The fact that military personnel do not repay loans almost a quarter less often than the average borrower was not a surprise,” Ponomarev says. “But students are usually not expected to be obligated, but meanwhile, cases of credit defaults are 10% less common than the average for the base.”

The study of big data allows scoring for insurers as well. Established in 2016, IDX is engaged in remote identification and online verification of documents. These services are in demand among freight insurers who are interested in the loss of goods as little as possible. Before insuring the transportation of goods, the insurer, with the consent of the driver, checks for reliability, explains Jan Sloka, commercial director of IDX. Together with a partner – the St. Petersburg company “Risk Control” – IDX has developed a service that allows you to check the identity of the driver, passport data and rights, participation in incidents related to the loss of cargo, etc. After analyzing the database of drivers, the company identified a “risk group”: most often, cargo is lost among drivers aged 30–40 years with a long driving experience, who have often changed jobs recently. It also turned out that the cargo is most often stolen by drivers of cars, the service life of which exceeds eight years.

Retailers have a different task – to identify customers who are ready to make a purchase, and determine the most effective ways to bring them to the site or store. To this end, the programs analyze the profile of customers, data from their personal account, the history of purchases, search queries and the use of bonus points, the contents of electronic baskets that they started filling out and abandoned. Data analytics allows you to segment the entire database and identify groups of potential buyers who may be interested in a particular offer, says Kirill Ivanov, director of the data office of the M.Video-Eldorado group.

For example, the program identifies groups of customers, each of which likes different marketing tools – an interest-free loan, cashback, or a discount promo code. These buyers receive an email newsletter with the corresponding promotion. The probability that a person, having opened the letter, will go to the company’s website, in this case increases significantly, Ivanov notes.

Data analysis also allows you to increase sales of related products and accessories. The system, which has processed the order history of other customers, gives the buyer recommendations on what to buy along with the selected product. Testing of this method of work, according to Ivanov, showed an increase in the number of orders with accessories by 12% and an increase in the turnover of accessories by 15%.

Retailers are not the only ones striving to improve the quality of service and increase sales. Last summer, MegaFon launched a “smart” offer service based on the processing of data from millions of subscribers. Having studied their behavior, artificial intelligence has learned to form personal offers for each client within the tariffs. For example, if the program notes that a person is actively watching video on his device, the service will offer him to expand the amount of mobile traffic. Taking into account the preferences of users, the company provides subscribers with unlimited traffic for their favorite types of Internet leisure – for example, using instant messengers or listening to music on streaming services, chatting on social networks or watching TV shows.

“We analyze the behavior of subscribers and understand how their interests are changing,” explains Vitaly Shcherbakov, director of big data analytics at MegaFon. “For example, this year, AliExpress traffic has grown 1,5 times compared to last year, and in general, the number of visits to online clothing stores is growing: 1,2–2 times, depending on the specific resource.”

Another example of the work of an operator with big data is the MegaFon Poisk platform for searching for missing children and adults. The system analyzes which people could be near the place of the missing person, and sends them information with a photo and signs of the missing person. The operator developed and tested the system together with the Ministry of Internal Affairs and the Lisa Alert organization: within two minutes of orientation to the missing person, more than 2 thousand subscribers receive, which significantly increases the chances of a successful search result.

Big data analysis has also found application in industry. Here it allows you to forecast demand and plan sales. So, in the Cherkizovo group of companies, three years ago, a solution based on SAP BW was implemented, which allows you to store and process all sales information: prices, assortment, product volumes, promotions, distribution channels, says Vladislav Belyaev, CIO of the group ” Cherkizovo. The analysis of the accumulated 2 TB of information not only made it possible to effectively form the assortment and optimize the product portfolio, but also facilitated the work of employees. For example, preparing a daily sales report would require a day’s work of many analysts – two for each product segment. Now this report is prepared by the robot, spending only 30 minutes on all segments.

“In industry, big data works effectively in conjunction with the Internet of things,” says Stanislav Meshkov, CEO of Umbrella IT. “Based on the analysis of data from the sensors that the equipment is equipped with, it is possible to identify deviations in its operation and prevent breakdowns, and predict performance.”

In Severstal, with the help of big data, they are also trying to solve rather non-trivial tasks – for example, to reduce injury rates. In 2019, the company allocated about RUB 1,1 billion for measures to improve labor safety. Severstal expects to reduce the injury rate by 2025% by 50 (compared to 2017). “If a line manager — foreman, site manager, shop manager — noticed that an employee performs certain operations unsafely (does not hold on to the handrails when climbing stairs at the industrial site or does not wear all personal protective equipment), he writes out a special note to him – PAB (from “behavioral security audit”),” says Boris Voskresensky, head of the company’s data analysis department.

After analyzing data on the number of PABs in one of the divisions, the company’s specialists found that safety rules were most often violated by those who had already had several remarks before, as well as by those who were on sick leave or on vacation shortly before the incident. Violations in the first week after returning from vacation or sick leave were twice as high as in the subsequent period: 1 versus 0,55%. But working on the night shift, as it turned out, does not affect the statistics of PABs.

Creating algorithms for processing big data is not the most difficult part of the work, company representatives say. It is much more difficult to understand how these technologies can be applied in the context of each specific business. This is where the Achilles’ heel of company analysts and even external providers lies, which, it would seem, have accumulated expertise in the field of big data.

“I often met big data analysts who were excellent mathematicians, but did not have the necessary understanding of business processes,” says Sergey Kotik, director of development at GoodsForecast. He recalls how two years ago his company had the opportunity to participate in a demand forecasting competition for a federal retail chain. A pilot region was chosen, for all goods and stores of which the participants made forecasts. The forecasts were then compared with actual sales. The first place was taken by one of the Russian Internet giants, known for its expertise in machine learning and data analysis: in its forecasts, it showed a minimal deviation from actual sales.

But when the network began to study his forecasts in more detail, it turned out that from a business point of view, they are absolutely unacceptable. The company introduced a model that produced sales plans with a systematic understatement. The program figured out how to minimize the probability of errors in forecasts: it is safer to underestimate sales, since the maximum error can be 100% (there are no negative sales), but in the direction of overforecasting, it can be arbitrarily large, Kotik explains. In other words, the company presented an ideal mathematical model, which in real conditions would lead to half-empty stores and huge losses from undersales. As a result, another company won the competition, whose calculations could be put into practice.

Big data technologies are relevant for many industries, but their active implementation does not occur everywhere, Meshkov notes. For example, in healthcare there is a problem with data storage: a lot of information has been accumulated and it is regularly updated, but for the most part this data has not yet been digitized. There is also a lot of data in government agencies, but they are not combined into a common cluster. The development of a unified information platform of the National Data Management System (NCMS) is aimed at solving this problem, the expert says.

However, our country is far from the only country where in most organizations important decisions are made on the basis of intuition, and not the analysis of big data. In April last year, Deloitte conducted a survey among more than a thousand leaders of large American companies (with a staff of 500 or more) and found that 63% of those surveyed are familiar with big data technologies, but do not have all the necessary infrastructure to use them. Meanwhile, among the 37% of companies with a high level of analytical maturity, almost half have significantly exceeded business goals in the past 12 months.

The study revealed that in addition to the difficulty of implementing new technical solutions, an important problem in companies is the lack of a culture of working with data. You should not expect good results if the responsibility for decisions made on the basis of big data is assigned only to the analysts of the company, and not to the entire company as a whole. “Now companies are looking for interesting use cases for big data,” says Miftakhov. “At the same time, the implementation of some scenarios requires investments in systems for collecting, processing and quality control of additional data that have not been analyzed before.” Alas, “analytics is not yet a team sport,” the authors of the study admit.