Table of Contents
1. Preface
2. Introduction
3. Survey Summary
4. Survey Questions Summary
5. Survey Data Summary
6. Methodology Design
7. Who is the Survey Intended for
8. Publishing Cadence
9. Purpose of the Survey Results
10. How are the Results and Insights Compiled
11. Conclusion
12. Contributors
Preface
On behalf of Loxz Digital, we are presenting the methodology behind our Survey which will be updated every quarter after the release of our findings.
The survey focuses on whether a company is ready for the implementation of machine learning in their organization and is structured to help you define four major barriers to assist in your efforts.
At the end of the survey, you will receive a readiness score and actionable insights you can immediately utilize to prepare yourself to implement Machine Learning in your organization.
Introduction
Machine learning is transforming and shaping the world, and more and more companies are adopting and adapting the machine learning trend.
Today, machine learning has impressive long-term consequences for various industries, including the software industry, education industry, healthcare industry, banking industry, transportation industry, the marketing industry, retail industry, etc. With the introduction of machine learning technology, both the cognitive ability and the working efficiency of companies are highly improved.
We designed and published the ML Readiness Survey to help companies identify their current ML roles across industries and gain insights into how people react to the machine learning adoption process and efforts.
In our Q2 report, we analyze and evaluate the relationship between industries and ML roles and how different variables affect ML role identification. In the Q3 report, we will dig into how ML helps companies recover from the COVID-19 pandemic and explore the dynamic change of ML role identifications.
Survey Summary
While enterprise adoption of ML is increasing and is demonstrating value to businesses and customers, the disparity in successful ML value capture between ML leaders and non-leaders is significant. The path to ML adoption is challenging and filled with barriers across the pillars of:
Although many ML readiness and adoption surveys have focused on IT, executive and business leader perspectives, we seek to also understand the broader perspectives of individuals in other data-driven roles. This is relevant because we see a growing number of Business Analysts and Data Scientists developing their roles as decision makers on teams. Executives and decision makers, as well as Data, Business, Marketing and other professionals across ML- aware organizations, gain insight into their organizational ML readiness
This survey provides insights about organizational readiness to ML-aware and ML-adopting companies of all sizes across multiple industries, primarily in the United States. Based on the specificity of the questions, we believe this report will become one of the more accurate reports in the industry on ML Survey preparedness.
4 major segments of businesses in terms of ML readiness
OBSERVERS have not yet adopted ML; they may be evaluating ML or may have plans for future adoption which have not yet come to fruition. We recognize Observers as companies that are still in research mode, discovering a wide variety of vendors but no models have been implemented into production. Observers are studying the landscape of MLOps tools, and end-to-end platforms
PERFORMERS have begun to deploy pilot projects or select few production ML solutions. Performers are usually actively identifying opportunities in ML adoption and incorporating ML as part of daily tasks.
INNOVATORS excel at ML adoption or ML tool development and have strong data teams as well as ML infrastructure and organizational frameworks. Innovators may include innovative small to medium-sized businesses which produce ML tools and services but lack large data quantities, or which effectively utilize ML tools to disrupt their industries, as well as large non-technology industry leaders who have undergone transformations to become powerful ML users.
LEADERS are often, but not exclusively technology-focused businesses with exceptional ML, data and engineering talent, which adopt, produce and acquire cutting-edge and industry-leading ML solutions, tools, and knowledge.
Survey Questions Summary
"Open-ended questions and close-ended questions provide insight into the development of ML adoption in different companies and industries and how people react to it."
To ensure conciseness and systematicness, 80% of survey questions are closed-ended questions and 20% of survey questions are open-ended questions. And among all the 6 open-ended, there are 4 short text questions and 2 long text questions.
Specific survey questions provide granular insight of how companies react to the ML adoption.
The survey includes various specific types of questions to get full insight into ML readiness. Out of 43 questions, 7 informative questions are designed to collect basic information about the companies and the survey takers; 9 Sentiment Analysis questions provide sentiment insights into the ML readiness; 8 recommended questions collect opinions about the company's current ML adoption and expected improvement; 16 scoring questions are the questions that will be answered in numeric values.
Survey Data Summary
Adopting ML technology is an upward trend, and more than 60% of the companies
are actively implementing and adopting ML tools."
Results of the survey appearing in Figure 3 show that across the 4 major segments of businesses, ML performers saw the highest proportion of readiness. 44% of respondents indicate their company is actively implementing and adopting ML tools. 17% of respondents lead the ML adoption. trend and are actively encouraging innovations, but only 2% of respondents indicate they are exploring or taking innovative actions during ML adoption. In addition, there are 36% of respondents indicating their company is taking a wait-and-see attitude towards ML adoption.
Among all observers, 88.9% indicate that their company is a start-up company and 11.1% of them indicate their company is an industry leader. As shown in Figure 5, 20% of observers are in the Education industry, 20% of them are in the Media and Entertainment industry, 10% of them are Electronics, Computers or Hardware industry, and another 10% of them are in other industries. The reason only 2% of respondents are confident that they are exploring and taking innovative actions during ML adoption is that machine learning is about models and data, and it is difficult to innovate regarding static data and existing mature models. In addition, small and medium-sized companies prefer investing in advertisement and products.
ML Readiness Performers vary across industries, but leaders are mainly in the
software or digital products industry and the healthcare industry.
Machine Learning is revolutionizing software and digital development and many companies realize that ML can transform the entire software development life cycle, so they are encouraging and leading the development of ML. The COVID-19 pandemic can be one of the triggering aspects encouraging Healthcare and/or Bio-pharmaceuticals companies to adopt ML technologies to infer vital clinical insights. Among survey respondents, Healthcare and/or Bio-pharmaceuticals companies see the second-highest proportion among leaders, accounting for 6.12%. From the survey responses among companies we've identified as leaders, innovators, and performers regarding ML readiness, software or digital product accounts for the highest proportion of adoption. Other industries see a development lag in ML adoption.
Respondents from the software or digital industry show the most positive
attitude
towards ML capabilities and adoption efforts. The most negative attitudes are in
the consulting or professional services industry across industries among all
survey respondents.
As shown in Figure 11, we found that there are 10.87% of all respondents from the software or digital industry indicating they are impressed by the company's ML capabilities and adoption efforts.
Over 20% of respondents from the software or digital industry are very positive about the company's ML capability and efforts.
However, in the consulting or professional service industry, we see the percentage of fair plus negative is almost equal to positive attitudes. 100% of respondents from the education industry indicate that ML capabilities and efforts are just fair
ML is recognized as a vital aspect by industry leaders, innovators and performers.
Now, many industries and companies are adopting and developing machine learning because they realize that machine learning tools enable organizations to more quickly identify profitable opportunities and potential risks. The application of machine learning boosts companies to drive business results. Machine learning is recognized as a very important aspect by industry leaders, performers and innovators. Using a numeric value from 1 as lowest to 5 as highest, 35.3% of industry leaders marked machine learning a score of 5 regarding importance as a differentiator to the company, and about 47.1% of industry leaders marked machine learning a score of 4 regarding importance as a differentiator to the company. 100% of innovators recognize machine learning as a score of 5. 52.6% of ML performers value machine learning a score of 5, but 50% of observers marked machine learning under a score of 3 regarding importance.
More than 60% of startup companies mark ML as a vital differentiator to the company.
Machine learning is not only vital to industry leaders but also very important to startup companies.
There are about 42.9% of startup companies showing that machine learning is a vital differentiator to their companies, marking machine learning a score of 5 and 30.6% of them marked machine learning a score of 4.
Only 8.2% of startup companies marked it under a score of 2. The scale was a numeric value between 1-5.
Leaders and performers have more proportion of "More than 500 data professionals" than other ML roles. Overall, companies that are recognized as leaders have more data professionals than other ML roles.
ML leaders and ML performers show more data professionals in their organizations than innovators and observers do, but ML leaders have a higher proportion of more than 500 data professionals than performers. Among all the respondents, there are 19.05% of respondents who were identified as leaders with more than 500 data professionals, and 19.05% have 100-500 data professionals. There are 19.05% of respondents who were identified as performers who have 100-500 data professionals, but only 4.76% have more than 500 data professionals in their organization.
Compared to leaders and performers, ML innovators and observers have fewer data professionals in their organizations. Among all the respondents, there are only 4.76% of respondents who were identified as innovators having 100-500 data professionals and there are 4.76% of respondents who were identified as observers showing that they have 0 data professionals and none of the observers show that they have more than 500 data professionals in their organizations
Machine learning is playing an important role in businesses and products.
Machine learning helps businesses improve scalability and gain insight from data. As machine learning increasingly plays a vital part in business development, many companies recognize machine learning as a feature of the business or as a core to the product. As shown in Figure 11, 46.66% of leaders recognize machine learning as an interpretable core to the product. Observers usually don't know the details of machine learning, but they still recognize machine learning as a feature of the product. 37.51% of performers mark machine learning as a core of the product and 53.13% regard machine learning as interpretable.
However, 100% of observers, 46.87% of performers, 50% of innovators and 53.34% of leaders mark machine learning as a black box. It is not only because some companies are just implementing fully-developed machine learning technology and they don't fully understand the principles behind it, but also because of the nature of machine learning. In machine learning, interpretable models are created to provide people with a better understanding of how the model works and how predictions were made.
However, there are a lot of black-box models that are generated directly from the data by an algorithm. These black-box models cannot be interpreted or understood by humans. Performers are usually actively encouraging and developing machine learning technologies, and they are willing to try different models and algorithms in their products and businesses. That's the reason why some of the respondents who are performers mark machine learning as a black box, but also a feature of the product.
Methodology Design
"4 broad areas THAT determine an Organizations ML"
The ML Readiness survey was designed to determine an organization’s ML readiness by encompassing information across 4 general areas, namely: organization strategy, data governance, talent and team culture.
Number of Questions N=43
Out of 43 survey questions, 30% of those questions ask about team culture, 28% discuss organizational strategy, 19% discover why data governance is critical and 23% ask about organizational talent surrounding ML. Most questions are not limited to only one particular question type. For example, questions related to team culture are correlated to talent. The pervasiveness of team culture, and data governance and the lack of critical talent pools within data science and MLOps continues to be an area of focus. There seems to be a blur between BI Analysts and Data Scientists as more companies focus on predictive modeling more than historical data sets.
The goal of the survey is to make it as accurate as possible, without the typical general insights you find from other ML Surveys. The survey methodology consists of a literature review, questionnaire creation, pilot/validation, building/selection of tools as detailed below:
- Literature review identification of issues not addressed by other ML Readiness Surveys
- Creation of draft questionnaire & logic
- Identify goals and target demographic/group
- Selection of target respondents
- Strategy for sharing and marketing survey
- Visualization/ dashboard implementation
- Pilot test and validation
- Final draft of survey following pilot
- Develop scoring system/ rubric based on pilot data
- Crafting of Survey Terms & Conditions
- Final decision on Custom-Build including visualizations
- Verification of Survey Data by data cleaning, cross-referencing and other methods (Pending)
- Iteration of Survey Scoring Algorithm, Visualizations, etc. as amount of data collected grows beyond responses from 300-1000 companies
Who Is The Survey Intended For
1. For Survey Creators
With ML adoption rising across many industries, digital service providers have access to increasing opportunities to provide ML services. However, distinguishing and validating opportunities that are a good fit remains challenging without information about the prospective client’s ML plans, needs and capabilities. Therefore, this survey will not only serve as a tool for lead generation and customer/ client engagement, but also as a leading indicator of Machine Learning Readiness for companies who are not sure if they are prepared take on ML in their environment.
2. For Survey Respondents
Although many excellent ML readiness or adoption surveys exist, we see a gap in terms of an ML readiness survey that provides up-to-date, immediate, compelling, and visually exciting feedback to the survey responders. ML Surveys which cater broadly to multiple industries, organization sizes and which seek holistic organizational perspectives beyond those of Business and IT/ Data leaders are also lacking.
3. Target Respondents
This ML Readiness Survey is intended for people playing a broad range of roles within ML- aware and ML-adopting companies across all industries and company sizes, primarily in the United States (US). Data, Business, Marketing professionals & decision-makers across ML-aware organizations will have the opportunity to gain insights into their organizational readiness by completing this Survey. Given that this is an English-language survey being conducted by a US company, we expect that the first survey’s responses will be largely but not exclusively US-based. As international results are collected, we may publish future international survey results.
Publishing Cadence
Purpose Of The Survey Results
This survey fills an important gap by providing rigorous, real-time and visually compelling, data-science ML-enabled insights into the hallmarks of ML readiness in the US and beyond. Importantly, it will address a range of industries, company sizes, and perspectives across the organization.
Identify the current role regarding ML readiness
Obtain sentiments about ML adoption
For each question, there will be a radar chart showed, as shown, to provide a tone score for each question generated by our sentiment analysis tool. For the sentiment analysis tool, we used IBM ToneAnalyzer and called the interface to generate tones and scores to provide insights behind topics. With sentiment scores and tones, you will be able to gain insights into how people react to ML adoption and how they feel about it.
How Are The Results And Insights Complied
Conclusion
Overall, from current survey results, 64% of the respondents show their organizations have adopted, or are actively adopting, ML. Among them, 3% are actively innovating in machine learning technology, and 17% of them are in the leading position in ML development and adoption. While the result shows a favorable trend in ML adoption, innovators are still a small minority
By industry, respondents show significant ML differentiation. And respondents in high-tech are more likely to get a higher ML Score, because according to the survey result, the ML Score is closely related to the number of data professionals.
Future editions will include analysis framework re-development, along with more survey responses, readable charts categorizing industry and business size along with weighted questions, and a more accurate ML readiness score. As additional survey responses and data are propagated, we expect less bias in the weightings and a more accurate big-picture view of the ML climate within each industry.
We see potential development of the healthcare industry in ML adoption, as is the case with startup companies. It seems that ML leaders are unleashing market potential and are a driving force for ML development. Startup companies are actively following the driving force and utilizing their own advantages to remain relevant in a dynamic machine learning market.
About Us
Loxz Digital Group is a Machine Learning Collective located in Berkeley, CA. Established in December of 2020, our focus is on building accurate machine learning models with diverse ensemble techniques for private and government entities.
We have partnered with esteemed organizations such as AWS, Splice Machine, and TurboSBIR to help us build machine learning models efficiently and coordinate with government entities as a gateway for the commercialization of our products and services. Collectively, the current assembled team has over 40 years of ML experience, housing 9 data scientists, all located in the United States and Canada. The data acquired from this survey is exclusively first-party data.
Contributors
Chen Song, Data Scientist
Lead Contributor, Author
Asha Shakeran, Ph.D-Data Science
Survey Methodology Engineering, Question Logic and Designr
Yiming Zhang-Data Science Lead
Feature Engineering, Mapping and Weightings
Abhishek Santra, Ph.D
Survey Methodologyr
Ara Baghdassarian, Full Stack Developer
Data Architecture, Mapping
Yumi Koyanagi, Designer
Survey Designer
Monica Steward
Editor
