Table of Contents
1. Preface
2. Key Insights
3. Conclusion
4. References
5. Appendix
6. Contributors
Preface
As machine learning (ML) continues to transform how the fastest growing companies conduct business (MIT Technology Review Insights, 2021), early adopters are beginning to reap steeper gains from their efforts. Ubiquitous adoption of machine learning is not only a global trend but a necessity for remaining competitive with industry peers, but many organizations struggle to understand how to get the most from their ML models.
However, the costs associated with adopting new technical infrastructures to leverage machine learning demand that firms are ready to allocate the appropriate resources to support talent tasked with using data to improve operational efficiencies all while reducing costs.
Loxz Digital continues to address these organizational risks by providing you with our machine learning readiness (MLR) score. However, as technology adapts, so does the Loxz Digital Organizational survey. It is our pleasure to announce that the MLR score is now accompanied with a series of diagnostic sub-scores indicative of how you can enhance ML within your firm. By highlighting potential pressure points that are common across industries, we allow you to understand how well your ML models are serving your purposes. Regardless of whether you are currently adopting, integrating, or maintaining machine learning, our sub-scores (see Table 1), give you the information you need to succeed.
| TABLE 1 | |
| Data Preparedness (DP) | Quantifies an organization’s ability to efficiently and effectively locate, integrate, and leverage business resources to achieve its machine learning objectives. |
| Model Development (MD) | Measures the frequency and strategy behind how an organization leverages its resources to construct machine learning models to be as accurate as possible. |
| Deploying Models (DM) | Assesses the infrastructure, scalability, and methodology an organization uses to integrate machine learning models into systems that are in development or already part of their existing technical infrastructure. |
| Model Monitoring (MM) | Provides a basis for understanding the approach an organization takes in leveraging technical resources to maintain, monitor, and retrain machine learning models that are in production. |
| Business Value (BV) | Represents the alignment of strategic initiatives and use of machine learning models to enhance one's business. |
Taken together, results from the Loxz Digital Organizational MLR survey not only provide diagnostic information about your organization’s readiness to undertake machine learning projects, but help you understand your competencies, across any industry.
Responding to the Loxz Digital Organizational assessment allows you to understand the technical maturity of your organization and identifies nuanced elements of ML which you can use to assess where to direct resources and how your organization compares to respondents from around the world. Our scores help you optimize and adjust your machine learning strategy and ensure your desired technical proficiencies are realized.
The Loxz digital survey was designed by domain experts, including the Loxz Digital data science team, to provide immediate insight into your ML strengths and weaknesses. The data source of this report is first-party respondent data from the beta versions of the ML Readiness Survey conducted by Loxz Digital. This report provides insights into the relationships between ML scores and subscores, ML plans and challenges, ML roles, industries and business values.
Key Insights
In addition to providing you with access to the industry leading machine learning readiness (MLR) assessment, the team at Loxz is proud to present insights from our most recent data collection wave.
Companies are readily preparing data for machine learning but struggle developing and deploying models.
Results in Figure 1 suggest even companies presently seeking to adopt machine learning have access to the data they need. However, bifurcation of their MLR maturity occurs when companies who have adopted machine learning are experienced enough to maintain a higher trend relative to their counterparts still looking to adopt ML. Interestingly, while those experienced in ML score higher than those looking to adopt it in terms of their development and deployment, they still struggle. Indeed, what really sets those who have adopted ML apart from those becoming acquainted with the industry, it’s their ability to monitor their models which maximizes business value. Regardless of the industry, companies that provide ML consultation services will want to focus on helping those seeking to adopt ML in developing and appropriately monitoring models.
In addition to providing you with access to the industry leading machine learning readiness (MLR) assessment, the team at Loxz is proud to present insights from our most recent data collection wave.
Industry leaders tend to outshine counterparts across the ML lifecycle, with the largest differences existing for deploying models.
What sets an industry leader apart is their ability to successfully deploy ML models at scale. Indeed, our data can be interpreted to mean that while innovators may even be on par or slightly better than leaders for model monitoring, the gap (Figure 2) is fairly consistent and remains the largest for deploying models. Deploying models requires institutional experience and technical talent uniformly optimizing their approaches toward leveraging resources, it stands to reason that leaders will continue to optimally reap the benefits of their ML models.
In addition to providing you with access to the industry leading machine learning readiness (MLR) assessment, the team at Loxz is proud to present insights from our most recent data collection wave.
The greatest challenge organizations face in adopting machine learning is ensuring their data is of high quality, quantity, and readily accessible.
As noted in Figure 1, the gap between those seeking to adopt ML and those who already have is smallest for data preparedness. As a result, the issues central to data for machine learning are well understood and readily identified across respondents. Although organizations largely know what they need, establishing these data feeds and ensuring that the data is of the appropriate quantity and quality (Figure 3) continue to persist as organizations develop, deploy, and monitor their models.
In addition to providing you with access to the industry leading machine learning readiness (MLR) assessment, the team at Loxz is proud to present insights from our most recent data collection wave.
our most recent data collection wave.
The pressure points differ across industries.
As noted in Figure 5, Media and Entertainment, Transportation and Logistics, as well as the Energy or Utility industry do not perceive data quality, quantity, and timeliness to be the greatest challenge for their industries. Instead, Media and Entertainment organizations indicate they are having the most difficulty building optimal models, while the respondents in Energy and Utility report trouble integrating ML into a scalable, user-friendly solution. This mirrors a wider trend given access to technical resources across these industries. For example, the most successful entertainment companies have been crowdsourcing data for years (TechCrunch, 2016). Similarly, major metros often collect massive amounts of data on transit systems and logistics. However, leveraging their triumphs in collecting data may have resulted in a technical bottleneck for many organizations.
In addition to providing you with access to the industry leading machine learning readiness (MLR) assessment, the team at Loxz is proud to present insights from our most recent data collection wave.
Companies emphasizing neither experimentation and reproducible quality are anchored on data preparedness.
Regardless of emphasis, organizations tend to have a similar proportion of their sub-score distributions across industries. However, organizations that are not focusing on either experimentation, quality, or an even mixture are generally focusing on preparing data. As a result, their business value is lower, despite the fact that model monitoring is fairly consistent across emphasis. Companies that do not understand the benefits of experimentation and who do not have a grasp on achieving reproducibility quality may have their priorities and resources misaligned. Companies who tend to emphasize experimentation tend to have marginally higher scores for model development and deploying models, but this may come at a slight cost of business value as actively experimenting requires the organization to understand how to assess the quality of their research and successfully integrate their findings into their decision making processes. Organizations that are focusing on quality and reproducibility are most similar, in terms of their proportional scores, to those who have achieved an even mixture of experimentation and quality.
In addition to providing you with access to the industry leading machine learning readiness (MLR) assessment, the team at Loxz is proud to present insights from our most recent data collection wave.
Relative to their MLR, industries tend to excel in capturing business value, but exactly where they struggle most varies. As depicted in Figure 6, all companies understood the business value of machine learning, mirroring a trend of widespread adoption of machine learning throughout the world (Loxz Digital, 2021). And while most industries struggle across deploying and developing models, there are core differences in data preparedness and model monitoring. For example, electronics companies are less likely to need assistance in data preparedness, and more likely to struggle with model monitoring, whereas e-commerce and retail experience the opposite problem. As researchers are able to better track adoption and use of ML across industries, insights into how these bottlenecks arise and what industry partners can do will be illuminated.
Conclusion
The machine learning readiness (MLR) and accompanying sub-scores provide a quick and direct assessment of your machine learning maturity throughout the ML lifecycle. While the scores incorporate many dimensions of machine learning (resources, processes, strategy), the Loxz Digital MLR sets the industry standard by providing the most accurate way for you to understand how to take on machine learning at your firm (see our Appendix for additional information).
Throughout this report we provided industry insights which speak to the face validity of our instrument, and provide a basis for understanding key industry trends. Clearly, the current bottleneck across industry presents itself in developing or deploying models. However, despite these overall trends, important differences across industries, especially in terms of their perceived challenges, helps understand what stage of the ML lifecycle that the industry as a whole is currently situated within.
Using the data obtained from the first three quarters, Loxz Digital has developed a revised assessment that is designed to provide more nuanced insights and even more granular diagnostic metrics. We invite you to join us in learning more about this assessment and take a peak at more key insights in December when we release the Q4 report.
Loxz not only differentiates itself by providing the only MLR assessment that diagnoses your pressure points, but by providing you with the recommendations you need to understand how you can capitalize on the full investments of your models. Whether you are just exploring your proficiencies or seeking to scope your competition, the Loxz Digital Organizational MLR provides the insights you need to maintain your advantage!
Loxz Digital Group is a Machine Learning Collective located in Berkeley, CA. Established in December of 2020, our focus is on building and deploying accurate machine learning models with diverse ensemble techniques for enterprise and government entities.
We have partnered with esteemed organizations such as AWS, Splice Machine, and TurboSBIR to help us build machine learning models efficiently and coordinate with government entities as a gateway for the commercialization of our RealTimeML predictive products.
Specifically, realtimeML is at the bedrock of what we do. Collectively, the current assembled team has over 40 years of ML experience, housing 9 data scientists, all located in the United States and Canada. The data acquired from this survey is exclusively first-party data.
References
Guan, D. (2016, November 11). Five industries that should take a cue
from Netflix
and crowdsource parts of its Tech.
TechCrunch. Retrieved October 26, 2021, from https://techcrunch.com/2016/11/10/5-industries-that-should-take-a-cue
Insights, M. I. T. T. R. (2021, October 20). Machine learning in the
cloud is
helping businesses innovate.
MIT Technology Review. Retrieved October 26, 2021, from https://www.technologyreview.com/2021/10/15/1037291/
machine-learning-in-the-cloud-is-helping-businesses-innovate/.
Loxz Digital (2021). Mining ML readiness insights with subscores.
Retrieved October 26, 2021, from http://resources.loxz.com/reports/mining-ml-insight-subscores
Appendix
To better understand how each industry compares in its overall MLR score relative to its sub-score, we are pleased to provide a series of visuals to showcase these gaps.
To better understand how each industry compares in its overall MLR score relative to its sub-score, we are pleased to provide a series of visuals to showcase these gaps.
To better understand how each industry compares in its overall MLR score relative to its sub-score, we are pleased to provide a series of visuals to showcase these gaps.
Contributors
Chen Song
Data Scientist, Lead Author
Justin Chase, Head of User Research
Editor, Coauthor
