![\"Lessons](\"https:\/\/solutionsreview.com\/business-intelligence\/files\/2023\/06\/Getting-the-Most-from-Machine-Learning.jpg\")
<\/p>\n<\/p>\n
Solutions Review\u2019s Premium Content Series is a collection of contributed articles written by industry experts in enterprise software categories. In this feature, DoiT International<\/a> Machine Learning Architect Jared Burns offers a brief on getting the most from machine learning.<\/strong><\/i><\/p>\n In an effort to become more data-driven, companies have been experimenting with machine learning (ML). A subset of artificial intelligence (AI), ML takes in historical data, identifies patterns and outputs new values, enabling applications to quickly predict outcomes without the need for detailed programming.<\/p>\n Unfortunately, many early efforts have failed to fully harness ML due to incomplete plans to action that data, threatening investments and undermining progress. In a survey of corporate executives by NewVantage Partners, a resounding 99 percent of respondents said their firms had active investments in big data and AI. Yet just 39 percent said they are managing data as an asset and only 24.4% claimed to have created a data culture within their firms.<\/p>\n The following approaches to ML can help companies develop more effective data strategies and best use ML to drive actionable data insights that will produce valuable output.<\/p>\n It was just a few years ago that companies began venturing into ML. However, many rushed in too quickly and did not have the foundational aspects of their ML lifecycle set up properly. With the AI hype in full force, the speed with which companies moved led to major mistakes and insufficient planning.<\/p>\n For instance, even when teams had the right skill sets in place, there would often be engineers working separately on the same data tasks. This could corrupt data, return inconsistent conclusions and waste investments. While a number of companies have found some technical success with ML, the business value was not nearly as strong as expected, and many are looking for a more practical approach to ML technology based on long term, sustainable AI infrastructures.<\/p>\n The first steps to any data analysis strategy are to create a plan and build a strong foundation. Teams need to understand what value they\u2019re looking to extract, how much they can invest and the expected timeframe for results. They should also have ample checkpoints built in to ensure they\u2019re getting accurate, valuable results.<\/p>\n The ability for teams to collaborate effectively is key to success. Strategies to foster this should include:<\/p>\n The types of data that companies can extract value from vary, and the ways they process and store the data varies too. These data types can include:<\/p>\n Fortunately, with all the technology and resources that Google and AWS have invested in, it\u2019s easy to dive into these areas. In fact, with just a few clicks, a model can be created to analyze and understand data, making it easy for a company with any level of sophistication to begin to parse.<\/p>\n For customers seeking a more customized approach, tools like Python can help build a bespoke ML system from scratch that better aligns with their workflow. Ideally, a company should be able to blend the two pathways, building custom models while enlisting pre-built models to pressure test algorithms for accuracy.<\/p>\n Above all else, the quality of the data and model is paramount \u2013 high quality input results in high quality output. Frequent testing is vital to operating a strong ML system and extracting real value.<\/p>\n If you don\u2019t have a high performing data engineering team, you\u2019re not going to develop good models. Data needs to be well cataloged and transformed in a way that\u2019s easy for ML teams to adopt and audit. Monitoring data quality that feeds ML algorithms is critical. All of the features that go into a model need to maintain a similar output in the real world versus what was trained. If the model is delivering output that is different from reality, then either it or the data is flawed.<\/p>\n Along with tracking data and features that go into a model, validating models is also an important aspect of training and using it in a production scenario. Validation refers to the process of confirming the model actually achieves its intended purpose. In most situations, this will involve confirmation that it is predictive under the conditions of its planned use. You may have times when models find unintended consequences if not well tested.<\/p>\n An approach has to be balanced between short and long term value. Every time you train a model, you need to think about ROI and if it\u2019s worth investing more resources. The goal is always to operationalize a model that can reteach itself on its own and deliver insights that pay for themselves.<\/p>\n Models can be set up to automatically train and deploy based on changes to incoming data, new model architectures, or on a scheduled cadence. The resulting scalability means you can adopt new use cases or drive something more accurate. However, automation is not a substitute for human oversight. Consistent evaluation is essential to understanding if it\u2019s worth the investment and effort of maintaining. Also, model explainability is critical to understanding whether the model\u2019s predictions are causing bias and introducing unintended consequences, which also needs to be incorporated into the ROI of training a model.<\/p>\n Some data streams are very active and require constant evaluation and maintenance. Others are less urgent and data can be processed at less frequent intervals. Determining frequency is a balance of risk (what are the consequences if the AI is making mistakes?), input speed (how often is new data coming in?), and cost\/value (how much is an AI operation costing each time it runs and how valuable is the output?).<\/p>\n Always keep in mind the cloud-cost element and understand each component of that cost (automated or custom).<\/p>\n It\u2019s important to consider both storage of the data and training time\/budget of the model, as training the model will burn through compute budget just to retrain. Also, if using an automated ML model, remember that it\u2019s easy to \u201cfall asleep at the wheel\u201d and for costs to mount.<\/p>\n If customizing a model, you need to think about data transformations, training and deployment, as each of these have different volumes of spend. Does a particular use case require an online scenario (constant input\/output of data) or a batch scenario (generating predictions nightly\/weekly)? You might eventually even scale up to using Graphics Processing Units (GPUs) and Tensor Processing Units (TPUs), which can get very expensive. Consider enlisting partners that can help pin down ML spending and control operations to ensure you don\u2019t receive any surprise bills.<\/p>\n Blogs from Google and Amazon offer a number of ways to get a quick start. Google, for example, has a quick lab repository of public data to quickly train and deploy a model. Both<\/a> providers<\/a> also have training tools that can help get project leaders up to speed.<\/p>\n Next, it\u2019s important to build a strong team of data engineers experienced in ML. Once this is in place, you\u2019ll be able to lean on this unit to build an intuitive model accessible to the wider team. The time frame for getting an ML program started varies based on scale, type and quality of data. Companies well positioned with a smart plan can get one up and running within a matter of weeks.<\/p>\n Tools are available within both Google and Amazon to help explain data and pinpoint issues. This can enable a company to tweak and adjust a model accordingly. But ultimately, the model will only be as good as its data, which will require ongoing care and validation.<\/p>\n\t\t\t![\"Download](\"https:\/\/solutionsreview.com\/business-intelligence\/files\/2020\/02\/BI_BG_SB_800.gif\")
<\/a><\/p>\n<\/div>\n\t\t<\/div><\/div>\nH0w to Get the Most from Machine Learning<\/strong><\/h2>\n
Build a Strong Foundation<\/strong><\/h4>\n
\n
Make the Most of Your Data<\/strong><\/h3>\n
\n
Strive for High Quality<\/strong><\/h4>\n
Find a Balance Between Short & Long-Term Value<\/strong><\/h4>\n
Pin Down Cloud Costs<\/strong><\/h4>\n
Lean on Resources and a Solid Team to Get Started<\/strong><\/h4>\n
![\"\"](\"https:\/\/solutionsreview.com\/business-intelligence\/files\/2021\/05\/21_BI_Matrix_SB_800.gif\")
<\/a><\/p>\n<\/div>\n\t\t<\/div><\/div>\n","protected":false},"excerpt":{"rendered":"