SpinDance

What to Know about MLaaS in 2021

Having worked as a software developer and with software developers for over a decade now, one of the things I have learned to appreciate is just how much developers dislike inefficiency. Anything we can do to automate our jobs and make them faster and easier will inevitably be done. Think back to how much work it was to build and host your own website a decade ago versus now. The manual build and deploy steps have been gradually replaced with automated builds, testing, and deployments across multiple environments with fantastic scalability. 

As a technology moves along the hype cycle into maturity, frameworks, tooling, and methodologies rise and fall until we begin arriving at the things that truly make a technology useful and efficient. Machine learning has seen an explosion of development in the last few years and shows no signs of slowing. Just as in other software development areas, machine learning is beginning to find its stride in the development track, making it much more accessible than ever before, thanks to MLaaS.

What is Machine Learning as a Service (MLaaS)

So, what is machine learning as a service? Simply put, MLaaS is when you use someone else’s tooling and infrastructure to enable machine learning development or deployment, usually at a price. MLaaS is a more specific version of Software as a Service (SaaS). In the olden days, if you wanted network storage, you bought or built a server, put it in a server rack, and attached it to your network. Now you can pay to use someone else’s server and let them handle redundancy, scalability, and maintenance, so you don’t have to. Using these other servers is where much of the efficiency in as-a-service offerings come from; they help customers accelerate solutions. Economies of scale often make this solution faster to set up, easier to maintain, and generally more cost-effective over time. In the same way, if you wanted to do machine learning development a few years ago, you had to jump through a number of hoops. First, you needed to hire a machine learning expert. Second, put $2000+ into a high-end GPU packed Linux box. Third, try to piece together several disparate frameworks and tooling, hoping you didn’t have any conflicting dependencies. Lastly, wrangle your data into some custom format until you could get a model training. While that may still be the right solution in some cases, for many, we have much better options now.

What ML Services Exist

Not surprisingly, the most prominent players in the cloud computing industry are also some of the most prominent players in the MLaaS space. Amazon Web Services, Google Cloud Platform, and Microsoft Azure are updating and releasing new and improved machine learning tooling at a lighting fast pace.

The types of services that these cloud providers offer include things like:

• Virtual machines for training models
• Data storage
• Data versioning
• Data labeling/ground-truthing tools
• Hosting options for models
• Pre-trained models for deployment such as:
  • Models for fraud detection
  • Models for detecting various objects in images
  • Models for doing sentiment analysis on text
  • Recommendation engines
  • Anomaly detection
• Development environments for data scientists and software developers

On top of these general offerings, we see a surge in very specific offerings targeted at use cases in certain industries. One example is Amazon’s Lookout for Equipment. This offering is targeted at the industrial sector, which has generally struggled with adopting machine learning. This industry’s struggle is partly due to the lack of experts available to get companies started and the high cost of entry into the ML space. Specific services like these reduce the need for in-house expertise, lower the barrier to entry, and start at a very low cost. AWS has gone so far with this that they offer devices, such as Monitron, that work with their cloud infrastructure to further reduce these barriers to entry. 

Along with the big names in cloud computing, we see very specialized companies entering this space and providing solutions that were hard to imagine 5 years ago. One great example of this is Edge Impulse. They are focused on bringing machine learning to edge devices, which has traditionally been incredibly difficult and required both a high level of expertise in embedded systems and machine learning. With their services, what used to take weeks of development time can be reduced down to days or even hours. 

With these types of technologies, it is no wonder that companies are further embracing machine learning into the future. A recent article in Forbes highlights some of the significant shifts in the industry and also points to some of the challenges for companies that are ahead. 

With everyone scrambling to get a piece of the pie, it can leave companies with a lot of questions like:

• Should I use a service platform or do the work in-house?
• If I do use a platform, which one is the best?
• Should I use a pre-canned specific solution or do something more general?
• How expensive is all of this?

While we can’t answer every question in this post, let’s look at a few high-level things to consider.

How to Get Started with MLaaS

There are a few high-level trade-offs to be considered.

• Generally, an MLaaS solution improves speed but tends to decrease flexibility in frameworks, versions, or the ability to adapt and tweak models to get the best solution.
• Depending on the amount of training you need to do, sometimes building in-house infrastructure may be a cheaper option.
• While MLaaS solutions tend to improve the speed of getting started, they can also be slower during actual development due to the large amounts of data moving around the cloud.
• Some solutions will promise the world but need to be vetted by someone with some machine learning experience and domain knowledge of your problem. Be wary of silver bullet sales pitches.
• Make sure you are considering the full machine learning process. If the service doesn’t work with the way you collect and store data, that is a problem. The ability to easily deploy a trained model from this service is an important consideration. If you have no way to monitor your model’s performance, that can be a significant issue for your solution’s long-term success.

There are also a few high-level questions to answer to help decide how to move forward. 

How unique is the problem?

Some problems in machine learning are pretty well understood and have solutions to them. If you are just looking to find people in an image, implement fraud detection, or recommend products to a user, you can probably find something off the shelf that will help you accelerate your solution quickly. However, if you work in a unique domain, such as optimizing feeding patterns on your grasshopper farm, you may struggle to find a solution that cleanly fits your needs. The more specific the service offering is, the more closely your problem will need to match it. More general services, such as using Amazon SageMaker to create your model from scratch, will take more time and expertise but will ultimately be more flexible.

Is in-house technical expertise available?

If you have a team of data scientists and developers already working on the problem, their expertise may be able to provide better solutions at a lower cost than trying to move to an MLaaS solution. This is particularly true if the problem you are trying to solve has a lot of nuances or needs a lot of flexibility. Often, an in-house expert will be able to quickly assess a service to know if it will work in your particular environment. If you don’t have this expertise, it will likely be wise to engage a third party to evaluate the right solution.

Is there in-house infrastructure to support it?

If you already have a lot of in-house infrastructures to support data storage, training, and deployment, it is probably worth leveraging that. However, if you want to integrate some of this into other machine learning services, be careful about which tools allow what types of external integration. This can be a major headache, even if a solution claims that it offers 3rd party integration. Many times, these integrations can be cumbersome, buggy, and fragile. 

Where should the model run? (cloud or edge)

This question is going to drive a lot of decisions. Generally, running models in the cloud is easier. However, it comes with a lot of limitations that may or may not be an issue. For instance, if you have a model that inspects the quality of a part on a manufacturing line, you may not have enough time for data to be collected, sent up to the cloud, processed, and sent back while still maintaining your cycle time. If this is a safety-critical application, you can’t depend on a wireless connection all the time to get results. While cloud providers are working hard to make sure their services can move into this domain, it may be better to find tooling specifically targeted to your edge application. 

How sensitive is the data/application?

If you are working with highly sensitive data, then this needs to be a significant consideration on how you choose to work with machine learning. Cloud platforms are becoming more and more secure and providing better options for end-to-end security than ever before. However, anytime data moves from one location to another, there is always increased risk. Each service being considered should be carefully scrutinized to know if or how it should be used in your scenario.

Will the problem statement change significantly in the future roadmap?

Rarely do you train up a model that will work perfectly from now into eternity. Inputs change. Business problems change. Customer’s needs change. Tying yourself to a very specific machine learning service might work great today but could be a hindrance down the road. Although we can’t predict the future, having a good roadmap of where you think your product or problem is going can help you make informed decisions today. 

Once you’ve answered some of these questions, you are ready to explore the options that are out there. Keep in mind that there will be trade-offs with any service that you use. Understanding what you are gaining and what you are losing is key to finding the right solution.

Set Yourself Up for Machine Learning Success

• Adopt a fail-fast methodology. What is the bare minimum you can try to see if a particular service will fit your need? Experiment quickly and move on quickly if things aren’t going in the right direction.
• Take advantage of free tier offerings, trials, and demos. Most machine learning services providers want you to buy their products and try to make the barrier to entry lower through low to no-cost trial periods. Try it out. If you don’t like it, try something different.
• Never trust a machine learning sales pitch. Machine learning can often be a black box that feels like magic. It can often be too easy for a sales demo to cherry-pick the right data to make their service look even more magical. Whenever you can, try the product for yourself and look for successful real-world use cases.
• Think about your problem holistically. If you are using multiple models, make sure the service you choose will support all of them. If you need other services like monitoring, data storage, or an API to hit a machine learning endpoint, it is probably better to choose a platform that provides all of these things, so you don’t have to learn more technologies and maintain different accounts.
• Try to understand what you don’t know and don’t be afraid to ask for help. If you don’t know how something works, it is better to understand it sooner rather than later. A few hours with an expert consultant could save you thousands or hundreds of thousands of dollars down the road. Know your limits and approach problems humbly.

Learn from Experience

Based on the trends over the last few years and the projections moving forward, I suspect many more machine learning services will hit the market in 2021 and beyond. Some will last. Some will fail. Navigating through all of them can be a big undertaking. Finding the right solution could be just what your business needs to get to market sooner or the golden ticket that sets you apart from the competition. There are risks, but the market is showing that there is also great reward. Picking the right service or set of services will start you off on the right foot and offer much greater efficiency than trying to do it yourself. 

If you start on this journey and feel over-your-head or would just like a set of expert eyes on your problem, SpinDance is here to help. We offer assessment, road mapping, general consulting, and development services for Machine Learning and IoT.  We can take your vision and build out your machine learning model, pipelines, and deployment strategies. You can leverage our years of experience to be most efficient and cost-effective. Contact SpinDance for a technical discovery session. 

About the Author

Ben Jacques, Engineering Manager at SpinDance, has over 10 years of experience in software development, with experience in backend development, user interface design, computer vision, system integration, and machine learning. When he’s not managing developers or testing out the latest machine learning algorithms, he enjoys spending time with family, working on home renovations, and spending as much time outdoors as possible. Connect with Ben