MMS • RSS
Article originally posted on Data Science Central. Visit Data Science Central
The future of artificial intelligence is brimming with exciting possibilities where data science, knowledgeable teams, and advanced tools to work together to push the ever-expanding limits of technology. But the path that leads data to a successful AI project is not a straight line.
Here’s a fun fact for you: Gartner estimates that 85% of Big Data projects fail. Technology giants like Microsoft have learned the hard way when their innocent chatbot unleashed on Twitter. Like most things in life, AI is hard to get but easy to mess up.
This does not mean that you are doomed to find yourself as Microsoft and end your beloved AI after months of hard work. To give you a hand, here are seven basic tips to consider when building an AI that can revolutionize your organization.
1. Clearly define the purpose of the AI project
If you can not summarize the ultimate goal of your AI in one sentence, it is not clear enough. Identifying your target customers and defining what makes your AI unique are key questions that will guide your approach and increase your chances of success. Here are some tips if you start with your own AI.
Understand your consumers
Here’s where you ask who benefits from your AI solution? What problems can you solve for them? Consider defining key use cases with a group of actual representatives of your target audience to get specific information about their needs. If there is no real need, there will be no adoption and no return on investment.
Measure your capabilities
This is where you really define what your solution entails and what you need to achieve it (data, knowledge, technologies, etc.). This will give you a clear idea of how the requirements meet your capabilities and your technology.
Evaluate your competition
The ultimate goal of your solution is to be a better alternative to what already exists. This means that your artificial intelligence project must be a step forward from existing solutions. So, what makes your project special?
Define the required quality
What is the quality of your AI for it to be useful? It’s time to define the level of precision your customers need and the steps to take to achieve it. You should also think about the results matrix to get quality results so that you can optimize your optimizations around this matrix.
2. Follow a proven methodology
AI is not something you want to improvise along the way. Following a proven methodology will ensure the reliability and success of your data science project.
The most common methods are SEMMA and CRISP-DM. We save you the Google search and give you a quick overview.
SEMMA stands for Sample, Explore, Modify, Model and Assess. It is an iterative process of data mining using rigorous modeling techniques. Although it is considered the standard methodology, it focuses on procedures rather than results and leaves out the commercial aspects. This is where CRISP-DM comes in.
CRISP-DM stands for “Standard Data Crossover Process for Data Mining”. Unlike SEMMA, this methodology includes a “business-understanding” phase that focuses on goals from the business perspective in relation to definitions of data mining.
3. Find data from a trusted source
There is no alternative, to create an AI, machine learning algorithms need data. Before going any further, you need to define how much data you need and how you want it.
Data Scientists have some options when creating training sets to power their algorithms: they can buy datasets, use artificial data, or turn to intelligent outsourcing solutions. The last option essentially acts as an extension of your internal resources.
It is, of course, possible to annotate the training data yourself, but not everyone has time for it.
4. Choose your algorithms for machine learning
Now, for the big question: what machine learning algorithm should you use? According to the Microsoft guide on the choice of algorithms, it depends on your project. Here are some considerations to help you reduce it:
- Accuracy of results
- Training time
- Use of linearity
- Number of parameters
- Number of features
The algorithms are not lacking, but you will, of course, choose the one that best suits your project. As you may already know, the majority of practical learning uses supervised learning. Some popular examples of supervised machine learning algorithms include linear regression for regression problems and support vector machines for classification problems.
However, if you do not plan on the desired results, you will want to use unsupervised learning. The most common examples of these algorithms include the k-means for clustering problems and the apriori algorithm for association rules learning problems.
For computer vision algorithms, artificial neural networks, such as the Convolutional Neural Network (CNN), are better suited for tagging, annotating, and segmenting images. Whereas the recursive neural network (RNN) is the best for language analysis. Finally, Multi-Layer Perceptron (MLP) is ideal for voice recognition and machine translation.
5. Design and build your infrastructure
Building an AI infrastructure is a strategic decision where you have to consider things like data storage, computing resources, budget, and time. A useful tutorial series by Intel explains the infrastructure you can choose:
Internal equipment (on-prem)
Building and maintaining your own IT infrastructure in-house requires much more effort, but it also gives you more freedom. With the on-site infrastructure, you can choose which GPU to use. There are predefined DL servers such as DGX Systems from Nvidia or you can have a custom workstation built with companies like Lambda Labs and AMAX. Another option is to create a DL workstation from scratch.
A cloud provider (like AWS, GCP, and Microsoft Azure) makes sense when you’re new to the business. You can get your first training model on a high-performance GPU for a lower initial investment than on-site, with the added benefit of up-to-date technology and uninterrupted maintenance. You can also use ML-specific vendors (such as Paperspace) that adapt their infrastructure offerings to better support deep learning workflows.
As with everything on this list, you need to answer some questions before selecting an infrastructure that will properly support your AI projects. For example, what is the size of your dataset? Do you have a team that can devote their time to maintaining the systems on site? Do you train a model from scratch or do you use a preset template? Answer these questions now so you do not have to deal with switching infrastructures later.
6. Test and validate your model
The AI must be trained before it can be useful. This means that you must run your AI application through a training dataset to create a model and rerun it on a new set to test the accuracy of the results.
This sounds simple in theory, but there are dangers such as data bias that causes poor functionality (and bad press). You may have seen the media storm surrounding the biased facial recognition software or the racial failure of the beauty contest but, Beauty.AI.
You will get a strong indication that something is wrong with your model if it miraculously generates a 100% accuracy. Over-fit is a classic AI challenge where your app stores training data and work poorly on real-world data. On the other hand, if you get disappointing results that do not model the training data or generalize the new data, you look at a case of under-equipment. It never ends.
In all honesty, training may take longer than actual development, but it may be the most important step in your AI strategy. A trained and tested model is a useful model.
7. Constantly monitor and retrain your model
Once you have a model that has finally been formed and validated, it can be tempting to lean back and call it a day. But the reality of your model monitors is dynamic, which means that your model should be too.
Originally Published @ https://www.techtiq.co.uk