Why is my training score so low?

Understanding and troubleshooting the performance of your AI Block

AI Blocks in Levity receive a Performance Score, also known as the training score. In a nutshell, this number represents how well your AI model will perform when classifying real data. The higher the number (up to 100), the more likely it is to get the predictions correct.

If this number seems low, there are a few reasons this might happen, and if you want to improve it, there are simple steps you can take to do this.

You aren't using enough training data

Using the minimum amounts of training data is one of the potential causes of a low training score. Adding more data and clicking Retrain is one way to give your model a boost in knowledge, and a potential improvement in score. 

You aren't using good quality data

Depending on your model's use, you might be able to improve your score by using better vetted training data. For example, if you are using Levity for visual quality inspection and checking for defects in images of products going through your production line, using clearer photographs with a higher resolution may improve the performance. 

You aren't using balanced data

An imbalanced model will negatively affect your training score. For example, if you are training an AI model to categorize inbox content and email attachments, and you use 1000 pieces of training data to illustrate resume attachments, but only 50 pieces of data for invoices, this would be an imbalanced dataset. Ideally, you would want to have similar amounts of data for the different 'labels' you were training the model on.

You need to simplify your model

If you are unable to increase the volume of training data acquired, another approach is to reduce the number of outcomes the model is looking for. A human doesn't need much information to know the difference between a sports car and a minibus, but if you ask them to define various models of similar looking sports cars, the amount of reference materials needed increases. AI is similar - as the number of labels increase and the variance between each diminishes, so too does the need for greater volumes of training data per label.

Consider if your model will still provide a benefit to you with less granularity to what it is trying to detect, and simplify your labels accordingly.

You need to add Human Review

The key to getting a better performance score is improving the training that the model gets. Levity's Human Review feature allow you the option to be part of the workflow, and have the model contact you when it needs clarification on a classification. You can set the error rate for this and adjust how much data you are asked to review. 

Our blog features more insights into using and improving on your machine learning models. Click here for more information.