1. Knowledge Base
  2. Human in the loop

How much data will I have to look at with Human Review?

Understanding the time cost of HITL

Re-training your model on the same training dataset will not yield better results.After selecting a maximum error rate, you will see an estimate of the percentage of data that will need to be reviewed manually.

errorrate

This percentage is influenced by the difficulty of your classification task and by how much quality-data you provided to train your classifier. 

Note: Re-training your model on the same training dataset will not yield better results - training will only improve with the addition of new data.

There's always a trade-off between the amount of data you will have to review and the amount of mistakes your classifier will make. You can increase the maximum error rate to lower the percentage of manual reviews vice versa.

The percentage you see is only an estimate based on your training data and is subject to change. For example, if the actual data points differ from those you trained with, the percentage you need to review might be higher or lower than expected. Also, the model will learn from your decisions and become more accurate over time, which means that you will have to gradually review less data manually while keeping the maximum error rate constant. Therefore, you might want to come back to decrease the maximum error rate after some time.