What is human-in-the-loop (HITL or human review)?

HITL refers to systems that allow humans to give direct feedback to a model for predictions below a certain level of confidence.

In practice, you need to determine what level of confidence is acceptable for the process: If it is ok to have wrong predictions "slipping through", you can set threshold rather low – which, in turn, requires much less manual intervention through human labor. In other cases, you want to be sure that the system only records "correct" predictions. Essentially this puts you in the driver seat on how your predictions are classified - and how or when you want the model to ask for your feedback on edge cases.

Did this answer your question?