There are many settings for machine learning algorithms that must be manually set up. These tuning options are called hyperparameters, and are used to tune algorithms when optimizing performance or balancing bias and variance. Model tuning is typically understood as a trial and error process where you tune hyperparameters, rerun algorithms, and then check and compare which set of hyperparameters yields the best outcome.
All machine learning algorithms have a set of default hyperparameters, which are external and unpredictable. For example, typical hyperparameters of a deep neural network model include learning rate, and the size and number of the hidden layers. As such, each machine learning models have their own hyperparameters, and performance depends on their values. And to discover the hyperparameter needed for optimal performance, studies and researches are continuously being conducted throughout the field. Our Click AI services offers automatic tuning for optimal hyperparameters according to input data and model selection. Let's take a look at some of the methods of hyperparameter tuning.
· Manual Search
In this method, machine learning developers directly apply hyperparameters based on their intuition, experience, and knowledge. The model is trained with manually selected hyperparameters and scored for validation data. This processes is repeated until satisfactory results are obtained.
· Grid Search
Grid search is a method of finding the optimal combination by applying pre-selected values to the hyperparameter set and calculating results for all combinations. This process can often be inefficient because it needs to train all hyperparameter combinations and could become a highly repetitive process.
· Random Search
Random search refers to a method of finding the optimal combination by repeatedly extracting random values between the predetermined minimum and maximum values and applying the value to the hyperparameter. It is similar to the grid search method, but less time consuming and more effective. That is because grid search is a method where the user chooses from the predetermined options, but random search tests different combinations in a predetermined range and may result in unexpected outcomes.
There is no need to worry about modeling tuning while using CLICK AI's automated machine learning platform. Hundreds of models with variety of hyperparameters are developed through CLICK AI in the model learning process, and CLICK AI will help you pick to best and most accurate models for you.