SparkBaseTrainValidIterator

class sparklightautoml.validation.base.SparkBaseTrainValidIterator(train)[source]

Bases: TrainValidIterator, Unpersistable, ABC

Implements applying selection pipeline and feature pipeline to SparkDataset.

apply_selector(selector)[source]

Select features on train data.

Check if selector is fitted. If not - fit and then perform selection. If fitted, check if it’s ok to apply.

Parameters:: selector (SparkSelectionPipeline) – Uses for feature selection.
Return type:: SparkBaseTrainValidIterator
Returns:: Dataset with selected features.