SparkLinearFeatures

class sparklightautoml.pipelines.features.linear_pipeline.SparkLinearFeatures(feats_imp=None, top_intersections=5, max_bin_count=10, max_intersection_depth=3, subsample=None, sparse_ohe='auto', auto_unique_co=50, output_categories=True, multiclass_te_co=3, **_)

Bases: SparkFeaturesPipeline, SparkTabularDataFeatures

Creates pipeline for linear models and nnets.

Includes:

• Create categorical intersections.

• OHE or embed idx encoding for categories.

• Other cats to numbers ways if defined in role params.

• Standartization and nan handling for numbers.

• Numbers discretization if needed.

• Dates handling.

• Handling probs (output of lower level models).

__init__(feats_imp=None, top_intersections=5, max_bin_count=10, max_intersection_depth=3, subsample=None, sparse_ohe='auto', auto_unique_co=50, output_categories=True, multiclass_te_co=3, **_)

Parameters:

• feats_imp (Optional[ImportanceEstimator]) – Features importances mapping.

• top_intersections (int) – Max number of categories to generate intersections.

• max_bin_count (int) – Max number of bins to discretize numbers.

• max_intersection_depth (int) – Max depth of cat intersection.

• subsample (Union[int, float, None]) – Subsample to calc data statistics.

• sparse_ohe (Union[str, bool]) – Should we output sparse if ohe encoding was used during cat handling.

• auto_unique_co (int) – Switch to target encoding if high cardinality.

• output_categories (bool) – Output encoded categories or embed idxs.

• multiclass_te_co (int) – Cutoff if use target encoding in cat handling on multiclass task if number of classes is high.

create_pipeline(train)

Create linear pipeline.

Parameters:

train (SparkDataset) – Dataset with train features.

Return type:

Union[SparkBaseEstimator, SparkBaseTransformer, SparkUnionTransformer, SparkSequentialTransformer]

Returns:

Transformer.