SLAMA (Spark version of LAMA)

This is a distributed version of LAMA library written on Spark framework. SLAMA brings LAMA functionality on Spark including:

• Automatic hyperparameter tuning, data processing.

• Automatic typing, feature selection.

• Automatic time utilization.

• Automatic report creation.

• Easy-to-use modular scheme to create your own pipelines.

• Support of Spark ML pipelines, including saving/loading.

• Caching and checkpointing of intermediate results

Known limitations: - Only the tabular preset is currently supported

Python API

• sparklightautoml.automl
• sparklightautoml.dataset
• sparklightautoml.ml_algo
• sparklightautoml.pipelines
• sparklightautoml.pipelines.selection
• sparklightautoml.pipelines.features
• sparklightautoml.pipelines.ml
• sparklightautoml.reader
• sparklightautoml.report
• sparklightautoml.tasks
• sparklightautoml.transformers
• sparklightautoml.validation

Running on YARN Cluster with spark-submit

• Running spark lama app on Spark YARN with spark_submit

Running on YARN Cluster (from source)

• Running spark lama app on Spark YARN

Running on Spark Standalone Cluster (from source)

• Running spark lama app on standalone cluster

Deploy on Minikube (from source)

• Deploy on Minikube
• Environment setup
• Run examples in minikube

Running on Kubernetes Cluster (from source)

• Running spark lama app on Kubernetes cluster