What is Uber Ludwig?
Ludwig is a framework for simplifying the processes of selecting, training and evaluating machine learning models. Think about configuring rather than coding. Ludwig provides a set of model architectures that can be combined to create an end-to-end optimized model. For example, using Ludwig, a data scientist can train a deep learning model by simply providing a CSV file that contains the training data as well as a YAML file with the inputs and outputs of the model.
Ludwig Design Principles:
- No coding required
- Generality: usable across many different use cases.
- Flexibility: experienced users have extensive control, while newcomers will find it easy to use.
- Extensibility: easy to add new model architecture and feature data types.
- Understandability: standard visualizations to understand performance and predictions of deep learning models
Ludwig History:
Uber released first version of Ludwig in 2019. It received a warm reception. Version after version, Ludwig incorporated new capabilities that adapt latest machine learning frameworks to its low-code engine.
Ludwig 0.3:
The biggest addition in Ludwig 0.3 is that it incorporates contributors outside Uber; notably, Stanford University. Here are some new features of Ludwig 0.3:
Hyperparameter Optimization: Finding the best combination of hyperparameters for a given machine learning problem. e.g., The command “ludwig hyperopt –config_file ” will output different values and scales for hyperparameters, which a data science team can use as a starting point.
Integration with Weights and Biases: Ludwig 0.3 integrates with the Weights and Biases (W&B), which provides a very visual interface for rapid experimentation and hyperparameter tuning in machine learning models.
Code-Free Transformers: In recent years, language-pretrained models and transformers have been at the center of major breakthroughs in deep learning for natural language processing. Transformers however are quite hard to use in machine learning applications. Ludwig 0.3 integrates support for transformers via its integration with Hugging Face’s Transformers repository.
TensorFlow 2 Backend: This release of Ludwig has undergone a major re-architecture based on TensorFlow 2. It allows Ludwig to take advantage of the many new capabilities of TensorFlow 2, and become much more streamlined and modular.
New Data Source Integration: In addition to CSVs and Pandas Dataframes as input datasets. Ludwig’s new version integrates easily with many data source formats such as excel, feather, fwf, hdf5, html tables, json, jsonl, parquet, pickle, sas, spss, stata and tsv.
Other Capabilities: Ludwig 0.3 supports noisy labels for weak supervision and k-fold cross-validation for training.
The idea of enabling machine learning by just using configuration files results is very appealing to many teams and organizations. Little by little, Ludwig is becoming one of the most impressive open-source low-code machine learning stacks in the market.