RecSys for B2B platform

I am working on RecSys to generate product recommendations for ABI’s B2B platform, BEEs. Some of the challenges include building AutoML for hyperparameter selection, distributed model training, feature store integration, a Python library of curated ML models with default configurations, and deploying models in cloud-native compute environments. I am excited to work on this with an amazing team.

Algorithm related challenges:

• Cross validation: How to perform cross-validation for RecSys, link statistical metrics with business KPIs, determine the balance between model fit and business KPIs, create a scoring function to compare models, and choose a data-splitting strategy that keeps models comparable.
• Model selection: Should we use a single model, a market-based model, or a hybrid that combines multiple models? Should we use time- or sequence-based models such as LSTM or GRU?
• Hyperparameter tuning: Which tuning framework can support GPUs (Wide and Deep), Spark (ALS), and CPUs (SAR, etc.)?
• KPIs: How can we evaluate and improve existing KPIs such as Map@K and NDCG@K?
• Hybrid model or mixture of models: What type of hybrid should we use: sequential, parallel, or weighted? Conceptually, there are two use cases.
• AutoML: What technology stack should we use for a multi-country setup that includes hybrid models and hyperparameter tuning?
• Model and data drift, retraining, and monitoring: How can we build a framework that integrates with the Python library to detect model and data drift, identify when retraining is needed, and monitor results for online and offline models?
• Other challenges: Backtesting, A/B testing, and linking online and offline evaluation.

Programming & Infra related challenges:

• Codespaces: How can a developer use a Codespace for a CPU-based day-to-day workflow? How can we manage multiple environments with Spark/GPU/CPU dependencies using a dev container? Can the same image be used in AML/ADB?
• AML + VS Code/Code Space Integration: Attach AML compute to VS Code as a terminal and Jupyter kernel. Run experiments in AML without leaving VS Code. Trigger multiple concurrent jobs in AML from VS Code and scale across nodes to run different models. Some of our classical models run multiple times as embarrassingly parallel workloads; results can be returned to cloud storage in a fan-out/fan-in pattern. We also want to use all cores on each node with joblib, so autoscaling should add nodes when a threshold is reached. Integrate mlflow with VS Code and AML.
• ADB + VS Code/Code Space Integration: Run experiments in ADB without leaving VS Code.
• Debugging: How can we use the VS Code visual debugger in distributed workflows in AML and ADB?
• Observability: How can we monitor aggregated logs from different nodes in VS Code?
• Testing: How can we run property-based tests for ML models in distributed compute environments?
• Library: How can we manage dependencies such as pyspark and system-level GPU and CPU requirements? How can we use JIT across models while accounting for the execution infrastructure? How can we make the library infrastructure-agnostic?

If you are excited about solving the challenges mentioned above, feel free to reach out to me.