GSoC 2026 Project Ideas

Currently, etna_core relies on a rigid, sequential list of layers (Vec<LayerWrapper>), making it impossible to build modern non-linear architectures like ResNets (which need skip connections), Transformers, or RNNs. This project involves a fundamental rewrite of the Etna core to replace manual list-based backpropagation with a define-by-run Automatic Differentiation (Autograd) engine. The student will implement a "Tape-based" gradient tracking system in Rust, similar to PyTorch's backend.

The primary objective is the full integration of a define-by-run autograd engine into Etna. This entails implementing the core Tensor struct to wrap data and gradient history, constructing a Wengert list (Tape) for dynamic operation recording, and completely replacing the existing SimpleNN implementation. Furthermore, the PyO3 bindings must be updated to expose this new Graph API, enabling the definition of complex models (such as ResNets) in Python while maintaining efficient execution in Rust.

Etna is designed for high-performance machine learning, but the current implementation relies on standard Rust iterators (input.iter().map(...)). While safe, this approach fails to utilize CPU vectorization (AVX2/AVX-512) and suffers from poor cache locality due to the nested Vec<Vec<f32>> memory layout. This project focuses on pure performance engineering to replace the memory backend with a flattened, contiguous array structure and implement SIMD (Single Instruction, Multiple Data) kernels.

This project necessitates refactoring etna_core to utilize a flattened 1D vector representation for N-dimensional tensors, thereby maximizing cache hits. The central task involves writing custom unsafe Rust kernels using std::arch intrinsics (AVX2/AVX-512) for critical operations such as Dot Product and Matrix Multiplication. Additionally, Rayon must be integrated to parallelize operations across mini-batches to ensure optimal core utilization, with the resulting speedup validated through a comprehensive benchmark suite.

Currently, etsi-watchdog operates strictly on batch data (static CSVs or DataFrames). To support modern MLOps pipelines, it must be capable of monitoring high-throughput, real-time data streams without manual intervention. This project will extend Watchdog to support real-time streaming drift detection by integrating with Apache Kafka.

The goal is to build a robust production system that transforms Watchdog into a live monitoring sentinel. This involves implementing a Kafka consumer to ingest feature streams and developing an efficient "sliding window" engine (Tumbling/Hopping windows) to calculate drift metrics incrementally. To ensure fault tolerance, Redis or RocksDB must be integrated to persist window state, preventing data loss during service restarts. The final solution is to be packaged within a production-ready docker-compose environment.