Building and Testing Navi (Rust)

Navi, X's high-performance machine learning model serving framework, is primarily written in Rust. This page outlines the process for building and testing its Rust-based components, namely segdense, dr_transform, and navi itself.

Overview of the Build Process

The navi project leverages Rust's cargo build system. While the repository's README indicates plans for a more complete Bazel build system in the future, current development and testing rely on cargo.

A key aspect of Navi's build is the integration with Protocol Buffers (protobufs) for inter-service communication, particularly with TensorFlow Serving and KFServing.

Protobuf Compilation (navi/navi/build.rs)

The navi/navi/build.rs file is a Rust build script↗ that executes during the cargo build process. Its primary function is to compile protobuf definitions into Rust code using the tonic_build crate.

This script ensures that the necessary Rust types for gRPC services and protobuf messages (e.g., PredictionService, PredictRequest, ModelInferRequest) are generated before the rest of the Navi codebase is compiled. These generated files are typically placed in the target/debug/build/<package_name>-<hash>/out/ directory during development.

Building Navi Components

To build the Rust components, navigate to the respective directories and use cargo build.

Building segdense

The segdense component (Navi - ML Model Serving Framework#segdense) is a library and a binary responsible for parsing and mapping model schemas.

1. Navigate to the segdense directory:

2. Build the project:
   This command will compile the segdense library and the executable defined in src/main.rs.

The segdense/src/main.rs file provides a simple entry point to load and parse a schema configuration from a JSON file. This is useful for validating schema definitions independently.

Building dr_transform

The dr_transform component (Navi - ML Model Serving Framework#dr_transform) handles the transformation of raw data into model-ready tensors.

1. Navigate to the dr_transform directory:

2. Build the project:
   This command compiles the dr_transform library. It does not have a main.rs for a standalone executable.

Building navi (the main service)

The navi crate contains the core logic for the ML model serving framework. Its entry point is navi/navi/src/main.rs.

1. Navigate to the navi directory:

2. Build the project:
   This command will compile the navi executable. The navi/navi/src/main.rs file contains the bootstrap function, which initializes the prediction service, model watchers, and metrics server.

Running Tests

To run the unit and integration tests for the Rust components:

Testing segdense

1. Navigate to the segdense directory:

2. Run tests:

Testing dr_transform

The dr_transform crate includes a test.rs module for its tests.

1. Navigate to the dr_transform directory:

2. Run tests:

Testing navi

1. Navigate to the navi directory:

2. Run tests: