Struct Interpreter

pub struct Interpreter { /* private fields */ }

net data holder. multiple sessions could share same net.

Implementations

impl Interpreter

pub fn from_file(path: impl AsRef<Path>) -> Result<Self>

Create an net/interpreter from a file.

path: the file path of the model

return: the created net/interpreter

pub fn from_bytes(bytes: impl AsRef<[u8]>) -> Result<Self>

Create an net/interpreter from a buffer.

bytes: the buffer of the model

return: the created net/interpreter

pub fn set_session_mode(&mut self, mode: SessionMode)

Set session mode

mode: the session mode

Warning: It should be called before create session!

pub fn resize_session(&self, session: &mut Session)

call this function to get tensors ready.

output tensor buffer (host or deviceId) should be retrieved after resize of any input tensor.

session: the session to be prepared

pub fn resize_session_reallocate(&self, session: &mut Session)

Resize session and reallocate the buffer.

session: the session to be prepared.

Note

NeedRelloc is default to 1, 1 means need realloc!

pub fn resize_tensor<T: TensorType>( &self, tensor: &mut Tensor<T>, dims: impl AsTensorShape, )

Resize the tensor using the given shape

pub fn resize_tensor_by_nchw<T: TensorType>( &self, tensor: &mut Tensor<T>, batch: u16, channel: u16, height: u16, width: u16, )

Resize tensor by

• N -> batch
• C -> channel
• H -> height
• W -> width

pub fn create_session(&mut self, schedule: ScheduleConfig) -> Result<Session>

Create a session with session config. Session will be managed in net/interpreter.

schedule : the config of the session

return: the created session

pub unsafe fn release_model(&mut self)

Release the model file buffer

Safety

This function is marked unsafe since it’s not clear what the safety guarantees are right now. With a simple test it caused a segfault so it’s marked unsafe

pub fn create_multipath_session( &mut self, schedule: impl IntoIterator<Item = ScheduleConfig>, ) -> Result<Session>

Create multi-path session with schedule configs and user-specified runtime. created session will be managed in net/interpreter.

schedule : the config of the session

return: the created session

pub fn model_print_io(path: impl AsRef<Path>) -> Result<()>

Print all input and output tensors info.

pub fn inputs<'i>(&self, session: &'i Session) -> TensorList<'i>

Get the input tensor of the session.

session: the session to get input tensor

return: List of input tensors

pub fn input<'s, H: HalideType>( &self, session: &'s Session, name: impl AsRef<str>, ) -> Result<Tensor<RefMut<'s, Device<H>>>>

Get the input tensor of the session by name.

session: the session to get input tensor from

name: the name of the input tensor

return: the input tensor

pub fn raw_input<'s>( &self, session: &'s Session, name: impl AsRef<str>, ) -> Result<RawTensor<'s>>

Get the raw input tensor of a session by name

pub unsafe fn input_unresized<'s, H: HalideType>( &self, session: &'s Session, name: impl AsRef<str>, ) -> Result<Tensor<RefMut<'s, Device<H>>>>

Safety

Warning We Still don’t know the safety guarantees of this function so it’s marked unsafe

pub unsafe fn input_unchecked<'s, H: HalideType>( &self, session: &'s Session, name: impl AsRef<str>, ) -> Tensor<RefMut<'s, Device<H>>>

Safety

Very unsafe since it doesn’t check the type of the tensor as well as the shape of the tensor

Panics if the name is not ascii Undefined Behavior if the tensor is not of type H

pub fn output<'s, H: HalideType>( &self, session: &'s Session, name: impl AsRef<str>, ) -> Result<Tensor<Ref<'s, Device<H>>>>

Get the output tensor of a session by name

session : the session to get output tensor from

name : the name of the output tensor

pub fn raw_output<'s>( &self, session: &'s Session, name: impl AsRef<str>, ) -> Result<RawTensor<'s>>

Get the raw output tensor of a session by name

pub fn run_session(&mut self, session: &Session) -> Result<()>

Run a session

pub fn run_session_with_callback( &mut self, session: &Session, before: impl Fn(&[RawTensor<'_>], OperatorInfo<'_>) -> bool + 'static, end: impl Fn(&[RawTensor<'_>], OperatorInfo<'_>) -> bool + 'static, sync: bool, ) -> Result<()>

Run a session with a callback

session : the session to run

before : a callback before each op. return true to run the op; return false to skip the op.

after : a callback after each op. return true to continue running; return false to interrupt the session.

sync : synchronously wait for finish of execution or not.

pub fn outputs<'o>(&self, session: &'o Session) -> TensorList<'o>

Get all output tensors of a session

pub fn set_cache_file( &mut self, path: impl AsRef<Path>, key_size: usize, ) -> Result<()>

If the cache exist, try to load cache from file. After createSession, try to save cache to file.

cache_file : the file path to save or load cache.

key_size : the size of key

Note

The API should be called before create session.

Key Depercerate, keeping for future use!

pub fn update_cache_file(&mut self, session: &mut Session) -> Result<()>

Update cache file

pub fn wait(&self, session: &Session)

Wait for all output tensors to be ready after computation

pub fn memory(&self, session: &Session) -> Result<f32>

Get memory usage of a session in MB

pub fn flops(&self, session: &Session) -> Result<f32>

Get float operation needed in session in M

pub fn resize_status(&self, session: &Session) -> Result<ResizeStatus>

Get the resize status

Trait Implementations

impl Debug for Interpreter

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for Interpreter

fn drop(&mut self)

Executes the destructor for this type.

impl Send for Interpreter