terra.tokenize_adata

Contents

terra.tokenize_adata#

terra.tokenize_adata(adata, model_folder_path, cache_directory_path, nproc=4, processing_mode='parallel', add_neigh_cell_ids=False, use_generator=True, keep_in_memory=False, include_special_tokens=False)#

Tokenize an AnnData object based on the parameters in the model config and return the tokenized Hugging Face dataset.

Parameters:

  • adata (AnnData) – AnnData object to be tokenized.

  • model_folder_path (str) – Path to the folder containing the model config, token dictionary, and normalization factors.

  • cache_directory_path (str) – Path where the cache is stored during dataset creation.

  • nproc (int (default: 4)) – Number of processes used.

  • processing_mode (Literal['sequential', 'parallel'] (default: 'parallel')) – Mode of processing.

  • add_neigh_cell_ids (bool (default: False)) – Whether neighbor cell IDs should be stored in tokenized data (used for perturbations).

  • use_generator (bool (default: True)) – Whether to use generator for dataset creation.

  • keep_in_memory (bool (default: False)) – Whether to keep dataset in memory.

  • include_special_tokens (bool (default: False)) – Whether to include special tokens in the tokenized output.

Returns:

  • dataset (Dataset) – The tokenized data stored in a Hugging Face dataset.

Contents