module core.dataset
Global Variables
- string_classes
function failsafe_collate
Puts each data field into a tensor with outer dimension batch size
class ResumableDistributedSampler
method __init__
__init__(
dataset: Dataset,
num_replicas: Optional[int] = None,
rank: Optional[int] = None,
shuffle: bool = True,
seed: int = 0,
drop_last: bool = False,
num_iters: int = None
) → None
property indices
method set_start_batch_idx
class DatasetNode
Automating DataLoader and DataSampler creation and maintainance.
method __init__
property board
the board writer of current task.
property device
the assigned device by current launcher.
property epoch_size
the size of current epoch.
property epoch_steps
the steps of current epoch.
property global_auto_steps
the global steps of current task.
property global_train_epochs
the global train epochs of current task.
property global_train_steps
the global train steps of current task.
property grad_acc_steps
the grad accumulator steps of current task.
property grad_scaler
the grad scaler of current task.
property launcher
the current launcher.
property name
the node name in the current activated ExecutableGraph.
property out_dir
the output directory of current task.
property run_id
the run id of current task.
property step_mode
whether current task is running by step (True) or by epoch (False).
property task
the current task.
property training
whether current task is training.