syft.core.remote_dataloader.remote_dataloader¶
Classes
|
|
|
- class syft.core.remote_dataloader.remote_dataloader.RemoteDataset(path, data_type='torch_tensor')[source]¶
Bases:
torch.utils.data.dataset.Dataset,syft.core.common.serde.serializable.Serializable- __init__(path, data_type='torch_tensor')[source]¶
- Parameters
path – information about where to get the raw data, for example, a file path,
or a directory path data_type – the type of data for example torch_tensor
For now, it’s should simply be a .pt file, which stores a Dataset object.
- load_dataset()[source]¶
Load the real Dataset object on DO’s machine. But for now, it’s just simply calling torch.load on a .pt file.
- static get_protobuf_schema()[source]¶
Return the type of protobuf object which stores a class of this type
As a part of serialization and deserialization, we need the ability to lookup the protobuf object type directly from the object type. This static method allows us to do this.
Importantly, this method is also used to create the reverse lookup ability within the metaclass of Serializable. In the metaclass, it calls this method and then it takes whatever type is returned from this method and adds an attribute to it with the type of this class attached to it. See the MetaSerializable class for details.
- Returns
the type of protobuf object which corresponds to this class.
- Return type
GeneratedProtocolMessageType
- class syft.core.remote_dataloader.remote_dataloader.RemoteDataLoader(remote_dataset, batch_size=1)[source]¶
Bases:
syft.core.common.serde.serializable.Serializable- __init__(remote_dataset, batch_size=1)[source]¶
TODO: now, only batch_size can be passed in by users, and it’s used when create DataLoader object in self.create_dataloader. In future steps, more augmentations should be supported, like shuffle, sampler, collate_fn, etc.
- static get_protobuf_schema()[source]¶
Return the type of protobuf object which stores a class of this type
As a part of serialization and deserialization, we need the ability to lookup the protobuf object type directly from the object type. This static method allows us to do this.
Importantly, this method is also used to create the reverse lookup ability within the metaclass of Serializable. In the metaclass, it calls this method and then it takes whatever type is returned from this method and adds an attribute to it with the type of this class attached to it. See the MetaSerializable class for details.
- Returns
the type of protobuf object which corresponds to this class.
- Return type
GeneratedProtocolMessageType