jacinle.io.fs

Classes

LSDirectoryReturnType

Functions

as_file_descriptor(fd_or_fname[, mode])	Convert a file descriptor or a file name to a file descriptor.
dump(filename, obj, **kwargs)	Dump a file with automatic file type detection.
dump_mat(filename, obj, **kwargs)	Dump a matlab file.
dump_npy(filename, obj, **kwargs)	Dump a npy numpy file.
dump_npz(filename, obj, **kwargs)	Dump a npz numpy file.
dump_pkl(fd_or_filename, obj, **kwargs)	Dump a pickle file.
dump_pklgz(filename, obj, **kwargs)	Dump a gziped pickle file.
dump_pth(filename, obj, **kwargs)	Dump a PyTorch file.
fs_verbose([mode])	A context manager to enable/disable verbose mode in file system operations.
link(path_origin, *paths[, use_relative_path])	Create a symbolic link to a file or directory.
load(filename, **kwargs)	Load a file with automatic file type detection.
load_h5(filename, **kwargs)	Load a HDF5 file.
load_mat(filename, **kwargs)	Load a matlab file.
load_npy(fd_or_filename, **kwargs)	Load a npy numpy file.
load_npz(fd_or_filename, **kwargs)	Load a npz numpy file.
load_pkl(fd_or_filename, **kwargs)	Load a pickle file.
load_pklgz(filename, **kwargs)	Load a gziped pickle file.
load_pth(filename, **kwargs)	Load a PyTorch file.
load_txt(filename, **kwargs)	Load a text file.
locate_newest_file(dirname, pattern)	Locate the newest file in a directory.
lsdir(dirname[, pattern, return_type, sort])	List all files in a directory.
mkdir(path)	Create a directory if it does not exist without raising errors when the directory already exists.
open(filename, mode, **kwargs)	Open a file.
open_gz(filename, mode)	Open a gzip file.
open_h5(filename, mode, **kwargs)	Open a HDF5 file.
open_txt(filename, mode, **kwargs)	Open a text file.
remove(file_or_dirname)	Remove a file or directory.
safe_dump(filename, data[, use_lock, ...])	Dump data to a file in a safe way.
set_fs_verbose([mode])	Enable/disable verbose mode in file system operations.
tempfile([mode, suffix, prefix])	A context manager that creates a temporary file and deletes it after use.

Class LSDirectoryReturnType

class LSDirectoryReturnType

Bases: JacEnum

__new__(value)

classmethod assert_valid(value)

Assert if the value is a valid choice.

classmethod choice_names()

Returns the list of the name of all possible choices.

classmethod choice_objs()

Returns the list of the object of all possible choices.

classmethod choice_values()

Returns the list of the value of all possible choices.

classmethod from_string(value)

Parameters:

value (str | JacEnum)

Return type:

JacEnum

classmethod is_valid(value)

Check if the value is a valid choice.

classmethod type_name()

Return the type name of the enum.

BASE = 'base'

FULL = 'full'

NAME = 'name'

REAL = 'real'

REL = 'rel'

Functions

as_file_descriptor(fd_or_fname, mode='r')

Convert a file descriptor or a file name to a file descriptor.

Parameters:

• fd_or_fname (str | IOBase) – a file descriptor or a file name.

• mode (str) – the mode to open the file if fd_or_fname is a file name.

Returns:

a file descriptor.

Return type:

IOBase

dump(filename, obj, **kwargs)

Dump a file with automatic file type detection.

dump_mat(filename, obj, **kwargs)

Dump a matlab file.

dump_npy(filename, obj, **kwargs)

Dump a npy numpy file.

Parameters:

filename (str)

dump_npz(filename, obj, **kwargs)

Dump a npz numpy file.

Parameters:

filename (str)

dump_pkl(fd_or_filename, obj, **kwargs)

Dump a pickle file.

Parameters:

fd_or_filename (str | IOBase)

dump_pklgz(filename, obj, **kwargs)

Dump a gziped pickle file.

Parameters:

filename (str)

dump_pth(filename, obj, **kwargs)

Dump a PyTorch file.

fs_verbose(mode=True)

A context manager to enable/disable verbose mode in file system operations.

link(path_origin, *paths, use_relative_path=True)

Create a symbolic link to a file or directory.

Parameters:

• path_origin (str) – the original file or directory.

• paths (str) – the symbolic links to be created.

• use_relative_path – whether to use relative path.

load(filename, **kwargs)

Load a file with automatic file type detection.

Parameters:

filename (str)

load_h5(filename, **kwargs)

Load a HDF5 file.

Parameters:

filename (str)

load_mat(filename, **kwargs)

Load a matlab file.

Parameters:

filename (str)

load_npy(fd_or_filename, **kwargs)

Load a npy numpy file.

Parameters:

fd_or_filename (str | IOBase)

load_npz(fd_or_filename, **kwargs)

Load a npz numpy file.

Parameters:

fd_or_filename (str | IOBase)

load_pkl(fd_or_filename, **kwargs)

Load a pickle file.

Parameters:

fd_or_filename (str | IOBase)

load_pklgz(filename, **kwargs)

Load a gziped pickle file.

Parameters:

filename (str)

load_pth(filename, **kwargs)

Load a PyTorch file.

load_txt(filename, **kwargs)

Load a text file.

locate_newest_file(dirname, pattern)

Locate the newest file in a directory. If there is no file matching the pattern, return None.

Parameters:

• dirname (str) – the directory name.

• pattern (str) – the file name pattern in glob format.

Returns:

the full path of the newest file.

Return type:

str | None

lsdir(dirname, pattern=None, return_type='full', sort=True)

List all files in a directory.

Parameters:

• dirname (str) – the directory name.

• pattern (str | None) – the file name pattern in glob format.

• return_type (str | LSDirectoryReturnType) – the return type. Can be one of the following: ‘base’: return the base name of the file. ‘name’: return the file name. ‘rel’: return the relative path of the file. ‘full’: return the full path of the file. ‘real’: return the real path of the file.

• sort (bool) – whether to sort the file names.

Returns:

a list of file names.

Return type:

List[str]

mkdir(path)

Create a directory if it does not exist without raising errors when the directory already exists.

open(filename, mode, **kwargs)

Open a file.

Parameters:

• filename (str)

• mode (str)

open_gz(filename, mode)

Open a gzip file.

open_h5(filename, mode, **kwargs)

Open a HDF5 file.

Parameters:

• filename (str)

• mode (str)

open_txt(filename, mode, **kwargs)

Open a text file.

remove(file_or_dirname)

Remove a file or directory.

Parameters:

file_or_dirname (str)

safe_dump(filename, data, use_lock=True, use_temp=True, lock_timeout=10)

Dump data to a file in a safe way. Basically, it will dump the data to a temporary file and then move it to the target file. This is to avoid the case that the target file is corrupted when the program is interrupted during the dumping process. It also supports file locking to avoid the case that multiple processes are dumping data to the same file at the same time.

Parameters:

• filename (str) – the target file name.

• data – the data to be dumped.

• use_lock – whether to use file locking.

• use_temp – whether to use a temporary file.

• lock_timeout – the timeout for file locking.

Returns:

If uses temp file, return True if the data is dumped to the temp file successfully, otherwise False. If not use temp file, return the result of the dump operation.

Return type:

bool

set_fs_verbose(mode=True)

Enable/disable verbose mode in file system operations.

Parameters:

mode (bool)

tempfile(mode='w+b', suffix='', prefix='tmp')

A context manager that creates a temporary file and deletes it after use.

Example

with tempfile() as f:
    f.write(b'hello world')
    f.seek(0)
    print(f.read())

Parameters:

• mode (str) – the mode to open the file.

• suffix (str) – the suffix of the file name.

• prefix (str) – the prefix of the file name.