Multi Filesystem¶

A MultiFS is a filesystem composed of a sequence of other filesystems, where the directory structure of each overlays the previous filesystem in the sequence.

One use for such a filesystem would be to selectively override a set of files, to customize behavior. For example, to create a filesystem that could be used to theme a web application. We start with the following directories:

`-- templates
    |-- snippets
    |   `-- panel.html
    |-- index.html
    |-- profile.html
    `-- base.html

`-- theme
    |-- snippets
    |   |-- widget.html
    |   `-- extra.html
    |-- index.html
    `-- theme.html

And we want to create a single filesystem that will load a file from templates/ only if it isn’t found in theme/. Here’s how we could do that:

from fs.osfs import OSFS
from fs.multifs import MultiFS

theme_fs = MultiFS()
theme_fs.add_fs('templates', OSFS('templates'))
theme_fs.add_fs('theme', OSFS('theme'))

Now we have a theme_fs filesystem that presents a single view of both directories:

|-- snippets
|   |-- panel.html
|   |-- widget.html
|   `-- extra.html
|-- index.html
|-- profile.html
|-- base.html
`-- theme.html

class fs.multifs.MultiFS(auto_close=True)[source]¶

A filesystem that delegates to a sequence of other filesystems.

Operations on the MultiFS will try each ‘child’ filesystem in order, until it succeeds. In effect, creating a filesystem that combines the files and dirs of its children.

__init__(auto_close=True)[source]¶

Create a new MultiFS.

Parameters:	auto_close (bool) – If `True` (the default), the child filesystems will be closed when `MultiFS` is closed.

add_fs(name, fs, write=False, priority=0)[source]¶

Add a filesystem to the MultiFS.

Parameters:

name (str) – A unique name to refer to the filesystem being added.
fs (FS or str) – The filesystem (instance or URL) to add.
write (bool) – If this value is True, then the fs will be used as the writeable FS (defaults to False).
priority (int) – An integer that denotes the priority of the filesystem being added. Filesystems will be searched in descending priority order and then by the reverse order they were added. So by default, the most recently added filesystem will be looked at first.

close()[source]¶

Close the filesystem and release any resources.

It is important to call this method when you have finished working with the filesystem. Some filesystems may not finalize changes until they are closed (archives for example). You may call this method explicitly (it is safe to call close multiple times), or you can use the filesystem as a context manager to automatically close.

Example

>>> with OSFS('~/Desktop') as desktop_fs:
...    desktop_fs.writetext(
...        'note.txt',
...        "Don't forget to tape Game of Thrones"
...    )

If you attempt to use a filesystem that has been closed, a FilesystemClosed exception will be thrown.

download(path, file, chunk_size=None, **options)[source]¶

Copy a file from the filesystem to a file-like object.

This may be more efficient that opening and copying files manually if the filesystem supplies an optimized method.

Note that the file object file will not be closed by this method. Take care to close it after this method completes (ideally with a context manager).

Parameters:	path (str) – Path to a resource. file (file-like) – A file-like object open for writing in binary mode. chunk_size (int, optional) – Number of bytes to read at a time, if a simple copy is used, or `None` to use sensible default. **options – Implementation specific options required to open the source file.

Example

>>> with open('starwars.mov', 'wb') as write_file:
...     my_fs.download('/Videos/starwars.mov', write_file)

Raises:	`fs.errors.ResourceNotFound` – if `path` does not exist.

get_fs(name)[source]¶

Get a filesystem from its name.

Parameters:	name (str) – The name of a filesystem previously added.
Returns:	the filesystem added as `name` previously.
Return type:	FS
Raises:	`KeyError` – If no filesystem with given `name` could be found.

getinfo(path, namespaces=None)[source]¶

Get information about a resource on a filesystem.

Parameters:	path (str) – A path to a resource on the filesystem. namespaces (list, optional) – Info namespaces to query. The `"basic"` namespace is alway included in the returned info, whatever the value of `namespaces` may be.
Returns:	resource information object.
Return type:	Info
Raises:	`fs.errors.ResourceNotFound` – If `path` does not exist.

For more information regarding resource information, see Resource Info.

getsize(path)[source]¶

Get the size (in bytes) of a resource.

Parameters:	path (str) – A path to a resource.
Returns:	the size of the resource.
Return type:	int
Raises:	`fs.errors.ResourceNotFound` – if `path` does not exist.

The size of a file is the total number of readable bytes, which may not reflect the exact number of bytes of reserved disk space (or other storage medium).

The size of a directory is the number of bytes of overhead use to store the directory entry.

getsyspath(path)[source]¶

Get the system path of a resource.

Parameters:	path (str) – A path on the filesystem.
Returns:	the system path of the resource, if any.
Return type:	str
Raises:	`fs.errors.NoSysPath` – If there is no corresponding system path.

A system path is one recognized by the OS, that may be used outside of PyFilesystem (in an application or a shell for example). This method will get the corresponding system path that would be referenced by path.

Not all filesystems have associated system paths. Network and memory based filesystems, for example, may not physically store data anywhere the OS knows about. It is also possible for some paths to have a system path, whereas others don’t.

This method will always return a str on Py3.* and unicode on Py2.7. See getospath if you need to encode the path as bytes.

If path doesn’t have a system path, a NoSysPath exception will be thrown.

Note

A filesystem may return a system path even if no resource is referenced by that path – as long as it can be certain what that system path would be.

gettype(path)[source]¶

Get the type of a resource.

Parameters:	path (str) – A path on the filesystem.
Returns:	the type of the resource.
Return type:	ResourceType
Raises:	`fs.errors.ResourceNotFound` – if `path` does not exist.

A type of a resource is an integer that identifies the what the resource references. The standard type integers may be one of the values in the ResourceType enumerations.

The most common resource types, supported by virtually all filesystems are directory (1) and file (2), but the following types are also possible:

ResourceType	value
unknown	0
directory	1
file	2
character	3
block_special_file	4
fifo	5
socket	6
symlink	7

Standard resource types are positive integers, negative values are reserved for implementation specific resource types.

geturl(path, purpose='download')[source]¶

Get the URL to a given resource.

Parameters:	path (str) – A path on the filesystem purpose (str) – A short string that indicates which URL to retrieve for the given path (if there is more than one). The default is `'download'`, which should return a URL that serves the file. Other filesystems may support other values for `purpose`: for instance, `OSFS` supports `'fs'`, which returns a FS URL (see FS URLs).
Returns:	a URL.
Return type:	str
Raises:	`fs.errors.NoURL` – If the path does not map to a URL.

hassyspath(path)[source]¶

Check if a path maps to a system path.

Parameters:	path (str) – A path on the filesystem.
Returns:	`True` if the resource at `path` has a syspath.
Return type:	bool

hasurl(path, purpose='download')[source]¶

Check if a path has a corresponding URL.

Parameters:	path (str) – A path on the filesystem. purpose (str) – A purpose parameter, as given in `geturl`.
Returns:	`True` if an URL for the given purpose exists.
Return type:	bool

isdir(path)[source]¶

Check if a path maps to an existing directory.

Parameters:	path (str) – A path on the filesystem.
Returns:	`True` if `path` maps to a directory.
Return type:	bool

isfile(path)[source]¶

Check if a path maps to an existing file.

Parameters:	path (str) – A path on the filesystem.
Returns:	`True` if `path` maps to a file.
Return type:	bool

iterate_fs()[source]¶: Get iterator that returns (name, fs) in priority order.

listdir(path)[source]¶

Get a list of the resource names in a directory.

This method will return a list of the resources in a directory. A resource is a file, directory, or one of the other types defined in ResourceType.

Parameters:	path (str) – A path to a directory on the filesystem
Returns:	list of names, relative to `path`.
Return type:	list
Raises:	`fs.errors.DirectoryExpected` – If `path` is not a directory. `fs.errors.ResourceNotFound` – If `path` does not exist.

makedir(path, permissions=None, recreate=False)[source]¶

Make a directory.

Parameters:	path (str) – Path to directory from root. permissions (Permissions, optional) – a `Permissions` instance, or `None` to use default. recreate (bool) – Set to `True` to avoid raising an error if the directory already exists (defaults to `False`).
Returns:	a filesystem whose root is the new directory.
Return type:	SubFS
Raises:	`fs.errors.DirectoryExists` – If the path already exists. `fs.errors.ResourceNotFound` – If the path is not found.

makedirs(path, permissions=None, recreate=False)[source]¶

Make a directory, and any missing intermediate directories.

Parameters:	path (str) – Path to directory from root. permissions (Permissions, optional) – Initial permissions, or `None` to use defaults. recreate (bool) – If `False` (the default), attempting to create an existing directory will raise an error. Set to `True` to ignore existing directories.
Returns:	A sub-directory filesystem.
Return type:	SubFS
Raises:	`fs.errors.DirectoryExists` – if the path is already a directory, and `recreate` is `False`. `fs.errors.DirectoryExpected` – if one of the ancestors in the path is not a directory.

open(path, mode='r', buffering=-1, encoding=None, errors=None, newline='', **kwargs)[source]¶

Open a file.

Parameters:	path (str) – A path to a file on the filesystem. mode (str) – Mode to open the file object with (defaults to r). buffering (int) – Buffering policy (-1 to use default buffering, 0 to disable buffering, 1 to select line buffering, of any positive integer to indicate a buffer size). encoding (str) – Encoding for text files (defaults to `utf-8`) errors (str, optional) – What to do with unicode decode errors (see `codecs` module for more information). newline (str) – Newline parameter. **options – keyword arguments for any additional information required by the filesystem (if any).
Returns:	a file-like object.
Return type:	io.IOBase
Raises:	`fs.errors.FileExpected` – If the path is not a file. `fs.errors.FileExists` – If the file exists, and exclusive mode is specified (`x` in the mode). `fs.errors.ResourceNotFound` – If the path does not exist.

openbin(path, mode='r', buffering=-1, **options)[source]¶

Open a binary file-like object.

Parameters:	path (str) – A path on the filesystem. mode (str) – Mode to open file (must be a valid non-text mode, defaults to r). Since this method only opens binary files, the `b` in the mode string is implied. buffering (int) – Buffering policy (-1 to use default buffering, 0 to disable buffering, or any positive integer to indicate a buffer size). **options – keyword arguments for any additional information required by the filesystem (if any).
Returns:	a file-like object.
Return type:	io.IOBase
Raises:	`fs.errors.FileExpected` – If `path` exists and is not a file. `fs.errors.FileExists` – If the `path` exists, and exclusive mode is specified (`x` in the mode). `fs.errors.ResourceNotFound` – If `path` does not exist and `mode` does not imply creating the file, or if any ancestor of `path` does not exist.

readbytes(path)[source]¶

Get the contents of a file as bytes.

Parameters:	path (str) – A path to a readable file on the filesystem.
Returns:	the file contents.
Return type:	bytes
Raises:	`fs.errors.FileExpected` – if `path` exists but is not a file. `fs.errors.ResourceNotFound` – if `path` does not exist.

readtext(path, encoding=None, errors=None, newline='')[source]¶

Get the contents of a file as a string.

Parameters:	path (str) – A path to a readable file on the filesystem. encoding (str, optional) – Encoding to use when reading contents in text mode (defaults to `None`, reading in binary mode). errors (str, optional) – Unicode errors parameter. newline (str) – Newlines parameter.
Returns:	file contents.
Return type:	str
Raises:	`fs.errors.ResourceNotFound` – If `path` does not exist.

remove(path)[source]¶

Remove a file from the filesystem.

Parameters:	path (str) – Path of the file to remove.
Raises:	`fs.errors.FileExpected` – If the path is a directory. `fs.errors.ResourceNotFound` – If the path does not exist.

removedir(path)[source]¶

Remove a directory from the filesystem.

Parameters:	path (str) – Path of the directory to remove.
Raises:	`fs.errors.DirectoryNotEmpty` – If the directory is not empty ( see `removetree` for a way to remove the directory contents). `fs.errors.DirectoryExpected` – If the path does not refer to a directory. `fs.errors.ResourceNotFound` – If no resource exists at the given path. `fs.errors.RemoveRootError` – If an attempt is made to remove the root directory (i.e. `'/'`)

scandir(path, namespaces=None, page=None)[source]¶

Get an iterator of resource info.

Parameters:	path (str) – A path to a directory on the filesystem. namespaces (list, optional) – A list of namespaces to include in the resource information, e.g. `['basic', 'access']`. page (tuple, optional) – May be a tuple of `(<start>, <end>)` indexes to return an iterator of a subset of the resource info, or `None` to iterate over the entire directory. Paging a directory scan may be necessary for very large directories.
Returns:	an iterator of `Info` objects.
Return type:	Iterator
Raises:	`fs.errors.DirectoryExpected` – If `path` is not a directory. `fs.errors.ResourceNotFound` – If `path` does not exist.

setinfo(path, info)[source]¶

Set info on a resource.

This method is the complement to getinfo and is used to set info values on a resource.

Parameters:	path (str) – Path to a resource on the filesystem. info (dict) – Dictionary of resource info.
Raises:	`fs.errors.ResourceNotFound` – If `path` does not exist on the filesystem

The info dict should be in the same format as the raw info returned by getinfo(file).raw.

Example

>>> details_info = {"details": {
...     "modified": time.time()
... }}
>>> my_fs.setinfo('file.txt', details_info)

upload(path, file, chunk_size=None, **options)[source]¶

Set a file to the contents of a binary file object.

This method copies bytes from an open binary file to a file on the filesystem. If the destination exists, it will first be truncated.

Parameters:	path (str) – A path on the filesystem. file (io.IOBase) – a file object open for reading in binary mode. chunk_size (int, optional) – Number of bytes to read at a time, if a simple copy is used, or `None` to use sensible default. **options – Implementation specific options required to open the source file.
Raises:	`fs.errors.ResourceNotFound` – If a parent directory of `path` does not exist.

Note that the file object file will not be closed by this method. Take care to close it after this method completes (ideally with a context manager).

Example

>>> with open('~/movies/starwars.mov', 'rb') as read_file:
...     my_fs.upload('starwars.mov', read_file)

validatepath(path)[source]¶

Validate a path, returning a normalized absolute path on sucess.

Many filesystems have restrictions on the format of paths they support. This method will check that path is valid on the underlaying storage mechanism and throw a InvalidPath exception if it is not.

Parameters:	path (str) – A path.
Returns:	A normalized, absolute path.
Return type:	str
Raises:	`fs.errors.InvalidPath` – If the path is invalid. `fs.errors.FilesystemClosed` – if the filesystem is closed. `fs.errors.InvalidCharsInPath` – If the path contains invalid characters.

which(path, mode='r')[source]¶

Get a tuple of (name, fs) that the given path would map to.

Parameters:	path (str) – A path on the filesystem. mode (str) – An `io.open` mode.

writebytes(path, contents)[source]¶

Copy binary data to a file.

Parameters:	path (str) – Destination path on the filesystem. contents (bytes) – Data to be written.
Raises:	`TypeError` – if contents is not bytes.

writetext(path, contents, encoding='utf-8', errors=None, newline='')[source]¶

Create or replace a file with text.

Parameters:	path (str) – Destination path on the filesystem. contents (str) – Text to be written. encoding (str, optional) – Encoding of destination file (defaults to `'utf-8'`). errors (str, optional) – How encoding errors should be treated (same as `io.open`). newline (str) – Newline parameter (same as `io.open`).
Raises:	`TypeError` – if `contents` is not a unicode string.