Multi Filesystem

A MultiFS is a filesystem composed of a sequence of other filesystems, where the directory structure of each overlays the previous filesystem in the sequence.

One use for such a filesystem would be to selectively override a set of files, to customize behavior. For example, to create a filesystem that could be used to theme a web application. We start with the following directories:

`-- templates
    |-- snippets
    |   `-- panel.html
    |-- index.html
    |-- profile.html
    `-- base.html

`-- theme
    |-- snippets
    |   |-- widget.html
    |   `-- extra.html
    |-- index.html
    `-- theme.html

And we want to create a single filesystem that will load a file from templates/ only if it isn’t found in theme/. Here’s how we could do that:

from fs.osfs import OSFS
from fs.multifs import MultiFS

theme_fs = MultiFS()
theme_fs.add_fs('templates', OSFS('templates'))
theme_fs.add_fs('theme', OSFS('theme'))

Now we have a theme_fs filesystem that presents a single view of both directories:

|-- snippets
|   |-- panel.html
|   |-- widget.html
|   `-- extra.html
|-- index.html
|-- profile.html
|-- base.html
`-- theme.html
class fs.multifs.MultiFS(auto_close=True)[source]

A filesystem that delegates to a sequence of other filesystems.

Operations on the MultiFS will try each ‘child’ filesystem in order, until it succeeds. In effect, creating a filesystem that combines the files and dirs of its children.

__init__(auto_close=True)[source]

Create a new MultiFS.

Parameters:auto_close (bool) – If True (the default), the child filesystems will be closed when MultiFS is closed.
add_fs(name, fs, write=False, priority=0)[source]

Add a filesystem to the MultiFS.

Parameters:
  • name (str) – A unique name to refer to the filesystem being added.
  • fs (FS or str) – The filesystem (instance or URL) to add.
  • write (bool) – If this value is True, then the fs will be used as the writeable FS (defaults to False).
  • priority (int) – An integer that denotes the priority of the filesystem being added. Filesystems will be searched in descending priority order and then by the reverse order they were added. So by default, the most recently added filesystem will be looked at first.
close()[source]

Close the filesystem and release any resources.

It is important to call this method when you have finished working with the filesystem. Some filesystems may not finalize changes until they are closed (archives for example). You may call this method explicitly (it is safe to call close multiple times), or you can use the filesystem as a context manager to automatically close.

Example

>>> with OSFS('~/Desktop') as desktop_fs:
...    desktop_fs.writetext(
...        'note.txt',
...        "Don't forget to tape Game of Thrones"
...    )

If you attempt to use a filesystem that has been closed, a FilesystemClosed exception will be thrown.

download(path, file, chunk_size=None, **options)[source]

Copy a file from the filesystem to a file-like object.

This may be more efficient that opening and copying files manually if the filesystem supplies an optimized method.

Note that the file object file will not be closed by this method. Take care to close it after this method completes (ideally with a context manager).

Parameters:
  • path (str) – Path to a resource.
  • file (file-like) – A file-like object open for writing in binary mode.
  • chunk_size (int, optional) – Number of bytes to read at a time, if a simple copy is used, or None to use sensible default.
  • **options – Implementation specific options required to open the source file.

Example

>>> with open('starwars.mov', 'wb') as write_file:
...     my_fs.download('/Videos/starwars.mov', write_file)
Raises:fs.errors.ResourceNotFound – if path does not exist.
get_fs(name)[source]

Get a filesystem from its name.

Parameters:name (str) – The name of a filesystem previously added.
Returns:the filesystem added as name previously.
Return type:FS
Raises:KeyError – If no filesystem with given name could be found.
getinfo(path, namespaces=None)[source]

Get information about a resource on a filesystem.

Parameters:
  • path (str) – A path to a resource on the filesystem.
  • namespaces (list, optional) – Info namespaces to query. The "basic" namespace is alway included in the returned info, whatever the value of namespaces may be.
Returns:

resource information object.

Return type:

Info

Raises:

fs.errors.ResourceNotFound – If path does not exist.

For more information regarding resource information, see Resource Info.

getsize(path)[source]

Get the size (in bytes) of a resource.

Parameters:path (str) – A path to a resource.
Returns:the size of the resource.
Return type:int
Raises:fs.errors.ResourceNotFound – if path does not exist.

The size of a file is the total number of readable bytes, which may not reflect the exact number of bytes of reserved disk space (or other storage medium).

The size of a directory is the number of bytes of overhead use to store the directory entry.

getsyspath(path)[source]

Get the system path of a resource.

Parameters:path (str) – A path on the filesystem.
Returns:the system path of the resource, if any.
Return type:str
Raises:fs.errors.NoSysPath – If there is no corresponding system path.

A system path is one recognized by the OS, that may be used outside of PyFilesystem (in an application or a shell for example). This method will get the corresponding system path that would be referenced by path.

Not all filesystems have associated system paths. Network and memory based filesystems, for example, may not physically store data anywhere the OS knows about. It is also possible for some paths to have a system path, whereas others don’t.

This method will always return a str on Py3.* and unicode on Py2.7. See getospath if you need to encode the path as bytes.

If path doesn’t have a system path, a NoSysPath exception will be thrown.

Note

A filesystem may return a system path even if no resource is referenced by that path – as long as it can be certain what that system path would be.

gettype(path)[source]

Get the type of a resource.

Parameters:path (str) – A path on the filesystem.
Returns:the type of the resource.
Return type:ResourceType
Raises:fs.errors.ResourceNotFound – if path does not exist.

A type of a resource is an integer that identifies the what the resource references. The standard type integers may be one of the values in the ResourceType enumerations.

The most common resource types, supported by virtually all filesystems are directory (1) and file (2), but the following types are also possible:

ResourceType value
unknown 0
directory 1
file 2
character 3
block_special_file 4
fifo 5
socket 6
symlink 7

Standard resource types are positive integers, negative values are reserved for implementation specific resource types.

geturl(path, purpose='download')[source]

Get the URL to a given resource.

Parameters:
  • path (str) – A path on the filesystem
  • purpose (str) – A short string that indicates which URL to retrieve for the given path (if there is more than one). The default is 'download', which should return a URL that serves the file. Other filesystems may support other values for purpose: for instance, OSFS supports 'fs', which returns a FS URL (see FS URLs).
Returns:

a URL.

Return type:

str

Raises:

fs.errors.NoURL – If the path does not map to a URL.

hassyspath(path)[source]

Check if a path maps to a system path.

Parameters:path (str) – A path on the filesystem.
Returns:True if the resource at path has a syspath.
Return type:bool
hasurl(path, purpose='download')[source]

Check if a path has a corresponding URL.

Parameters:
  • path (str) – A path on the filesystem.
  • purpose (str) – A purpose parameter, as given in geturl.
Returns:

True if an URL for the given purpose exists.

Return type:

bool

isdir(path)[source]

Check if a path maps to an existing directory.

Parameters:path (str) – A path on the filesystem.
Returns:True if path maps to a directory.
Return type:bool
isfile(path)[source]

Check if a path maps to an existing file.

Parameters:path (str) – A path on the filesystem.
Returns:True if path maps to a file.
Return type:bool
iterate_fs()[source]

Get iterator that returns (name, fs) in priority order.

listdir(path)[source]

Get a list of the resource names in a directory.

This method will return a list of the resources in a directory. A resource is a file, directory, or one of the other types defined in ResourceType.

Parameters:

path (str) – A path to a directory on the filesystem

Returns:

list of names, relative to path.

Return type:

list

Raises:
makedir(path, permissions=None, recreate=False)[source]

Make a directory.

Parameters:
  • path (str) – Path to directory from root.
  • permissions (Permissions, optional) – a Permissions instance, or None to use default.
  • recreate (bool) – Set to True to avoid raising an error if the directory already exists (defaults to False).
Returns:

a filesystem whose root is the new directory.

Return type:

SubFS

Raises:
makedirs(path, permissions=None, recreate=False)[source]

Make a directory, and any missing intermediate directories.

Parameters:
  • path (str) – Path to directory from root.
  • permissions (Permissions, optional) – Initial permissions, or None to use defaults.
  • recreate (bool) – If False (the default), attempting to create an existing directory will raise an error. Set to True to ignore existing directories.
Returns:

A sub-directory filesystem.

Return type:

SubFS

Raises:
open(path, mode='r', buffering=-1, encoding=None, errors=None, newline='', **kwargs)[source]

Open a file.

Parameters:
  • path (str) – A path to a file on the filesystem.
  • mode (str) – Mode to open the file object with (defaults to r).
  • buffering (int) – Buffering policy (-1 to use default buffering, 0 to disable buffering, 1 to select line buffering, of any positive integer to indicate a buffer size).
  • encoding (str) – Encoding for text files (defaults to utf-8)
  • errors (str, optional) – What to do with unicode decode errors (see codecs module for more information).
  • newline (str) – Newline parameter.
  • **options – keyword arguments for any additional information required by the filesystem (if any).
Returns:

a file-like object.

Return type:

io.IOBase

Raises:
openbin(path, mode='r', buffering=-1, **options)[source]

Open a binary file-like object.

Parameters:
  • path (str) – A path on the filesystem.
  • mode (str) – Mode to open file (must be a valid non-text mode, defaults to r). Since this method only opens binary files, the b in the mode string is implied.
  • buffering (int) – Buffering policy (-1 to use default buffering, 0 to disable buffering, or any positive integer to indicate a buffer size).
  • **options – keyword arguments for any additional information required by the filesystem (if any).
Returns:

a file-like object.

Return type:

io.IOBase

Raises:
readbytes(path)[source]

Get the contents of a file as bytes.

Parameters:

path (str) – A path to a readable file on the filesystem.

Returns:

the file contents.

Return type:

bytes

Raises:
readtext(path, encoding=None, errors=None, newline='')[source]

Get the contents of a file as a string.

Parameters:
  • path (str) – A path to a readable file on the filesystem.
  • encoding (str, optional) – Encoding to use when reading contents in text mode (defaults to None, reading in binary mode).
  • errors (str, optional) – Unicode errors parameter.
  • newline (str) – Newlines parameter.
Returns:

file contents.

Return type:

str

Raises:

fs.errors.ResourceNotFound – If path does not exist.

remove(path)[source]

Remove a file from the filesystem.

Parameters:

path (str) – Path of the file to remove.

Raises:
removedir(path)[source]

Remove a directory from the filesystem.

Parameters:

path (str) – Path of the directory to remove.

Raises:
scandir(path, namespaces=None, page=None)[source]

Get an iterator of resource info.

Parameters:
  • path (str) – A path to a directory on the filesystem.
  • namespaces (list, optional) – A list of namespaces to include in the resource information, e.g. ['basic', 'access'].
  • page (tuple, optional) – May be a tuple of (<start>, <end>) indexes to return an iterator of a subset of the resource info, or None to iterate over the entire directory. Paging a directory scan may be necessary for very large directories.
Returns:

an iterator of Info objects.

Return type:

Iterator

Raises:
setinfo(path, info)[source]

Set info on a resource.

This method is the complement to getinfo and is used to set info values on a resource.

Parameters:
  • path (str) – Path to a resource on the filesystem.
  • info (dict) – Dictionary of resource info.
Raises:

fs.errors.ResourceNotFound – If path does not exist on the filesystem

The info dict should be in the same format as the raw info returned by getinfo(file).raw.

Example

>>> details_info = {"details": {
...     "modified": time.time()
... }}
>>> my_fs.setinfo('file.txt', details_info)
upload(path, file, chunk_size=None, **options)[source]

Set a file to the contents of a binary file object.

This method copies bytes from an open binary file to a file on the filesystem. If the destination exists, it will first be truncated.

Parameters:
  • path (str) – A path on the filesystem.
  • file (io.IOBase) – a file object open for reading in binary mode.
  • chunk_size (int, optional) – Number of bytes to read at a time, if a simple copy is used, or None to use sensible default.
  • **options – Implementation specific options required to open the source file.
Raises:

fs.errors.ResourceNotFound – If a parent directory of path does not exist.

Note that the file object file will not be closed by this method. Take care to close it after this method completes (ideally with a context manager).

Example

>>> with open('~/movies/starwars.mov', 'rb') as read_file:
...     my_fs.upload('starwars.mov', read_file)
validatepath(path)[source]

Validate a path, returning a normalized absolute path on sucess.

Many filesystems have restrictions on the format of paths they support. This method will check that path is valid on the underlaying storage mechanism and throw a InvalidPath exception if it is not.

Parameters:

path (str) – A path.

Returns:

A normalized, absolute path.

Return type:

str

Raises:
which(path, mode='r')[source]

Get a tuple of (name, fs) that the given path would map to.

Parameters:
  • path (str) – A path on the filesystem.
  • mode (str) – An io.open mode.
writebytes(path, contents)[source]

Copy binary data to a file.

Parameters:
  • path (str) – Destination path on the filesystem.
  • contents (bytes) – Data to be written.
Raises:

TypeError – if contents is not bytes.

writetext(path, contents, encoding='utf-8', errors=None, newline='')[source]

Create or replace a file with text.

Parameters:
  • path (str) – Destination path on the filesystem.
  • contents (str) – Text to be written.
  • encoding (str, optional) – Encoding of destination file (defaults to 'utf-8').
  • errors (str, optional) – How encoding errors should be treated (same as io.open).
  • newline (str) – Newline parameter (same as io.open).
Raises:

TypeError – if contents is not a unicode string.