Implementing Filesystems

With a little care, you can implement a PyFilesystem interface for any filesystem, which will allow it to work interchangeably with any of the built-in FS classes and tools.

To create a PyFilesystem interface, derive a class from FS and implement the Essential Methods. This should give you a working FS class.

Take care to copy the method signatures exactly, including default values. It is also essential that you follow the same logic with regards to exceptions, and only raise exceptions in errors.

Constructor

There are no particular requirements regarding how a PyFilesystem class is constructed, but be sure to call the base class __init__ method with no parameters.

Thread Safety

All Filesystems should be thread-safe. The simplest way to achieve that is by using the _lock attribute supplied by the FS constructor. This is a RLock object from the standard library, which you can use as a context manager, so methods you implement will start something like this:

with self._lock:
    do_something()

You aren’t required to use _lock. Just as long as calling methods on the FS object from multiple threads doesn’t break anything.

Python Versions

PyFilesystem supports Python2.7 and Python3.X. The differences between the two major Python versions are largely managed by the six library.

You aren’t obligated to support the same versions of Python that PyFilesystem itself supports, but it is recommended if your project is for general use.

Testing Filesystems

To test your implementation, you can borrow the test suite used to test the built in filesystems. If your code passes these tests, then you can be confident your implementation will work seamlessly.

Here’s the simplest possible example to test a filesystem class called MyFS:

from fs.test import FSTestCases

class TestMyFS(FSTestCases):

    def make_fs(self):
        # Return an instance of your FS object here
        return MyFS()

You may also want to override some of the methods in the test suite for more targeted testing:

class fs.test.FSTestCases

Basic FS tests.

assert_bytes(path, contents)

Assert a file contains the given bytes.

Parameters:
  • path (str) – A path on the filesystem.
  • contents (bytes) – Bytes to compare.
assert_exists(path)

Assert a path exists.

Parameters:path (str) – A path on the filesystem.
assert_isdir(path)

Assert a path is a directory.

Parameters:path (str) – A path on the filesystem.
assert_isfile(path)

Assert a path is a file.

Parameters:path (str) – A path on the filesystem.
assert_not_exists(path)

Assert a path does not exist.

Parameters:path (str) – A path on the filesystem.
assert_text(path, contents)

Assert a file contains the given text.

Parameters:
  • path (str) – A path on the filesystem.
  • contents (str) – Text to compare.
destroy_fs(fs)

Destroy a FS instance.

Parameters:fs (FS) – A filesystem instance previously opened by make_fs.
make_fs()

Return an FS instance.

test_geturl_purpose()

Check an unknown purpose raises a NoURL error.

test_validatepath()

Check validatepath returns an absolute path.

Essential Methods

The following methods MUST be implemented in a PyFilesystem interface.

Non - Essential Methods

The following methods MAY be implemented in a PyFilesystem interface.

These methods have a default implementation in the base class, but may be overridden if you can supply a more optimal version.

Exactly which methods you should implement depends on how and where the data is stored. For network filesystems, a good candidate to implement, is the scandir method which would otherwise call a combination of listdir and getinfo for each file.

In the general case, it is a good idea to look at how these methods are implemented in FS, and only write a custom version if it would be more efficient than the default.

Helper Methods

These methods SHOULD NOT be implemented.

Implementing these is highly unlikely to be worthwhile.