makeprov.paths

Functions

`download_file`(url, dest_path, *[, ...])	Download a file via HTTP GET using urllib, streaming to disk with progress logging.
`urlopen`(url[, data, timeout, cafile, ...])	Open the URL url, which can be either a string or a Request object.

Classes

`CachedDownload`(url, cache_path, *[, ...])	Input wrapper that lazily downloads and records source metadata.
`InDir`(*paths)	Input directory that tracks files declared within it.
`InPath`(*paths)	Marker for input paths where `"-"` maps to stdin.
`OutDir`(*paths)	Output directory that tracks files declared within it.
`OutPath`(*paths)	Marker for output paths where `"-"` maps to stdout.
`Path`(args, *kwargs)	PurePath subclass that can make system calls.
`ProvPath`(*paths)	Filesystem path with first-class support for stream placeholders.
`Request`(url[, data, headers, ...])

Exceptions

`HTTPError`(url, code, msg, hdrs, fp)	Raised when HTTP error occurs, but also acts like non-error return
`URLError`(reason[, filename])

class makeprov.paths.CachedDownload(url, cache_path, *, headers=None, transform='prov:wasDerivedFrom')

Bases: InPath

Input wrapper that lazily downloads and records source metadata.

open(mode='r', *args, **kwargs)

Open the path for reading, honoring stdin streams.

Parameters:

mode (str) – File mode; defaults to read.
*args – Additional positional arguments forwarded to Path.open.
**kwargs – Additional keyword arguments forwarded to Path.open.

Returns:

Readable file-like object.

Return type:

IOBase

Examples

InPath("example.txt").open().read()

class makeprov.paths.InDir(*paths: str | bytes | ProvPath)

Bases: InPath

Input directory that tracks files declared within it.

The file() helper produces InPath instances rooted in the directory while recording them for provenance collection.

property children: tuple[InPath, ...]

file(name)

Return type:: InPath

class makeprov.paths.InPath(*paths: str | bytes | ProvPath)

Bases: ProvPath

Marker for input paths where "-" maps to stdin.

Examples

from makeprov.paths import InPath

src = InPath("data/input.txt")
with src.open() as handle:
    _ = handle.read()

open(mode='r', *args, **kwargs)

Open the path for reading, honoring stdin streams.

Parameters:

mode (str) – File mode; defaults to read.
*args – Additional positional arguments forwarded to Path.open.
**kwargs – Additional keyword arguments forwarded to Path.open.

Returns:

Readable file-like object.

Return type:

IOBase

Examples

InPath("example.txt").open().read()

class makeprov.paths.OutDir(*paths: str | bytes | ProvPath)

Bases: OutPath

Output directory that tracks files declared within it.

The file() helper produces OutPath instances rooted in the directory while recording them for provenance collection.

property children: tuple[OutPath, ...]

file(name)

Return type:: OutPath

class makeprov.paths.OutPath(*paths: str | bytes | ProvPath)

Bases: ProvPath

Marker for output paths where "-" maps to stdout.

Examples

from makeprov.paths import OutPath

dest = OutPath("data/output.txt")
dest.write_text("generated")

as_inpath()

Convert an output marker into an input marker.

Returns:: A new instance pointing to the same filesystem location.
Return type:: InPath
Raises:: ValueError – If the current path represents a stream.

Examples

from makeprov.paths import OutPath

OutPath("data/output.txt").as_inpath()

open(mode='w', *args, **kwargs)

Open the path for writing, creating parent directories when needed.

Parameters:

mode (str) – File mode; defaults to write.
*args – Additional positional arguments forwarded to Path.open.
**kwargs – Additional keyword arguments forwarded to Path.open.

Returns:

Writable file-like object.

Return type:

IOBase

Examples

with OutPath("output.txt").open("w") as handle:
    handle.write("hello")

class makeprov.paths.ProvPath(*paths: str | bytes | 'ProvPath')

Bases: PosixPath

Filesystem path with first-class support for stream placeholders.

Hyphen ("-") paths are treated as stdin/stdout streams but still behave like pathlib.Path instances for all other operations.

Examples

from makeprov.paths import ProvPath

p = ProvPath("-")
assert p.is_stream

property is_stream: bool

open(mode='r', *args, **kwargs)

Open the path while respecting stream semantics.

Parameters:

mode (str) – File open mode, passed through to Path.open when not operating on a stream.
*args – Additional positional arguments forwarded to Path.open.
**kwargs – Additional keyword arguments forwarded to Path.open.

Returns:

A file-like object for the requested mode.

Return type:

IOBase

Examples

from makeprov.paths import ProvPath

with ProvPath("output.txt").open("w") as handle:
    handle.write("hello")

property stream_name: str | None

makeprov.paths.download_file(url, dest_path, *, chunk_size=8192, headers=None)

Download a file via HTTP GET using urllib, streaming to disk with progress logging.

Return type:: None