arxiv.canonical.core module

Core interfaces for the canonical record.

class arxiv.canonical.core.ICanonicalSource(*args, **kwargs)[source]

Bases: typing_extensions.Protocol

Interface for source services, used to dereference URIs.

can_resolve(uri)[source]

Indicate whether or not the implementation can resolve an URI.

Parameters

uri (D.URI) –

Returns

Return type

bool

Return type

bool

load(key)[source]

Make an IO that waits to load from the record until it is read().

Parameters

key (D.URI) –

Returns

Yields bytes when read. This may be a lazy IO object, so that reading is deferred until the latest possible time.

Return type

IO

Return type

IO[bytes]

class arxiv.canonical.core.ICanonicalStorage(*args, **kwargs)[source]

Bases: arxiv.canonical.core.ICanonicalSource, arxiv.canonical.core.IManifestStorage, typing_extensions.Protocol

Interface for services that store the canonical record.

list_subkeys(key)[source]

List all of the subkeys (direct descendants) of key in the record.

Parameters

key (URI) –

Returns

Items are the relative names of the descendants of key. For filesystem-based storage, this may be equivalent to os.listdir.

Return type

list

Return type

List[str]

load_entry(key)[source]

Load a bitstream entry.

Parameters

key (URI) – Key that identifies the bitsream in the record.

Returns

  • RecordStream – The bitstream resource.

  • str – Checksum of the bitstream (URL-safe base64-encoded md5 hash).ß

Return type

Tuple[RecordStream, str]

store_entry(ri)[source]

Store a bitstream entry in the record.

This method MUST decompress the content of the entry if it is gzipped (as is sometimes the case in the classic system) and update the CanonicalFile (ri.record.stream.domain).

Parameters

ri (IStorableEntry) – A storable bitstream.

Return type

None

class arxiv.canonical.core.IEventStream(*args, **kwargs)[source]

Bases: typing_extensions.Protocol

Interface for the canonical event stream.

emit(event)[source]

Emit an Event on the stream.

Parameters

event (Event) –

Return type

None

listen(on_event)[source]

Listen for :class:`Event`s on the stream.

Parameters

on_event (callable) – This object will be called for every event that is received. It should accept a single argument, an Event, and is expected to return None.

Return type

None

class arxiv.canonical.core.IManifestStorage(*args, **kwargs)[source]

Bases: typing_extensions.Protocol

Manifest protocol.

This could conceivably be stored separately from the canonical record content, so it is defined separately.

load_manifest(key)[source]

Load an integrity manifest.

Parameters

key (Key) – Key used to identify manifest in storage.

Returns

Return type

Manifest

Return type

Manifest

store_manifest(key, manifest)[source]

Store an integrity manifest.

Parameters

  • key (Key) – Key used to identify manifest in storage.

  • manifest (Manifest) – The manifest record to store.

Return type

None

class arxiv.canonical.core.IPreservationAPI(*args, **kwargs)[source]

Bases: typing_extensions.Protocol

Interface for the daily preservation record API.

add_events(*events)[source]

Add new events to the preservation record.

Return type

None

load_package(date)[source]

Load the preservation package for a particular date.

Return type

IntegrityEntry

class arxiv.canonical.core.IRegisterAPI(*args, **kwargs)[source]

Bases: typing_extensions.Protocol

Interface for the canonical register API.

add_events(*events)[source]

Add new events to the register.

Return type

None

load_eprint(identifier)[source]

Load an EPrint from the record.

Return type

EPrint

load_event(identifier)[source]

Load an Event by identifier.

Return type

Event

load_events(selector)[source]

Load all :class:`.Event`s for a day, month, or year.

Return type

Tuple[Iterable[Event], int]

load_history(identifier)[source]

Load the event history of an EPrint.

Return type

Iterable[EventSummary]

load_listing(date, shard='listing')[source]

Load a Listing for a particulate date.

Return type

Listing

load_version(identifier)[source]

Load an e-print Version from the record.

Return type

Version

class arxiv.canonical.core.IStorableEntry(*args, **kwargs)[source]

Bases: typing_extensions.Protocol

Minimal interface for a bitstream interface that can be stored.

Services that implement ICanonicalStorage can assume that the attributes of this interface are available on objects passed for storing.

property checksum

URL-safe b64-encoded md5 hash.

Return type

str

name = None

Name of the entry.

property record

Reference to a RecordEntry.

Return type

RecordEntry[~_EDomain]

update_checksum()[source]

Update the integrity checksum for this entry.

Return type

None

arxiv.canonical.core.Month

alias of builtins.int

arxiv.canonical.core.Selector = typing.Union[int, typing.Tuple[int, int], datetime.date]

A selector can refer to a year, month, or a specific date.

arxiv.canonical.core.Year

alias of builtins.int

arxiv.canonical.core.YearMonth

alias of typing.Tuple

arxiv.canonical.core.dereference(sources, uri)[source]

Dereference an URI using a set of available sources.

Sources are checked one at a time for ability to resolve the URI. When one is found, the URI is loaded.

Parameters

  • sources (sequence) – Items are content sources that should conform to ICanonicalSource. They will be tried in the order provided.

  • uri (URI) – URI to dereference.

Returns

BytesIO object.

Return type

io

Raises

RuntimeError – Raised when the URI cannot be resolved.

Return type

IO[bytes]