Tar Utilities Python Module

Utilities and classes for unbundling and archiving a tar file.

class pacifica.ingest.tarutils.FileIngester(hashtype, hashcode, file_id)[source]

Class to ingest a single file from a tar file into the file archives.

__init__(hashtype, hashcode, file_id)[source]

Constructor for FileIngester class.

file_id = 0
fileobj = None
hashval = None
read(size)[source]

Read wrapper for requests that calculates the hashcode inline.

recorded_hash = ''
server = ''
upload_file_in_file(info, tar)[source]

Upload a file from inside a tar file.

validate_hash()[source]

Validate that the calculated hash matches the hash uploaded in the tar file.

exception pacifica.ingest.tarutils.HashValidationException[source]

Class to capture hashsum validation failures.

class pacifica.ingest.tarutils.MetaParser[source]

Class used to hold and search metadata.

__init__()[source]

Constructor.

clean_metadata()[source]

clean /data from filepaths.

file_count = -999
file_obj_count(meta_list)[source]

Count the file objects in metadata and keep the count.

files = {}
get_fname(file_id)[source]

Get the file name from the file ID.

get_hash(file_id)[source]

Return the hash string for a file name.

get_subdir(file_id)[source]

Get the sub directory element from the file ID.

load_meta(tar, job_id)[source]

Load the metadata from a tar file into searchable structures.

meta = None
meta_str = ''
post_metadata()[source]

Upload metadata to server.

read_meta(metafile, job_id)[source]

Read the metadata from metafile and assume it’s good.

start_id = -999
transaction_id = -999
class pacifica.ingest.tarutils.TarIngester(tar, meta)[source]

Class to read a tar file and upload it to the metadata and file archives.

__init__(tar, meta)[source]

Constructor for TarIngester class.

ingest()[source]

Ingest a tar file into the file archive.

meta = None
tar = None
pacifica.ingest.tarutils.file_count(tar)[source]

Retrieve the file count for a tar file.

Does not count metadata.txt as that is not uploaded to the file archive

pacifica.ingest.tarutils.get_clipped(fname)[source]

Return a file path with the data separator removed.

pacifica.ingest.tarutils.open_tar(fpath)[source]

Seek to the location of fpath, returns a file stream pointer and file size.

pacifica.ingest.tarutils.patch_files(meta_obj)[source]

Patch the files in the archive interface.