vera_ingest package

PDF parsing, chunking, and conversion to .vera archives.

vera_ingest

Source ingestion, chunking, and conversion adapters for VERA.

Modules:

• convert –

Classes:

• Chunk –

  Text segment produced by the extraction chunker.

• ParsedBlock –

  Layout block extracted from a PDF page.

• ParsedPage –

  Single page extracted from a PDF.

Functions:

• batch_convert –

  Convert PDFs from a directory scan or an explicit path list.

• parse_pdf –
• parse_pdf_structured –

  Parse a PDF into pages plus structured blocks.

• chunk_pages –
• build_chunks_from_blocks –

  Heading-aware chunking over structured blocks.

• detect_heading –
• figures –

  Return figure attachments produced during ingest.

• figures_for –

  Return figure attachments linked to a query result.

• get_page –

  Return ingest-provided viewer data for one page.

• get_blocks –

  Return ingest-provided layout blocks.

• get_chunk_regions –

  Return ingest-provided highlight regions for a chunk.

• regions_for –

  Return ingest-provided highlight regions for a query result.

• get_source_document –

  Return the attachment identified as the archive's source document.

• export_source_document –

  Write the source attachment to disk and return its path.

Chunk dataclass

Chunk(text: str, page_start: int, page_end: int, heading_path: str, token_count: int, block_ids: list[str] = list())

Text segment produced by the extraction chunker.

Attributes:

• text (str) –

  Chunk content.

• page_start (int) –

  First page number spanned by the chunk.

• page_end (int) –

  Last page number spanned by the chunk.

• heading_path (str) –

  Heading breadcrumb at chunk start.

• token_count (int) –

  Approximate token count.

• block_ids (list[str]) –

  Source layout block identifiers.

text instance-attribute

text: str

page_start instance-attribute

page_start: int

page_end instance-attribute

page_end: int

heading_path instance-attribute

heading_path: str

token_count instance-attribute

token_count: int

block_ids class-attribute instance-attribute

block_ids: list[str] = field(default_factory=list)

ParsedBlock dataclass

ParsedBlock(page_number: int, block_type: str, text: str, bbox: tuple[float, float, float, float] | None = None, heading_level: int | None = None, image_bytes: bytes | None = None, image_ext: str = '')

Layout block extracted from a PDF page.

Attributes:

• page_number (int) –

  1-based page number.

• block_type (str) –

  One of heading, paragraph, image, caption, or table.

• text (str) –

  Block text content.

• bbox (tuple[float, float, float, float] | None) –

  Bounding box (x0, y0, x1, y1) in page points, when available.

• heading_level (int | None) –

  Heading depth for heading blocks.

• image_bytes (bytes | None) –

  Raw image bytes for image blocks.

• image_ext (str) –

  Image format extension for image blocks.

page_number instance-attribute

page_number: int

block_type instance-attribute

block_type: str

text instance-attribute

text: str

bbox class-attribute instance-attribute

bbox: tuple[float, float, float, float] | None = None

heading_level class-attribute instance-attribute

heading_level: int | None = None

image_bytes class-attribute instance-attribute

image_bytes: bytes | None = None

image_ext class-attribute instance-attribute

image_ext: str = ''

ParsedPage dataclass

ParsedPage(page_number: int, width: float | None, height: float | None, text: str)

Single page extracted from a PDF.

Attributes:

• page_number (int) –

  1-based page number.

• width (float | None) –

  Page width in points, when available.

• height (float | None) –

  Page height in points, when available.

• text (str) –

  Concatenated native text for the page.

page_number instance-attribute

page_number: int

width instance-attribute

width: float | None

height instance-attribute

height: float | None

text instance-attribute

text: str

batch_convert

batch_convert(directory: str | None = None, *, paths: Sequence[str] | None = None, recursive: bool = False, overwrite: bool = False, model: str = 'hashing', parser: str = 'pymupdf', chunk_size: int = 500, overlap: int = 75, store_original: bool = True, ocr_mode: str = 'auto', ocr_language: str = 'eng', ocr_dpi: int = 300, progress: Callable[[int, int, str], None] | None = None, cancel: Any | None = None) -> dict[str, Any]

Convert PDFs from a directory scan or an explicit path list.

Parameters:

• directory (str | None, default: None ) –

  Root directory to scan for PDFs when paths is omitted.

• paths (Sequence[str] | None, default: None ) –

  Explicit PDF file paths to convert. When set, directory discovery is skipped and recursive is ignored.

• recursive (bool, default: False ) –

  When True, scan subdirectories (directory mode only).

• overwrite (bool, default: False ) –

  When True, replace existing .vera outputs.

• model (str, default: 'hashing' ) –

  Embedding model name passed to :func:convert.

• parser (str, default: 'pymupdf' ) –

  PDF parser backend passed to :func:convert.

• chunk_size (int, default: 500 ) –

  Target chunk size passed to :func:convert.

• overlap (int, default: 75 ) –

  Chunk overlap passed to :func:convert.

• store_original (bool, default: True ) –

  Whether to embed originals passed to :func:convert.

• ocr_mode (str, default: 'auto' ) –

  OCR mode passed to :func:convert.

• ocr_language (str, default: 'eng' ) –

  OCR language passed to :func:convert.

• ocr_dpi (int, default: 300 ) –

  OCR DPI passed to :func:convert.

• progress (Callable[[int, int, str], None] | None, default: None ) –

  Optional (current, total, filename) callback.

• cancel (Any | None, default: None ) –

  Optional cancellation token.

Returns:

• dict[str, Any] –

  A report dict with converted, skipped, failed, and related

• dict[str, Any] –

  fields. directory is the scan root, or the common parent of

• dict[str, Any] –

  paths.

Raises:

• NotADirectoryError –

  When directory is not a directory.

• FileNotFoundError –

  When a path in paths is missing.

• ValueError –

  When neither directory nor paths is usable.

parse_pdf

parse_pdf(path: str) -> list[ParsedPage]

parse_pdf_structured

parse_pdf_structured(path: str, *, ocr_mode: str = 'auto', ocr_language: str = 'eng', ocr_dpi: int = 300, diagnostics: dict[str, Any] | None = None, cancel: Any | None = None) -> tuple[list[ParsedPage], list[ParsedBlock]]

Parse a PDF into pages plus structured blocks.

Detects headings via font size/weight relative to body text, keeps paragraphs as text blocks, and captures embedded images as image blocks.

chunk_pages

chunk_pages(pages, chunk_size: int = 500, overlap: int = 75) -> list[Chunk]

build_chunks_from_blocks

build_chunks_from_blocks(blocks: list[tuple[str, ParsedBlock]], chunk_size: int = 500, overlap: int = 75) -> list[Chunk]

Heading-aware chunking over structured blocks.

detect_heading

detect_heading(text: str, current: str) -> str

figures

figures(document: VeraDocument, page_start: int | None = None, page_end: int | None = None, include_data: bool = False) -> list[dict[str, Any]]

Return figure attachments produced during ingest.

figures_for

figures_for(document: VeraDocument, result: QueryResult, include_data: bool = False) -> list[dict[str, Any]]

Return figure attachments linked to a query result.

get_page

get_page(document: VeraDocument, page_number: int) -> dict[str, Any] | None

Return ingest-provided viewer data for one page.

get_blocks

get_blocks(document: VeraDocument, page_number: int | None = None) -> list[dict[str, Any]]

Return ingest-provided layout blocks.

get_chunk_regions

get_chunk_regions(document: VeraDocument, chunk_id: str) -> list[dict[str, Any]]

Return ingest-provided highlight regions for a chunk.

regions_for

regions_for(document: VeraDocument, result: QueryResult) -> list[dict[str, Any]]

Return ingest-provided highlight regions for a query result.

get_source_document

get_source_document(document: VeraDocument) -> AttachmentRecord

Return the attachment identified as the archive's source document.

export_source_document

export_source_document(document: VeraDocument, path: str | PathLike[str] | None = None) -> str

Write the source attachment to disk and return its path.

Conversion writes through VeraDocument. Viewer helpers interpret ingest-produced attachments and metadata. See the conversion guide and figures and regions.

convert

Functions:

• convert –

  Convert a PDF into a validated .vera archive.

• batch_convert –

  Convert PDFs from a directory scan or an explicit path list.

convert

convert(input_path: str, output_path: str, *, model: str = 'hashing', parser: str = 'pymupdf', chunk_size: int = 500, overlap: int = 75, store_original: bool = True, ocr_mode: str = 'auto', ocr_language: str = 'eng', ocr_dpi: int = 300, cancel: Any | None = None) -> str

Convert a PDF into a validated .vera archive.

Parses the PDF, chunks extracted text, embeds chunks, and writes the result through :class:~vera.document.VeraDocument. The archive is validated before the temporary file is published atomically.

Parameters:

• input_path (str) –

  Source PDF path.

• output_path (str) –

  Destination .vera path.

• model (str, default: 'hashing' ) –

  Embedding model name (default "hashing").

• parser (str, default: 'pymupdf' ) –

  PDF parser backend (currently only "pymupdf").

• chunk_size (int, default: 500 ) –

  Target chunk size in characters.

• overlap (int, default: 75 ) –

  Character overlap between consecutive chunks.

• store_original (bool, default: True ) –

  When True, embed the original PDF as an attachment.

• ocr_mode (str, default: 'auto' ) –

  "auto" (default), "off", or "force".

• ocr_language (str, default: 'eng' ) –

  Tesseract language code (default "eng").

• ocr_dpi (int, default: 300 ) –

  Rasterization DPI for OCR.

• cancel (Any | None, default: None ) –

  Optional cancellation token with raise_if_cancelled().

Returns:

• str –

  The output_path string.

Raises:

• FileNotFoundError –

  When input_path does not exist.

• ValueError –

  When no searchable text is extracted or the parser is unsupported.

batch_convert

batch_convert(directory: str | None = None, *, paths: Sequence[str] | None = None, recursive: bool = False, overwrite: bool = False, model: str = 'hashing', parser: str = 'pymupdf', chunk_size: int = 500, overlap: int = 75, store_original: bool = True, ocr_mode: str = 'auto', ocr_language: str = 'eng', ocr_dpi: int = 300, progress: Callable[[int, int, str], None] | None = None, cancel: Any | None = None) -> dict[str, Any]

Convert PDFs from a directory scan or an explicit path list.

Parameters:

• directory (str | None, default: None ) –

  Root directory to scan for PDFs when paths is omitted.

• paths (Sequence[str] | None, default: None ) –

  Explicit PDF file paths to convert. When set, directory discovery is skipped and recursive is ignored.

• recursive (bool, default: False ) –

  When True, scan subdirectories (directory mode only).

• overwrite (bool, default: False ) –

  When True, replace existing .vera outputs.

• model (str, default: 'hashing' ) –

  Embedding model name passed to :func:convert.

• parser (str, default: 'pymupdf' ) –

  PDF parser backend passed to :func:convert.

• chunk_size (int, default: 500 ) –

  Target chunk size passed to :func:convert.

• overlap (int, default: 75 ) –

  Chunk overlap passed to :func:convert.

• store_original (bool, default: True ) –

  Whether to embed originals passed to :func:convert.

• ocr_mode (str, default: 'auto' ) –

  OCR mode passed to :func:convert.

• ocr_language (str, default: 'eng' ) –

  OCR language passed to :func:convert.

• ocr_dpi (int, default: 300 ) –

  OCR DPI passed to :func:convert.

• progress (Callable[[int, int, str], None] | None, default: None ) –

  Optional (current, total, filename) callback.

• cancel (Any | None, default: None ) –

  Optional cancellation token.

Returns:

• dict[str, Any] –

  A report dict with converted, skipped, failed, and related

• dict[str, Any] –

  fields. directory is the scan root, or the common parent of

• dict[str, Any] –

  paths.

Raises:

• NotADirectoryError –

  When directory is not a directory.

• FileNotFoundError –

  When a path in paths is missing.

• ValueError –

  When neither directory nor paths is usable.

batch_convert

batch_convert(directory: str | None = None, *, paths: Sequence[str] | None = None, recursive: bool = False, overwrite: bool = False, model: str = 'hashing', parser: str = 'pymupdf', chunk_size: int = 500, overlap: int = 75, store_original: bool = True, ocr_mode: str = 'auto', ocr_language: str = 'eng', ocr_dpi: int = 300, progress: Callable[[int, int, str], None] | None = None, cancel: Any | None = None) -> dict[str, Any]

Convert PDFs from a directory scan or an explicit path list.

Parameters:

• directory (str | None, default: None ) –

  Root directory to scan for PDFs when paths is omitted.

• paths (Sequence[str] | None, default: None ) –

  Explicit PDF file paths to convert. When set, directory discovery is skipped and recursive is ignored.

• recursive (bool, default: False ) –

  When True, scan subdirectories (directory mode only).

• overwrite (bool, default: False ) –

  When True, replace existing .vera outputs.

• model (str, default: 'hashing' ) –

  Embedding model name passed to :func:convert.

• parser (str, default: 'pymupdf' ) –

  PDF parser backend passed to :func:convert.

• chunk_size (int, default: 500 ) –

  Target chunk size passed to :func:convert.

• overlap (int, default: 75 ) –

  Chunk overlap passed to :func:convert.

• store_original (bool, default: True ) –

  Whether to embed originals passed to :func:convert.

• ocr_mode (str, default: 'auto' ) –

  OCR mode passed to :func:convert.

• ocr_language (str, default: 'eng' ) –

  OCR language passed to :func:convert.

• ocr_dpi (int, default: 300 ) –

  OCR DPI passed to :func:convert.

• progress (Callable[[int, int, str], None] | None, default: None ) –

  Optional (current, total, filename) callback.

• cancel (Any | None, default: None ) –

  Optional cancellation token.

Returns:

• dict[str, Any] –

  A report dict with converted, skipped, failed, and related

• dict[str, Any] –

  fields. directory is the scan root, or the common parent of

• dict[str, Any] –

  paths.

Raises:

• NotADirectoryError –

  When directory is not a directory.

• FileNotFoundError –

  When a path in paths is missing.

• ValueError –

  When neither directory nor paths is usable.