Arrow Data Containers with Strict Schemas

Quivr provides Table containers for Arrow data with strict schema enforcement. The Tables are backed by Apache Arrow’s columnar memory format and designed for applications requiring type safety and immutable data structures.

Built on the Arrow memory model, Quivr Tables provide consistent performance characteristics and efficient memory usage for data processing applications.

Features

Type Safety

• Strict schema enforcement at runtime
• Static column definitions
• Immutable data structures

Performance

• Zero-copy data operations
• Consistent performance characteristics
• Reduced memory footprint compared to pandas DataFrames

Integration

• Arrow memory model for interoperability
• Streaming I/O support
• Parquet serialization/deserialization

Examples

import quivr as qv

# Define strict schema with type enforcement
schema = qv.Schema([
    qv.Field('object_id', qv.string()),
    qv.Field('ra', qv.float64()),
    qv.Field('dec', qv.float64()),
    qv.Field('magnitude', qv.float64()),
])

# Create immutable table with data validation
table = qv.Table(schema, {
    'object_id': ['2013 RR165', '2024 YR4'], 
    'ra': [123.456, 234.567],
    'dec': [45.678, -12.345],
    'magnitude': [18.2, 19.7]
})

# Fast, memory-efficient operations
filtered = table.filter(table['magnitude'] < 19.0)

Used for data processing in ADAM’s computational pipeline.