Skillversitas

Structured Arrays

A structured array gives every element a small schema - named fields, each with its own dtype - so one array can hold records like (name, age, score) instead of a homogeneous list of numbers. It is the array-level answer to "a table of rows with different-typed columns," and the bridge to reading fixed-width or legacy data files and to passing structured records into C. The trade: field access is convenient, but arithmetic and most of the vectorized toolkit expect homogeneous arrays, so structured arrays are a data-carrying format, not a computation format.

Defining a Structured dtype

import numpy as np

dt = np.dtype([("name", "U10"), ("age", "i4"), ("score", "f8")])
print(dt)                # [('name', 'U10'), ('age', '<i4'), ('score', '<f8')]
print(dt.names)          # ('name', 'age', 'score')
print(dt.itemsize)       # bytes per record

rec = np.array([("ada", 30, 91.5), ("bo", 25, 88.0)], dtype=dt)
print(rec.shape)         # (2,) - two records
print(rec.dtype)         # the structured dtype

The dtype is a list of (name, type) pairs; U10 is a fixed-width 10-character string, i4 a 32-bit int, f8 a float64. Each element is one packed record of itemsize bytes, laid out contiguously - the same memory model as any array, just with a richer element. Building the array from a list of tuples is the straightforward path; the same dtype also describes how to interpret a raw byte buffer (below).

Field Access

print(rec["age"])          # [30 25] - the whole field, as an array
print(rec["name"])         # ['ada' 'bo']
print(rec[0]["score"])     # 91.5 - one record, one field

# a field view is itself an array - vectorized ops work on it
ages = rec["age"]
print(ages.max(), ages.mean())

# assign into a field
rec["score"] = rec["score"] * 1.0
print(rec[0])              # the record, all fields

Selecting a field - rec["age"] - returns a plain homogeneous array over that field, which is where the vectorized toolkit comes back: max, mean, comparisons, all of it. The workflow for structured data is therefore split: carry records as structured, compute on per-field arrays. Multi-field conditions ((rec["age"] > 20) & (rec["score"] > 90)) combine the per-field boolean arrays with the usual bitwise operators.

Record Arrays: Objects with Attributes

rec_arr = rec.view(np.recarray)
print(rec_arr[0].name)         # attribute-style access
print(rec_arr[rec_arr.age > 26])   # boolean filter returns records

# record arrays add attribute access; the memory is identical
print(rec_arr.base is rec)     # a view over the same buffer

A recarray is the same buffer viewed with attribute-style access (rec_arr[0].name instead of rec_arr[0]["name"]) - ergonomics, not a different storage. It is the form that loadtxt/genfromtxt return when you ask for named columns, and it makes record-centric code read naturally. Under the hood it is still a structured array: same dtype, same memory, same per-field array views.

Interpreting Raw Buffers

# a raw sequence of (int32, float64) pairs, e.g. from a C file
raw = np.frombuffer(
    bytes([0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 20, 8, 0, 0, 0, 2,
           0, 0, 0, 0, 0, 0, 40, 8, 0, 0, 0, 3]),
    dtype=np.dtype([("n", "i4"), ("v", "f8")]))
print(raw)                     # the records, decoded from bytes
print(raw["n"], raw["v"])      # fields extracted

# the reverse: pack Python data into a compact binary record array
packed = np.array([(1, 10.5), (2, 20.25)], dtype=np.dtype([("n", "i4"), ("v", "f8")]))
print(packed.tobytes())        # the exact on-disk layout

The killer feature: a structured dtype is a decoder for a raw byte stream. Give NumPy the field layout and it splits a flat buffer into records - the standard move for reading fixed-width or binary C data without a line-by-line parser. tobytes() does the reverse, the exact layout a C program would see. This is the "schema as dtype" idea: the dtype is not just metadata, it is the format definition.

When to Use One

  • Use structured arrays to carry records between Python and C, to decode fixed-width or binary dumps, or when you want one compact buffer for a homogeneous set of records.
  • Use pandas instead when the table is the working model - labels, missing values, joins, and heterogeneous columns are pandas's home turf.
  • Avoid dtype=object as a substitute: an object array of tuples keeps per-element Python objects and loses the contiguous layout; a structured array packs the same data into one buffer with typed fields.

Summary

Model records with a structured dtype - a list of (name, type) pairs - and build the array from tuples or by viewing a raw buffer with that dtype. Access fields as whole arrays (rec["age"]) to bring the vectorized toolkit back, combine per-field masks with bitwise operators for multi-field filters, and use recarray for attribute-style ergonomics over the same memory. The structured dtype is the format definition: it decodes byte streams into records and repacks them for C. Use it for data-carrying and interop; use pandas when the table itself is the model.