sem_cython12/README.md

sem_cython12

OpenMP-parallel numerical kernel library for Python. Pre-built Linux shared object included; no compilation required at install time.

Contents

• sem_cython12/sem_core12.cpython-312-x86_64-linux-gnu.so - compiled extension (Linux, CPython 3.12, x86_64).
• sem_cython12/wrapper.py - Python API.
• sem_cython12/__init__.py - package entry.

Requirements

• Linux x86_64.
• CPython 3.12.
• numpy >= 1.23 (see requirements.txt).
• A modern glibc + libgomp. Both ship with Ubuntu 20.04 LTS and later. No other system libraries needed.

The Windows / macOS binaries are not included in this distribution.

Install

git clone https://git.sevana.biz/vvs/sem_cython12.git
cd sem_cython12
pip install -r requirements.txt
# Make the package importable, either:
pip install -e .                      # if pyproject.toml/setup.py is added
# or just put the package on PYTHONPATH:
export PYTHONPATH=$PWD:$PYTHONPATH

Quick start

import numpy as np
from sem_cython12 import wrapper as cy

# Sanity check
assert cy.available(), "compiled extension did not load"
print("backend:", cy.backend())

# Thread count (defaults to ~50% of logical cores; set explicitly via
# either the SEM_NUM_THREADS env var or set_num_threads()):
cy.set_num_threads(8)
print("threads:", cy.get_num_threads())

# Example workload
rng = np.random.default_rng(0)
Q = rng.standard_normal((1000, 32))         # 1000 queries
M = rng.standard_normal((5000, 32))         # 5000 reference points

# For each query: max similarity to any reference, with kernel scale lam.
sim = cy.batch_max_similarity(Q, M, lam=1.0)
print(sim.shape, sim.dtype)                 # (1000,) float64

API reference

All functions accept either Python lists or numpy arrays; inputs are internally cast to contiguous float64. Outputs are numpy arrays.

Configuration

Function	Purpose
available() -> bool	True iff the compiled extension loaded
backend() -> str	'cython12' or 'python-fallback'
get_num_threads() -> int	Active OpenMP worker count
set_num_threads(n: int)	Set OpenMP worker count (n >= 1)

Distance / similarity

Function	Inputs	Output
batch_max_similarity(X_query, X_members, lam)	(Q, D), (M, D), lam > 0	(Q,) - per-query similarity score in [0, 1] against the closest member
concept_support_matrix(X_query, member_mats, lam)	(Q, D), list of (M_k, D), lam > 0	(Q, K) - one similarity column per member set
pairwise_distances(X)	(N, D)	(N, N) - symmetric distance matrix between rows
nn_distances(X)	(N, D)	(N,) - min positive distance per row; inf if none

Best-tradeoff filtering

Function	Inputs	Output
pareto_core_mask(S)	(N, k)	(N,) byte mask: rows that survive the multi-objective best-tradeoff filter
one_sided_mask(S)	(N, k)	(N, k) byte mask: rows contributing meaningfully on a single column only
non_redundant_witnesses(S)	(N, k)	int32 array of row indices contributing meaningfully across multiple columns

Vector reduction

Function	Inputs	Output
extend_frontier_kernel(cur_centers, cur_radii, new_emb, cur_arity)	(F, D), (F,), (A, D), int	(flat_centers (F*A, D), flat_radii (F*A,))

See the wrapper docstrings for exact semantics of each function.

Performance notes

Threads are configured globally per process; calling set_num_threads(n) updates the OpenMP team size for all subsequent calls. The default uses approximately 50% of the host's logical cores so other processes are not starved on shared machines.

For workloads dominated by pairwise_distances and pareto_core_mask, near-linear scaling up to ~8 threads is typical on commodity x86 hardware. batch_max_similarity is BLAS-friendly and benefits most from larger M (reference set) at fixed D.

Memory / threading model

• All arrays are processed in shared memory; no inter-process serialisation.
• Each routine releases the GIL during its inner loops, so calling it concurrently from Python threads is safe.
• The compiled extension links against the system OpenMP runtime (libgomp); avoid mixing with conda's intel-openmp in the same process if possible.

Diagnostics

backend() returns 'python-fallback' only when the .so failed to import (wrong architecture, glibc too old, missing libgomp). In that state, every numerical function raises RuntimeError; check available() before each batch to fail loudly rather than silently fall back.

Licence

The Software is licensed under the terms contained in the LICENSE file in this repository.

In short:

• Research and non-commercial use: granted free of charge under the conditions in section 2 of the LICENSE.
• Commercial use: requires a separate written commercial licence from the Licensor. Contact licensing@sevana.biz.
• No warranty: the Software is provided strictly "AS IS", without warranty of any kind. The Licensor's total aggregate liability is limited to zero.

Please read the LICENSE file in full before using the Software.

Support

Open an issue at https://git.sevana.biz/vvs/sem_cython12.