SneppX-ALG Documentation

SneppX-ALG is a cognitive processing system implementing neural architecture search, hierarchical state spaces, mixture of experts, and a full S0–S9 security layer. Written in C11 + C++20 with CUDA acceleration, targeting x86-64 with Python bindings. This documentation covers installation, architecture, security, algorithms, and API usage.

1.2 Installation

pip install sneppx-alg

git clone https://github.com/ammar49-cyber/sneppx-alg.gitcd sneppx-alg mkdir build && cd build cmake .. -DSNEPPX_BUILD_PYTHON=ONcmake --build .

-DSNEPPX_BUILD_CUDA=ON       # Enable CUDA kernels-DSNEPPX_BUILD_TESTS=ON      # Build test suite-DSNEPPX_USE_LTO=ON          # Link-time optimization-DSNEPPX_BUILD_VULKAN=ON     # Vulkan compute backend

docker pull ghcr.io/ammar49-cyber/sneppx-alg:cpu-latest docker pull ghcr.io/ammar49-cyber/sneppx-alg:cuda-latest

1.3 Quick Start

Create a tensor, build a transformer model, and run inference in a few lines.

import sneppx as spx# Create a tensorx = spx.tensor([[1, 2, 3], [4, 5, 6]], dtype=spx.float32)print(x.shape)  # (2, 3)# Build a modelmodel = spx.nn.Transformer( vocab_size=32000, dim=4096, n_layers=32, n_heads=32, dim_head=128, )# Run inferenceoutput = model(x)print(output.shape)  # (2, 3, 32000)

1.4 CLI Tools

SneppX-ALG ships with four CLI tools for evaluation, quantization, RLHF, and serving.

sneppx-eval --model llama-7b --benchmarks mmlu,gsm8k

sneppx-quantize --input model.gguf --output model-q4.gguf --mode int4

sneppx-rlhf --model llama-7b --method dpo --dataset anthropic-hh

sneppx-serve --model llama-70b --quantize int4 --port 8080

sneppx-hub upload org/model@v1.0.0 ./weights --task text-generation

1.5 Model Hub & Sharing

The SNEPPX Model Hub (`sneppx-hub`) provides a centralized model registry and sharing platform supporting search, versioning, organization management, leaderboards, and C/Python client integrations.

from model_hub import Hub hub = Hub(url="http://localhost:8100", api_key="...") hub.upload("org/model", "./weights", version="v1.0.0", task="text-generation") hub.download("org/model@v1.0.0", "./cache")

2.1 L0 — Kernel

The kernel layer provides the foundational compute substrate: a multi-dimensional tensor engine with automatic differentiation, a secure memory pool, and a thread pool for parallel execution. All higher layers depend on this layer.

// C11 tensor APISNEPPXTensor* t = SNEPPX_tensor_create( (size_t[]){4, 256, 256}, 3, SNEPPX_F32); SNEPPX_tensor_random_uniform(t, -1.0, 1.0); SNEPPXTensor* r = SNEPPX_matmul(t, t);

SNEPPX_tape_begin(); SNEPPXTensor* loss = SNEPPX_cross_entropy(output, target); SNEPPX_backward(loss);float grad = SNEPPX_gradient(param); SNEPPX_tape_end();

SNEPPXMemoryPool* pool = SNEPPX_memory_pool_create(1024 * 1024 * 1024);void* block = SNEPPX_pool_alloc(pool, 4096); SNEPPX_pool_free(pool, block); SNEPPX_memory_pool_destroy(pool);

2.2 L1 — Neural Architecture

Layer 1 implements five core algorithms: ARC (adversarial robustness), SER (mixture of experts), HSS (hierarchical state space), NPE (neural program execution), and FM (federated memory). Each algorithm is differentiable and hardware-accelerated. See the Algorithms section for detailed documentation.

from sneppx.algorithms import ARC, SER, HSS, NPE, FM arc = ARC(epsilon=0.1, steps=40) ser = SER(num_experts=8, top_k=2) hss = HSS(state_dim=128, num_levels=4) npe = NPE(instruction_set="standard") fm = FM(strategy="ring")

2.3 L2 — Model Zoo

Pre-configured model architectures for LLaMA 2/3, Mistral, Qwen2, and DeepSeek V2. Each model config specifies the full architecture: number of layers, attention heads, hidden dimensions, RoPE parameters, and MoE configuration. Models can be loaded from HuggingFace-compatible weights.

from sneppx.models import LLaMA, Mistral, Qwen2, DeepSeekV2# Load LLaMA 3 8Bmodel = LLaMA.from_pretrained("meta-llama/Meta-Llama-3-8B")# Load Mistral 7Bmodel = Mistral.from_pretrained("mistralai/Mistral-7B-v0.3")# Load Qwen2 72Bmodel = Qwen2.from_pretrained("Qwen/Qwen2-72B")# Load DeepSeek V2 Litemodel = DeepSeekV2.from_pretrained("deepseek-ai/DeepSeek-V2-Lite")

2.4 L3 — Training

The training layer orchestrates distributed training across multiple GPUs and nodes. Supports mixed precision (FP16/BF16), gradient checkpointing, gradient accumulation, and dynamic loss scaling. Integrates with ZeRO-1/2/3, tensor/pipeline/expert parallelism.

from sneppx import training as T trainer = T.Trainer( model=model, optimizer="adamw", lr=3e-4, mixed_precision="bf16", gradient_checkpointing=True, gradient_accumulation_steps=4, zero_stage=2, ) trainer.fit(dataset, epochs=3)

2.5 L4 — Quantization

The quantization layer reduces model precision to minimize memory footprint and accelerate inference. Supports symmetric/asymmetric INT8, packed INT4, FP8 (E4M3/E5M2), AWQ, and GPTQ. All quantized formats are compatible with the serving engine. See the Quantization section for detailed documentation.

from sneppx.quantization import quantize_int8, quantize_int4, QuantMode model_int8 = quantize_int8(model, mode=QuantMode.SYMMETRIC) model_int4 = quantize_int4(model, pack=True)

2.6 L5 — Serving

Production inference server with continuous batching, paged KV cache, and quantized inference. Implements a REST API compatible with the OpenAI API specification. Supports dynamic batching, request priority queuing, and model warm-up.

from sneppx.serving import InferenceServer server = InferenceServer( model=model, quantize="int4", max_batch_size=64, max_seq_len=8192, ) server.start(port=8080)# curl http://localhost:8080/v1/completions -d '{"prompt": "Hello", "max_tokens": 100}'

2.7 L6 — Security

The full S0–S9 security stack is integrated at this layer: hardware security, memory hardening, obfuscation, cryptography, network security, AI safety, key management, secure updates, formal verification, and penetration testing. Every primitive is implemented in C with no external dependencies. See the Security section for details.

from sneppx.security import SecureAllocator, CryptoContext ctx = CryptoContext() ctx.keygen(algorithm="kyber-1024") ciphertext, shared_secret = ctx.encapsulate(public_key)

2.8 L7 — Hardware Backends

Hardware backends provide accelerated computation on CUDA (Hopper/Ampere), CPU (AVX-512, AVX2, NEON), ROCm, and Vulkan. Each backend implements the same operator interface, enabling transparent switching. The CUDA backend includes Flash Attention v2/v3, fused GEMM, and NCCL collectives.

from sneppx.backends import set_backend, get_backend_info set_backend("cuda")          # Switch to CUDAset_backend("cpu")            # Switch to CPU (AVX-512)set_backend("vulkan")         # Switch to Vulkan computeprint(get_backend_info())   # Current backend details

3.1 S0 — Hardware Security

Hardware-level cryptographic acceleration using CPU instruction set extensions. AES-NI for symmetric encryption, SHA-NI for hashing, and secure enclave support for trusted execution environments. Constant-time operations prevent side-channel attacks.

// Hardware-accelerated AES-GCMSNEPPX_AES256GCM_ctx ctx; SNEPPX_aes256gcm_init(&ctx, key, nonce); SNEPPX_aes256gcm_encrypt(&ctx, plaintext, ciphertext, aad);// Uses AES-NI when available, falls back to portable

3.2 S1 — Memory Hardening

Secure memory allocator with guard pages, stack canaries, ASLR, memory quarantine, W^X enforcement, and freelist integrity verification. All allocations are zeroed on free. Constant-time memory comparison prevents timing attacks.

// Secure allocation with guard pagesSNEPPXSecureAllocator* alloc = SNEPPX_secure_allocator_create();void* buf = SNEPPX_secure_alloc(alloc, 4096);// buf is preceded and followed by guard pagesSNEPPX_secure_free(alloc, buf);// Memory is zeroed before return to pool

3.3 S2 — Obfuscation

Code obfuscation engine implementing control flow flattening, instruction substitution, string encryption, opaque predicates, and virtual machine obfuscation. Anti-debug measures detect ptrace, TLS callbacks, and SEH handlers. White-box AES protects cryptographic keys in untrusted environments.

// Obfuscate a function at build timeSNEPPX_OBFUSCATE_STARTvoid secure_function(uint8_t* data, size_t len) // Compiler applies CFG flattening + substSNEPPX_OBFUSCATE_END

3.4 S3 — Cryptography

Full cryptographic suite with no external dependencies. Includes Kyber KEM (post-quantum key encapsulation), Dilithium and SPHINCS+ signatures, AES-256-GCM, ChaCha20-Poly1305, Ed25519, X25519, SHA-3, BLAKE3, Argon2id, HKDF, and Hash_DRBG. All primitives are constant-time.

// Kyber KEM encapsulationSNEPPXKyberKeypair kp; SNEPPX_kyber_keygen(&kp, SNEPPX_KYBER_1024); SNEPPXKyberCiphertext ct;uint8_t shared_secret[32]; SNEPPX_kyber_encapsulate(&ct, shared_secret, kp.public); SNEPPX_kyber_decapsulate(shared_secret, &ct, kp.secret);

3.5 S4 — Network Security

Network security layer implementing TLS 1.3, Noise protocol (NK/XX/IK), QUIC multiplexing, mTLS, OCSP stapling, Certificate Transparency, DNS-over-HTTPS, and WireGuard. Includes a NIDS, rate limiter, port knocking daemon, and gRPC auth interceptor.

// Noise protocol handshake (XX pattern)SNEPPXNoiseSession session; SNEPPX_noise_init(&session, SNEPPX_NOISE_XX); SNEPPX_noise_handshake(&session, transport);// Encrypted transport established

3.6 S5 — AI Safety

AI sanitization layer: semantic prompt injection detection across 12 languages, jailbreak detection, model inversion defense, data extraction prevention, training data sanitization, watermarking, adversarial smoothing, bias measurement, and a policy DSL for prompt restrictions.

from sneppx.security.safety import PromptFilter filter = PromptFilter(rules=["no-jailbreaks", "no-injections"]) result = filter.check("Ignore previous instructions and...")print(result.flagged, result.score)  # True, 0.97

3.7 S6 — Key Vault

Secure key storage with HSM-backed key store, Shamir secret sharing, key ceremony workflows, automatic rotation, web dashboard, threat visualization, policy DSL, compliance reporting, and a tamper-evident audit chain.

from sneppx.security.vault import KeyVault vault = KeyVault(backend="hsm") vault.generate_key("model-weights", algorithm="aes-256") vault.rotate("model-weights") audit_log = vault.get_audit_log()

3.8 S7 — Secure Updates

TUF-compliant update system with multi-role signing, bsdiff delta updates, A/B partition management, manifest verification, TPM attestation, canary rollout, offline bundles, and dependency resolution.

from sneppx.security.updates import UpdateManager updater = UpdateManager() manifest = updater.verify_update("sneppx-v1.1.0.bin") updater.apply(manifest, strategy="canary")

3.9 S8 — Formal Verification

TLA+ parser, LTL model checking with counterexample generation, symbolic execution, loop invariant inference, data flow taint analysis, and Lean 4 proof export. Verifies safety properties of the architecture.

from sneppx.security.verification import ModelChecker mc = ModelChecker(spec="sneppx.tla") property = "[] (state = ready => response_valid)"result = mc.check_ltl(property)print(result.satisfied, result.counterexample)

3.10 S9 — Penetration Testing

Automated penetration testing framework: CVE vulnerability scanner, network fuzzer, API scanner, dependency checker, static analysis, supply chain audit, cryptographic test suite, red team simulation, compliance auto-checker, and a self-audit framework that validates the security implementation.

from sneppx.security.pentest import SelfAuditor auditor = SelfAuditor() report = auditor.run_full_audit()print(report.summary, report.vulnerabilities)# Network fuzzerfuzzer = auditor.network_fuzzer(target="localhost:8080") fuzzer.run(duration=300)

4.1 ARC — Adversarial Robustness

Adversarial Robustness Classifier implements FGSM and PGD attacks for adversarial training, gradient obfuscation via randomized smoothing, and certified robustness bounds. All operations are differentiable.

from sneppx.algorithms import ARC arc = ARC( epsilon=0.1,          # Perturbation budgetsteps=40,             # PGD iterationsstep_size=0.01,       # Per-step magnitudeattack="pgd",        # "fgsm" or "pgd"smoothing=0.5,       # Randomized smoothing sigma) adversarial = arc.attack(model, x, y) robust_model = arc.adversarial_train(model, dataset)

4.2 FM — Federated Memory

Federated Memory implements ring and butterfly all-reduce for distributed gradient averaging, gradient compression via top-k sparsification and quantization, and federated averaging (FedAvg) with secure aggregation.

from sneppx.algorithms import FM fm = FM( strategy="ring",         # "ring" or "butterfly"compression=0.01,      # Top-k fraction to keepsecure_aggregation=True, num_clients=8, ) global_model = fm.federated_averaging(local_models)

4.3 HSS — Hierarchical State Space

Implements selective scan (Mamba/S6), S4 kernels, HiPPO initialization, and hierarchical softmax. Multi-resolution sequence processing with linear-time parallel scan that replaces quadratic attention for long-range dependencies.

from sneppx.algorithms import HSS hss = HSS( state_dim=128,           # Latent state dimensionnum_levels=4,           # Hierarchy depthscan_type="selective",   # "s4" or "selective"hippo_order=64,         # HiPPO projection order) output = hss(x)  # (batch, seq, dim)

4.4 NPE — Neural Program Execution

Differentiable program synthesis and execution engine. Defines 70+ instructions (arithmetic, memory, control flow, I/O) in a register-based VM. Programs are differentiable instruction sequences learned through gradient descent.

from sneppx.algorithms import NPE npe = NPE( instruction_set="standard", num_registers=16, max_program_length=256, vm_optimize=True, )# Compile a programprogram = npe.compile(""" LOAD R1, input_0 LOAD R2, input_1 ADD  R3, R1, R2 STORE output, R3 """) result = npe.execute(program, inputs={x, y})

4.5 SER — Sparse Mixture of Experts

Sparse Mixture of Experts with top-k gating, expert dispatch, load balancing loss, and expert all-to-all communication. Supports dynamic expert routing with auxiliary losses for balanced utilization.

from sneppx.algorithms import SER ser = SER( num_experts=8,            # Total expertstop_k=2,                # Active experts per tokenexpert_dim=2048,         # Hidden dimension per expertload_balancing=True,     # Auxiliary loss coefficientz_loss=0.001,            # Router z-loss for stability) output = ser(x)  # (batch, seq, dim)

5.1 LLaMA 2/3

Full LLaMA architecture support including LLaMA 2 (7B, 13B, 70B) and LLaMA 3 (8B, 70B). Implements RoPE, SwiGLU activation, grouped-query attention, and RMSNorm.

from sneppx.models import LLaMA# Available configsmodel_7b = LLaMA.from_pretrained("meta-llama/Llama-2-7b") model_13b = LLaMA.from_pretrained("meta-llama/Llama-2-13b") model_70b = LLaMA.from_pretrained("meta-llama/Llama-2-70b") model_3_8b = LLaMA.from_pretrained("meta-llama/Meta-Llama-3-8B") model_3_70b = LLaMA.from_pretrained("meta-llama/Meta-Llama-3-70B")# Manual configconfig = LLaMA.config_llama3_8b() model = LLaMA(config)

5.2 Mistral

Mistral 7B with sliding window attention, rolling buffer KV cache, and pre-fill chunking. Includes a weight converter from HuggingFace safetensors format.

from sneppx.models import Mistral model = Mistral.from_pretrained("mistralai/Mistral-7B-v0.3")# Convert weights from HuggingFacefrom sneppx.models.mistral import convert_weights convert_weights("path/to/hf/weights", "path/to/output.gguf")

5.3 Qwen2

Qwen2 7B and 72B with SwiGLU, RoPE, and QKV bias. Supports the full Qwen2 architecture including the 72B MoE variant.

from sneppx.models import Qwen2 model_7b = Qwen2.from_pretrained("Qwen/Qwen2-7B") model_72b = Qwen2.from_pretrained("Qwen/Qwen2-72B")

5.4 DeepSeek V2

DeepSeek V2 architecture with Multi-head Latent Attention (MLA), fine-grained MoE, and KV cache compression. Supports Lite and Full configs.

from sneppx.models import DeepSeekV2 model_lite = DeepSeekV2.from_pretrained("deepseek-ai/DeepSeek-V2-Lite") model_full = DeepSeekV2.from_pretrained("deepseek-ai/DeepSeek-V2")

6.1 INT8 Symmetric/Asymmetric

INT8 quantization with symmetric (per-tensor or per-channel) and asymmetric (per-tensor) modes. Uses min-max or percentile calibration. Reduces model size by 4x with minimal accuracy loss.

from sneppx.quantization import quantize_int8, QuantMode model_int8 = quantize_int8( model, mode=QuantMode.SYMMETRIC, per_channel=True, calibration="percentile", percentile=99.9, )

6.2 INT4 Packed Quantization

Packed INT4 quantization stores two 4-bit values per byte. Supports Q4_0, Q4_1, Q4_K_M, and Q4_K_S formats (GGML-compatible). Group size is configurable.

from sneppx.quantization import quantize_int4 model_int4 = quantize_int4( model, format="q4_k_m",      # GGML formatgroup_size=32, pack=True, )

6.3 FP8 E4M3 / E5M2

FP8 quantization with two formats: E4M3 (higher precision, 4 exponent + 3 mantissa bits) for weights and E5M2 (wider dynamic range, 5 exponent + 2 mantissa bits) for gradients. Hardware-accelerated on Hopper GPUs.

from sneppx.quantization import quantize_fp8 model_fp8 = quantize_fp8( model, weight_format="e4m3",    # Weights: E4M3grad_format="e5m2",     # Gradients: E5M2)

6.4 AWQ — Activation-aware Weight Quantization

AWQ identifies salient weight channels by analyzing activation distributions and applies per-channel scaling before quantization. Preserves accuracy better than naive INT4 at the same bit width.

from sneppx.quantization import AWQ awq = AWQ( bits=4, group_size=128, zero_point=True, n_samples=128,          # Calibration samples) model_awq = awq.quantize(model, calib_dataset)

6.5 GPTQ — Post-training Quantization

GPTQ applies optimal brain quantization with Hessian-based weight updates. Processes weights column-by-column, updating remaining weights to compensate for quantization error. Supports INT4 and INT3.

from sneppx.quantization import GPTQ gptq = GPTQ( bits=4, group_size=128, damp_percent=0.01, desc_act=True,         # Descending activation order) model_gptq = gptq.quantize(model, calib_dataset)

6.6 QuantizedLinear Layer

A drop-in replacement for nn.Linear that stores weights in quantized format and dequantizes on-the-fly during forward pass. Supports all quantization modes and is compatible with the autodiff engine.

from sneppx.quantization import QuantizedLinear layer = QuantizedLinear( in_features=4096, out_features=4096, bits=4, group_size=32, mode="awq", ) output = layer(x)  // Auto-dequantizes on forward

7.1 ZeRO-1/2/3

ZeRO (Zero Redundancy Optimizer) partitions optimizer states (ZeRO-1), gradients (ZeRO-2), and parameters (ZeRO-3) across devices. Enables training of models with trillions of parameters by distributing memory. Communicates via NCCL all-reduce and all-gather collectives.

from sneppx.distributed import ZeRO zero = ZeRO( stage=2,                # ZeRO stage (1, 2, or 3)world_size=8, offload=True,          # Offload to CPU/NVMeoffload_device="cpu", ) optimizer = zero.wrap_optimizer(model, adamw)

7.2 Tensor Parallel

Tensor parallelism splits linear layers and attention heads across devices using row/column partitioning. Implements all-reduce for synchronization after partitioned computation. Communication overhead is O(d_model) per transformer layer.

from sneppx.distributed import TensorParallel tp = TensorParallel( tp_size=4, parallel_mode="column_row",   # Column + Row split) model_tp = tp.wrap_model(model)

7.3 Pipeline Parallel

Pipeline parallelism partitions transformer layers across devices using the 1F1B (one-forward-one-backward) schedule. Minimizes bubble overhead by interleaving micro-batches. Each device holds a contiguous subset of layers.

from sneppx.distributed import PipelineParallel pp = PipelineParallel( pp_size=4, num_micro_batches=8,               # Micro-batches for 1F1Bschedule="1f1b", ) model_pp = pp.wrap_model(model, layer_partition=list(range(8, 32)))

7.4 Expert Parallel

Expert parallelism distributes MoE experts across devices. Uses all-to-all communication to dispatch tokens to the devices hosting their assigned experts. Minimizes idle compute by balancing expert load.

from sneppx.distributed import ExpertParallel ep = ExpertParallel( ep_size=8, num_experts=64, dispatch_mode="all_to_all", load_balancing=True, ) model_ep = ep.wrap_moe_model(model)

7.5 DDP — Distributed Data Parallel

Bucket-based gradient all-reduce using NCCL. Groups parameters into buckets by size and overlaps gradient computation with communication. Supports gradient compression and asynchronous all-reduce.

from sneppx.distributed import DDP ddp = DDP( model=model, bucket_size=250 * 1024 * 1024,   # 250MB bucketsgradient_compression=True, compression_ratio=0.01, )

8.1 Flash Attention v2/v3

Memory-efficient exact attention using online softmax and tiling. Flash Attention v2 minimizes non-matrix-multiply FLOPs and improves parallelism. v3 adds GQA (grouped query attention) support and paged KV cache for continuous batching. Achieves O(N²) compute with O(N) memory.

from sneppx.cuda import flash_attention output = flash_attention( query, key, value, causal=True, softmax_scale=0.088,    # 1/sqrt(d_head)version=3,              # v2 or v3)  # (batch, heads, seq, dim)

8.2 GEMM Kernels

CUDA tensor-core matrix multiplication with 128x128 tiling, warp-level matrix fragments, and automatic kernel selection. Supports FP16, BF16, and INT8 inputs with FP32 accumulation. Implements split-k for reduced register pressure.

from sneppx.cuda import gemm c = gemm( a, b, transa=False, transb=False, alpha=1.0, beta=0.0, dtype="bf16", )

8.3 Fused Optimizers

GPU-fused implementations of AdamW, Lion, and LAMB optimizers. Fuses the gradient computation, update step, and weight decay into a single kernel launch. Reduces memory traffic and improves training throughput by 10–30%.

from sneppx.cuda.optim import FusedAdamW, FusedLAMB optimizer = FusedAdamW( model.parameters(), lr=3e-4, betas=(0.9, 0.95), weight_decay=0.1, fused=True, )

8.4 Memory Pool

Pre-allocated GPU memory pool with block-based allocation and stream-ordered deallocation. Eliminates cudaMalloc/cudaFree overhead during training. Supports memory pooling across streams and event-based synchronization.

from sneppx.cuda.memory import CUDAMemoryPool pool = CUDAMemoryPool( size=16 * 1024 * 1024 * 1024,   # 16GBblock_size=4096, ) tensor_gpu = pool.alloc((4096, 4096), dtype=spx.float16) pool.free(tensor_gpu)

9.1 tensor — Tensor Class

Core tensor data structure supporting N-dimensional arrays with hardware-accelerated operations. Dtypes: float32, float64, int8, int16, int32, int64, uint8, bfloat16.

x = spx.tensor([[1, 2], [3, 4]], dtype=spx.float32, device="cuda")print(x.shape, x.dtype, x.device)  # (2, 2) float32 cuda:0y = x + x                           # Element-wise addz = x @ x.T                         # Matrix multiplys = z.sum()                         # Reductions.backward()                        # Autodiffprint(x.grad)                        # Gradient tensor

9.2 nn — Neural Network Modules

Neural network module library with Module, Linear, Embedding, Dropout, LayerNorm, and Transformer building blocks. All modules support autodiff and parameter registration.

import sneppx as spxclass MyModel(spx.nn.Module):def __init__(self):super().__init__() self.layers = spx.nn.Sequential( spx.nn.Linear(768, 4096), spx.nn.GELU(), spx.nn.Linear(4096, 768), spx.nn.Dropout(0.1), )def forward(self, x):return self.layers(x) model = MyModel() output = model(x) loss = output.mean() loss.backward()

9.3 optim — Optimizers

Optimizers for gradient-based training: SGD, AdamW, Lion, and LAMB. All support weight decay, gradient clipping, and learning rate scheduling. LAMB is optimized for large-batch training.

from sneppx.optim import AdamW, Lion, CosineAnnealingLR optimizer = AdamW( model.parameters(), lr=3e-4, betas=(0.9, 0.95), weight_decay=0.1, ) scheduler = CosineAnnealingLR(optimizer, T_max=1000, eta_min=1e-6)# Training loopfor batch in dataloader: loss = model(batch) optimizer.zero_grad() loss.backward() optimizer.clip_grad_norm(1.0) optimizer.step() scheduler.step()

9.4 data — Data Loading

Data loading pipeline with Dataset, DataLoader, Tokenizer, and streaming support. DataLoader supports multi-worker prefetching, shuffle, and batch collation. Tokenizer supports BPE, Unigram, and WordPiece models.

from sneppx.data import Dataset, DataLoader, Tokenizer# Custom datasetclass MyDataset(Dataset):def __getitem__(self, idx):return self.data[idx], self.labels[idx] dataset = MyDataset.from_jsonl("data.jsonl") loader = DataLoader( dataset, batch_size=32, shuffle=True, num_workers=4, )# Tokenizertokenizer = Tokenizer.from_pretrained("llama-3") tokens = tokenizer.encode("Hello, world!")

9.5 quantization — Quantization API

High-level quantization interface with QuantMode enum and helper functions for INT8, INT4, FP8, AWQ, and GPTQ.

from sneppx.quantization import ( QuantMode, quantize_int8, quantize_int4, quantize_fp8, AWQ, GPTQ, QuantizedLinear, )# QuantMode optionsprint(QuantMode.SYMMETRIC)    # [-127, 127] rangeprint(QuantMode.ASYMMETRIC)   # [0, 255] range with zero-pointmodel_q = quantize_int8(model, mode=QuantMode.SYMMETRIC, per_channel=True)

9.6 lora — LoRA Adapters

Low-Rank Adaptation (LoRA) and QLoRA for parameter-efficient fine-tuning. Includes DPO (Direct Preference Optimization) and GRPO (Group Relative Policy Optimization) trainers for alignment.

from sneppx.lora import LoRALinear, QLoRALinear, DPOTrainer# Apply LoRA to a modelconfig = LoRAConfig(r=16, alpha=32, dropout=0.1) model = LoRALinear.wrap(model, config)# DPO trainingtrainer = DPOTrainer( model=model, ref_model=reference_model, beta=0.1, ) trainer.train(dataset)

9.7 eval_harness — Evaluation Harness

Standardized evaluation framework supporting MMLU, GSM8K, HellaSwag, ARC, and custom tasks. Supports few-shot prompting, log-probability scoring, and generation-based evaluation.

from sneppx.eval_harness import EvalHarness, ExactMatchTask, MMLU harness = EvalHarness(model, tokenizer)# Run MMLU (5-shot)mmlu = MMLU(subjects=all, n_shot=5) results = harness.evaluate(mmlu)print(results.accuracy)  # 0.724# Custom tasktask = ExactMatchTask( name="my-task", dataset=dataset, metric="f1", ) result = harness.evaluate(task)

9.8 serving — Inference Server

Production inference server with OpenAI-compatible REST API, continuous batching, paged KV cache, and quantized inference. Supports streaming, prefix caching, and dynamic batching.

from sneppx.serving import InferenceServer server = InferenceServer( model_path="models/llama-70b-q4.gguf", backend="cuda", max_batch_size=64, max_seq_len=32768, kv_cache_size=16,           # GBcontinuous_batch=True, ) server.start(port=8080, host="0.0.0.0")# Python clientfrom sneppx.serving.client import InferenceClient client = InferenceClient("http://localhost:8080") response = client.complete("The capital of France is", max_tokens=50)print(response.text)