Performance Benchmarks

Last updated: December 2025 | Model: ConstantSense v1 Preview

Model Specifications

Specification	Value
Model Name	`constantsense-v1-preview`
Parameters	413,472
Architecture	φ-Spectral + BERT embeddings (frozen)
Quantization	FP32 (no quantization)
Model Size	1.6 MB
Hardware	CPU (Intel/AMD x86) or GPU (CUDA)

Accuracy Comparison

Sentiment Analysis Task (Binary Classification)

Model	Parameters	Accuracy	F1 Score
ConstantSense v1	413K	91.51%	0.91
BERT-base-uncased	110M	92-94%	0.93
DistilBERT	66M	90-92%	0.91
TinyBERT	14.5M	89-91%	0.90

Note: BERT baselines from Hugging Face models. Exact numbers vary by dataset and task.

Latency Benchmarks

Single request inference time (mean ± std)

Model	CPU (Intel i7)	GPU (NVIDIA T4)	Batch Size
ConstantSense v1	42 ± 5 ms	12 ± 2 ms	1
BERT-base (HF)	85 ± 10 ms	25 ± 3 ms	1
DistilBERT (HF)	45 ± 6 ms	15 ± 2 ms	1

Hardware: Intel Core i7-10700K @ 3.80GHz, NVIDIA Tesla T4 16GB
Input: 128 tokens average
Measured over 1000 requests

Efficiency Metrics

Metric	ConstantSense v1	vs BERT-base
Parameter Efficiency	413K params	266× smaller
Memory Footprint	1.6 MB	275× smaller
Training Time	6 minutes	50× faster
Inference Latency (CPU)	42 ms	2× faster
Cost per 1M tokens	~$0.10	10× cheaper

Throughput Testing

Requests per second (sustained load)

Configuration	RPS (Requests/sec)	Latency (p95)
Single CPU core	~24 RPS	52 ms
4 CPU cores	~85 RPS	48 ms
GPU (T4)	~140 RPS	15 ms

Benchmark Methodology

Dataset

IMDb sentiment analysis dataset
25,000 training samples
25,000 test samples
Binary classification (positive/negative)

Evaluation Protocol

5-fold cross-validation
Stratified sampling
Fixed random seed (42) for reproducibility
Metrics: Accuracy, F1, Precision, Recall

Hardware Configuration

CPU: Intel Core i7-10700K @ 3.80GHz (8 cores)
GPU: NVIDIA Tesla T4 16GB
RAM: 32GB DDR4
OS: Ubuntu 22.04 LTS

Software Versions

PyTorch: 2.1.0
Transformers: 4.35.0
CUDA: 12.1
Python: 3.9.18

Trade-offs & Limitations

What We Optimize For

✅ Parameter efficiency (266× reduction)
✅ Fast inference (2× faster than BERT)
✅ Low memory footprint (1.6MB)
✅ Training speed (6 minutes)

Current Limitations

⚠️ Accuracy: ~1-3% lower than BERT-base on some tasks
⚠️ Task-specific: Optimized for sentiment analysis
⚠️ Context length: 512 tokens max
⚠️ Preview status: API may change

Reproducibility

All benchmarks are reproducible. Model code and evaluation scripts available upon request for academic verification.

Try the API Request Benchmark Details