Most peoplefine-tune models.We build them.
Tantra AI Labs builds open-weight language models from scratch — every layer, every weight, every token — plus the reasoning datasets and production infrastructure to run them.
Open-weight, from scratch.
AstraGPT-7B
7B decoder-only LLM. Custom BPE tokenizer, GQA (28Q/4KV), RoPE, SwiGLU. Native <think> chain-of-thought. 32K context.
AstraGPTCoder-7B
Coding-specialized sibling. LoRA fine-tuned for code generation with think-tag reasoning. Runs in ~6GB VRAM at 4-bit.
Every layer, hand-built.
No AutoModel. No copy-paste. AstraGPT's transformer, BPE tokenizer, and supervised fine-tuning loop were all written from first principles in PyTorch.
Token IDs │ ▼ Token Embedding [64k → 4096] ▼ ×28 AstraGPT Blocks ┌─────────────────────────┐ │ RMSNorm (pre-norm) │ │ → GQA (28Q / 4KV) │ │ RoPE θ=1M · KV-cache │ │ → residual │ │ RMSNorm │ │ → SwiGLU FFN │ │ → residual │ └─────────────────────────┘ ▼ Final RMSNorm ▼ LM Head [4096 → 64k] ▼ Logits → next token
The anatomy of AstraGPT-7B.
28 transformer blocks, grouped-query attention with 4× fewer KV heads, rotary embeddings at θ=1M and a SwiGLU feed-forward — every component implemented and understood from first principles.
Token IDs │ ▼ Token Embedding [64k → 4096] ▼ ×28 AstraGPT Blocks ┌─────────────────────────┐ │ RMSNorm (pre-norm) │ │ → GQA (28Q / 4KV) │ │ RoPE θ=1M · KV-cache │ │ → residual │ │ RMSNorm │ │ → SwiGLU FFN │ │ → residual │ └─────────────────────────┘ ▼ Final RMSNorm ▼ LM Head [4096 → 64k] ▼ Logits → next token
Reasoning, at scale.
AstraGPT Reasoning Dataset
Instruction–reasoning–answer triples with explicit cognitive traces. Apache-2.0.
Download dataset →High-Reasoning Dataset v1
Multi-AI knowledge-distilled. Deep CoT, math proofs, production code, edge cases. MIT.
Download dataset →From training rigs to production gateways.
AstraGPT Training
Bare-metal VPS, 2× RTX 4090 (48GB). Custom PyTorch loop — BF16, grad accumulation, cosine LR, W&B logging.
Astra AI Gateway
Production LLM gateway. 40+ models, multi-channel routing, fallbacks, HTTP/2 zero-buffer streaming, cost analytics.
vLLM Serving
Self-hosted inference with prefix caching. Optimized connection pooling cuts per-request latency ~40%.
Try the gateway — one OpenAI-compatible endpoint, 40+ models
curl https://api.aisubscription.shop/v1/chat/completionsBuilt by one engineer.
Tantra AI Labs is founded by Aditya Wakharkar — an AI infrastructure and cybersecurity engineer from Nanded, India. From training 7B models on a private GPU rig to shipping production gateways and ethical-hacking tools, the lab operates on one belief: open weights, built honestly.