Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A

Runs LLaMA with Extremely HIGH speed

LLM inference in Fortran

The bare metal in my basement

Portable LLM - A rust library for LLM inference

Wrapper for simplified use of Llama2 GGUF quantized models.

eLLM Infers Qwen3-480B on a CPU in Real Time

V-lang api wrapper for llm-inference chatllm.cpp

Simple large language model playground app

VB.NET api wrapper for llm-inference chatllm.cpp

C# api wrapper for llm-inference chatllm.cpp

Nim api-wrapper for llm-inference chatllm.cpp

Simple bot that transcribes Telegram voice messages. Powered by go-telegram-bot-api & whisper.cpp Go bindings.

🧠 A comprehensive toolkit for benchmarking, optimizing, and deploying local Large Language Models. Includes performance testing tools, optimized configurations for CPU/GPU/hybrid setups, and detailed guides to maximize LLM performance on your hardware.

rust api wrapper for llm-inference chatllm.cpp

lua api wrapper for llm-inference chatllm.cpp

gemma-2-2b-it int8 cpu inference in one file of pure C#

kotlin api wrapper for llm-inference chatllm.cpp

Java-port of qwen3.c

The Ark Project: Selecting the perfect AI model to reboot civilization from a 64GB USB drive. Comprehensive analysis of open-source LLMs under extreme constraints, with final recommendation: Meta Llama 3.1 70B Instruct (Q6_K GGUF). Includes interactive tools, detailed comparisons, and complete implementation guide for offline deployment.