Tool profile
openvinotoolkit/model_server
github.com
A scalable C++ inference server for serving generative AI and classic deep learning models optimized with OpenVINO.
What it does
openvinotoolkit/model_server
OpenVINO Model Server is a production-grade C++ inference server that exposes ML models over standard network APIs, including an OpenAI-compatible API for LLMs, embeddings, image generation and audio, plus KServe APIs for classic models. It is aimed at teams deploying generative AI, agentic and classic deep learning workloads on Intel CPU, GPU and NPU hardware, running via Docker, bare metal, Kubernetes or Windows. It is offered as open source from the OpenVINO toolkit.
PricingOpen source
RatingN/A
Bookmarks0
Tags
Categories
A concise summary has not been added yet.
FAQ
Frequently asked questions
- What is openvinotoolkit/model_server?
- OpenVINO Model Server is a production-grade C++ inference server that exposes ML models over standard network APIs, including an OpenAI-compatible API for LLMs, embeddings, image generation and audio, plus KServe APIs for classic models. It is aimed at teams deploying generative AI, agentic and classic deep learning workloads on Intel CPU, GPU and NPU hardware, running via Docker, bare metal, Kubernetes or Windows. It is offered as open source from the OpenVINO toolkit.
- How much does openvinotoolkit/model_server cost?
- openvinotoolkit/model_server uses a Open source pricing model. Check the official website for current plans.
- What is openvinotoolkit/model_server used for?
- openvinotoolkit/model_server is listed on TolsAI under: Open Source.
- Where can I use openvinotoolkit/model_server?
- openvinotoolkit/model_server is available on its official website, github.com.
Explore
More in Open Source
An open-source agentic harness for a self-improving AI agent that plans and executes tasks you describe.
A vectorless, reasoning-based RAG engine that indexes documents as trees and retrieves with LLM reasoning.
A PyTorch-like Python library to build and auto-optimize LLM workflows such as chatbots, RAG and agents.
A modular web UI for generating images, video and audio with AI models like Stable Diffusion, Flux and Wan.
A step-by-step guide to designing AI systems built on LLMs, RAG, and AI agents.
Open-source AI pipeline engine for building, debugging and deploying LLM workflows and agent orchestration from your IDE.
Discussion
Discussion
Log in to join the discussion
No comments yet. Start the discussion.