I have the question to the guys in inference world, I am building some stuff connected to the inference in my work, and I started wondering, what do actually big guys use in their inference work? Do you prefer standard (and what is standard lol) solutions like vLLM and LMCache, or you additionally add Mooncake, or use vllm native? Happy to have a discussion what is better and why as well, thanks!

Comments URL: https://news.ycombinator.com/item?id=50302109

Points: 1

# Comments: 0