MIDREAL
Menu

Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium

https://github.com/RunanywhereAI/trnrt (github.com) OVERHYPED

54 points | 4d ago | Discuss on Hacker News | Back to Radar

Comments

bookernath 4d ago on HN
Tokens per minute is a strange metric, I usually see tok/s. A per-second figure would be more accessible in comparing this to alternatives.
badmonster0 4d ago on HN
how does this compare with sglang
shubham2802 4d ago on HN
They don't have support running trn2 afaik especially no glm 5.3 flash . they did release an update -5 days back but again its for trn3 nothing trn2
kevo1ution 4d ago on HN
curious how the token/$ on trainuim compare to nvidia chips and amd. is the future "chipless" inference?
sanchitmonga22 (author) 4d ago on HN
We are able to prove out better token/$ for Trn2 vs H200!

There is a reason why Anthropic and OpenAI are choosing to run their inference on Trainium, it's highly gated right now, but we're trying to add support in the open and for open models!

coredog64 4d ago on HN
Good on these folks for doing something useful with Trainium. I'm always surprised that the Tranium ecosystem is as poor as it is. Amazon's market cap is nearly $3T and they can't throw a couple L6s and a modest token budget at this? Making Trainium work as seamlessly as possible as an Nvidia substitute is a clear flywheel play.
sanchitmonga22 (author) 4d ago on HN
facts! Trn2 and Trn3, is equivalent of Hopper and Blackwell, but the software support is poor!

We want to build the best engine in the open, welcoming any contributions!

aiexplore369 2d ago on HN
That 'd be great , there will better and competitive options and edge as well !

Comments are loaded live from Hacker News and are not stored by Mid or Real.