mikehostetler
ReqLLM - Composable LLM client built on Req
Hey everyone!
I’m excited to share ReqLLM - a new approach to LLM interactions in Elixir that I’ve been working on. After building agent systems with various LLM clients, I kept running into the same frustrations: they either lacked Elixir’s composability principles or didn’t integrate well with existing HTTP pipelines.
Why Another LLM Client?
While building out Jido features, I needed a lower-level API for making LLM requests. ReqLLM is built on Req, with each Provider built as a Req plugin that handles provider-specific wire formats. It’s designed to compose naturally with your existing Req-based applications.
Core Architecture
Plugin-Based Providers: Each LLM provider (Anthropic, OpenAI, Google, etc.) is a composable Req plugin.
Typed Data Structures: Every interaction uses proper structs (Context, Message, StreamChunk, Tool, ContentPart) that implement Jason.Encoder - no more wrestling with nested maps.
Two Client Layers: High-level helpers for quick wins (generate_text/3, stream_text/3, generate_object/4, etc) plus low-level Req plugin access when you need full control.
Built-in Observability: Usage and cost tracking on every response based on metadata sync’d from https://models.dev
Quick Example
# Simple approach
ReqLLM.put_key(:anthropic_api_key, "sk-ant-...")
{:ok, text} = ReqLLM.generate_text!("anthropic:claude-3-sonnet", "Hello")
# Tool calling with structured responses
weather_tool = ReqLLM.tool(
name: "get_weather",
description: "Get weather for a location",
parameter_schema: [location: [type: :string, required: true]],
callback: fn args -> {:ok, "Sunny, 72°F"} end
)
{:ok, response} = ReqLLM.generate_text(
"anthropic:claude-3-sonnet",
"What's the weather in Paris?",
tools: [weather_tool]
)
Current Status
ReqLLM 1.0-rc is available on Hex with 45+ providers and 665+ models (auto-synced from models.dev). I’m using it in production for Jido agent systems and it’s been solid. Planning to add Ollama/LocalAI support and enhanced streaming soon.
Resources
- Hex Package: req_llm | Hex
- Documentation: ReqLLM v1.5.1 — Documentation
- Getting Started Guide: Getting Started — ReqLLM v1.5.1
- GitHub:
I’d love to hear your thoughts and see what you build with it! The plugin architecture makes it pretty straightforward to add new providers if there’s one you need.
Most Liked
mikehostetler
ReqLLM 1.0.0-rc.7 Released
I’ve published the 1.0.0-rc.7 release of ReqLLM with Elixir 1.19 support on Hex.
It’s been a journey to build this package. LLM Providers are all over the place - but with this release, I’ve been able to successfully prove that the high level API now supports 136 models via a custom fixture testing system I developed.
Several companies and projects have already begun integrating and battle testing this package. With this release, I’m now focused on getting to a stable 1.0 release.
What’s New
Test Coverage: 136 models validated across 10 providers with fixture-based testing against 10 scenarios:
- Core: basic generation, streaming, token limits, usage metrics
- Capabilities: multi-tool calling, tool round-trip, tool avoidance, object generation (streaming/non-streaming), reasoning tokens
Providers: Added Amazon Bedrock (streaming, multi-model, AWS SigV4 auth) and Z.AI (GLM-4.5/4.6, 204K context).
Architecture: Simplified tool call handling with ReqLLM.ToolCall helpers, provider normalize_model_id/1 callback, improved Context API with append/2 and prepend/2.
Examples: Full agent implementation and 16+ runnable scripts covering embeddings, multimodal analysis, tool calling, object generation, reasoning tokens, context reuse.
Validation: Run mix mc to see coverage status. Use REQ_LLM_FIXTURES_MODE=record to test against live APIs.
Links
josevalim
This is fantastic! Elixir does come with many of the building blocks for rolling your own agentic system but the “talking to different LLMs” is definitely one of the more time consuming bits. I also like that you have a purely data driven API for defining tools.
My only request would be to make the bit where providers automatically fetch keys optional (either opt-in or opt-out), as turning that on/off would be important for folks writing services where you BYOK (like Tidewave).
Also have you implemented the APIs for dealing with reasoning tokens? OpenAI, Anthropic, OpenRouter, etc all have different APIs for them, which is also annoying to deal with.
PS: You forgot to make Credo a dev-only dependency. 
mikehostetler
Thanks!
jido_ai will be moving to use ReqLLM very shortly - and we will be dropping support for Langchain and Instructor.
API’s will be maintained - so you should be good to start and not worry too much about things changing.
LLM API’s are pretty dynamic, so I can’t promise 100% - but we’re doing our best to build a solid foundation for the Elixir community.
mikehostetler
PS: You forgot to make Credo a dev-only dependency.
Thanks for the catch! So tough to pull the trigger on a release
My only request would be to make the bit where providers automatically fetch keys optional (either opt-in or opt-out), as turning that on/off would be important for folks writing services where you BYOK (like Tidewave).
Absolutely, I had similar ideas - implemented it twice - didn’t feel right yet. Agree this is necessary.
Also have you implemented the APIs for dealing with reasoning tokens?
Yes, this is currently unreleased - I hit another weird snag and pulled back - but it will be there by the formal 1.0.
mikehostetler
Great question - and I agree - I’ll make a PR
This is only due to my eyes glazing over after sprinting on this for a week!!!
Popular in Announcing
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Build Elixir applications with speed and confidence.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Practical resources that improve the lives of professional developers.
Enabling companies to succeed by building software people love.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"