sonic182
Llm_composer - Elixir library for building LLM-based applications
Hi everyone, at Doofinder we have been building llm_composer for some new apps, and we thought it could be useful to share it with the community.
llm_composer is an Elixir library that simplifies working with large language models (LLMs) like OpenAI’s GPT, OpenRouter, Ollama, AWS Bedrock, and Google (Gemini).
It provides a streamlined way to build and execute LLM-based applications or chatbots, with features such as:
- Multi-provider support (OpenAI, OpenRouter, Ollama, Bedrock, Google Gemini/Vertex AI).
- System prompts and message history management.
- Streaming responses.
- Function calls with auto-execution.
- Structured outputs with JSON schema validation.
- Built-in cost tracking (currently for OpenRouter).
- Easy extensibility for custom use cases.
A key feature is the provider router that handles failover automatically.
It will use one provider until it fails, then fall back to the next provider in the list, applying an exponential backoff strategy. This makes it resilient in production environments where provider APIs can become temporarily unavailable.
Under the hood, llm_composer uses Tesla as the HTTP client.
For production setups, especially when using streaming, it is recommended to run it with Finch for optimal performance.
More info and docs:
HexDocs: LlmComposer — llm_composer v0.13.0
GitHub:
Most Liked
Jskalc
sonic182
We started this lib about a year ago (we’d been using it internally even before that) and only now decided to share it here.
Both libraries aim at the same thing: wrapping LLM APIs behind a unified HTTP client.
The main differences are:
-
HTTP layer: we use Tesla, ReqLLM builds on Req.
-
Routing: llm_composer includes a router with fallback + retries.
-
API style: each library exposes a slightly different way to configure and work with “bots” or requests, so it depends which style you prefer.
mercyf
This looks great 
sonic182
LlmComposer v0.12.0 released
This version removes deprecated configuration options and adds built-in cost tracking for multiple providers.
Removed (breaking changes)
- Removed deprecated
:providerand:provider_optskeys. Use the:providerslist instead. - Removed global
:api_keyinSettings. Each provider should define its own key. - Dropped backward compatibility handling for these deprecated options.
Added
- Automatic cost tracking for OpenAI, Google, and OpenRouter:
- Fetches real-time pricing from provider APIs (
models.devand OpenRouter). - Adds a
CostInfostruct in responses with token usage and cost breakdowns. - Supports both automatic and manual pricing configuration.
- Fetches real-time pricing from provider APIs (
Changelog: https://github.com/doofinder/llm_composer/blob/master/CHANGELOG.md
Docs: https://hexdocs.pm/llm_composer/readme.html
Popular in Announcing
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Build Elixir applications with speed and confidence.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
Courses that'll move you from confusion to "Aha, now I get it!"