LLMs

caution

You are currently on a page documenting the use of text completion models. Many of the latest and most popular models are chat completion models.

Unless you are specifically using more advanced prompting techniques, you are probably looking for this page instead.

Features (natively supported)

All LLMs implement the Runnable interface, which comes with default implementations of all methods, ie. invoke, batch, stream, map. This gives all LLMs basic support for invoking, streaming, batching and mapping requests, which by default is implemented as below:

• Streaming support defaults to returning an AsyncIterator of a single value, the final result returned by the underlying LLM provider. This obviously doesn't give you token-by-token streaming, which requires native support from the LLM provider, but ensures your code that expects an iterator of tokens can work for any of our LLM integrations.
• Batch support defaults to calling the underlying LLM in parallel for each input. The concurrency can be controlled with the maxConcurrency key in RunnableConfig.
• Map support defaults to calling .invoke across all instances of the array which it was called on.

All LLMs

Name	Description
AI21	You can get started with AI21Labs' Jurassic family of models, as well...
AlephAlpha	LangChain.js supports AlephAlpha's Luminous family of models. You'll ...
AWS SageMakerEndpoint	LangChain.js supports integration with AWS SageMaker-hosted endpoints...
Azure OpenAI	[Azure
Bedrock	Amazon Bedrock is a fully managed
ChromeAI	This feature is experimental and is subject to change.
Cloudflare Workers AI	This will help you get started with Cloudflare Workers AI [text
Cohere	This will help you get started with Cohere completion models (LLMs)
Deep Infra	LangChain supports LLMs hosted by Deep Infra through the DeepInfra wr...
Fireworks	Fireworks AI is an AI inference platform to run
Friendli	Friendli enhances AI application performance and optimizes cost savin...
Google Vertex AI	Google Vertex is a service that
Gradient AI	LangChain.js supports integration with Gradient AI. Check out Gradien...
HuggingFaceInference	Here's an example of calling a HugggingFaceInference model as an LLM:
Layerup Security	The Layerup Security integration allows you to secure your calls to a...
Llama CPP	Only available on Node.js.
MistralAI	Mistral AI is a platform that offers hosting for
Ollama	This will help you get started with Ollama [text completion models
OpenAI	OpenAI is an artificial
RaycastAI	Note: This is a community-built integration and is not officially sup...
Replicate	Here's an example of calling a Replicate model as an LLM:
Together AI	You are currently on a page documenting the use of Together AI models...
WatsonX AI	LangChain.js supports integration with IBM WatsonX AI. Checkout Watso...
Writer	LangChain.js supports calling Writer LLMs.
YandexGPT	LangChain.js supports calling YandexGPT LLMs.