Skip to content
/ llm.nvim Public

Free large language model (LLM) support for Neovim, provides commands to interact with LLM (like ChatGPT, ChatGLM, kimi).

9 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

Kurama622/llm.nvim

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

59 Commits
lua/llm
lua/llm
 
 
plugin
plugin
 
 
README.md
README.md
 
 
stylua.toml
stylua.toml
 
 

Repository files navigation

llm.nvim

Important

This is a universal plugin for a large language model (LLM), designed to enable users to interact with LLM within neovim.

You can customize any LLM (such as glm, kimi) you wish to use.

You can customize some useful tools to complete your tasks more effectively.

Finally, and most importantly, you can use various free models (whether provided by Cloudflare or others).

Screenshots

Chat

You can converse with it just like you would with ChatGPT.

llm-chat

Quick Translate

Select the text to translate quickly.

llm-translate

Explain Code

Can't understand the code? Don't worry, AI will explain every code snippet for you.

llm-explain-code

Customizable LLM application tools

You can customize some useful tools to complete your tasks more effectively. Detailed tutorial can be found on wiki.

Optimize Code

Let AI optimize your code. wiki: create-a-tool-to-help-optimize-your-code

llm-optimize-code

Translate

Your next translator is not a translator. wiki: create-a-translator-tool

llm-trans

Installation

cloudflare

  1. You need sign up on cloudflare and get your account and API key. Then you will find all models on cloudflare, where the models labeled as beta are free.

  2. Set ACCOUNT and LLM_KEY in your zshrc or bashrc

export ACCOUNT=<Your ACCOUNT>
export LLM_KEY=<Your API_KEY>
  • lazy.nvim
  {
    "Kurama622/llm.nvim",
    dependencies = { "nvim-lua/plenary.nvim", "MunifTanjim/nui.nvim" },
    cmd = { "LLMSesionToggle", "LLMSelectedTextHandler" },
    config = function()
      require("llm").setup()
    end,
    keys = {
      { "<leader>ac", mode = "n", "<cmd>LLMSessionToggle<cr>" },
      { "<leader>ae", mode = "v", "<cmd>LLMSelectedTextHandler 请解释下面这段代码<cr>" },
      { "<leader>t", mode = "x", "<cmd>LLMSelectedTextHandler 英译汉<cr>" },
    },
  },

ChatGLM (智谱清言)

  1. You need sign up on https://open.bigmodel.cn/, and get your account and API key.

  2. LLM_KEY in your zshrc or bashrc

export LLM_KEY=<Your API_KEY>
  • lazy.nvim
  {
    "Kurama622/llm.nvim",
    dependencies = { "nvim-lua/plenary.nvim", "MunifTanjim/nui.nvim" },
    cmd = { "LLMSesionToggle", "LLMSelectedTextHandler" },
    config = function()
      require("llm").setup({
        max_tokens = 512,
        url = "https://open.bigmodel.cn/api/paas/v4/chat/completions",
        model = "glm-4-flash",
        prefix = {
          user = { text = "😃 ", hl = "Title" },
          assistant = { text = "⚡ ", hl = "Added" },
        },

        save_session = true,
        max_history = 15,

        -- stylua: ignore
        keys = {
          -- The keyboard mapping for the input window.
          ["Input:Submit"]      = { mode = "n", key = "<cr>" },
          ["Input:Cancel"]      = { mode = "n", key = "<C-c>" },
          ["Input:Resend"]      = { mode = "n", key = "<C-r>" },

          -- only works when "save_session = true"
          ["Input:HistoryNext"] = { mode = "n", key = "<C-j>" },
          ["Input:HistoryPrev"] = { mode = "n", key = "<C-k>" },

          -- The keyboard mapping for the output window in "split" style.
          ["Output:Ask"]        = { mode = "n", key = "i" },
          ["Output:Cancel"]     = { mode = "n", key = "<C-c>" },
          ["Output:Resend"]     = { mode = "n", key = "<C-r>" },

          -- The keyboard mapping for the output and input windows in "float" style.
          ["Session:Toggle"]    = { mode = "n", key = "<leader>ac" },
          ["Session:Close"]     = { mode = "n", key = "<esc>" },
        },
      })
    end,
    keys = {
      { "<leader>ac", mode = "n", "<cmd>LLMSessionToggle<cr>" },
      { "<leader>ae", mode = "v", "<cmd>LLMSelectedTextHandler 请解释下面这段代码<cr>" },
      { "<leader>t", mode = "x", "<cmd>LLMSelectedTextHandler 英译汉<cr>" },
    },
  },

Customized Large Language Model

  1. Add the requested URL.
  2. Specify the model you will be using.
  3. Customize the streaming processing function (used for parsing the model output).

wiki: example

  • lazy.nvim
  {
    "Kurama622/llm.nvim",
    dependencies = { "nvim-lua/plenary.nvim", "MunifTanjim/nui.nvim" },
    cmd = { "LLMSesionToggle", "LLMSelectedTextHandler" },
    config = function()
      require("llm").setup({
        max_tokens = 8000,
        url = "https://api.moonshot.cn/v1/chat/completions",
        model = "moonshot-v1-8k", -- "moonshot-v1-8k", "moonshot-v1-32k", "moonshot-v1-128k"

        streaming_handler = function(chunk, line, output, bufnr, winid, F)
          if not chunk then
            return output
          end
          local tail = chunk:sub(-1, -1)
          if tail:sub(1, 1) ~= "}" then
            line = line .. chunk
          else
            line = line .. chunk

            local start_idx = line:find("data: ", 1, true)
            local end_idx = line:find("}]", 1, true)
            local json_str = nil

            while start_idx ~= nil and end_idx ~= nil do
              if start_idx < end_idx then
                json_str = line:sub(7, end_idx + 1) .. "}"
              end
              local data = vim.fn.json_decode(json_str)
              if not data.choices[1].delta.content then
                break
              end

              output = output .. data.choices[1].delta.content
              F.WriteContent(bufnr, winid, data.choices[1].delta.content)

              if end_idx + 2 > #line then
                line = ""
                break
              else
                line = line:sub(end_idx + 2)
              end
              start_idx = line:find("data: ", 1, true)
              end_idx = line:find("}]", 1, true)
            end
          end
          return output
        end
      })
    end,
    keys = {
      { "<leader>ac", mode = "n", "<cmd>LLMSessionToggle<cr>" },
    },
  }

Default Configuration

  • floating window
window key mode desc
Input ctrl+g i submit your question
Input ctrl+c i cancel dialog response
Input ctrl+r i Rerespond to the dialog
Input ctrl+j i select the next session history
Input ctrl+k i select the previous session history
Output+Input <leader>ac n toggle session
Output+Input <esc> n close session
  • split window
window key mode desc
Input <cr> n submit your question
Output i n open the input box
Output ctrl+c n cancel dialog response
Output ctrl+r n Rerespond to the dialog

Configuration

llm.nvim comes with the following defaults, you can override them by passing config as setup param.

llm.nvim/lua/llm/config.lua

Lines 26 to 166 in 51350dc

M.configs = {
prompt = "",
max_tokens = 512,
model = "@cf/qwen/qwen1.5-14b-chat-awq",
url = nil,
streaming_handler = nil,
prefix = {
user = { text = "## User \n", hl = "Title" },
assistant = { text = "## Assistant \n", hl = "Added" },
},
history_path = HOME .. "/.local/state/nvim/llm-history",
max_history_files = 15,
max_history_name_length = 10,
save_session = true,
input_box_opts = {
relative = "editor",
position = {
row = "85%",
col = input_box_start,
},
size = {
height = "5%",
width = input_box_width,
},
enter = true,
focusable = true,
zindex = 50,
border = {
style = "rounded",
text = {
top = " Enter Your Question ",
top_align = "center",
},
},
win_options = {
winblend = 0,
winhighlight = "Normal:Normal,FloatBorder:FloatBorder",
},
},
output_box_opts = {
style = "float", -- right | left | above | below | float
relative = "editor",
position = {
row = "35%",
col = output_box_start,
},
size = {
height = "65%",
width = output_box_width,
},
enter = true,
focusable = true,
zindex = 20,
border = {
style = "rounded",
text = {
top = " Preview ",
top_align = "center",
},
},
win_options = {
winblend = 0,
winhighlight = "Normal:Normal,FloatBorder:FloatBorder",
},
},
history_box_opts = {
relative = "editor",
position = {
row = "35%",
col = history_box_start,
},
size = {
height = "65%",
width = history_box_width,
},
zindex = 70,
enter = false,
focusable = false,
border = {
style = "rounded",
text = {
top = " History ",
top_align = "center",
},
},
win_options = {
winblend = 0,
winhighlight = "Normal:Normal,FloatBorder:FloatBorder",
},
},
popwin_opts = {
relative = "cursor",
position = {
row = -7,
col = 10,
},
size = {
height = 10,
width = "60%",
},
enter = true,
focusable = true,
zindex = 50,
border = {
style = "rounded",
text = {
top = " Explain ",
top_align = "center",
},
},
win_options = {
winblend = 0,
winhighlight = "Normal:Normal,FloatBorder:FloatBorder",
},
},
-- stylua: ignore
keys = {
-- The keyboard mapping for the input window.
["Input:Submit"] = { mode = "i", key = "<C-g>" },
["Input:Cancel"] = { mode = "i", key = "<C-c>" },
["Input:Resend"] = { mode = "i", key = "<C-r>" },
-- only works when "save_session = true"
["Input:HistoryNext"] = { mode = "i", key = "<C-j>" },
["Input:HistoryPrev"] = { mode = "i", key = "<C-k>" },
-- The keyboard mapping for the output window in "split" style.
["Output:Ask"] = { mode = "n", key = "i" },
["Output:Cancel"] = { mode = "n", key = "<C-c>" },
["Output:Resend"] = { mode = "n", key = "<C-r>" },
-- The keyboard mapping for the output and input windows in "float" style.
["Session:Toggle"] = { mode = "n", key = "<leader>ac" },
["Session:Close"] = { mode = "n", key = "<esc>" },
},
}

Example Configuration

  {
    "Kurama622/llm.nvim",
    dependencies = { "nvim-lua/plenary.nvim", "MunifTanjim/nui.nvim" },
    cmd = { "LLMSesionToggle", "LLMSelectedTextHandler" },
    config = function()
      require("llm").setup({
        prompt = "请用中文回答",
        max_tokens = 512,
        model = "@cf/qwen/qwen1.5-14b-chat-awq",
        prefix = {
          user = { text = "😃 ", hl = "Title" },
          assistant = { text = "⚡ ", hl = "Added" },
        },

        save_session = true,              -- if false, history box will not be showed
        max_history = 15,                 -- max number of history
        history_path = "/tmp/history",    -- where to save history

        input_box_opts = {
          relative = "editor",
          position = {
            row = "85%",
            col = 15,
          },
          size = {
            height = "5%",
            width = 120,
          },

          enter = true,
          focusable = true,
          zindex = 50,
          border = {
            style = "rounded",
            text = {
              top = " Enter Your Question ",
              top_align = "center",
            },
          },
          win_options = {
            -- set window transparency
            winblend = 20,
            -- set window highlight
            winhighlight = "Normal:Normal,FloatBorder:FloatBorder",
          },
        },
        output_box_opts = {
          style = "float", -- right | left | above | below | float
          relative = "editor",
          position = {
            row = "35%",
            col = 15,
          },
          size = {
            height = "65%",
            width = 90,
          },
          enter = true,
          focusable = true,
          zindex = 20,
          border = {
            style = "rounded",
            text = {
              top = " Preview ",
              top_align = "center",
            },
          },
          win_options = {
            winblend = 20,
            winhighlight = "Normal:Normal,FloatBorder:FloatBorder",
          },
        },

        history_box_opts = {
          relative = "editor",
          position = {
            row = "35%",
            col = 108,
          },
          size = {
            height = "65%",
            width = 27,
          },
          zindex = 70,
          enter = false,
          focusable = false,
          border = {
            style = "rounded",
            text = {
              top = " History ",
              top_align = "center",
            },
          },
          win_options = {
            winblend = 20,
            winhighlight = "Normal:Normal,FloatBorder:FloatBorder",
          },
        },

        -- LLMSelectedTextHandler windows options
        popwin_opts = {
          relative = "cursor",
          position = {
            row = -7,
            col = 20,
          },
          size = {
            width = "50%",
            height = 15,
          },
          enter = true,
          border = {
            style = "rounded",
            text = {
              top = " Explain ",
            },
          },
        },

        -- stylua: ignore
        keys = {
          -- The keyboard mapping for the input window.
          ["Input:Submit"]  = { mode = "n", key = "<cr>" },
          ["Input:Cancel"]  = { mode = "n", key = "<C-c>" },
          ["Input:Resend"]  = { mode = "n", key = "<C-r>" },

          -- only works when "save_session = true"
          ["Input:HistoryNext"]  = { mode = "n", key = "<C-j>" },
          ["Input:HistoryPrev"]  = { mode = "n", key = "<C-k>" },

          -- The keyboard mapping for the output window in "split" style.
          ["Output:Ask"]  = { mode = "n", key = "i" },
          ["Output:Cancel"]  = { mode = "n", key = "<C-c>" },
          ["Output:Resend"]  = { mode = "n", key = "<C-r>" },

          -- The keyboard mapping for the output and input windows in "float" style.
          ["Session:Toggle"] = { mode = "n", key = "<leader>ac" },
          ["Session:Close"]  = { mode = "n", key = "<esc>" },
        },
      })
    end,
    keys = {
      { "<leader>ac", mode = "n", "<cmd>LLMSessionToggle<cr>" },
      { "<leader>ae", mode = "v", "<cmd>LLMSelectedTextHandler 请解释下面这段代码<cr>" },
      { "<leader>t", mode = "x", "<cmd>LLMSelectedTextHandler 英译汉<cr>" },
    },
  },

Finally, here is my personal configuration for reference.

https://github.com/Kurama622/.lazyvim/blob/main/lua/plugins/llm.lua

About

Free large language model (LLM) support for Neovim, provides commands to interact with LLM (like ChatGPT, ChatGLM, kimi).

Topics

neovim nvim glm gpt neovim-plugin kimi nvim-lua llm chatgpt chatglm

Resources

Readme
Activity

Stars

9 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages