Comments (7)
Hi, the problem is that you use Program.ChatHistory
to initialize the session and again feed it to the session when generating output. Thus, the system message is fed to the session after the chat has started, which is not allowed. Please construct a new ChatHistory
or HistoryMessage
object in ProcInput
, rather than reuse Program.ChatHistory
there.
I wrote these words on my phone so some spellings may be not correct. I believe you could find them according to the semantic. :)
from llamasharp.
Thanks for the quick response, I still get the same error unfortunately.
I have not been able to get LlamaSharp to work since version 0.0.2 and the old versions do not support GGUF models.
public async static void ProcInput()
{
string userPrompt = Program.MainForm.txtInput.Text; // Get the user's input from the TextBox
Program.MainForm.txtOutput.AppendText("\nUser: " + userPrompt + "\n");
MainForm.txtInput.Text = "";
Program.chatHistory = new ChatHistory(); // Reset chat history to start fresh
// Concatenate the user prompt to the existing prompt
Program.chatHistory.AddMessage(AuthorRole.System, "Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision.");
Program.chatHistory.AddMessage(AuthorRole.User, "Hello, Bob.");
Program.chatHistory.AddMessage(AuthorRole.Assistant, "Hello. How may I help you today?");
await foreach (var text in Program._session.ChatAsync(new ChatHistory.Message(AuthorRole.User, userPrompt), new InferenceParams
{
MaxTokens = 1024,
AntiPrompts = new List<string> { "User: " },
Temperature = 0.7f,
TopK = 40,
RepeatPenalty = 1.1f,
MinP = 0.05f,
TopP = 0.95f
}))
if (Program.MainForm.txtOutput.Lines.Length > 0)
{
// Remove the last line
int lastLineIndex = Program.MainForm.txtOutput.Lines.Length - 1;
Program.MainForm.txtOutput.Text = string.Join(Environment.NewLine, Program.MainForm.txtOutput.Lines.Take(lastLineIndex));
}
}
from llamasharp.
Hi, please don't add system message in ProcInput
. Instead, add it only when initializing _session
and only add user message when chatting
from llamasharp.
Hi, please don't add system message in ProcInput
. Instead, add it only when initializing _session
and only add user message when chatting
from llamasharp.
Thanks, there are errors now but it's not sending any output at all regardless of which model I pick. Each time LlamaSharp is updated I try each version in the hope I can get it to work in c# but to no avail.
I tried to ask for some help in the discord again. I feel very pessimistic, is there another simpler version of LlamaSharp or a fork for dummies?
from llamasharp.
I'd recommend to pick an example in LLama.Example folder to see if you could get it to work, then try to integrate it into your project.
Of course, asking it in discord is also okay.
from llamasharp.
I'm going close this issue since there's no activity. Please feel free to ask to reopen this issue if there's any other problem.
BTW, now I have my PC available and maybe you could try the following code, if the problem still exists.
public async static void InitAI()
{
// Load a model
var parameters = new ModelParams(Program.modelPath)
{
ContextSize = 1024,
Seed = 1337,
GpuLayerCount = 80
};
// Initialize chat history
Program.chatHistory = new ChatHistory(); // Reset chat history to start fresh
// Concatenate the user prompt to the existing prompt
Program.chatHistory.AddMessage(AuthorRole.System, "Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision.");
Program.chatHistory.AddMessage(AuthorRole.User, "Hello, Bob.");
Program.chatHistory.AddMessage(AuthorRole.Assistant, "Hello. How may I help you today?");
InferenceParams inferenceParams = new InferenceParams()
{
MaxTokens = 256, // No more than 256 tokens should appear in answer. Remove it if antiprompt is enough for control.
AntiPrompts = new List<string> { "User:" } // Stop generation once antiprompts appear.
};
var model = LLamaWeights.LoadFromFile(parameters);
var context = model.CreateContext(parameters);
var executor = new InteractiveExecutor(context);
ChatSession session = new ChatSession(executor, Program.chatHistory);
foreach(var msg in Program.chatHistory.Messages){
session = await session.AddAndProcessMessage(msg);
}
Program._session = session;
}
public async static void ProcInput()
{
string userPrompt = Program.MainForm.txtInput.Text; // Get the user's input from the TextBox
Program.MainForm.txtOutput.AppendText("\nUser: " + userPrompt + "\n");
MainForm.txtInput.Text = "";
var msg = new ChatHistory.Message(AuthorRole.User, userPrompt);
// Call ChatAsync with the correct arguments
await foreach (var text in Program._session.ChatAsync(msg, true, new InferenceParams() { Temperature = 0.9f, AntiPrompts = new List<string> { "User:" } }))
{
Program.MainForm.txtOutput.AppendText(text);
}
if (Program.MainForm.txtOutput.Lines.Length > 0)
{
// Remove the last line
int lastLineIndex = Program.MainForm.txtOutput.Lines.Length - 1;
Program.MainForm.txtOutput.Text = string.Join(Environment.NewLine, Program.MainForm.txtOutput.Lines.Take(lastLineIndex));
}
}
from llamasharp.
Related Issues (20)
- CentOS x86_64 Failed Loading 'libllama.so' HOT 4
- System.TypeInitializationException: 'The type initializer for 'LLama.Native.NativeApi' threw an exception.' HOT 12
- How do I continously print the answer word for word when using document ingestion with kernel memory? HOT 1
- How to rebuild LLamaSharp backends HOT 3
- Namespace should be consistent HOT 2
- Mamba HOT 11
- Android Backend HOT 5
- [Feature] Allow async model loading and cancellation
- [CI] Add more unit test to ensure the the outputs are reasonable HOT 3
- Take multiple chat templates into account
- [Feature]: Support for Function Calling or Tools HOT 4
- [BUG]: DefragThreshold default is not matching llama.cpp and probably not intended HOT 6
- [BUG]: Answer stop abruptly after contextsize, even with limiting prompt size HOT 1
- [BUG]: Linux cuda version detection could be incorrect HOT 2
- [BUG]: WSL2 has problem running LLamaSharp with cuda11
- Add unit test about long context HOT 3
- Add debug mode of LLamaSharp
- How to better provide system information for LLMs HOT 3
- LLAVA Configuration HOT 4
- [Feature]: 不同的LLM模型,代码要以怎样的方式融合到项目里 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from llamasharp.