Comments (3)
We also face the same issues when we try to build custom evaluations with the CustomPromptEval
. Sometimes we get an error:
"2024-08-29 19:10:21.296 | ERROR | uptrain.operators.language.custom:evaluate_local:144 - Error when processing payload at index 0: None"
and from what I saw from the debugging it's because the format of the response that it tries to parse
'```json\n{\n "Choice": "Correct",\n "Explanation": "...."\n}\n```'
is not the expected one.
Apart from that, we would also like to be able to get the "Explanation" and not only the score.
Thank you!
from uptrain.
Related Issues (20)
- Return tokens count/cost for checks HOT 1
- Azure LLM evaluation - Deployment Error HOT 7
- llama.cpp local eval support
- docker image to multi architecture HOT 2
- Issue Creating new project HOT 2
- Use Local LLM HOT 1
- Support to use locally hosted HuggingFace LLMs as an evaluator
- Event loop is closed when running factual accuracy evaluations HOT 1
- Error when sending request to LLM API: Task got bad yield: WaitTaskRescheduled(abort_func=<function sleep_forever.<locals>.<lambda> at 0x1413db380>)
- Extend Visualization Module to Support Bubble, Violin, and Box Plots
- Response Matching requires context field because the field gets dropped by polars HOT 1
- Error when processing payload
- Error when parsing JSON: Empty strings are not legal JSON5 HOT 1
- Polars version
- Getting TLSV1_ALERT_INTERNAL_ERROR when attempting to hit demo.uptrain.ai
- using custom llm
- RuntimeError: Event loop is closed when running dashboard on docker
- Experiment page never renders on the dashboard
- Experiments page not getting rendered
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from uptrain.