wzf2000 / recllmsim

A dataset for task-based recommendation conversation.

RecLLMSim: A Task-based Recommendation Conversation Dataset

A dataset for task-based recommendation conversation.

RecLLMSim Dataset V2

We provide all the data files under the V2 folder.

The conversation dataset is divided into several parts according to the task. Each conversation is stored in a .json file under the corresponding task folder.

The format of the .json file is:

{
    "task": "The name of the task for this conversation.",
    "preference_id": "The ID of the user profile used in this conversation.",
    "task_context_id": "The ID of the task description used in this conversation.",
    "preference": "Preference used in this conversation.",
    "preference_zh": "Preference used in this conversation translated in Chinese.",
    "task_context": "The scenario-specific task description used in this conversation",
    "task_context_zh": "The scenario-specific task description used in this conversation translated in Chinese.",
    "history": [
        {
            "role": "user",
            "content": "Some text generated by user simulator LLM.",
            "content_zh": "Some text generated by user simulator LLM translated in Chinese.",
            "intent": "The intent annotation of the user simulator LLM.",
            "intent_zh": "The intent annotation of the user simulator LLM translated in Chinese.",
        },
        {
            "role": "assistant",
            "content": "Some text generated by assistant LLM.",
            "content_zh": "Some text generated by assistant LLM translated in Chinese.",
            "hallucination": {
                "hallucination": "Whether the assistant hallucinates in this turn.",
                "memo": "The memo of the hallucination.",
            },
        },
        // ...
    ],
    "conflict": false,
    "preference_summary": "The summary of the user simulator's preference extracted from the conversation.",
    "rating": {
        "Preference Alignment": "The rating of preference alignment.",
        "Additional Preferences": "The rating of additional preferences.",
        "Role-Playing Completeness": "The rating of role-playing completeness.",
        "Memorization": "The rating of memorization.",
        "Detail Level": "The rating of detail level.",
        "Practical Usefulness": "The rating of practical usefulness.",
        "Diversity": "The rating of diversity.",
    },
}

RecUserSim Framework

We provide the code for the framework RecUserSim in the RecUserSim folder.

Check the readme.md file in the RecUserSim folder for more details.