dirkmjk / limepy Goto Github PK

JSONDecodeError Traceback (most recent call last)
in
8 sid = '156133'
9
---> 10 csv = download.get_responses(base_url, user_name, password, user_id, sid)
11 path = Path('../data/responses.csv')
12 path.write_text(csv)

5 frames
/usr/lib/python3.8/json/decoder.py in raw_decode(self, s, idx)
353 obj, end = self.scan_once(s, idx)
354 except StopIteration as err:
--> 355 raise JSONDecodeError("Expecting value", s, err.value) from None
356 return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I'm getting the following error while trying to execute download.get_responses

~/.virtualenvs/sandbox/lib/python3.6/site-packages/limepy/download.py in get_responses(base_url, user_name, password, user_id, sid, lang, document_type, completion_status, heading_type, response_type, from_response_id, to_response_id, fields)
84 document_type, completion_status, heading_type,
85 response_type, from_response_id, to_response_id,
---> 86 fields)
87 release_session_key(base_url, session_key, user_id)
88 return csv

~/.virtualenvs/sandbox/lib/python3.6/site-packages/limepy/download.py in export_responses(user_id, base_url, session_key, sid, lang, document_type, completion_status, heading_type, response_type, from_response_id, to_response_id, fields)
39 req = requests.post(api_url, json=payload)
40 result = req.json()['result']
---> 41 csv = base64.b64decode(result.encode()).decode('utf-8-sig')
42 return csv
43

AttributeError: 'dict' object has no attribute 'encode'

When try to connect with LDAP authentication : you ca n not set Plugin param

See : https://manual.limesurvey.org/RemoteControl_2_API#get_session_key

There are possibility to have different URL for authentication

Authentication url can be different : see https://gitlab.com/SondagesPro/Authentification/AuthRemoteToken : plugins/unsecure&plugin=AuthRemoteToken

Related LimeSurvey issue (reason to use different url): https://bugs.limesurvey.org/view.php?id=18177

I'm using LimeSurvey version 2.62. with two languages. If I create a Question object my_question and then print my_question.question and my_question.summary I get the question title only in one language, and the answers for each language. Is there a way to access the languages explicitly?

Mussten Sie Informationen angeben, um Zugriff auf das Korpus zu erhalten?

      Count  Percent  Valid Percent
Yes    11.0     30.6           73.3
No      4.0     11.1           26.7
Ja     11.0     30.6           73.3
Nein    4.0     11.1           26.7

The beginning of my survey.lss file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<document>
 <LimeSurveyDocType>Survey</LimeSurveyDocType>
 <DBVersion>261</DBVersion>
 <languages>
  <language>de</language>
  <language>en</language>
 </languages>
 <answers>
  <fields>
   <fieldname>qid</fieldname>
   <fieldname>code</fieldname>
   <fieldname>answer</fieldname>
   <fieldname>assessment_value</fieldname>
   <fieldname>sortorder</fieldname>
   <fieldname>language</fieldname>
   <fieldname>scale_id</fieldname>
  </fields>
  <rows>
   <row>
    <qid><![CDATA[501861]]></qid>
    <code><![CDATA[A1]]></code>
    <answer><![CDATA[Yes]]></answer>
    <assessment_value><![CDATA[0]]></assessment_value>
    <sortorder><![CDATA[1]]></sortorder>
    <language><![CDATA[en]]></language>
    <scale_id><![CDATA[0]]></scale_id>
   </row>
   <row>
    <qid><![CDATA[501861]]></qid>
    <code><![CDATA[A1]]></code>
    <answer><![CDATA[Ja]]></answer>
    <assessment_value><![CDATA[0]]></assessment_value>
    <sortorder><![CDATA[1]]></sortorder>
    <language><![CDATA[de]]></language>
    <scale_id><![CDATA[0]]></scale_id>
   </row>

I'm using limepy to create a PDF report for a survey.

Iterating through the questions, I have a Question object for every question, which is usually enough to build the report. However, it looks like some details are missing, e.g. for multiple choice questions with an Other option, I'm unable to find the freeform entries users made. I'd be happy to look them up manually in the survey's dataframe, but I don't see a way to get the question code from a Question object.

Maybe I'm overlooking something obvious – but if not: Could you add a code property to the Question class?

Sorry but I am not sure what to enter for user_id?

I can use the login on my base url and visit index.php/admin/remotecontrol.

I tried to enter my username twice:

from limepy import download

download.get_session_key(
    base_url,
    "my_username",
    "my_pw",
    "my_username",
)

But I just get:

{'id': "my_username'', 'result': {'status': 'Invalid user name or password'}, 'error': None}

I'm sure that I entered the username and pw correctly as they are the same as for the login via base url + /admin, right?

As I wasn't able to find anything related to an user_id on https://api.limesurvey.org/classes/remotecontrol_handle.html or on https://manual.limesurvey.org/RemoteControl_2_API#Python_example_and_glue

Maybe a small note on the README page could be helpful for new users?

Currently, limepy assumes multiple choice options are checked if they don’t have a missing value. This does not take into account changes caused by manual data editing of LimeSurvey responses. Specifically, a checkbox array option normally has a value 1 when checked and a missing value when not checked, but after manual editing, options that are not checked may get a 0.

In order to prevent duplicated question data when a survey uses more than one language, the data structure has been changed in LimeSurvey 4. As a result of the changes, creating a Survey object with limepy currently fails.

Hi,

Nice library.

Here's my problem: I have a survey structure file (LSS format), without any answer yet.

I'd like to use this library to explore the survey structure. But from the example given on the main page and from the source code, my understanding is that it's necessary to have some dataframe containing the answers, in order to read the structure file.

Am I correct, or is there a way to read the LSS file on its own?

If it's not currently possible, it would be a pretty useful feature!

Thanks.

While trying to build a simple pipeline, I encountered an error that I can't seem to fix on my end.
The error being raised:
/python3.10/site-packages/limepy/wrangle.py", line 134, in <dictcomp> if item['language'] == language
TypeError: string indices must be integers

The section of code mentioned:

            if not isinstance(items, list):
                items = [items]
            question_l10ns = {
                item['qid']:item['question']
                for item
                in items
                if item['language'] == language

My code can be found here.

I'm a little bit lost and maybe I'm missing something, while initializing some questions I receive a "KeyError". I'm still new to Python, maybe it is something obvious I have overlooked.

Hi -- I noticed that LimePy does not have a license in this repo, but in PyPI it is stated that it is MIT license. It might be useful to provide a license file in the github repository as well, just to avoid issues.

Hi,

I am not able to create a survey object from my data (manually exported), despite I carefully set heading type to 'code' and reponse type to 'short'.

My code is:

from limepy.wrangle import Survey, Question
import pandas as pd

df = pd.read_csv('results-survey618718_code.csv', sep=',')
with open('limesurvey_survey_618718.lss') as f:
    my_structure = f.read()

my_survey = Survey(df, my_structure)

And I get:

  File "/home/xxxxxxx/anaconda3/lib/python3.9/site-packages/limepy/wrangle.py", line 262, in create_readable_df
    colname = colname.replace('\n', ' ')

AttributeError: 'NoneType' object has no attribute 'replace'

Is it a bug? Or am I misusing the tool? Thanks for your help.

Here is the CSV file I use:

"id","submitdate","lastpage","startlanguage","startdate","datestamp","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","matiere_SQ001","matiere_SQ002","matiere_SQ003","matiere_SQ004","matiere_SQ005","matiere_SQ006","suggestion","general_SQ001","general_SQ002","general_SQ003","general_SQ004","general_SQ005","ptspositifs_SQ001_SQ001","ptspositifs_SQ002_SQ001","ptspositifs_SQ003_SQ001","ptsnegatifs_SQ001_SQ001","ptsnegatifs_SQ002_SQ001","ptsnegatifs_SQ003_SQ001","commentaire","poisson"
"10","2022-04-01 07:55:43","17","fr","2022-04-01 07:38:41","2022-04-01 07:55:43","A2","A3","A1","A2","A2","A2","+ d'applications en cinématiques ","A2","A1","A1","A1","A1","A1","DS parfois trop long","A2","A2","A2","A2","A2","A2","","A2","A1","A1","A1","A1","A1","","A2","A1","A1","A1","A1","A1","","A2","A2","A1","A1","A1","A1","","A1","A1","A1","A1","A1","A1","","A2","A1","A2","A2","A2","A2","","A1","A1","A1","A1","A1","A1","","A1","A1","A2","A1","A1","A2","","A2","A2","A1","A1","A1","A1","","A2","A2","A1","A1","A2","A1","","A1","A2","A1","A1","A1","A2","","A2","A1","A1","A1","A1","A1","","A2","A3","A4","A4","A2","A2","Rendre les cours plus concrets","A1","A2","A2","A1","A1","Professionalisation ","Bonne ambiance ","Échelle humaine","","","","","Bonne idée ! "
"11","2022-04-01 07:49:16","17","fr","2022-04-01 07:38:42","2022-04-01 07:49:16","A2","A2","A4","A4","A4","A4","Problème de compréhension  avec le prof ","A2","A2","A2","A2","A2","A2","","A1","A1","A1","A1","A1","A1","","A2","A1","A2","A2","A2","A2","","A2","A2","A2","A2","A2","A2","","A1","A1","A1","A1","A1","A1","","A2","A2","A2","A2","A2","A2","","A2","A2","A2","A2","A2","A2","","A2","A2","A2","A2","A3","A2","","A2","A2","A3","A3","A2","A2","","A2","A2","A2","A2","A2","A2","","A1","A1","A1","A1","A1","A1","","A2","A2","A3","A2","A2","A2","","A3","A3","A3","A2","A2","A2","","A2","A2","A2","A2","A2","A2","","A1","A1","A1","A1","A3","Le niveau ","L'ambiance ","Soirée ","Le contenu","Certains profs ","Horaire ","","C'est une bonne idée "

Hi! I'm using download.get_responses and the connection to remote-control web if working perfectly, but for some sid's I have the error 'charmap' codec can't encode character '\U0001f449' in position 1064935: character maps to <undefined>
I tried to redefine the encoding using sys.getdefaultencoding(), 'replace but I'm thinking how can I add this line to the reading process before I could do the path.write_text(csv)
This is happening with some of the surveys I had on my website, not all of them.
If you can help me with that or give me some ideas, I'll appreciate it a lot. Thank you.

The url set is OK only if you use apache.

Not with IIS or nginx server (it's /index.php?r=admin/remotecontrol )

Workaround : set the url to https://example.org/limesurvey/index.php?r=admin/remotecontrol&

Maybe an option to avoid the workaround ?

Hi,
first of all, thanks for your amazing library, it helps me a lot for my bachelors thesis.
But i noticed one problem: some questions dont work with the write_open_ended function.
The question itself gets written but without answers.
As of right now i cannot figure out why that happens, but i'll take a look.
If i print the corresponding df everything looks fine and not different from questions that gets written properly.

Any ideas?

With single-language surveys languages will be a string, but it is expected to be a list. This will cause problems when creating a Survey object without specifying the language.

The creation of a Survey object may yield an error on parsing the lss survey structure if elements such as question_attributes are missing.

Hi,
I manually downloaded my CSV using Question code and Answer codes as settings and used your sample code to build my survey object:

df = pd.read_csv('code-results-survey543837.csv', sep=';')
with open('limesurvey_survey_872183.lss') as f:
    my_structure = f.read()
my_survey = Survey(df, my_structure)

But the my_survey object is failing. Here's the traceback:

KeyError                                  Traceback (most recent call last)
/tmp/ipykernel_824/587664489.py in <module>
      3     my_structure = f.read()
      4 
----> 5 my_survey = Survey(df, my_structure)
~/.local/lib/python3.8/site-packages/limepy/wrangle.py in __init__(self, dataframe, structure, language, strip_tags)
     52         self.language = language
     53         self.strip_tags = strip_tags
---> 54         self.questions, self.groups = self.parse_structure(structure)
     55         self.question_list = self.create_question_list()
     56         self.readable_df = self.create_readable_df()
~/.local/lib/python3.8/site-packages/limepy/wrangle.py in parse_structure(self, structure)
    108                 if item['language'] == language
    109             }
--> 110             question_l10ns_help = {
    111                 item['qid']:item['help']
    112                 for item
~/.local/lib/python3.8/site-packages/limepy/wrangle.py in <dictcomp>(.0)
    109             }
    110             question_l10ns_help = {
--> 111                 item['qid']:item['help']
    112                 for item
    113                 in items
KeyError: 'help'

Any idea what I'm doing wrong?
Thanks!