How to force clear the org-ql's cache? about org-ql

[georgewsinger]

Killing buffers fixes this issue. That means the cache is at fault somehow. Is there some way to force-clear the org-ql-cache? Or to disable caching altogether?

[alphapapa]

Hello eeg_bert. ;)

1. If there is a bug in caching, clearing the cache would not fix the bug.
2. There are no known bugs in the cache, and it's unlikely to be one. But if you've found one, we should fix it.
3. As you can see, member-if-not is not used in org-ql. That warning is caused by your own function.
4. "Before first heading" errors don't come from org-ql itself but from org code that is called from many places. org-ql shouldn't be causing such an error; if it were, it would be a bug that needs to be fixed, which wouldn't have anything to do with caching.
5. "Killing buffers fixes this issue. That means the cache is at fault somehow." Possibly, but probably not. Killing buffers has many side effects.
6. I think it's likely that your problem is caused by macro expansion. Note that org-ql-agenda is a macro, and you're calling it from a function with arguments from runtime. There are several layers here, so I'm not sure if this is the problem. But try using the function org-ql-agenda--agenda instead.

[georgewsinger]

@alphapapa Thanks for helping me with this. For reference:

(cl-defun org-ql-agenda--agenda (buffers-files query &key sort buffer narrow super-groups)
  "FIXME: Docstring"
  (declare (indent defun))
  (when (and super-groups (not org-super-agenda-mode))
    (user-error "`org-super-agenda-mode' must be activated to use grouping"))
  ;; I think it's reasonable to use `eval' here.
  (let* ((org-super-agenda-groups super-groups)
         (entries (--> (eval `(org-ql ',buffers-files
                                ,query
                                :sort ,sort
                                :markers t
                                :narrow ,narrow))
                       (mapcar #'org-ql-agenda--format-element it)
                       (cond ((bound-and-true-p org-super-agenda-mode) (org-super-agenda--group-items it))
                             (t it))
                       (s-join "\n" it)))
         (inhibit-read-only t))
    (with-current-buffer (org-ql-agenda--buffer buffer)
      (erase-buffer)
      (insert entries)
      (pop-to-buffer (current-buffer))
      (org-agenda-finalize)
      (goto-char (point-min)))))

It's not clear to me what format query is supposed to be in. For example (todo "TODO") causes emacs to complain that todo is not a function (since it's not, and macro magic is causing it to get converted into something more appropriate).

Is there a straightforward way to convert my function into one that works with org-ql-agenda--agenda?

[alphapapa]

It's not clear to me what format query is supposed to be in. For example (todo "TODO") causes emacs to complain that todo is not a function (since it's not, and macro magic is causing it to get converted into something more appropriate).

Macros aren't magic. ;) They're just functions called at expansion time instead of at runtime, and they're expected to return a sexp for runtime evaluation.

But an important difference to note is that, unlike function arguments, macro arguments are not evaluated before calling the macro. So when changing from the macro version to the function version, you should quote the query argument, like '(todo "TODO"), to prevent it from being evaluated.

[georgewsinger]

This appears to have fixed the issue, but I won't know for sure until I use it a bit more (to see if weird behavior erupts over time). Thanks for help! org-ql is extremely useful.

[alphapapa]

You're welcome. Please let me know if you have any more feedback. I want to put the package on MELPA, but I think it needs more widespread testing and polishing first.

[georgewsinger]

My function:

(defun query-ql (files res-list min max)
  (org-ql-agenda--agenda files
    `(and
    (todo "TODO" "QUERY" "WAITING")
      (when (property "R")
        (if ,'res-list
            (seq-some #'identity (mapcar (lambda (arg) (string-match-p arg (property "R"))) ,'res-list))
          t)
        )
      (when (property "E")
        (let* ((e-min (first-number-from-string (property "E")))
              (e-max (last-number-from-string (property "E")))
              )
          (if (or (< ,max e-min) (> ,min e-max))
              nil
              t
              )))
    )
    ))

is unfortunately still exhibiting weird behavior:

Debugger entered--Lisp error: (error "Before first headline at position 199 in buffer george-functions.el")
  signal(error ("Before first headline at position 199 in buffer george-functions.el"))
  error("Before first headline at position %d in buffer %s" 199 #<buffer george-functions.el>)
  (condition-case nil (outline-back-to-heading invisible-ok) (error (error "Before first headline at position %d in buffer %s" (point) (current-buffer))))
  org-back-to-heading(t)
  (cond (beg) ((or (not (featurep 'org-inlinetask)) (org-inlinetask-in-task-p)) (org-back-to-heading t)) (t (progn (defvar org-called-with-limited-levels) (defvar org-outline-regexp) (defvar outline-regexp) (defvar org-outline-regexp-bol) (let* ((org-called-with-limited-levels t) (org-outline-regexp (org-get-limited-outline-regexp)) (outline-regexp org-outline-regexp) (org-outline-regexp-bol (concat "^" org-outline-regexp))) (org-back-to-heading t)))))
  (let ((beg (cond (beg) ((or (not (featurep 'org-inlinetask)) (org-inlinetask-in-task-p)) (org-back-to-heading t)) (t (progn (defvar org-called-with-limited-levels) (defvar org-outline-regexp) (defvar outline-regexp) (defvar org-outline-regexp-bol) (let* ((org-called-with-limited-levels t) (org-outline-regexp (org-get-limited-outline-regexp)) (outline-regexp org-outline-regexp) (org-outline-regexp-bol (concat "^" org-outline-regexp))) (org-back-to-heading t))))))) (forward-line) (if (looking-at-p org-planning-line-re) (progn (forward-line))) (cond ((looking-at org-property-drawer-re) (forward-line) (cons (point) (progn (goto-char (match-end 0)) (line-beginning-position)))) (force (goto-char beg) (org-insert-property-drawer) (let ((pos (save-excursion (search-forward ":END:") (line-beginning-position)))) (cons pos pos)))))

# ..

I have minimized my function down to a simple call which is causing problems:

(org-ql-agenda--agenda "/home/george/SimulaOKR/SimulaDebug.org"
  `(and
    (todo "TODO" "QUERY" "WAITING")))

and I get weird "Before first heading" messages printed to the Message buffer when I even delete all of the contents of SimulaDebug.org (the file I'm searching against). I restart emacs and I also get the same behavior.

EDIT: If I delete the buffer george-function.el that the function is being called from (even though the buffer the function is being called from isn't in a buffer that is being searched), then the call suddenly works. Very strange behavior.

[alphapapa]

when I even delete all of the contents of SimulaDebug.org (the file I'm searching against).

1. Look at the backtrace. The error occurs in the function org-back-to-heading.
2. Switch to a new, empty buffer and eval (org-back-to-heading t).

[georgewsinger]

(2) generates the error. So I guess my function can only be called from an org file? If so, that's not so bad.

[alphapapa]

No, the point is that that error is raised when you try to search an empty buffer.