exist-db / documentation Goto Github PK

I am quoting the General information paragraph from the Upgrade guide on /exist/apps/doc/build/doc-0.4.8/upgrading.xml

General Information

Never install a new version of eXist-db into the same directory as an older version:

1. Create a backup of your data. If the new version is binary compatible with the old version, keep the data directory (by default in webapp/WEB-INF/data) of the old version. Note: A running instance of eXist-db needs to be stopped before copying files from the data dir.
2. Install the new version into a different location.
3. If the new version is binary compatible, replace the data directory of the new install with the one from the old one.
   Otherwise you need to do a full restore of the data.

As to a new user this upgrade guide paragraph is not clear enough, at least not on the first reading.

First of all, is it safe to retain old data directory and during the new install just to point the location of the data directory to the old one? I tried it and it seems that this procedure is OK, although doing this exception is raised. If this is safe to do, it would be nice to state it in this guide (and to take care of that exception).

Regarding aforementioned quote, if it is not safe to point to old data directory, than it should be pointed to a new one. In step 1 copying of files is mentioned and I don't get it why. The old directory is retained and not deleted during second installation. Maybe to move the data that way emptying the old data directory, and later bring all data back by copying to it.

In step 3 it states that I should replace the data directory. Does it mean to replace the contents of the directory, or to make changes in conf.xml to point to another location? What is safe? Maybe both?

Application Server Configuration page at exist/apps/doc/build/doc-0.4.8/configuration.xml should be updated. Well, it would be nice.

• The contents are broken and not complete.
• Paragraph eXist-db Configuration: Editing conf.xml states that there are four sections, but there are more.
• Some elements should be updated, e.g. I believe there is no element under in the conf file.
• It would be helpful to better emphasize headings that represent xml elements so they could be more noticeable in the document
• etc.

The title of all the pages is "eXist-db Documentation".

When browsing there is no easy way to find and return to a specific
section in the history.

It would be nice, if the pages contained the subject in their title, eg.
"Quick start" or "Lucene" etc.

Problem

When venturing beyond the basic how-to's of exist-db many competing sources of documentation are available. Case in point the xqdoc.xml competes with Adam's book p. 127f, and a post by Wolfgang from 2007.

Solution

Having a central place with easy means for users to flag information in need of updates might help everyone to more efficiently arrive at the most recent and applicable documentation.

Hello,
a colleague and me looked at the "Application Server Configuration" documentation today and wondered if the part about the indexer and case sensitivity is correct. This concerns line 1215 up to line 1230 of the configuration.xml where it says: "specifies whether string comparisons are to be case-sensitive" ... "Setting case-sensitive='yes' violates the XQuery specs" -- shouldn't this be: "Setting case-senstive='no' violates the XQuery specs"? (as the XPath functions mentioned are case-sensitive by default and the default value for this in the conf.xml was already 'yes').
Best,
Ulrike

documentation/data/upgrading.xml mentions changes for going from 1.4 to 2.2. I m not familiar enough with the under the hood changes to create a pull-request with detailed instructions for upgrading from 2.2. to 3.0. See #84 this should really be on the exist-db web-page, to be honest.

How whitespace is preserved (or not) and its configuration in eXist-db is not clear to many people, for example see: eXist-db/exist#548

Several improvements relevant to 3.1.0 since the last release on Feb 7 are worth including in a new release of the documentation app.

Proposed steps:

• Bump version
• Update release notes in repo.xml to note significant changes
• Release to GitHub and public-repo

@class="app:xyz" I m really not the person to go over the documentation here. HTML templating has defeated me many times.

Maybe someone with a better grasp could take another look at it.

Many parts of the documentation still reference the SourceForge SVN respository, most notably the page header . Sometimes only the URL needs to be updated, but sometimes entire sections need to be rewritten, including the new screenshots: http://exist-db.org/exist/apps/doc/building.xml?#svn-repo
Also is there a decision on whether bug reports should from how on be reported here: https://github.com/eXist-db/exist/issues or still here: http://sourceforge.net/p/exist/bugs/ . If it is the first one, then the documentation header needs to be updated about this as well.

At http://exist-db.org/exist/apps/doc/api/index.html, the first paragraph ends with this sentence:

For more information, refer to the Javadocs for this API.

That link leads to:

HTTP ERROR 404

Problem accessing /exist/apps/doc/api/index.html. Reason:

Document /db/apps/doc/api/index.html not found

I do not think that this merits its own issue, but what can I do?

In the doc at http://exist-db.org/exist/apps/doc/lucene.xml#D2.2.4 it reads

<fulltext default="none" attributes="no"/>

but maybe should read instead

<fulltext default="none" attributes="false"/> ?

Perhaps the parser understands it both ways, but the xsd schema does not and all the other references in the documentation use "false" too.

Placeholder, PR will follow. The troubleshooting instructions for macOS don't know about .dmg apps yet. Came up in eXist-db/exist#1432

From @dizzzz on eXist-open, referring to http://atomic.exist-db.org/HowTo/JettySSL:

that document is from 2008 ! it is probably outdated :-/ (classic issue with wikis)

See http://exist-db.org/exist/apps/doc/xacml.xml.

XACML was removed from eXist in eXist-db/exist#996.

configuration.xml is missing documentation about conf.xml's <query-pool> element. Here is the comment from conf.xml and default values:

           <!--                                                                        
                Configure the query pool.

                - max-stack-size:                                                        
                    maximum number of queries in the query-pool.                         

                - size:                                                                  
                    number of copies of the same query kept in the query-pool.           
                    Value "-1" effectively disables caching. Queries cannot be shared     
                    by threads, each thread needs a private copy of a query.             

                - timeout:                                                               
                    amount of time that a query will be cached in the query-pool.        

                - timeout-check-interval:                                                
                    time between checking for timed out queries. For value "-1"          
                    the time out is switched off, resulting cached queries to remain     
                    in the cache forever.                                                
            -->
        <query-pool max-stack-size="5" size="128" timeout="120000"                
                    timeout-check-interval="30000"/>

I seem to remember from the notes of a conference call that the devs would like more help with keeping the documentation up to date. I think that folks are rightfully hesitant to outright delete outdated sections or files from the documentation. This is further compounded by the fact that there are no clear instructions for contributors for how to proceed. Maybe it would help if the repo had:

1. a contribution.md outlining the general strategy
2. a depreciated folder

The folder wouldn't be an ideal solution, but it might be a good stop gap. It could be simply kept around as archive, or cleaned up by the core team as they see fit. It seems much easier to move stuff around, then to create a PR with lots of deletions. Thoughts? I'd be happy to have a go at the contribution.md if it would help.

try { 'a' + 7 } catch * { concat($err:code, ": ", $err:description }

should be

try { 'a' + 7 } catch * { concat($err:code, ": ", $err:description) }

If you look for example at this section of the documentation:

http://exist-db.org/exist/apps/doc/xquery.xml#serialization

you see that each item in the list of serialization parameters is truncated and terminated with an ellipsis. This makes it impossible to learn what the actual parameters are; you have to guess. If it's necesssary to truncate them, they could at least be supplied with an @title attribute so we could find out the actual param name by mouseover.

There is a lot of documentation for eXist, but navigation is not easy at all.

I'd propose to add a Table of Contents (linked or even embedded-but-collapsed) on each page, this would be a good improvement.

Maybe... using the FullText/lucene index we'd could think of a generated index, maybe 'tag cloud' displayed.

When the q & field parameters are provided, the word highlighted seems to be repeated. For example, notice how "backup" is repeated on this page:

http://exist-db.org/exist/apps/doc/backup.xml?q=backup&field=all&id=D2.2.4#D2.2.8.5.

There are many instances of "backupbackup" and one instance of "backupbackupbackup", e.g.:

As with the Admin Client, if the backup uses a different password 
for the "admin" user than the current session, you must specify 
the backupbackupbackup password using the -P.

In the HTML, each instance of "backup" in is wrapped in a <span class="hi"> - which makes me think that the KWIC module is generating some extra copies of the matched word.

E.g., to the "Learning XQuery and eXist-db" article at http://exist-db.org/exist/apps/doc/learning-xquery.xml. (Walmsley is listed.)

What is the Problem

OOTB, the only thing that new users have to decide is an admin pw (trivial) and the memory and cache configuration for their instance (far from trivial). I learned this as trible knowledge passed on from experienced eXist-db devs. Enter monex, for the cache settings we now have this:

But there is no documentation how to interpret it, after a couple of years, I barely understand 40% of whats going on in this panel.

Solution

Rewrite the configuration guidelines, so that people can make more effective use of monex. What is dom.dbx, and where can i see it? Should I be worried that my collection cache isn't full? How many fails are ok?

I realize that there is no simple one size fits all answer to how to set the basic memory config parameters, but i feel that the documentation could do a much better job at helping users get help, by talking about how monex displays relevant information for the topics which are otherwise hidden in different parts of the documentation.

Any core devs takers?

It should be discussed whether or not to recode site:expand-links() on page.html

because they do not work if the documentation app is installed elsewhere than on exist-db.org, and

because the function needlessly involves searching all collections in db for abbrev attributes.

http://markmail.org/message/5xb6eyikef4oxjcj

The same applies to the other core apps using the same function.

I can browse documentation, but the search option does not work. Search result for any term is Found 0 results. Online version works fine.

This page:

http://exist-db.org/exist/apps/doc/building.xml

currently says:

Install the standard distribution

Get a standard distribution from the download page on Sourceforge.
Install it into a directory of your choice.
Make sure you include the source package.

"download page" is linked to https://sourceforge.net/projects/exist/files/Stable/2.0/. I believe it should be linked to the 2.1 stable version.

Hello, the demo page at the bottom of http://exist-db.org/exist/apps/demo/examples/templating/templates.html?n1=2&n2=8 (i.e. http://exist-db.org/exist/restxq/page?user=Sam&language=it) returns a

HTTP ERROR 405
Problem accessing /exist/restxq/page. Reason:
HTTP method GET is not supported by this URL

error.

as written in eXist-db/exist#389

As noted in a thread on exist-open http://markmail.org/message/y4jjzyspepkjbxgy, the URL rewriting documentation does not explain the use of <error-handler>.

Live site: http://exist-db.org/exist/apps/doc/urlrewrite.xml
GitHub source: https://github.com/eXist-db/documentation/blob/master/data/urlrewrite.xml

Now that eXist-db/exist#1419 has been merged, we need to document how to disable the _query and _xupdate query parameters via the xquery-submission and xupdate-submission parameters in webapp/WEB-INF/web.xml.tmpl. Candidate destinations for this information would be https://github.com/eXist-db/documentation/blob/master/data/production_web_proxying.xml and https://github.com/eXist-db/documentation/blob/master/data/production_good_practice.xml.

Should we not have a develop branch?

http://exist-db.org/exist/apps/doc/devguide_log4j.xml

The term "XXXXXXXXX" is used several times, I suppose it should be replaced with "eXist-db".

This was removed from the code-base in eXist-db/exist@d0c03f1

for example eXist
still mentions
sudo tools/wrapper/bin/exist.sh install

eXist-db/documentation contains the current (v3.0) info for
tools/yajsw/bin/installDaemon.sh

To lower the threshold of editing the documentation XML, it might me a nice idea to add a 'edit' button/link to each page, that fires-up exude with that page.

Probably this link should be visible only when logged in as admin or so.

As discussed in eXist-db/exist#1549, the existing directions for setting up an eXist data source in oXygen are out of date. They should be replaced with reference to the "Create eXist-db XML connection wizard" that has been available in the last several major versions of oXygen.

See http://markmail.org/message/mqladaa6ey2s73b5

http://exist-db.org/exist/apps/doc/versioning.xml#D2.2.4.3

Also raised the question of where the documentation for an extension module be kept? And how that could be incorporated into the eXist-db website.

See feature added in eXist-db/exist#1130.

This should probably go into http://exist-db.org/exist/apps/doc/xquery.xml#serialization

highlight-matches= both | elements | attributes | none
when querying text with the full text or n-gram extensions, the query engine tracks the exact position of all matches inside text content. The serializer can later use this information to mark those matches by wrapping them into an element <exist:match>.
Setting highlight-matches=both will enable this feature for every kind of indexable node.

util:expand() is also covered in http://exist-db.org/exist/apps/doc/kwic.xml, but I'm not sure the kwic module itself knows how to deal with matches on attributes.

The Lucene article at http://exist-db.org/exist/apps/doc/lucene.xml mentions <exist:match>:

Or you can ask eXist-db to highlight each match with an exist:match tag, which you can later use to locate the matches within the text.

I seem to recall that the new range index highlights matches, but this is not covered in http://exist-db.org/exist/apps/doc/newrangeindex.xml.

From @dizzzz in eXist-db/exist#1396:

I realise that the documentation [namely, the JMX article] is not complete; it should be added that the JMX information is only accessible via the HTTP interface when

• the client access the API on localhost
• a token is provided when accessing the API from remote

On the security implications of these JMX ports:

So your data is safe. Exist-db does not provide a way of having parts of the HTTP interface exposed via different HTTP ports, I guess that is not the intention of an application server. For these use cases it is a better to look at a nginx based reverse proxy, and have the 8080 port not externally exposed. This is a typical setup for production environments anyway.

Going back to the doc startpage is not obvious to do.

I'd think of a 'doc home' button that brings the reader back to "doc/"

current docs describe the html admin interface to view the versioning info;

in exist-2.2 ( eXist-db/exist#337 eXist-db/dashboard#23 ) the admin interface has been removed...

for example the web.xml config code-listing says: fixme!/ljo @ljo it seems that a proper xml listing should be there instead.

I would guess that the list of jars to include for embedded might also need another go over for version 3.x. Do we still need to include lib/core/sunxacml.jar I thought xacml is depreciated?

Not sure if any changes to REST should be mentioned here.

One of the ideas kicking around for making eXist's documentation easier to contribute to and maintain is to turn it into Markdown. For recent discussion of this, see #34.

I've started a proof of concept at #90. You can clone my markdown branch at https://github.com/joewiz/documentation.git, and try browsing through the Markdown in your web browser at https://github.com/joewiz/documentation/blob/markdown/data/documentation.md. It's a very early step. It's just an experiment to help us think through whether this route seems useful to pursue further.

But besides actual code experiments, this issue is about the general topic of pros and cons of moving to Markdown. I'll start us off:

Pros:

• Markdown—particularly GitHub-flavored Markdown (GFM)—is essentially the native format in GitHub's web interface. GFM is easier than just about any other document format to view, author, edit, and diff in GitHub's web interface. GFM files are automatically rendered as HTML on GitHub, with working links and images, so the source control repository becomes more useful as a viewer. See https://github.com/joewiz/documentation/blob/markdown/data/documentation.md for an example of the Markdown conversion of http://exist-db.org/exist/apps/doc/.
• Adopting GFM could encourage everything from quick edits to major contributions by lowering barrier for contributing to the documentation.

Cons:

• Markdown doesn't have as rich a vocabulary as DocBook... but were we really using DocBook to its full potential? We never transformed it into anything but HTML.
• Isn't XML and thus lacks the rich infrastructure for structural validation; but our DocBook corpus was already marked by significant variation.
• To be searchable in eXist, the Markdown will need to be transformed back into XML (behind the scenes, probably as a post-install step or as a trigger), and we don't know yet if the transformed XML will be as readily queried as the DocBook.

Ultimately, moving from DocBook to Markdown is just technological musical chairs. It doesn't magically improve the content by itself. That's where the real work is needed. The question is: Is this a better starting point from which to begin improving the content? And will it encourage enough contributions to be worth the effort?

As discussed on TEI-L (https://listserv.brown.edu/archives/cgi-bin/wa?A2=ind1703&L=TEI-L&D=0&P=21706), many users are unaware that eXist's REST interface includes the ability to submit arbitrary queries. Locking this down on public servers should be documented in the prose documentation, probably http://exist-db.org/exist/apps/doc/production_good_practice.xml#D2.2.5.5. Currently you have to know to dig into web.xml at https://github.com/eXist-db/exist/blob/develop/webapp/WEB-INF/web.xml.tmpl#L87-L101 - or have access to Retter & Siegel chapter 8.

bin\startup.bat

yields into (java7- running develop)

Exception in thread "main" java.lang.UnsupportedClassVersionError: org/exist/start/Main : Unsupported major.minor version 52.0
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)

(Issue with solution provided)

I just wanted to note that documentation (which I probably didn't read anyway, oops) should instruct you to remove existing 2.2 or other previous exist before installing 3.0. The installer asks if you want to overwrite the files, but even if you say yes, it doesn't overwrite certain files. I had to update my java to 1.8 to run the 3.0 installer. The problem then emerges of it trying to use the wrong version of java to run the files even though you have installed with the correct version.
e.g. bin/startup.sh will remain pointing to the previous java version resulting in:
"Exception in thread "main" java.lang.UnsupportedClassVersionError: org/exist/start/Main : Unsupported major.minor version 52.0" as a problem.

Even if you run the startup.sh then having changed it to point to the right Java, other files give you problems. e.g.
"java.lang.NoClassDefFoundError: com/github/krukow/clj_lang/ITransientCollection"
or
" err:XPST0081 error found while loading module restxq: Error while loading module modules/restxq.xql: No namespace defined for prefix text [at line 271, column 40]" if trying to go to the dashboard.

Solution:

sudo rm -rf /usr/local/eXist-db/

then make sure you are running the right version of java with:
java -version

then run the installer for 3.0RC again.

In this line:

https://github.com/eXist-db/documentation/blob/master/data/urlrewrite.xml#L467

"does only need" should be "only needs".

With the recent new release of 3.4.1, the Troubleshooting documentation needs an update for its section on Port Conflicts.
If you want to change the default ports, that page directs us to /tools/jetty/etc/jetty.xml
to change this line: <Set name="port"><SystemProperty name="jetty.port" default="8080"/></Set>

After a bit of grepping, I found that line is located on another file in the same directory: jetty-http.xml

The Troubleshooting documentation indicates I should be looking to change <Set name="confidentialPort">8443</Set>
and

<Set name="Port">8443</Set>
I think the syntax must have changed for this; at least I haven't located the attribute value confidentialPort in the new jetty files.

For more details, see this exchange with me and @adamretter (advising me on how to set up and run two separate instances of eXist on the same server.