GithubHelp home page GithubHelp logo

m-web-server's Introduction

M Web Server

This source tree represents a web (HTTP) server implemented in the M language.

Purpose

The MUMPS Advanced Shell MUMPS Web Server (MWS) provides for a way to serve web services from the MUMPS Database. It does that by mapping URLs to MUMPS procedures that retrieve or save the data depending on the request. The mapping is dynamic and depends on a Fileman-compatible file which allows you to configure security on each web service. The MUMPS Web Server is independent of VISTA and does not need any part of VISTA in order to run--not even Fileman.

MWS provides the following features:

  • It is completely stateless.
  • It runs plain RESTful web services rather than implementing a custom protocol.
  • It does not introduce any new data structures. Fileman data structures are used as the source of truth.
  • It fully supports JSON out of the box; XML is also supported.
  • It provides Meaningful URLs to VISTA data to make it easy to program against VISTA.
  • It is integrated with VISTA's security primitives.
  • It is simple to deploy.

This project aims to provide standardized and easy to deploy RESTful web services from M and from VISTA. The software can also serve file-system based resources that can take advantage of the web services.

See M-Restful-Services-White-Paper.md for more information.

Install & Dependencies

See INSTALL.md.

Developer Documentation

See the doc folder.

To make a new version, see doc/packaging.md.

To set-up TLS, see doc/tls-setup.md.

Testing Documentation

There are extensive unit tests covering 80% of the code.

Future work

A lot of work needs to be put it to make this software more user friendly.

A list of issues can be found here.

XINDEX Note

The code does not pass XINDEX checks due to its extensive use of OS specific calls. It should be treated as a vendor utility for the purposes of SAC Compliance.

Links

m-web-server's People

Contributors

chathaway-codes avatar christopheredwards avatar dlwicksell avatar frederickmarshall avatar glilly avatar jensli avatar ksbhaskar avatar lindamry avatar nars1 avatar osehr-agent avatar shabiel avatar whatisthehumanspirit avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

m-web-server's Issues

Question

Would like to use this with WorldVista to integrate with SQL Server SSIS using Web Service Task. Looking at what will need to be modified.

Non UTF-8 byte sequences in body causes crashes

When running GTM in UTF-8 mode we experience crashes when HTTP request and response bodies contain non UTF-8 byte sequences.

This is because in UTF-8 mode the string manipulation routines, for example $length and $extract consider such byte sequences to be invalid and generate errors.

For GTM the solution is to use for example $zlength and $zextract instead. I will submit a pull request with these changes for GTM. I'm not sure if these changes mess up the Cache support however, so I don't know if then will be useable. But maybe they can serve as a starting point.

Example:

GTM>w $l($zc(255))
%GTM-E-BADCHAR, $ZCHAR(255) is not a valid character in the UTF-8 encoding form

GTM>w $zl($zc(255))
1

Environment

M Web Server version: 1.1.2

GTM>w $zversion
GT.M V7.0-000 Linux x86_64
GTM>w $zchset
UTF-8

Specially crafted JSON can cause incorrect results with the "\s" node

This test isn't valid JSON (and gives no errors as such) and based on validators it is working correctly. Just a placeholder incase this comes up in the future.

Test Case:

YDB>r JSON
{"name": {"\\s": "ok"}, "name": 42}
YDB>d DECODE^%webjson("JSON","OUT","ERR") ZWR OUT
OUT("name")=42
OUT("name","\s")="ok"

YDB>d ENCODE^%webjson("OUT","ENCODE","ERR") W ENCODE(1)
{"name":"42"}

Security Hole: DUZ not properly removed for subsequent requests

I discovered a security hole in M Web Server. It looks like if the DUZ is set, it remains set across subsequent requests. I did a test for this by creating a non-authenticated endpoint that first raises an exception if the DUZ is set upon entry and if not sets the DUZ to nonzero to see if the value sticks across subsequent requests. Sure enough, that is exactly what happened. I¹m addressing this by NEWing the DUZ in my endpoint, but you might want to look at this.

%GO routine no longer works on Cache after running WWWINIT

I noticed that after running WWWINIT in my Cache environment, the %GO routine no longer works. This is because WWINIT maps the %W* routines away from %SYS, which masks several built-in Cache routines:
%WebStress.Recorder %Wgdisp %Wmdc %Wpglo %Wprim %Wprima
At least one of these is used by %GO.

https server

Hi!

It is possible to force TLS 1.2 only?

I am using key and certificate, this standard way defined in documentation.

I can connect to server with TLS 1.0, 1.1, but it looks security team force to use 1.2.

Kind regards
michal abramowicz

Investigate why trailing slashes are needed below

URLMAP ;
        ;;GET ping ping^%webapi
        ;;GET api/regions/{region}/ getRegion^%ydbguiRest
        ;;GET api/dashboard/getAll getDashboard^%ydbguiRest
        ;;DELETE api/regions/{region}/ deleteRegion^%ydbguiRest
        ;;POST api/regions/{region}/createDb createDb^%ydbguiRest
        ;;POST api/regions/{region}/extend extendRegion^%ydbguiRest
        ;;POST api/regions/{region}/journalSwitch journalSwitch^%ydbguiRest
        ;;zzzzz

If a URL starts with /, it's not matched appropriately.

^%W(17.6001,0)="WEB SERVICE URL HANDLER^17.6001S^1^1"
^%W(17.6001,1,0)="GET"
1)="/multiply/{multiplier}/{multiplicand}"
2)="M^ZZGMUL"
^%W(17.6001,"B","GET","/multiply/{multiplier}/{multiplicand}","M^ZZGMUL",1)=""

George said to leave out the initial '/' in the URI, and that seems to work. Moving n to PUT and POST.

Log to stdout

This can help when the web server is ran in docker containers and NOGBL is set.

Zipping .woff2 returns incorrect data to the web browser/client

Bad:

Access-Control-Allow-Origin: *
Cache-Control: max-age=604800
Content-Encoding: gzip
Content-Length: 250388
Content-Type: font/woff2
Date: Wed, 20 Jul 2022 13:38:22 GMT

Correct:

Content-Encoding: gzip
Content-Length: 250752
Content-type: application/octet-stream
Date: Wed, 20 Jul 2022 13:36:37 GMT
Last-Modified: Wed, 20 Jul 2022 13:25:53 GMT
Server: SimpleHTTP/0.6 Python/2.7.13

Errors in NOGBL mode are not reported

When an M error happens during the execution of the Web Server, LOGERR doesn't set the global that contains the error information because we are running in "No Globals" mode; and as a result, they are suppressed and never sent back to the client.

Add support for CORS

CORS requires specific headers to be sent back and forth for web pages to use web services hosted elsewhere

Error installing in YottaDB r1.34: %YDB-E-SETECODE, Non-empty value assigned to $ECODE (user-defined error trap)

I have a newly-installed Debian 11 system, with a fresh installation of YottaDB r1.34. When I follow the installation instructions for the webinit routine, it terminates with:

[... lots of warnings ...]
%YDB-W-BLKTOODEEP, Block level too deep
%YDB-E-SETECODE, Non-empty value assigned to $ECODE (user-defined error trap)

Here's my complete process from a fresh Debian 11 install:

YottaDB installation (building from source because it seems to be unhappy with ICU version otherwise):

$ sudo apt update && \
    sudo apt upgrade -y && \
    sudo apt install -y build-essential icu-devtools git curl cmake pkg-config tcsh {libconfig,libelf,libicu,libncurses,zlib1g,libicu}-dev
 [...]
$ sudo reboot
 [... a short while later ...]
$ curl https://gitlab.com/YottaDB/DB/YDB/raw/master/sr_unix/ydbinstall.sh > /tmp/ydbinstall.sh
$ chmod +x /tmp/ydbinstall.sh
$ sudo /tmp/ydbinstall.sh --utf8 default --from-source --branch r1.34
 [...]
Since you are running systemd we changed memory settings and you need to restart
the systemd-logind service using this command "systemctl restart systemd-logind".

If your Desktop session is managed by the systemd-logind service (e.g., a Gnome
or Plasma desktop), restarting systemd-logind will restart your GUI. Save all
your work prior to issuing "systemctl restart systemd-logind".
YottaDB version r1.34 installed successfully at /usr/local/lib/yottadb/r134
YottaDB pkg-config file installed successfully at /usr/share/pkgconfig/yottadb.pc
$ sudo systemctl restart systemd-logind

YottaDB environment:

$ source /usr/local/lib/yottadb/r134/ydb_env_set
$ yottadb -version
YottaDB release:         r1.34
Upstream base version:   GT.M V6.3-011
Platform:                Linux x86_64
Build date/time:         2022-08-29 20:02
Build commit SHA:        f2f77cc184d9fe2e272d7b1d39b64434e9769804

Restoring webinit:

$ curl -L https://github.com/shabiel/M-Web-Server/releases/download/1.1.5/webinit.rsa > /tmp/webinit.rsa
$ yottadb -dir

YDB>W $ZRO
/home/mwilson/.yottadb/r1.34_x86_64/o/utf8*(/home/mwilson/.yottadb/r1.34_x86_64/r /home/mwil
son/.yottadb/r) /usr/local/lib/yottadb/r134/utf8/libyottadbutil.so
YDB>D ^%RI

Routine Input Utility - Converts RO file to *.m files.

Formfeed delimited <No>? 
Input device: <terminal>: /tmp/webinit.rsa

Routines

Output directory : /home/mwilson/.yottadb/r

webinit   

Restored 525 lines in 1 routine.

Running webinit:

YDB>D ^webinit
         Q:$$CACHE $ZU(168)
                    ^-----
                At column 13, line 66, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-INVFCN, Invalid function name
         I $$CACHE S %=$ZU(168,^%SYS("TempDir")) QUIT  ; Cache
                        ^-----
                At column 17, line 73, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-INVFCN, Invalid function name
         I $$CACHE N % S %=$ZU(168,DIR) QUIT
                            ^-----
                At column 21, line 79, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-INVFCN, Invalid function name
         I $P($P($ZV,") ",2),"(")<2012 S NMSP=$ZU(5)
                                               ^-----
                At column 40, line 91, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-INVFCN, Invalid function name
         I $P($P($ZV,") ",2),"(")>2011 S NMSP=$NAMESPACE
                                               ^-----
                At column 40, line 92, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-INVSVN, Invalid special variable name
         ZN "%SYS" ; Go to SYS
         ^-----
                At column 2, line 96, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-INVCMD, Invalid command keyword encountered
         N % S %=##Class(Config.Namespaces).Get(NMSP,.PROP) ; Get all namespace properties
                 ^-----
                At column 10, line 100, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EXPR, Expression expected but not found
         I '% W !,"Error="_$SYSTEM.Status.GetErrorText(%) S $EC=",U-CONFIG-FAIL," QUIT
                                  ^-----
                At column 27, line 101, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-SPOREOL, Either a space or an end-of-line was expected but not found
         S %=##class(Config.Configuration).GetGlobalMapping(NMSP,G,"",DBG,DBG)
             ^-----
                At column 6, line 110, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EXPR, Expression expected but not found
         I '% S %=##class(Config.Configuration).AddGlobalMapping(NMSP,G,"",DBG,DBG)
                  ^-----
                At column 11, line 111, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EXPR, Expression expected but not found
         I '% W !,"Error="_$SYSTEM.Status.GetErrorText(%) S $EC=",U-CONFIG-FAIL," QUIT
                                  ^-----
                At column 27, line 112, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-SPOREOL, Either a space or an end-of-line was expected but not found
         N % S %=##Class(Config.MapRoutines).Get(NMSP,G,.PROPRTN)
                 ^-----
                At column 10, line 115, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EXPR, Expression expected but not found
         I '% S %=##Class(Config.MapRoutines).Create(NMSP,G,.PROPRTN)
                  ^-----
                At column 11, line 117, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EXPR, Expression expected but not found
         I '% W !,"Error="_$SYSTEM.Status.GetErrorText(%) S $EC=",U-CONFIG-FAIL," QUIT
                                  ^-----
                At column 27, line 118, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-SPOREOL, Either a space or an end-of-line was expected but not found
         ZN NMSP ; Go back
         ^-----
                At column 2, line 119, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-INVCMD, Invalid command keyword encountered
         N $NAMESPACE S $NAMESPACE="%SYS"
            ^-----
                At column 5, line 127, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-INVSVN, Invalid special variable name
         N $NAMESPACE S $NAMESPACE="%SYS"
                         ^-----
                At column 18, line 127, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-INVSVN, Invalid special variable name
         n % s %=##class(Security.SSLConfigs).Exists("encrypt_only",.config,.status) ; check if conf
ig exists
                 ^-----
                At column 10, line 129, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EXPR, Expression expected but not found
         . s %=##class(Security.SSLConfigs).Create("encrypt_only",.prop) ; create a default ssl conf
ig
               ^-----
                At column 8, line 132, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EXPR, Expression expected but not found
         . i '% w $SYSTEM.Status.GetErrorText(%) s $ec=",u-cache-error,"
                         ^-----
                At column 18, line 133, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-SPOREOL, Either a space or an end-of-line was expected but not found
         . s %=##class(Security.SSLConfigs).Exists("encrypt_only",.config,.status) ; get config
               ^-----
                At column 8, line 134, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EXPR, Expression expected but not found
         e  s %=config.Activate()
                      ^-----
                At column 15, line 135, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-SPOREOL, Either a space or an end-of-line was expected but not found
         d config.TestConnection("encrypted.google.com",443,.rtn)
                 ^-----
                At column 10, line 139, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-SPOREOL, Either a space or an end-of-line was expected but not found
         set httprequest=##class(%Net.HttpRequest).%New()
                         ^-----
                At column 18, line 151, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EXPR, Expression expected but not found
         . set httprequest.Https=1
                          ^-----
                At column 19, line 153, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EQUAL, Equal sign expected but not found
         . set httprequest.SSLConfiguration="encrypt_only"
                          ^-----
                At column 19, line 154, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EQUAL, Equal sign expected but not found
         set httprequest.Server=server
                        ^-----
                At column 17, line 159, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EQUAL, Equal sign expected but not found
         if port set httprequest.Port=port
                                ^-----
                At column 25, line 160, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EQUAL, Equal sign expected but not found
         set httprequest.Timeout=5
                        ^-----
                At column 17, line 161, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EQUAL, Equal sign expected but not found
         new status set status=httprequest.Get(filepath)
                                          ^-----
                At column 35, line 162, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-SPOREOL, Either a space or an end-of-line was expected but not found
         new response set response=httprequest.HttpResponse.Data
                                              ^-----
                At column 39, line 163, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-SPOREOL, Either a space or an end-of-line was expected but not found
         new sysfile set sysfile=##class(%Stream.FileBinary).%New()
                                 ^-----
                At column 26, line 164, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EXPR, Expression expected but not found
         set status=sysfile.FilenameSet(filename)
                           ^-----
                At column 20, line 165, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-SPOREOL, Either a space or an end-of-line was expected but not found
         set status=sysfile.CopyFromAndSave(response)
                           ^-----
                At column 20, line 166, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-SPOREOL, Either a space or an end-of-line was expected but not found
         set status=sysfile.%Close()
                           ^-----
                At column 20, line 167, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-SPOREOL, Either a space or an end-of-line was expected but not found
         D $SYSTEM.Process.SetZEOF(1) ; Cache stuff!!
           ^-----
                At column 4, line 262, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-LABELEXPECTED, Label expected in this context
         O ROPATH:("RS"):0 E  S $EC=",U-ERR-OPEN-FILE,"
                   ^-----
                At column 12, line 267, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-DEVPARPARSE, Error parsing device parameter specification
         O DIR_"test.txt":"NWS":0
                          ^-----
                At column 19, line 300, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-DEVPARPARSE, Error parsing device parameter specification
         C DIR_"test.txt":"D"
                          ^-----
                At column 19, line 304, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-DEVPARPARSE, Error parsing device parameter specification
         . O TCPIO:(:PORT:"ACT"):2
                    ^-----
                At column 13, line 385, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-DEVPARPARSE, Error parsing device parameter specification
         D CHKTF^%ut('$D(^$R("%ZVEMD")))
                          ^-----
                At column 19, line 451, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-GBLNAME, Either an identifier or a left parenthesis is expected after a ^ in this con
text
         N NS S NS=$NAMESPACE
                    ^-----
                At column 13, line 460, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-INVSVN, Invalid special variable name
         ZN "%SYS"
         ^-----
                At column 2, line 461, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-INVCMD, Invalid command keyword encountered
         S %=##class(Config.Configuration).AddGlobalMapping(NS,"%Z*","",NS,NS)
             ^-----
                At column 6, line 462, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EXPR, Expression expected but not found
         S A("Database")=NS S %=##Class(Config.MapRoutines).Create(NS,"%Z*",.A)
                                ^-----
                At column 25, line 463, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-EXPR, Expression expected but not found
         ZN NS
         ^-----
                At column 2, line 464, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-INVCMD, Invalid command keyword encountered
         D CHKTF^%ut($D(^$R("%ZVEMD")))
                         ^-----
                At column 18, line 467, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-GBLNAME, Either an identifier or a left parenthesis is expected after a ^ in this con
text
         D:$D(^$R(R))  F  S R=$O(^$R(R)) Q:R=""  Q:($P(R,NMSP,2)="")  D
               ^-----
                At column 8, line 475, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-E-GBLNAME, Either an identifier or a left parenthesis is expected after a ^ in this con
text
         . X "ZR  ZS @R"
           ^-----
                At column 4, line 476, source module /home/mwilson/.yottadb/r/webinit.m
%YDB-W-BLKTOODEEP, Block level too deep
%YDB-E-SETECODE, Non-empty value assigned to $ECODE (user-defined error trap)

Question: Project status and maturity?

Hello!

This project is interesting to us for implementing a network interface to a GTM application. The project looks professionally developed, tested and documented.

Before we start relying on this projects we'd like to ask its authors the following questions:

  • In your opinion, is this project mature enough to be used in a production environment?
  • What is the activity level in the project? Will critical bugs be fixed in a reasonable time?

Regardless, we appreciate that you have published such a nice looking project and contributed to the GTM ecosystem!

Decoder crashes on bad input

FOIA201805-SYN>k  s in="{""test"":[4,3,2]}}"

FOIA201805-SYN>d decode^%webjson("in","out")
%YDB-E-LVUNDEF, Undefined local variable: VVSTACK(0)
		At M source location DIRECT+27^%webjsonDecode

FOIA201805-SYN>zsh
DIRECT+27^%webjsonDecode    ($ZTRAP)
    (Direct mode) 
DIRECT+26^%webjsonDecode
DIRECT+24^%webjsonDecode
+1^GTM$DMOD    (Direct mode) 

VVERR="^TMP(""%webjsonerr"",$J)"
VVERRORS=0
VVIDX=18
VVINPUT(1)="{""test"":[4,3,2]}}"
VVJSON="VVINPUT"
VVLINE=1
VVMAX=100
VVPROP=0
VVROOT="out("
VVSTACK=0
VVSTACK(1)="test"
VVSTACK(2)=3
VVTYPE="}"
XVV("$T")=1
XVV("ID")=0
in="{""test"":[4,3,2]}}"
out("test",1)=4
out("test",2)=3
out("test",3)=2

Re-add paging support

Paging can be useful for certain data, however the old implementation was JDS specific

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.