schwarzit / sap-usi-logging-api

An easy-to-use, object-oriented encapsulation around the SAP application log (Transaction SLG1)

USI Logging API

Purpose

This reusable component allows the creation of complete, comprehensive logs with almost no effort and extends SAP's standard logging API by powerful features.

Logging error situations instead of debugging them can:

• significantly reduce support efforts
• boost developer productivity
• speed up the processing of tickets
• aid in analyzing bugs that are hard to debug

The component is:

• based on SAPs application log (Transaction SLG1)
• easy to use
• well documentend
• controlled by customizing (Log levels will control the detail level of the logs)
• completely object-oriented

The component enhances the capabilities of the SAP standard by so-called data containers, that can be used to attach virtually any type of data to log messages making them even more valuable. A variety of data containers for common use cases already exists, but new containers can easily be added whenever needed. The screenshot below shows how seamlessly they are integrated into the SAP standard. Messages with data containers have a detail button that opens a popup and the data can be accessed directly from SLG1.

Installation Guide

Please refer to the installation guide in our wiki.

How to contribute

Please check our contribution guidelines to learn more about this topic.

Further information

Please refer to the wiki.

sap-usi-logging-api's Issues

Outdated comment in Report /USI/BAL_DELETE_ORPHAN_LOG_DAT

The report header claims the API would be backwards compatible with 7.31.

This information became obsolete with version 1.2.0.
The API now requires SAP BASIS 740 SP 02.

Cleanup of demo report /USI/BAL_DEMO_02

The program currently has type "1" (Executable program) but since it uses dynpros it has to be started using the transaction /USI/BAL_DEMO_02.

• Program Type has to be changed to "M" (Module pool)
• Event INITIALIZATION has to be removed

Expose class /USI/CL_BAL_DC_STRUCTURE via package interface /USI/BAL_DATA_CONTAINERS_REUSE

The data container /USI/CL_BAL_DC_STRUCTURE was added with v1.4.0, but it was not added to the package interface /USI/BAL_DATA_CONTAINERS_REUSE. Systems with active package check will report package errors, if the container is used.

The data container has to be added to the package interface.

Display [unsaved] logs

So far two developers asked, if it was possible to display an unsaved log.

The intention was to collect messages during runtime without saving them.
After processing the log should have been displayed to create a comprehensive output.

This is currently not possible, as /usi/cl_bal_lstate_active clears the message buffer after saving.
In order to display an unsaved or partly saved log, saved messages need to be reloaded from the database.
Additionally function module /usi/bal_popup_message_details relies on the data containers being saved to the database.

As this seems to be a valid use-case it should be made possible.

Structure-Container /USI/CL_BAL_DC_STRUCTURE: Includes and structured fields are not recognized

The following code would produce an empty structure container in the log.

DATA(logger) = /usi/cl_bal_factory=>get_instance( )->create_new_logger( i_log_object = 'MY_LOG_OBJECT'
                                                                        i_sub_object = 'MY_SUB_OBJECT' ).
DATA(token) = logger->claim_ownership( ).

logger->add_free_text( i_free_text = |This won't work|
                       i_details   = NEW /usi/cl_bal_dc_structure( i_structure = VALUE p0001( begda = sy-datum ) ) ).

logger->save( token ).
logger->free( token ).

The reason can be found in method /usi/cl_bal_dc_structure->build_alv_output

    LOOP AT structure_description->get_components( )
        REFERENCE INTO DATA(component)
        WHERE type->kind EQ cl_abap_typedescr=>kind_elem.

      ASSIGN COMPONENT component->name OF STRUCTURE <structure> TO <field>.
      IF sy-subrc NE 0.
        CONTINUE.
      ENDIF.

      INSERT VALUE #( fieldname = component->name
                      value     = <field> )
          INTO TABLE alv_output->*.
    ENDLOOP.
  ENDMETHOD.

The problem is, that structure P0001 contains only includes and that method CL_ABAP_STRUCTDESCR->GET_COMPONENTS will return an internal table, that contains structure descriptions for the included structures, but not their respective fields.

Since all fields without an elementary type are excluded from the processing, the method will return an empty table, that will then be logged.

=> The results of method CL_ABAP_STRUCTDESCR->GET_COMPONENTS need to be processed by a recursive logic, that will resolve structured types into their respective fields. This should work for included types and for structured fields.

Usage of ABAP4_COMMIT_WORK

The reports /USI/BAL_DELETE_ORPHAN_LOG_DAT and /USI/BAL_DELETE_OUTDATED_CUST are using the function module ABAP4_COMMIT_WORK that was not releases by SAP and is not exposed in any package interfaces. The reports currently have package errors that need to be fixed.

Update Codebase to ABAP 7.40 SP 02

With release 1.2.0 ABAP 7.40 SP 02 was defined as the required language version. After changing the abaplint.json accordingly, more than 300 findings referring to outdated syntax were reported.

The codebase should be updated to ABAP 7.40 SP 02.

Structure-Container /USI/CL_BAL_DC_STRUCTURE: Potential Loss of Data for deep structures

The data of deep structures might be lost, in some cases.
The following test report reproduces the issue.

TYPES: BEGIN OF ty_deep_structure,
         field_01 TYPE char10,
         field_02 TYPE string,
       END   OF  ty_deep_structure.

DATA(logger) = /usi/cl_bal_factory=>get_instance( )->create_new_logger( i_log_object = 'Z_MY_LOG_OBJECT'
                                                                        i_sub_object = 'Z_MY_SUB_OBJECT' ).
DATA(token) = logger->claim_ownership( ).

DATA(structure_container) = NEW /usi/cl_bal_dc_structure(
                                    i_structure = VALUE ty_deep_structure( field_01 = 'THIS DATA'
                                                                           field_02 = 'WILL BE LOST!' ) ).

logger->add_free_text( i_message_type = /usi/cl_bal_enum_message_type=>information
                       i_free_text    = 'Data of structure will be lost'
                       i_details      = structure_container ).

logger->save( token ).
logger->free( token ).

The debugger reveals, that the data is temporarily available inside the constructor.

Right after leaving the constructor, the data reference inside the data container will still be bound, but the dereferrenced structure will be initial!

The constructor of the class should be adjusted to avoid unexpected loss of data.
Replacing this

structure = REF #( i_structure ).

with that

CREATE DATA structure LIKE i_structure.
ASSIGN structure->* TO FIELD-SYMBOL(<structure>).
<structure> = i_structure.

fixes the issue.

This might not be ideal in terms of performance, so this might be changed again in the future, but for now I'd rather accept an insignificant hit to performance than an unexpected loss of data.

Sub-Logs

The API is currently enforcing the One-Log-Approach.
This is nice for analyzing issues, as all relevant messages will be in one log.

It is not so nice anymore, when you want to analyze the behavior of a reusable component, that would create its own logs, which are then "abducted" by the calling applications, as they are opening logs of their own.

A compromise that works for both scenarios would be nice.

Idea:

• Every call to /USI/CL_BAL_FACTORY=>/USI/IF_BAL_FACTORY->CREATE_NEW_LOGGER should create a new logger instance
• If the calling application already opened a log, then both logs should be "linked"
  • A trace-message should be created in the main-log and in the sub-log
  • The trace-messages should have a new data container (Type: Navigation; Target: The other log)
  • Top-Down-Navigation (Parent to child) in SLG1 should work WITHOUT authority checks - if a log was created in the context of an application and somebody is allowed to see the main log, the sub-logs should also be visible. Otherwise support might become impossible, which would render the entire API useless.
  • Bottom-Up-Navigation MUST have an authoritiy check in place. An authorization for a low-level-component could otherwise grant access to logs of various applications and potentially sensitive data.
• Sub-Logs have to inherit their configuration (e.g. retention times) from their respective parent log. Otherwise sub-logs might be deleted before the parent, which would violate the One-Log-Approach.

Some points still need some clarification:

• How to deal with sub-logs of reusable components, if the component does not close them?
  • Saving a parent log should always save and destroy child logs
  • The token passed by the parent log has to be valid for the sub-logs too

Add Structure-Container

The API currently offers no data container for structures. If e.g. a key-structure passed to a method needs to be logged, the structure has to be converted into an internal table, so that the itab-container can be "abused".

This workaround has two major drawbacks:

1. It is unnecessarily inconvenient for the using developer
2. It wastes space on the database, since the itab-container does not only store the converted structure itself, but also metadata describing the data (Fieldcatalog)

A simple structure-container should be added. The data should be stored and displayed as a table of fieldnames and values.

Add Data Container for JSON documents

The logging API has no data container for JSON documents so far.

Abusing the HTML container to store JSON documents is technically possible, but the document would be displayed as a string. There would be no line-breaks or indentations and it would not be possible to expand or collapse nodes.

A "pretty" JSON container should be added to the API.

Data Containers: Catch Transformation Errors

The following code causes a short dump.

TYPES: BEGIN OF ty_table_line,
         field TYPE n LENGTH 10,
       END   OF ty_table_line,
       ty_table TYPE STANDARD TABLE OF ty_table_line WITH EMPTY KEY.

DATA: table_line TYPE ty_table_line,
      table      TYPE ty_table.

table_line = CONV char10( |{ space WIDTH = 9 }0| ).
INSERT table_line INTO TABLE table.

/usi/cl_bal_factory=>get_instance( )->get_existing_logger( )->add_free_text(
    i_free_text = 'test'
    i_details   = NEW /usi/cl_bal_dc_collection( )->insert(
                          NEW /usi/cl_bal_dc_itab( i_internal_table = table ) ) ).

Method /USI/CL_BAL_SERIALIZER=>SERIALIZE_FIELDS does not catch CX_SY_CONVERSION_NO_NUMBER, that is thrown, because table[ 1 ]-field contains value 0 which is not feasible for a field of type NUMC10. Since the exception is not caught, the entire application is terminated with a short dump.

Method /USI/CL_BAL_SERIALIZER=>SERIALIZE_FIELDS should catch cx_root and convert it into /USI/CX_BAL_INVALID_INPUT. The exception /USI/CX_BAL_ROOT needs to be propagated by the marked methods.

This would result in corrupt itab containers being ignored.

Function module /USI/BAL_POPUP_MESSAGE_DETAILS - Data Containers missing in tree

The following test program creates a log having one message with two structure containers.

DATA(logger) = /usi/cl_bal_factory=>get_instance( )->get_existing_logger( ).
DATA(token)  = logger->claim_ownership( ).

logger->add_free_text(
    i_message_type = /usi/cl_bal_enum_message_type=>information
    i_free_text    = 'Just a test'
    i_details      = NEW /usi/cl_bal_dc_collection( )->insert(
                             NEW /usi/cl_bal_dc_structure( i_structure = VALUE sflight( carrid = 'AA'
                                                                                        connid = '123' ) )
                     )->insert(
                             NEW /usi/cl_bal_dc_structure( i_structure = VALUE sflight( carrid = 'BB'
                                                                                        connid = '234' ) ) ) ).

logger->save( token ).
logger->free( token ).

Since the data container allows multiple instances for the same log message, both structures should be available in SLG1, but only the second instance is offered in the navigation tree.

The bug is caused by the following type declaration.

Logging multiple instances of the same data container having the same description will cause the issue. Passing a different title for each data container instance would prevent this and is highly recommended anyway, but this is still a bug in the UI.

Data Container Persistency: Activate Compression

Before saving a log message:

1. All data containers of that message will be serialized (ABAP => XML).
2. The single XML documents are combined into one big XML document.
3. The XML document is converted into binary data and saved to the database.

The XML-to-Binary-Conversion happens in method /USI/CL_BAL_DATA_CONT_COLL_DAO->CONVERT_XML_TO_DB.

But since the statement EXPORT ... TO DATA BUFFER ... does NOT use the addition COMPRESSION ON, the data will NOT be compressed.

Since all containers contain human-readable data (char-like), and char-like data can be massively compressed, enabled compression could reduce the needed disk space by over 90%.

=> The compression should be activated.

Optimize the ITAB-Container /usi/cl_bal_dc_itab

1. The ITAB-Container should be able to deal with internal tables, having a non-DDIC-line-type.
2. Elementary line types (e.g. MATNR, PERNR_D) should be supported as well.

Bugfix for data container /usi/cl_bal_dc_retcode_and_msg

Opened by mistake.

ITAB-Container /USI/CL_BAL_DC_ITAB: Support for Data References

The ITAB-Container currently supports structured or elementary line types.

Internal tables having a line type TYPE REF TO struct or TYPE REF TO data_element are currently NOT supported.

Dereferencing a data reference should at least be tried. If the reference is bound and the bound data object is compatible with the container, it should be logged.

Client-specific log level

The API currently knows two different log levels:

• the Product speciffc Log Level
• the User specific Log Level

They are suitable for most cases, but at rare occasions you might want to increase the log level of an entire client as a current setting. A valid use-case for that would be the rollout of a new solution, that includes a hypercare-phase. Setting a client-wide log level during that phase might be a good idea.

As an abuse of that functionality might lead to ridiculous amounts of data being written, the usage has to be limited to a reasonable extend.

The User specific Log Level already enforces an expiration date, that must not be more than 14 days in the future. The client specific Log Level should default to a maximum of three days in the future.

As the needed validity periods might differ between systems, clients or customers, they should be made flexible using BAdIs. A default implementation shipped with the API should define reasonable defaults (14 days; 3 days), that can be overwritten, if needed.

ITAB-Container /USI/CL_BAL_DC_ITAB: Move ITAB-Normalization from CONSTRUCTOR to SERIALIZE

The constructor of the ITAB-Container will "normalize" the format of the internal table.

That means:

• Any given table will be converted into TYPE STANDARD TABLE ... WITH NON-UNIQUE DEFAULT KEY
• Components, that cannot be rendered by an ALV-Grid (e.g. object references) will be removed from the line type
• The line type of the normalized table will be a structure in any case - if the source table used a data element as the line type, it will be converted into a structure having exactly one field of that type

All these optimizations are needed to achieve compatibility with the ALV Grid, that will render the data later or to avoid wasting space on the database. However this should not happen in the constructor, since the container might still be irrelevant, if the log level is too low - so we might be doing some heavy lifting for nothing!

The normalization should be moved to the serialize-method. That would give us a lightweight constructor while preserving all advantages of the table format normalization.