sassoftware / sas-studio-custom-steps

Repository to share SAS Studio Custom Steps. A custom step enables you to create a user interface for SAS Studio users at your site to complete a specific task.

SAS Studio Custom Steps

Overview

A custom step enables you to create a user interface for SAS Studio on SAS Viya users at your site to complete a specific task. Custom steps are saved to SAS Content, so they can be shared with others at your site. The Steps panel lists all SAS steps available to a user, including custom steps. Watch this SAS video Create Custom Steps with SAS Studio on SAS Viya for a quick introduction. For more details see SAS Documentation on Custom Steps in SAS Studio.

List of Contributions

List of custom steps in this repository

Getting Started - Making a Custom Step from this repository available in SAS Studio

Follow the instructions on Using a Custom Step from this GitHub repository in SAS Studio to download custom steps from this repository and make them available for use in your SAS Studio environment.

Contributing

We welcome contributions! Please read CONTRIBUTING.md for details on how to submit contributions to this project.

License

This project is licensed under the Apache 2.0 License.

Additional Resources

• SAS Documentation on Custom Steps in SAS Studio
• SAS Demo - Create a Custom Step in SAS Studio on SAS Viya (YouTube)

sas-studio-custom-steps's Issues

Append Step help - Does it have support for multiple input tables?

Is it possible to use Append Table step to append multiple source tables into one (big) output table? I see only one input port.

Separator not working accurately in some cases.

Determination of separator not always accurate, especially when there are blanks in the variable name.

# and ! as separators

Need DIAP to also check for hash (#) and exclamation point (!) as possible separators automatically.

NLS chars not accurately dealt with

NLS characters are not dealt with accurately, need to use k-functions.

SAS Job Fails due to possible error in airflow

[2023-10-15T20:02:01.265+0100] {logging_mixin.py:120} INFO - Waiting for Job to run to completion...
[2023-10-15T20:17:50.735+0100] {logging_mixin.py:120} INFO - HTTP Call failed with error "('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))" from /jobExecution/jobs/e6335a5e-dd62-4498-a14d-bea035299caf/state. Will set jobstatus=unknown and continue checking...
[2023-10-15T20:17:33.840+0100] {base_job.py:232} ERROR - LocalTaskJob heartbeat got an exception
Traceback (most recent call last):
And then a lot of trace info, please see the log for the details

[2023-10-16T02:16:44.084+0100] {taskinstance.py:1401} INFO - Marking task as SUCCESS. dag_id=GDW_PostETL, task_id=Check_Scheduled_SourceSystems, execution_date=20231015T010500, start_date=20231016T010504, end_date=20231016T011644
[2023-10-16T02:16:45.115+0100] {standard_task_runner.py:100} ERROR - Failed to execute job 595174 for task Check_Scheduled_SourceSystems ((psycopg2.errors.DeadlockDetected) deadlock detected
DETAIL: Process 1879381 waits for ShareLock on transaction 155630385; blocked by process 1845775.
Process 1845775 waits for ShareLock on transaction 155630384; blocked by process 1879381.

Looking at the two DAG-logs attached I see the following reference:

"Background on this error at: https://sqlalche.me/e/14/e3q8":

---

OperationalError

Exception raised for errors that are related to the database’s operation and not necessarily under the control of the programmer, e.g. an unexpected disconnect occurs, the data source name is not found, a transaction could not be processed, a memory allocation error occurred during processing, etc.

This error is a DBAPI Error and originates from the database driver (DBAPI), not SQLAlchemy itself.

The OperationalError is the most common (but not the only) error class used by drivers in the context of the database connection being dropped, or not being able to connect to the database. For tips on how to deal with this, see the section Dealing with Disconnects.

This indicates an error in the underlying metadatabase that Airflow uses (typically MySQL or Postgress). See https://towardsdatascience.com/sigterm-signal-fix-airflow-486ab704b126 for how to tweek the configuration of Airflow (requires Medium membership to view).

[BUG] Import - CSV with long column names - Unexpected column names generated

Describe the bug
@nachorodriguez79
Using a CSV file with long names where the first 32 characters are the same, seem to result in generated names like VAR3 instead of an abbreviated name for VAR3.
I realize this is an edge case, so no need to fix. But might be an idea to add a section "Known Issues" to the readme and list issue.

To Reproduce
Steps to reproduce the behavior:

1. Use the custom step and read the attached CSV file
   CSV-with-really-long-column-names.csv

Expected behavior
All columns should have an abbreviated version of the A clear and concise description of what you expected to happen.

Screenshots
Here is a screenshot of the CSV file:

And a screenshot of the Preview Data tab for the output port of the step:

Environment (please complete the following information):

• Name of custom step: Import - GCSFS File Reader/Import - GCSFS File Reader.step
• SAS Viya version: 2023.12
• Browser: Chrome

Additional context
Add any other context about the problem here.

[BUG] DQ - Match Code "Match column not propagated when empty"

Describe the bug
In custom step "DQ - Match Code" when "match column" is left empty...

... the table preview shows the column with *_MC ...

... but in the column structure is not updated.