tmartin14 / ucc_migration_test

NOTE: This code is no longer supported. While this may still work for many TA's, it's no longer a sure thing so YMMV.

ucc Migration Test

This file contains source for a Unix shell script to perform a migration of modular inputs from a Splunk Add-On Builder(AOB) generated source base into a new Splunk UCC-based Add-On (a.k.a. TA). Splunk Universal Configuration Console (UCC) format provides a consistent user interface across all Splunk Add-Ons.

The script uses the helper functions from AOB's python3 libraries and merges the souce code from AOB-generated Inputs into a single file for each input. There are a number of edits performed using sed, awk and grep that create a new TA in the package directory.

This script copies the globalConfig.json file to the root directory providing the proper setup to execute the addonfactory-ucc-generator utility.

The script will make modifications to your python files and then prompt you to run ucc-gen, package the application for Splunkbase and submit the package to appInspect. Each of these actions are recommended but optional and separate scripts are included to package the add-on and submit it to appInspect at later points in time.

Notes

• This utility has been tested for migrating Modular Inputs (both custom and REST calls) as well as Alert Actions.
• The ucc-gen utility can be found here: https://github.com/splunk/addonfactory-ucc-generator
• ucc-gen is installed using $ pip3 install splunk-add-on-ucc-framework

Instructions

1. Clone this repository to your local machine
2. Copy your TA's entire directory into this directory /ucc-migration_test/Splunk_TA_XXX
3. Run the convert.sh script from the root directory of this repository passing in the name of the TA’s directory ./convert.sh Splunk_TA_XXX Your new source code will now be located in the /ucc-migration_test/package directory and is setup to run ucc-gen.
4. Accept the prompt to run ucc-gen with the --ta-version parameter OR exit and execute ucc-gen command from the /ucc-migration_test directory later. The new TA will be written to a ucc-migration_test/output directory.
5. Accept the prompt package your TA into a .tgz file OR exit and package the TA later using the package_app.sh script included.
6. Upload your new TA from ucc-migration_test/output/ to a Splunk server and test it out.
7. Accept the prompt to submit your app to Splunk's appInspect process OR exit and submit the TA later using the submit_appInspect.sh script included

Helpful Link

Splunk Cloud Vetting

ucc_migration_test's Issues

TA with no modular inputs does not message cleanly.

Sharing this output with you. Did the convert on an alert action only TA. This one does not have any modular inputs, the script did not handle this gracefully. Also Creating lib/requirements.txt was not behaving as expected with the messages shown below. Version checking indicates different values but show they are the same.

`abc@def:~/LearnUCC/ucc_migration_test$ ./convert.sh TA-ms-teams-alert-action

Converting TA-ms-teams-alert-action...

removing existing ./package directory
Creating a new ./package directory

Updating globalConfig.json. Adding "template":"input_with_helper" to each "service"

---

 Creating lib/requirements.txt file

---

mkdir: cannot create directory ‘./package/lib’: File exists
cat: './package/bin/input_module_.py': No such file or directory
cat: './input_module_.py': No such file or directory

Please check ./package/lib/requirements.txt file for additional libraries you may need to include for your
modular inputs to work. This script collects libraries that may or may not be needed and comments them all in
/package/lib/requirements.txt:
splunktaucclib>=4.1.0

The following list may also be required for your modular inputs to work.

Uncomment those that are required.

---

 Processing Alert Actions...

---

Processing alert action named: ms_teams_publish_to_channel
Done.
Processing alert action named: ms_teams_publish_to_channel_replay
Done.
Finished with Alert Actions

---

 Processing Modular Inputs...

---

ls: cannot access '*.cc.json': No such file or directory

Checking for potential issues with tabs and/or indentation. (python3 doesn't like mixing tabs and spaces)
grep: input_module_*.py: No such file or directory
no tab/indentation issues found.

ls: cannot access 'input_module_*.py': No such file or directory
Finished with Modular Inputs

sed: can't read s/This is an add-on powered by the Splunk Add-on Builder./This is an add-on powered by the Splunk Universal Configuration Console (UCC)./g: No such file or directory

---

 Cleaning Up... Removing files that are no longer needed

---

rm: cannot remove './package/default/addon_builder.conf': No such file or directory
rm: cannot remove './package/aob_events_in_meta.json': No such file or directory
rm: cannot remove './package/default/data/ui/views/inputs.xml': No such file or directory
rm: cannot remove './package/default/inputs.conf': No such file or directory
rm: cannot remove './package/README/addon_builder.conf.spec': No such file or directory
Done.

---

Version Checking
appserver/static/js/build/globalConfig.json --> [
1.1.5
app.manifest --> 1.1.5
default/app.conf --> 1.1.5
1.1.5

ERROR: Versions are out of synch. Please edit so that all 3 files reflect the same version number
Exiting now`

globalConfig.json version error

I was getting a version checking error when running the convert script:

Version Checking
appserver/static/js/build/globalConfig.json --> 3.2.1,
app.manifest --> 3.2.1
default/app.conf --> 3.2.1

ERROR: Versions are out of synch. Please edit so that all 3 files reflect the same version number
Exiting now

Looks like the regex was picking up the ',' after the version number in the globalConfig.json file. I changed line 385 to read:

CURR_VERSION=grep "version" ./globalConfig.json | grep -o '[^: d]*$' | sed 's/,//;s/"//g'

Which seems to have resolved that issue:
Version Checking
appserver/static/js/build/globalConfig.json --> 3.2.1
app.manifest --> 3.2.1
default/app.conf --> 3.2.1
all version settings match

Version Checking issue

Version Checking
appserver/static/js/build/globalConfig.json --> 1.0.1
app.manifest --> 1.0.1
default/app.conf --> 1.0.1
1.0.1

ERROR: Versions are out of synch. Please edit so that all 3 files reflect the same version number
Exiting now