¶ 1 Overview
RED-PSS is the automation stack for the RED Framework and runs on a dedicated RED server which is based on RHEL 8 (or compatible operating systems). The RED-PSS is completely self-contained which means that on top of a RHEL minimal installation all necessary libraries and tools are installed in /app independent of Python and Openssl versions on the machine itself.
RED-PSS is part of the RED Automation Module (see Automation Module)
¶ 2 RED-PSS Environment
¶ 2.1 Instances
The RED Server may contain multiple independent RED-PSS instances. Each instance controls one dedicated Exasol database with a corresponding RED Metadata Dictionary (RED-MDD). Different instances can have different versions.
¶ 2.2 Directory Structure
/app/
- python/
- openssl/
- redftp/
- bin/
- env/
- <instance>
- <user>
- redsys/
- bin/
- etc/
- lib/
- log/
- tmp/
- env/
- <instance>
- bin/
- etc/
- lib/
- log/
- DB/
- int/
- tmp/
- data/
- input/
- work/
- error/
- archive/
¶ 2.2.1 redsys Directory Tree
This tree contains the complete installation of RED-PSS except for base libraries like Openssl or Python. All objects are owned by redsys user.
- redsys/bin
- redsys/etc
- redsys/lib/
- redsys/log
- redsys/tmp: Tools and Logs for non-instance specific administration purposes
- redsys/env: Contains a directory for each instance with exactly the same subtree and tools (depending on the technical version (TV).
- redsys/env/<instance>/data: Landing zone for files
¶ 2.2.2 redftp Directory Tree
- redftp/bin: Admin Tools
- redftp/env/<instance>: Top level directory for FTP users for a specific instance owned by root
While FTP-users are named like redftp_<instance>_<user> the directory in the env-directory represents only the <user>-part.
¶ 2.2.3 Environment Variables
| Variable | Explanation |
|---|---|
| REDHOME | /app/redsys |
| REDENV | the current instance (upper case) |
| REDENV_HOME | $REDHOME/env/<instance> |
¶ 2.3 Security
¶ 2.3.1 Application Management
For administration purposes the Application Manager must sudo to redsys and change into the target environment for the instance:
[user@red01 ~]$ sudo su - redsys
[redsys@red1 ~]$ tst1
---------------------------------
current environment: TST1
exit with CTRL-D
---------------------------------
TST1 [redsys@red01 tst1]$ pwd
/app/redsys/env/tst1
TST1 [redsys@red01 tst1]$
¶ 2.3.2 sFTP Users
sFTP users reside in a change root environment (see 2.2.2) authenticated by public key. Uploaded data are made available for redsys via softlinks in $REDENV_HOME/data/input. redsys and the sFTP users share the same group redftp.
sFTP users are bound to the UID range 1200 – 1299.
In order to add or remove sFTP-users use the commands redftp_useradd or redftp_userdel, respectively. The commands may be executed as redsys or root.
¶ 2.3.3 REST API
The REST-API is available through https only. Certificate and Private Key must exist in $REDENV_HOME/etc/ssl/REST:
- certificate.pem
- private_key.pem
¶ 2.4 Start/Stop of RED-PSS
The complete stack is automatically started on machine startup by systemd (redsys.service). However, only instances are started where the (empty) file $REDENV_HOME/.autostart exists.
redsys.service calls $REDHOME/bin/redsysctl which may be used as well as redsys-User in order to start and stop all instances or just one.
DEV1 [redsys@red01 bin]$ redsysctl -h
USAGE:
Manage the current instance (uses REDENV):
redsysctl {start | stop | status | restart}
Manage a specific instance:
redsysctl {start | stop | status | restart}
instance Start/stop all instances:
redsysctl {start-all | stop-all | restart-all | status-all}
DEV1 [redsys@red01 bin]$
redsysctl in turn calls $REDENV_HOME/bin/redctl and $REDENV_HOME/bin/restctl.
¶ 2.5 Suspended Mode
redctl may be started in suspended mode by using the -S option. In this case no jobs and watchers will be started and, therefore, no load processes will be executed. All other services, however, are up and running.
The suspended mode can be ended with the resume command or by restarting redctl (restart command).
¶ 2.6 Cloning a DB into an existing RED-environment
Before starting the Exasol cloning process (i.e. restoring a backup from another DB) you must execute redctl preclone. By doing this, important data and metadata are backed up in $REDENV_HOME/tmp/preclone.
After the restore, redctl takes care of restoring these data to the clone and, furthermore, all logs (in the DB and on the filesystem) are cleaned up.
Note: The audit trail of the cloned DB will not be deleted by redctl. Only the SYS-user can do so.
¶ 3 Command Line Tools
¶ 3.1 redctl
redctl is the main commandline utility for the RED-PSS
$ redctl -h
usage: redctl [options] command [command-options]
options:
-h, --help show this help message and exit
-D, --debug show debug output
commands:
{version,start,stop,abort,restart,start_trace,status,reload,suspend,
resume,show,watch,we_list_error,we_reset_error,start_db_job,list_db_
jobs,toggle_db_job,list_db_watcher,toggle_db_watcher,start_job,precl
one}
| Commands | Explanation |
|---|---|
| version | show versions |
| start | start server |
| stop | stop server |
| abort | abort server |
| restart | restart server |
| start_trace | start server with output on STDOUT and STDERR |
| status | show server status |
| reload | reload job config |
| suspend | suspend all jobs |
| resume | resume all jobs |
| show | show status for jobs and processes |
| watch | watch status for jobs and processes |
| we_list_error | list errors of watcher executors |
| we_reset_error | reset errors of watcher executors |
| start_db_job | start a DB job immediately |
| list_db_jobs | list DB jobs |
| toggle_db_job | toggle DB job between active and inactive |
| list_db_watcher | list DB watcher |
| toggle_db_watcher | toggle DB watcher between active and inactive |
| start_job | start an already scheduled job immediately |
| preclone | save state before cloning from another instance |
¶ 3.2 redinfo
¶ 3.3 restctl
restctl starts and stops the GUnicorn application server for the REST-API
DEV1 [redsys@red01 bin]$ restctl -h
USAGE: restctl {start | stop | status | restart}
DEV1 [redsys@red01 bin]$
¶ 4 Process Architecture
¶ 4.1 JMON
The Job Monitor scans the views REDSYS.ALL_ADM_JOBS and REDSYS.ALL_ADM_WATCHER and schedules jobs and watchers, respectively, as defined therein. JMON monitors if they are running or scheduled and reschedules if necessary. Every 60 seconds JMON writes back statistics into REDSYS in the tables ADM_JMON_LOAD and ADM_JMON_JOBLOG.
When a job or watcher is scheduled JMON hands it over to the scheduler (SCHED) which executes the job then in a worker pool. JMON also recognizes if a job or watcher was activated or deactivated and sets the proper action. If the interval or cron attributes are changed redctl has to be restarted in order to bring the changes into effect.
¶ 4.2 SCHED
The scheduler master process starts workers for 3 worker pools:
- L – loader queue (for all load processes)
- A – action queue (for application actions)
- D – the default queue for all jobs
Jobs and Watchers log information into ADM_JOB_LOG and ADM_WATCHER_LOG, respectively, of the target schema.
Watcher executor errors are logged into ADM_WATCHER_EXEC_ERROR.
¶ 4.3 DBLS
The database log server receives log buffers from the DB and writes them into $REDENV_HOME/log/<target schema>. Errors and alerts are recorded in the database log-table (LOG_MESSAGES) as well.
¶ 4.4 JEXEC
The job executor runs external jobs as defined in an application action. Based on the action definition the proper script in $REDENV_HOME/cmd is executed.
¶ 4.5 CFISRV
The CFI-server presents files in the data-tree to the RED framework in the DB. This server uses internally the FTPs protocol for communication with the DB.
CFISRV is optional and can be deactivated.
¶ 4.6 JSON2CSV
The JSON-to-CSV process converts JSON input files into CSV format before they are recognized by CSVSRV.
JSON2CSV is active only if CFISRV is active.
¶ 4.7 PARQ2CSV
The Parquet-to-CSV process converts Parquet input files into CSV format before they are recognized by CSVSRV.
PARQ2CSV is active only if CFISRV is active.
¶ 4.8 SENDMAIL
The sendmail server sends buffers received from the DB to the defined mail relay.
¶ 4.9 SENDFILE
The sendfile server exports a CSV or Parquet file from a table or view in the DB and sends it via mail or sFTP to the defined destination.
¶ 4.10 MAIL2CMD
Mail-to-command interface. MAIL2CMD is optional and can be deactivated.
¶ 5 Configuration
The configuration of RED-PSS can be found in $REDENV_HOME/etc/red_params.conf.
¶ 5.1 [DEFAULT] Section
Default values for all other sections.
| VAriables | default values |
|---|---|
| log_level | DEBUG, INFO, WARN, ERROR, ALERT |
| exa_connection | Exasol connect (DSN) |
| exa_username | REDSYS |
| exa_password | password of REDSYS |
| exa_dbname | DB name |
¶ 5.2 [redctl] Section
| Variables | Explanation |
|---|---|
| sched_service_port | internal service port of the scheduler master |
| sched_default_pool_size | number of workers in the default pool |
| sched_loader_pool_size | number of workers in the loader pool |
| sched_action_pool_size | number of workers in the action pool |
| sendmail_host | interface for internal service |
| sendmail_port | port for internal service |
| sendmail_smtp_host | host of target SMTP server |
| sendmail_smtp_port | port of target SMTP server |
| sendmail_smtp_user | SASL auth username |
| sendmail_smtp_pass | SASL auth password |
| sendmail_smtp_from | sender mail address |
| sendmail_smtp_encr | transport encryption: STARTTLS, PLAIN |
| sendfile_host | interface for internal service |
| sendfile_port | port for internal service |
| dbls_host | interface for internal service |
| dbls_port | port for internal service |
| jexec_host | interface for internal service |
| jexec_port | port for internal service |
| cfi_enabled | enable or disable CFISRV: true/false |
| cfi_host | interface for internal service |
| cfi_port | port for internal service |
| cfi_pasv_port_start | start of the passive port range |
| cfi_pasv_port_end | end of the passive port range |
| json2csv_json_suffix | suffix of incoming JSON files |
| json2csv_csv_suffix | suffix of converted CSV files |
| json2csv_retention | retention (in days) of JSON input files |
| parq2csv_parq_suffix | suffix of incoming Parquet files |
| parq2csv_csv_suffix | suffix of converted CSV files |
| parq2csv_retention | retention (in days) of Parquet input files |
| mail2csv_enabled | enable or disable MAIL2CSV: true/false |
| mail2csv_host | interface for internal service |
| mail2csv_port | port for internal service |
| log_retention | log retention (in days) for all server, job and DB-logs |
| log_level | log level for all server processes |
¶ 5.3 [redinfo] Section
| Variables | Explanation |
|---|---|
| log_level | log level for redinfo |
¶ 5.4 [REST] Section
| Variables | Explanation |
|---|---|
| rest_enabled | start REST service on startup: true/false |
| exa_username | TU (Technical User) for the base API connection |
| exa_password | TU (Technical User) password |
| log_level | log level for the API |
| gu_host | GUnicorn listener interface |
| gu_port | GUnicorn listener port |
| gu_workers | number of GUnicorn workers |
¶ 6 Logging
¶ 6.1 Server Logs
The redctl and JMON logs go to redctl.log
¶ 6.2 REST-API Logs
See REST subdirectory
¶ 6.3 Database Logs
The top level directory for DB logs is the DB subdirectory. Within that subdirectories which are named like the respective target DB-schema can be found. ERROR and ALERT messages generated by DB-processes (i.e. LUA Scripts) are written into the respective LOG_MESSAGES tables as well.
All log messages of all schemas can be found in REDSYS.ALL_LOG_MESSAGES.
¶ 7 Monitoring View in DB
The status of RED-PSS can be queried directly from the DB.
- ALL_ADM_JOBS
- ALL_ADM_JOB_LOG
- ALL_ADM_JOB_STATS
- ALL_ADM_JOB_STATUS
- ALL_ADM_WATCHER
- ALL_ADM_WATCHER_EXEC_ERROR
- ALL_ADM_WATCHER_EXEC_STATUS
- ALL_ADM_WATCHER_LOG
- ALL_ADM_WATCHER_STATUS
- ALL_LOG_MESSAGES
- STA_ADM_JOB_OVERVIEW
- STA_ADM_WATCHER_OVERVIEW