Alchemist Notifications¶
A key feature of Alchemist is its comprehensive notification system, designed to keep you informed every step of the way.
During its operation, Alchemist generates notifications that could be informational, warnings, or errors. These notifications are an integral part of Alchemist's functionality and are produced for all operations, including parsing, analysis, and conversion.
Always Check Notifications
Always check the notifications after running any Alchemist operation. They provide context about the operation and its results, including tips on unsupported constructs, potential issues, and other important information.
Notifications also influence the return code from the Alchemist CLI. See how to configure this below.
Accessing Notifications¶
There are multiple ways to access notifications:
- Console output: Some Notifications are printed to the console during the operation, depending on the notification configuration.
- Alchemist UI: Notifications are displayed in the Alchemist UI, if you are using the Alchemist Desktop application.
- Reports: Notifications are included as a separate table in the report generated by Alchemist (e.g. as one of the analysis outputs).
Elements of Notifications¶
A notification consists of the following elements:
- Level: The level of notification severity, which could be
info,warning, orerror. - Code: A unique code that identifies the notification.
- Operation: The operation that generated the notification.
- Type: The high-level category of the notification.
- Message: The message that provides information about the notification.
- Origin: The origin in the source file if notification can be directly attributed to a particular point in a source.
Notification Types¶
- Information: General information, non-critical notifications.
- Unsupported: A source constructs that is unsupported, e.g. a function that can't be automatically converted.
- Missing Dependency: A required dependency is missing from the sources, e.g. missing dataset from a metadata export file.
- Config Error: There is an error in the configuration.
- Upstream Error: An error caused by an earlier error.
- Invalid Source: A genuine error in the source, usually an invalid code.
- Internal Error: An unexpected internal Alchemist error.
- Imprecise Conversion: The conversion is not imprecise, such as those described in conversion nuances.
- Invalid Template: The provided conversion template is invalid.
Notification Codes¶
| Code | Operation | Dialect | Meaning |
|---|---|---|---|
| G000 | General | Generic | General notification (unspecified). |
| P00_001 | Parse | Generic | Can't read source file. |
| P01_000 | Parse | SAS | Unspecified SAS parser notification. |
| P01_001 | Parse | SAS | Unspecified SAS Code parser notification. |
| P01_010 | Parse | SAS | Incorrect SAS program or unsupported macro nesting. |
| P01_020 | Parse | SAS | Option parsing error. |
| P01_101 | Parse | SAS | A comment parse error. |
| P02_000 | Parse | SAS EG | Unspecified SAS EG parser notification. |
| P02_001 | Parse | SAS EG | Unsupported SAS EG version. |
| P02_002 | Parse | SAS EG | Can't resolve EG ID as warning. |
| P02_003 | Parse | SAS EG | Can't resolve EG ID as error. |
| P02_004 | Parse | SAS EG | Can not parse DNA. |
| P02_005 | Parse | SAS EG | No parser available. |
| P02_006 | Parse | SAS EG | Unsupported element. |
| P02_007 | Parse | SAS EG | SAS Code for node is not available. |
| P02_008 | Parse | SAS EG | Bad Link. |
| P02_009 | Parse | SAS EG | Node collision. |
| P02_010 | Parse | SAS EG | Import task warning. |
| P02_020 | Parse | SAS EG | Potential project inconsistency due to user driven adhoc manual execution. |
| P02_101 | Parse | SAS EG | Unable do read source due to unknown encoding. |
| P03_000 | Parse | SAS Meta | Unspecified SAS Meta parser notification. |
| P03_002 | Parse | SAS Meta | Unexpected, probably corrupted data in SPK package. |
| P03_003 | Parse | SAS Meta | Unsupported SPK construct. |
| P03_004 | Parse | SAS Meta | Conflicts between multiple exported metadata packages. |
| P03_005 | Parse | SAS Meta | SPK exported without dependecies. |
| P03_101 | Parse | SAS Meta | SAS Marker parse error (detailed). |
| P03_102 | Parse | SAS Meta | SAS Marker parse error. |
| R00_001 | Resolve | Generic | Unexpected error during resolution. |
| R01_001 | Resolve | SAS | Resolver unable to infer/predict types. |
| R01_002 | Resolve | SAS | Resolver unable to coerce types. |
| R02_001 | Resolve | SAS EG | Unexpected error during resolution. |
| R03_001 | Resolve | SAS Meta | Unexpected error during resolution. |
| A00_001 | Analyze | Generic | Unexpected error during analysis. |
| A00_201 | Analyze | Generic | Mismatched source DDS table sets. |
| A01_001 | Analyze | SAS | Unexpected error during analysis. |
| A01_202 | Analyze | SAS | Invalid data in source DDS tables. |
| A01_203 | Analyze | SAS | Non-critical data quality issues during DDS validation. |
| A01_204 | Analyze | SAS | Irrecoverable data quality issues during DDS validation. |
| A02_000 | Analyze | SAS EG | Unspecified SAS EG analyzer notification. |
| A02_001 | Analyze | SAS EG | Failed to get the stable order of tasks within a flow. |
| A03_001 | Analyze | SAS Meta | Two jobs forms a reference loop via job reference transforms. |
| C00_001 | Convert | Generic | Conversion configuration error. |
| C00_002 | Convert | Generic | Unexpected error during conversion. |
| C00_003 | Convert | Generic | Unexpected error during rendering. |
| C00_004 | Convert | Generic | Invalid template. |
| C00_005 | Convert | Generic | Unsupported construct. |
| C04_001 | Convert | Databricks | Unexpected error during conversion. |
| C04_021 | Convert | Databricks | Missing information necessary for conversion. |
| C04_031 | Convert | Databricks | Skipped/filtered element |
| C04_032 | Convert | Databricks | Skipped inactive element |
| C04_033 | Convert | Databricks | Skipped unnecessary (doesn't affect result) element |
| C05_001 | Convert | Prophecy | Unexpected error during conversion |
| C05_002 | Convert | Prophecy | Alias is not supported |
| C05_003 | Convert | Prophecy | Missing dependency |
| C05_004 | Convert | Prophecy | Can't replace matched node type. Template definition error. |
| C05_005 | Convert | Prophecy | Can't parse content of template file |
| C05_006 | Convert | Prophecy | Multiple target datasets are not supported by SQL Statement Gem. Template definition error. |
| C05_007 | Convert | Prophecy | Source gem is not converted. Missing dependency. |
| C05_009 | Convert | Prophecy | Unsupported transform type, passthrough gem will be generated |
| C05_010 | Convert | Prophecy | Error when converting transform, passthrough gem will be generated |
| C05_011 | Convert | Prophecy | SAS Extract transform has more than one output or source, skipped |
| C05_012 | Convert | Prophecy | Source dialect for SAS Extract transform is missing, skipped |
| C05_013 | Convert | Prophecy | Target dataset SAS Extract transform is missing, skipped |
| C05_014 | Convert | Prophecy | SAS Extract transform has set operations in query, skipped |
| C05_015 | Convert | Prophecy | SAS Append transform has more than one target dataset, skipped |
| C05_016 | Convert | Prophecy | SAS Append transform has missing source dependency, skipped |
| C05_017 | Convert | Prophecy | SAS Append transform has missing target dataset, skipped |
| C05_018 | Convert | Prophecy | SAS Loader transform has more than one source or target dataset, skipped |
| C05_019 | Convert | Prophecy | SAS Loader transform has missing source or target dataset, skipped |
| C05_020 | Convert | Prophecy | SAS Lookup transform has no lookup datasets, skipped |
| C05_021 | Convert | Prophecy | SAS Lookup transform source dataset is missing, skipped |
| C05_022 | Convert | Prophecy | SAS Lookup transform target dataset is missing, skipped |
| C05_023 | Convert | Prophecy | SAS SQL transform has more than one target dataset, skipped |
| C05_024 | Convert | Prophecy | SAS Custom transform prototype is missing, skipped |
| C05_025 | Convert | Prophecy | SAS Custom transform prototype has no code, skipped |
| C05_026 | Convert | Prophecy | SAS User Written transform has no code, skipped |
| C05_027 | Convert | Prophecy | SAS Split transform has more than one source dataset, skipped |
| C05_028 | Convert | Prophecy | SAS Split transform number of target datasets doesn't match number of queries, skipped |
| C05_029 | Convert | Prophecy | Unsupported expression, skipped |
| C05_030 | Convert | Prophecy | Deduplicate gem unsupported column alias, skipped |
| C05_031 | Convert | Prophecy | Aggregate expression unsupported column alias, skipped |
| C05_032 | Convert | Prophecy | Skipped inactive element |
| C05_033 | Convert | Prophecy | Skipped unnecessary (doesn't affect result) element |
| C05_034 | Convert | Prophecy | SAS Sort transform has more than one source or target dataset, skipped |
| C05_035 | Convert | Prophecy | Delta MERGE operation is only supported for Delta files & tables |
| C05_036 | Convert | Prophecy | Unable to parse prophecy project dataset json |
| C05_037 | Convert | Prophecy | SAS Sort transform has unknown duplicates option, deduplication skipped |
| C05_038 | Convert | Prophecy | SAS Sort transform has unknown sortseq (collating) option, default ordering used |
| C05_039 | Convert | Prophecy | SAS SQL Join transform has unsupported join type, SQL Gem will be generated |
| C05_040 | Convert | Prophecy | SAS Lookup transform has unsupported action with custom expression, ignored |
| C05_041 | Convert | Prophecy | Cannot enforce unknown column type |
| C06_101 | Convert | Spark | Found expressions with unknown type, this may lead to incorrect conversion |
| C06_102 | Convert | Spark | Found unsupported expressions that require template for conversion |
| C06_010 | Convert | Spark | The source construct can't be converted to a 100% equivalent in Spark |
Notification CLI Report Configuration¶
A user can configure which notifications are report. i.e displayed on the console and what is the return code of the Alchemist CLI based on the notifications.
Setting the Exit Code Based on Notifications¶
After command has completed, Alchemist will assess all generated notifications. Unless the config defines that some notifications should be ignored, all notifications are considered and if at least one error notification is raised, Alchemist will exit with code 1.
There are a couple of ways how this behavior can be modified:
- If
treat_warnings_as_errorsis set toTrue, Alchemist will exit with code 1 if any warning is raised. - Notifications can be ignored based on their code (via
ignorekey) or operation.
Here is a full annotated example of the notification configuration in the alc_config.yml file related to exit codes:
notifications:
# Treat warnings as errors.
# If set to True, Alchemist will exit with code 1 if any warning is raised.
treat_warnings_as_errors: false # Default: False
# Ignore notifications with these codes.
# Ignored notifications do not affect alchemist exit code and are not printed in CLI report.
ignore: ["P01_000"] # Default: []
# Ignore all parsing notifications.
# Ignored notifications do not affect alchemist exit code and are not printed in CLI report.
parse_ignore_all: false # Default: False
# Ignore all resolving notifications.
# Ignored notifications do not affect alchemist exit code and are not printed in CLI report.
resolve_ignore_all: false # Default: False
# Ignore all transforming notifications.
# Ignored notifications do not affect alchemist exit code and are not printed in CLI report.
transform_ignore_all: false # Default: False
# Ignore all analysis notifications.
# Ignored notifications do not affect alchemist exit code and are not printed in CLI report.
analyze_ignore_all: false # Default: False
# Ignore all conversion notifications.
# Ignored notifications do not affect alchemist exit code and are not printed in CLI report.
convert_ignore_all: false # Default: False
Reporting Notifications to the CLI¶
Alchemist will report (print) notifications to the CLI based on the configuration. By default, Alchemist will report a summary with notification counts as well as all non-ignored non-developer notifications one by one. The following configuration options can be used to modify this behavior:
notifications:
# Report notifications count to the CLI.
# also available via ALC_SHOW_NOTIFICATION_COUNT env var
cli_notification_count_report: true # Default: True
# If you are not interested in the notifications count, you can disable it:
# cli_notification_count_report: false
# Report notifications from the selected level and up with details to the CLI.
# also available via ALC_CLI_REPORT env var
cli_report: INFO # Default: "INFO"
# E.g. to report only errors and warnings:
# cli_report: WARNING
# If you want ignored notifications to still be printed to the CLI.
report_ignored: true # Default: False
# If you want to get developer notifications in the report as well.
# set to true to include developer notifications in the report:
include_dev: true # Default: False