Manage externalized files

Learn how to work with externalized files within YAML configuration.

4 minute read

Some entities have fields that contain the content of a file, for example, a script or a certificate. In the XML federated configuration, these files are embedded within the XML, which makes it hard to edit them. In a YAML configuration, the content of these files is stored in separate external files, which provides the following advantages:

  • They can be easily read without unnecessary encoding.
  • You can create and edit these files using a text editor (IDE) of your choice.
  • Files and directory can have long names by default, but you can rename your external file as you wish.

For example, the following YAML file contains a field Set Backend message, which content is stored in a JSON file:

---
type: FilterCircuit
fields:
  name: Get organizationId
children:
- type: ChangeMessageFilter
  fields:
    name: Set Backend message
    outputContentType: application/json
    body: '{{file "Get organizationId - Set Backend message.json"}}' # the message has been externalized

Content of Get organizationId - Set Backend message.json:

{
  "name": "${targetName}",
  "serviceType": "rest",
  "description": "Streamed backend",
  "version": "1.0",
  "basePath": "https://${StreamsSSEVHOST}",
  "resourcePath": "/subscribers",
  "models": {},
  "consumes": [],
  "produces": [],
  "organizationId": "${backend.organizationId}"
}

File externalization is managed with the placeholder {{file "my_file" "file_option"}}:

  • my_file is the name of the externalized file. The location can be:

    • Absolute, the path begins with /.
    • Relative to the directory of the YAML file referencing it. In the above example, both files are in the same directory.
  • file_option can be:

    • empty: {{file "my_file"}}
    • binary: {{file "my_file" "binary"}}
    • pem: {{file "my_file" "pem"}}

Externalized files default naming scheme

During conversion from FED, externalized file names are generated as follows:

Entity EntityType Field file_option Externalized file default name scheme Possible extensions Additional rules Environmentalization possible inside file content
Script language / Scripts (in Resources) JavaScriptFilter, Script script N/A <Parent entity>-Files/<entity field "name"> .groovy, .js, .nashorn.js, .py Yes
Set Message (Filter) ChangeMessageFilter body N/A <Parent entity>-Files/<entity field "name"> .json, .html, .xml, .txt Extension depends on content type content. Yes
HTTP Redirect (Filter) RedirectFilter content N/A <Parent entity>-Files/<entity field "name"> .json, .html, .xml, .txt Extension depends on content type content. Yes
Retrieve from or Write to Database (Filter) Query sqlStatement N/A <Parent entity>-Files/<entity field "name"> .sql Yes
JSON Add Node (Filter) JSONAddNodeFilter content N/A <Parent entity>-Files/<entity field "name"> .json Yes
Add XML Node (Filter) InsertXMLNode nodeContent N/A <Parent entity>-Files/<entity field "name"> .xml Yes
JSON Schema JSONSchema contents N/A <entity field "name"> .json Yes
Certificate Certificate content pem if file contains PEM headers and footers. N/A, otherwise <entity field "dname">-cert .pem How to add a new certificate No
key pem if file contains PEM headers and footers. N/A, otherwise <entity field "dname">-key .pem How to add a private key No
XML Schema Document Bundles / WSDL Document Bundles ResourceBlob content binary <entity fields "ID"> (generated by Policy Studio but can be manually set) .xsd, .dtd, .wsdl Extension depends on content on type No
Server Settings → Security → Kerberos KerberosConfiguration configFile N/A <entity fields "name"> .krb5.conf No
Alert type → OPSEC  OpsecAlertSystem connectionInfo N/A <entity fields "name"> .conf No
Alert type → OPSEC → Files to upload UpLoadFile contents binary upload-files/<entity fields "name"> n/a No
Security Loadable Module OESSecurityRuntimeLoadableModule nameAuthorityDefinition N/A <entity fields "name"> .xml No
PGP Key Pair PGPKeyPair publicKey binary <entity field "alias">-pub .crt No
privateKey binary <entity field "alias">-sec .asc No
Key Pair KeyPair publicKey binary <entity field "alias"> .pub No
privateKey binary <entity field "alias"> .pem No
Authentication Repository Group → Authentication Repository Database → Query Query sqlStatement N/A <Grand Parent entity field "name">/<Parent entity field "name"> - <entity field "name"> .sql No
Tivoli Connection TivoliSettings configFile N/A <entity field "name"> .conf Tivoli configuration File No
configDatabaseFile binary <entity field "name"> .db.conf Tivoli Configuration Database File. No
sslStashFile binary <entity field "name"> .ssl.key SSL Stash file No
sslKeyFile binary <entity field "name"> .ssl.stash SSL Key file No
  • The entities are named as in Policy Studio.