Connector Configuration

Sitecore Configuration

Mandatory Settings (Basic)

Page Content Retrieval Settings

Configuration options for the retrieval of the content for web pages.

For Sitecore web pages, the HTML source code is downloaded and processed. Depending on the Sitecore template of the web page, a Jquery selector can be defined to select the relevant parts of the HTML source as the cotent of the web page. If a web page has a Sitecore template for which no selector is configured here, the default selector is applied. By default, this default selector is html, so that the whole HTML source code is used as the content of the web page.

The configuration consists of a list of template selectors, which are pairs of Sitecore template name and selector. The selector in a pair will be used to parse web pages that use the Sitecore template. The syntax to define Jquery selectors is explained in Jsoup.

Name Key Description

Default Selector

raytion.connector.agent.sitecore.
contentretrieval.defaultSelector

Selector used for web pages with a template name not configured in Template Selectors. The default value is html, meaning the whole HTML source will be used as content.

Sitecore template name

raytion.connector.agent.sitecore.
contentretrieval.templateSelectorEntries.templateName

Sitecore items with this template will be parsed with the corresponding selector.

Sitecore template selector

raytion.connector.agent.sitecore.
contentretrieval.templateSelectorEntries.selector

Sitecore items with the corresponding template will be parsed with this selector.

Sitecore Connection

Configuration options to connect to the Sitecore Host.

Name Key Description

Sitecore Host URL

raytion.connector.agent.sitecore.
connection.host

Full URL of the Sitecore Host. HTTP scheme, server name and, if needed, the port number of the server are required.

API User Domain

raytion.connector.agent.sitecore.
connection.apiUserDomain

Domain of the API User.

API User Name

raytion.connector.agent.sitecore.
connection.apiUserName

Name of the API User.

API User Password

raytion.connector.agent.sitecore.
connection.apiUserPassword

Password of the API User.

API Authentication Cookie Name

raytion.connector.agent.sitecore.
connection.apiAuthenticationCookieName

Name of the cookie set after successfull authentication to the Sitecore REST API. This name is usually ".ASPXAUTH".

Authentication Cookie Expiration Buffer

raytion.connector.agent.sitecore.
connection.
authenticationCookieExpirationBufferSeconds

Time in seconds before the authentication cookie will expire.

Sitecore Content Settings

Configuration options for Sitecore content.

Name Key Description

Sitecore Content Base Path

raytion.connector.agent.sitecore.
sitecore.contentBasePath

Base path of the content tree in Sitecore Content Editor.

Sitecore Media Base Path

raytion.connector.agent.sitecore.
sitecore.mediaBasePath

Base path of the media tree in Sitecore Content Editor.

Sitecore Content Languages

raytion.connector.agent.sitecore.
sitecore.languages.description

List of languages used in Sitecore whose items should be processed by the connector.

URL Manipulation Settings

Configuration options to manipulate Sitecore item URLs.

Configures a list of individual manipulation rules for URL modifications. Each rule cosists of a pattern specifying a regular expression for matching a URL and a value specifying a corresponding regular expression for replacing the URL.

Name Key Description

Regular Expression

raytion.connector.agent.sitecore.
url-manipulation.
manipulators.regex

Regular expression to match the Click and Download URL against.

Regex Replacement

raytion.connector.agent.sitecore.
url-manipulation.
manipulators.replace

Replacement for the regular expression matched.

Azure Connection Settings

Configuration options to connect to Azure services. Supported authentication types are "Application Secret" and "Client Certificate" (recommended).

Name Key Description

Tenant ID

raytion.connector.agent.sitecore.
azure.tenantId.description

Tenant ID of the Azure AD Host.

Azure Application ID

raytion.connector.agent.sitecore.
azure.applicationId

Id. of the registered Azure application.

Authentication Type

raytion.connector.agent.sitecore.
azure.authenticationType

Preferred authentication type, either "Application Secret" or "Client Certificate".

Application Secret

raytion.connector.agent.sitecore.
azure.applicationSecretProperties.
applicationSecret

For authentication type "Application Secret": The application secret of the registered Azure application.

Certificate Password

raytion.connector.agent.sitecore.
azure.applicationCertificateProperties.
certificatePassword

For authentication type "Client certificate": Password required to access the client certificate.

Azure Application Client Certificate

raytion.connector.agent.sitecore.
azure.applicationCertificateProperties.
appCertificate

For auhentication type "Client Certificate": The private key file of the certificate used to authenticate the Azure application.

Sitecore Item Filters

Include or exclude sitecore items based on their properties.

Name Key Description

All Include Filters Must Apply

raytion.connector.agent.sitecore.
filter.allWhitelistsApply

If checked, an item must match all include filter properties, if unchecked, the item must match at least one of the include filters to be included in the crawl results. If an item matches any of the exclude criteria it will not be included in the crawl results.

Item Path Include List

raytion.connector.agent.sitecore.
filter.itemPathIncludeList

Only items with paths matching these filters will be included in the crawl results.

Item Path Exclude List

raytion.connector.agent.sitecore.
filter.itemPathExcludeList

Items with paths matching these filters will not be included in the crawl results.

Metadata Field Name

raytion.connector.agent.sitecore.
filter.metadataFilterProperties.metadataName

Name of the metadata field the filter is applied to.

Metadata Value Include List

raytion.connector.agent.sitecore.
filter.metadataFilterProperties.includeList

Only items with metadata values matching these filters will be included in the crawl results.

Metadata Value Exclude List

raytion.connector.agent.sitecore.
filter.metadataFilterProperties.excludeList

Items with metadata values matching these filters will not be included in the crawl results.

Proxy Settings

Configures a proxy that is applied to all agent connections.

Name Key Description

Use Proxy

raytion.connector.agent.sitecore.
proxy.useProxy

If enabled, a proxy will be used for all connections used by the Agent.

Proxy URI

raytion.connector.agent.sitecore.
proxy.uri

The full URI of the proxy service including protocol, host, and port.

Use Authentication

raytion.connector.agent.sitecore.
proxy.authentication

If enabled, the proxy access uses authentication.

Username

raytion.connector.agent.sitecore.
proxy.username

The username used to authenticate the proxy service access.

Password

raytion.connector.agent.sitecore.
proxy.password

The password used to authenticate the proxy service access.

Optional Settings (Advanced)

Azure Client Settings

Configuration options for the Azure client.

Name Key Description

Response Page Size

raytion.connector.agent.sitecore.
azureclient.responsePageSize

Maximum number of results per page for Azure requests.

MemberOf Refetch Limit

raytion.connector.agent.sitecore.
azureclient.memberOfRefetchLimit

If the prefetched memberOf entries reaches this number, a full memberOf request will be done for the user or group.

Requests per second

raytion.connector.agent.sitecore.
azureclient.desiredRps

Maximum amount of requests which are started towards the Microsoft APIs.

Rate Limit Requests Per Second

raytion.connector.agent.sitecore.
azureclient.rateLimitRps

Maximum amount of requests per second which are started towards the Microsoft APIs while the API rate is near its limit.

User Agent HTTP Header

raytion.connector.agent.sitecore.
azureclient.userAgent

HTTP header which is sent with every request. It identifies the agent responsible for the request.

CSM Configuration

Required Configuration Properties

CSM Connection Settings

Configuration options for the connection to the target CSM instance.

Name Description

CSM endpoint

URL of the CSM instance to connect to.

CSM Authentication Settings

Configuration Options for the authentication against the target CSM instance.

Name Description

Username

Username of the technical user.

Password

Password of the technical user.

Optional Configuration Properties

CSM Connection Settings

Configuration options for fine-tuning the Http connection parameters.

Name Description

Concurrent Connections

Maximum number of concurrent open connections.

Requests Rate

Maximum number of requests per second.

Connect Timeout in Milliseconds

Timeout of the connect request.

Socket Timeout in Milliseconds

Timeout of the socket connected to CSM.

Request Timeout in Milliseconds

Timeout of a request to CSM.

CSM Ingestion Settings

Configuration options to specify how principals are ingested in the CSM.

Name Description

Domain

Namespace under which to ingest principals.

Apache Solr Configuration

Instance Configuration

Configuration options related to specifying the target Solr Instance and Collection including authentication/authorization settings.

Setting Description

Deployment

Deployment of the target Solr instance. Use Collection Server to feed against a standalone collection server. If your target instance is a Cloud Deployment, select the option Zookeeper.

Collection Server URL

For Standalone Collection Server, the URL to the collection server including the data path, e.g. http://localhost:8983/solr.

Zookeeper Instances

For Solr Cloud target, list of zookeeper instances including host and port.

Zookeeper Chroot

Znode Chroot of the cluster.

Collection ID

ID of the target collection.

Use Authentication

Enable this option to use Basic Authentication to authenticate against your Solr instance.

Username

Basic Authentication Username, if authentication is enabled.

Password

Basic Authentication Password, if authentication is enabled.

Use Proxy

If enabled, the connection to the Solr instance will be established through HTTP/HTTPS proxy.

Proxy Endpoint

Target proxy URL including protocol, host and port.

Proxy Authentication

If enabled, the connector uses the specified credentials to authenticate towards proxy.

Proxy Username

Proxy authentication username.

Proxy Password

Proxy authentication password. The value will be stored encrypted by the connector.

ACL Settings

Configuration options influencing the document ACL creation.

Setting Description

Domain Prefix

Prefix applied to access control entries of document ACLs. In case a domain considering CSM (Query type: 'sharepoint') is used with the connector, this prefix needs to match the CSM domain concatenated with the CSM domain separator (':'). Otherwise, leave the prefix blank.

Language Settings

Settings that define which metadata fields require a language specific processing. These fields will be suffixed with the language code, belonging to the document (example: 'content_en'). The language specific processing can be defined in the Apache Solr schema for fields with a dedicated language suffix.

Setting Description

Standard Fields

Select any arbitrary fields to append the language suffix. The list consists of standardized fields provided by the connector. The fields are: Content, Source, Title, Item Type, Keywords, Author, and Contributors.

Include Additional Fields

Enable this option to include also additional fields not listed in the standard field list, e.g. fields generated by the pipeline.

Additional Fields

List of field names to apply the suffix to.

Fallback Language

Fallback language code in ISO 639 format to apply for items with missing language information.
Adjustments to these configuration options, may require appropriate changes to the Apache Solr schema and a full content synchronization to index the altered metadata fields.

Advanced ACL Settings (Optional)

Advanced options for handling document ACL.

Setting Description

Everyone ACE

Identifier of the access control entry which marks a document as public.

Nobody ACE

Identifier of the access control entry which indicate that a document is not accessible by anyone.

Metadata Field Settings (Optional)

Define the names of Apache Solr specific fields.

Setting Description

Allow Document ACL Field

Metadata field name for the allow access control list.

Deny Document ACL Field

Metadata field name for the deny access control list.

Content Field

Metadata field name for the document’s content.

Advanced HTTP Settings (Optional)

Configuration options for fine-tuning the Http connection parameters.

Setting Description

Socket Timeout

Timeout value for receiving data from server.

Connection Timeout

Timeout value for establishing a connection to server.

Connection Request Timeout

Timeout value for requesting a connection from connection manager.

Max. Number of Connections

Max. number of connections maintained by the connection manager.

Max. Number Requests per Second

Max. number of requests send to the server per second.

Max. Number of Retries

The maximum amount of times failed request shall be retried. Infinite retries are not supported. The delay between retries follows the pattern: [1s, 5s, 30s, 30s, …​].

General Configuration

Database Configuration

Name Property Key Description

Configuration Type

raytion.connector.db.config.type

Supported are PostgreSQL, MS SQL Server, and JDBC URL configuration.

PostgreSQL

Name Property Key Description

Host

raytion.connector.db.config.postgres.host

Domain name or IP address of the database server.

Port

raytion.connector.db.config.postgres.port

Specifies the port number PostgreSQL is listening on, default is 5432.

Database Name

raytion.connector.db.config.postgres.name

Name of the database.

Username

raytion.connector.db.config.postgres.username

Username to authenticate with. The regarding user has to have read and write permissions to the database.

Password

raytion.connector.db.config.postgres.password

Password of the configured database user.

Add Custom Parameter

raytion.connector.db.config.postgres.addParameters

Enables the configuration of additional parameters.

MS SQL Server

Name Property Key Description

Host

raytion.connector.db.config.mssql.host

Domain name or IP address of the database server. Instance to connect to on server can be specified by '‹server_name>|<instance_name>'.

Port

raytion.connector.db.config.mssql.port

Specifies the port number MS SQL Server is listening on, default is 1433.

Database Name

raytion.connector.db.config.mssql.name

Name of the database.

Username

raytion.connector.db.config.mssql.username

Username to authenticate with. The regarding user has to have read and write permissions to the database.

Password

raytion.connector.db.config.mssql.password

Password of the configured database user.

Add Custom Parameter

raytion.connector.db.config.mssql.addParameters

Enables the configuration of additional parameters.

JDBC URL

Name Property Key Description

URL

raytion.connector.db.config.jdbc.url

JDBC URL for the target database. Out of the box, the connector will use H2 file database. For productive usage, use PostgreSQL specifying the URL in format: jdbc:postgresql:<host>:<port>/<database>

Username

raytion.connector.db.config.jdbc.username

Database Username to read and write to database.

Password

raytion.connector.db.config.jdbc.password

Database Password for the specified user

Traversal Configuration

Name Property Key Description

Traversal History Length

raytion.connector.agent.traversal
.store.historyLength

Max. number of traversals to store in the history. Once the limit is exceeded, the connector will automatically remove oldest entries in the history. (default: 100)

Include Checksum

raytion.connector.agent.traversal
.pipeline.includePipelineChecksum

If enabled, any changes made to the pipeline e.g. configuration, the subsequent incremental run triggers a refeed of all items.

Change Processing Interval

raytion.connector.agent.traversal
.changeprocessing.interval

Interval between change processing traversals.

Resume on Start

raytion.connector.agent.traversal
.resume.resumeTraversalOnRestart

If enabled, any traversals in paused state are automatically resumed after the connector restart. Otherwise, the traversal remains in paused state.

Number of Traversal Workers

raytion.connector.agent.traversal
.workers.worker

Number of workers to execute the traversal in parallel. Increasing this value might improve the performance, but will footprint higher memory consumption. It is recommended to keep the default value. (default: 10)

Traversal Job Poll Interval

raytion.connector.agent.traversal
.workers.jobPollInterval

Interval between the workers to be triggered to fetch and process the next tasks. (default: 10ms)

Completion Timeout

raytion.connector.agent.traversal
.workers.completionTimeout

If the search engine indexes the items asynchronously, there might be some processing still in-flight during the completion process of a traversal. This value specifies the timeout value until all asynchronous callbacks are expected to return before completing the traversal. (default: 10m)

Executor Size

raytion.connector.agent.traversal
.execution.executorSize

The executor size restricts the max. number of concurrent running traversals.

Queue Size

raytion.connector.agent.traversal
.execution.queueSize

The queue size restricts the max. number of queued traversals. If the value is exceeded, the connector rejects further traversal requests until the queue size is below the configured size.

Traversal Jobs

Name Property Key Description

Job Timeout Check Frequency

raytion.connector.job-broker.heartbeatPeriod

Configures how often the connector checks for timed out jobs.

Job Timeout

raytion.connector.job-broker.heartbeatTimeout

The duration for which a job can stay idle before it is timed out.

Job Cache Size

raytion.connector.job-broker.jobPollCacheSize

Max. cache size of Jobs waiting for processing in memory. When cache is empty, next batch is fetched.

Security Configuration

Request Restriction Settings

Name Property Key Description

Accepted Host Domains

raytion.connector.security
.requests.acceptedDomains

A list of domains (+ port) that are allowed as host names in the headers of HTTP requests to the connector. This means that you can access the connector only via a URL that employs one of the configured domains. Each entry must have the format domain:port. Examples:

  • localhost:16370

  • connector.company.com:16370

If no domains are configured (the default), then you can use any domain via which the connector host is reachable.

Principal Aliaser Configuration

Principal Aliasing is applied on user information as part of Content ACL processing during Content Synchronization and Principal processing during Principal Synchronization. It’s purpose is to map external source system user to the corresponding user in search engines domain. You can configure a list of aliasers in the connector which will be applied in sequence and in order on user ACEs and user principals. The Connector supports following custom aliasing mechanism.

Custom Aliaser Disabled

If the Custom Aliaser checkbox is not selected, the connector will process user information on ACE and user principals unchanged to Search Engine. If all relevant users in the source system can be found with the same identifier in the search engine, this setup is sufficient to reflect the same secure search experience in the search engine as defined by the policy in the source system. The connector uses this option as default to process user information.

Custom Aliaser Enabled

If custom aliasing is enable then there are four types of aliaser avaialble:

Simple XML Table Aliaser

Static mapping table which can be uploaded as XML file. The connector uses the uploaded file as lookup table to map a user in the source system to a user in the search engine. Users missing a record in the file will be dropped from the ACE and during Principal Synchronization. This option is only recommended for environment with a manageable amount of users as for each user the corresponding mapping entry needs to be specified in the file.

Name Description

XML Mapping File

Browse and upload or drag and drop.

Sample XML mapping file:

<?xml version="1.0" encoding="UTF-8"?>
<storeddata>
    <entry keyValue="user1">user1@raytion.com</entry>
    <entry keyValue="user2">user2@raytion.com</entry>
    <entry keyValue="user3">user3@raytion.com</entry>
</storeddata>
Regex Replacer Aliaser

Regex Replacer Aliaser computes aliases based on a regular expression. Principals that match the regular expression are replaced by the Substitution String.

Name Property Key Description

Pattern

raytion.connector.aliaser.aliasers[*]
.replacer.pattern

The regular expression to match, this is the part that will be replaced. If braces (…​) are used in the pattern then the matched value can be retrieved using $1

Substitute String

raytion.connector.aliaser.aliasers[*]
.replacer.substituteString

String to replace the matching part of the find string. Matched value is accessed by employing $1
Regex Extractor Aliaser

Regex Extractor Aliaser computes aliases based on a regular expression. Principals that match the regular expression are inserted into the Insert-Into String.

Name PropertyKey Description

Pattern

raytion.connector.aliaser.aliasers[*]
.extractor.pattern

The regular expression to match, this is the part that will be inserted into the new value. If braces (…​) are used in the pattern then the matched value can be retrieved using $$

Insert-Into String

raytion.connector.aliaser.aliasers[*]
.extractor.insertIntoString

String to replace the matching part of the pattern. Matched value is accessed by employing $$
LDAP Aliaser

Ldap Aliaser searches for an LDAP entry with the requested name in the input value and returns the specified output attribute.

Name Property Key Description

Host

raytion.connector.aliaser.aliasers[*]
.ldap.host

Fully Qualified Domain Name of an LDAP server

Port

raytion.connector.aliaser.aliasers[*]
.ldap.port

Port to use for LDAP connection, defaults are 389/636 or (recommended) 3268/3269 for simple/SSL

AccountDN

raytion.connector.aliaser.aliasers[*]
.ldap.bindAccountDN

AccountDN for bind to LDAP

Password

raytion.connector.aliaser.aliasers[*]
.ldap.password

Password part of credentials

Input Field

raytion.connector.aliaser.aliasers[*]
.ldap.inputField

The Active Directory attribute name for this equality filter

Search Root DN

raytion.connector.aliaser.aliasers[*]
.ldap.baseDN

Distinguished Name of the subtree which is searched. The smaller the subtree the better the performance but the higher the chance of encountering principals which are not part of this subtree

Output Field

raytion.connector.aliaser.aliasers[*]
.ldap.outputField

Attribute that should be returned in result entries