Job Priority
Job priority defines the order in which jobs are processed by the Strategy Intelligence Server. Jobs typically execute on a first-come, first-served basis, however situations arise whereby Strategy Intelligence Server will process certain Jobs before others.
Connection Management
Connection Management is the process of setting the optimal number of default, medium, and high priority warehouse connections to support the Job executions on a given environment. This is based on user group, Job cost and project profiles. The overall goal of Job prioritization and connection management is to prioritize Jobs such that they can be run in a specified order.
Defining Priorities and Connections
Defining Priorities and Connections apply to all projects which use the selected database instance. A user can define Job priorities and number of connections on the Job Prioritization tab in the Database Instance editor, as displayed in the screen shot below:
Administrators can, based on priority, specify the number of warehouse connections that are required for efficient Job processing. There are three possible priorities for a Job: high, medium and low.
Administrators are not required to set medium and high connections, but must set at least one low connection, because low priority is the default Job priority.
The optimal number of connections is dependent on several factors, however the main criterion to consider when setting the number of connections is the number of concurrent queries the Warehouse Database can support.
Strategy Intelligence Server processes a Job on a connection that corresponds to the Job’s priority. If no priority is specified for a Job, the Strategy Intelligence Server processes the job on a low priority connection. For example, Jobs with a high priority are processed by high priority connections; Jobs with a low or no priority are processed by a low priority connection.
In the case of multiple job priority entries matching the executing Job, the Strategy Intelligence Server will use the Job priority specified for the priority map entry that has the maximum number of matching factors as specified in the following Knowledge Base document:
Strategy Intelligence Server also engages in connection borrowing when processing Jobs. Connection borrowing occurs when Strategy Intelligence Server executes a Job on a lower priority connection because no connections with an equivalent priority is available for the Job at execution time. For example, high priority Jobs can run on both medium and low priority connections as well as the default high priority connections. In this way, users and administrators can be assured that the highest priority Jobs have access to all connection resources. Also, connection borrowing reduces the waste of resources caused by idle connections.
For example, if a Job with a medium priority is submitted and all medium priority connections are busy processing Jobs, Strategy Intelligence Server will process the Job on a low priority connection. When a Job is submitted and no connections are available to process it, either with the pre-selected priority or with a lower priority, Strategy Intelligence Server places the Job in queue and processes it when a connection becomes available.
Note that connection borrowing is not a guarantee that higher priority Jobs will always execute before lower priority Jobs. For example, suppose there is a high priority Job and a low priority Job in the pending queue, and a low priority connection becomes available. There is no guarantee that either the high or low priority Job will execute on the low priority thread. The Job that gets chosen to be executed on the low priority thread will depend on the order in which the Jobs were placed in the internal pending queue.
User should note that even though Job priority is set at the database instance level, in a clustered environment, it applies per Intelligence Server machine. For example, if Administrators configure to have 2 low, 2 medium and 2 high priority threads at the database instance level, it is expected to see a total of up to 6 connections on each Intelligence Server.
