Article last updated on the 16th of May, 2024.
Contents
1. Introduction
2. Requirements
3.1. Web Job Priorities
3.1.1. Integration
3.1.2. Site Provisioning
3.1.3. Management
3.1.4. Priority Based Queue (Low, Medium, High and Critical Jobs)
3.1.4.1. Critical
3.1.4.2. High
3.1.4.3. Medium
3.1.4.4. Low
3.2. Changing default queue for webjobs
3.3. Web Jobs in queue
1. Introduction
The WorkPoint Jobs Engine is a part of the WorkPoint platform that manages the execution of various tasks and operations related to WorkPoint sites and solutions. The Jobs Engine is in charge of tasks such as site synchronization, inheritance, aggregations, site collection sync, security replication, and system processes. The Jobs Engine operates on a queue-based system to organize jobs and their priorities and offers a user interface for job monitoring and management.
The WorkPoint 365 system has a vast number of different job types which makes the system function. Each job type is assigned to a specific job queue by default (jobs can be assigned to a different job queue if necessary). A couple of examples of web jobs could be the "Master site synchronization" job, the "Aggregation" job, and the "Site collection provisioning" job.
When any job is triggered, it will enter the job queue associated with that job type. From there, it will await execution by the Jobs engine.
WorkPoint 365 operates with four different web job queues:
- Integration: This queue is normally used for integration through the WorkPoint API.
- Site Provisioning: This queue is normally used for site provisioning through the WorkPoint Integration API or through Dynamic Site Creator when it runs in parallel mode.
- Management: This queue is normally dedicated to management jobs that are not assigned a priority.
- Priority Based Queue: This specific Job Queue prioritizes tasks based on their assigned priority weight and the time they have spent waiting in the queue.
Priority based queue
The Jobs Engine uses a single queue to store and process all web jobs assigned a priority, regardless of their type or source. The Jobs Engine also manages the different priorities of the web jobs and ensures that higher priority web jobs are executed before lower priority ones. To do this, the Jobs Engine uses a scoring system that calculates a score for each web job based on its priority and the time it has been in the queue. The priority of a web job is defined by a weight assigned to each priority level, such as high, medium, or low. The time in queue is measured by the difference between the current time and the time the web job was queued. The score of a web job is computed by multiplying the weight and the time in queue, and then normalizing the result by a factor. The Jobs Engine then sorts the web jobs by their scores in descending order and executes the web job with the highest score first.
The following diagram illustrates the functionality of the priority based job queue:
In the priority based queue there are three jobs; an Aggregation job, a Site creation job, and a Site collection provisioning job.
Each job has a score calculation based on the weight their job is associated with multiplied by the time (in minutes) they have been in the queue.
Based on the job's scores, they are executed in the following order: Site creation -> Aggregation -> Site collection provisioning -> Aggregation.
Moving job types between queues
Using the Job Queue Manager, it's possible to move each job types between queues. This means that you could decide to move a job which by default goes into the "Management" queue, to the "Site provisioning" queue.
It's also possible to move job types out of their default queues, and instead assign them to a priority weight, e.g., Low, Medium, and High. The specific job type will then follow the functionality of the Priority based queue as described above.
The default queue settings, however, work well in most scenarios, and you should only make changes if necessary.
2. Requirements
There are no requirements associated with the use of the Job Queue Manager.
3. Configuration
Note that the option to set a max number for concurrent jobs of the Integration- and the Site Provisioning job types are only available in WorkPoint 365 version 4.
To access the Webjob Queue Manager Settings page, follow these steps:
- In the WorkPoint 365 administration, expand the "Job Queue Manager" section in the left side panel and select "Webjob Queue Manager Settings".
The Webjob Queue Manager Settings allows administrators to define how many concurrent jobs of specific types can run. For some types of jobs, it is possible to finely tune how many concurrent jobs can run during specific time intervals (daytime, nighttime, and weekends).
The following page shows an example of the Job Queue Manager settings page:
Note that the numbers in the settings fields in the image above are the default values. If you need to reset to default values, please refer to the values in the image.
The Job Queue Manager settings page is split into two sections; the Web Job Priorities section and the Web Jobs in queue section.
3.1. Web Job Priorities
In the Web Job Priorities section, you can define how many concurrent jobs of specific types can run.
Note that you can click "Click here to view the jobs in each priority" text to expand the page and view which jobs are in which categories.
Note that all mentions of time intervals are in relation to the WorkPoint solution root site time zone.
3.1.1. Integration
This queue is used for integration through the WorkPoint API. To reduce the effects of throttling during integration / migration, you can define the number of concurrent running jobs that can be run during daytime hours and evening hours. The time settings are currently based on the time zone of the WorkPoint root site collection.
For this type, you can specify how many concurrent jobs can run during daytime (06:00 - 18:00), nighttime (18:00 - 06:00), and weekends (Saturday at 00:00 until Monday at 00:00).
The number you put into each field defines how many integration jobs can run concurrently during the specific time intervals.
3.1.2. Site Provisioning
This queue is used for site provisioning through the WorkPoint Integration API or through Dynamic Site Creator when it runs in parallel mode.
For this type, you can specify how many concurrent jobs can run during daytime (06:00 - 18:00), nighttime (18:00 - 06:00), and weekends (Saturday at 00:00 until Monday at 00:00).
The number you put into each field defines how many Site Provisioning jobs can run concurrently during the specific time intervals.
3.1.3. Management
This queue is dedicated to management jobs that are not assigned a priority.
For this type, you can specify how many concurrent jobs can run during daytime (06:00 - 18:00), nighttime (18:00 - 06:00), and weekends (Saturday at 00:00 until Monday at 00:00).
The number you put into the number field defines how many Management jobs can run concurrently during the specific time intervals.
3.1.4. Priority Based Queue (Low, Medium, High and Critical Jobs)
This queue is dedicated to jobs that are not assigned a specific queue. All jobs which are not assigned to a specific queue (Integration, Site Provisioning, or Management) will enter the Priority Based Queue, and use the weight associated with the priority category they are assigned to.
For this type, you can specify how many concurrent jobs can run during daytime (06:00 - 18:00), nighttime (18:00 - 06:00), and weekends (Saturday at 00:00 until Monday at 00:00).
The number you put into the number field defines how many jobs from the Priority Based Queue can run concurrently during the specific time intervals.
3.1.4.1. Critical
This priority can by used when you need a critical job to run immediately. No jobs get this priority assigned by default. The Critical priority category has a default weight of 100.
3.1.4.2. High
This Priority is used for the most business critical jobs and is the highest priority that is assigned to jobs by default. The High priority category has a default weight of 50.
3.1.4.3. Medium
The jobs in this category can be either short or long running. The end users typical do not have to wait for these jobs to finish. The Medium priority category has a default weight of 20.
3.1.4.4. Low
This Priority is primarily used for long running jobs. This can be either maintenance or jobs requested by an administrator. The Low priority category has a default weight of 5.
Note that you can click "Click here to view the jobs in each priority" text to expand the page and view which jobs are in which categories.
3.2. Changing default queue for webjobs
In this section, we will cover how to change the default queue for webjobs.
We will demonstrate this by changing the "Notification Cleanup Job" type from the Management queue to the "High" priority weight category:
- In the Webjob Queue Manager Settings page, we click the "View the jobs in each priority" link.
This expands the page with a section where all job types are displayed in their associated queues and priority categories:
- Next, we find and click the "Notification Cleanup Job" type to edit it.
This opens the "Change Priority" page for the Notification Cleanup Job type:
- In the "Priority/Queue Nane" setting, we can open the dropdown menu and select the queue or priority we want to assign to the Notification Cleanup Job type. In this instance, we select the "High" priority.
- To save the priority, we click the "Save" button.
We are then returned to the Webjob Queue Manager Settings page, where we can now inspect the changes:
As we can see, the Notification Cleanup Job type has successfully been moved to the "High" priority category, meaning such a job will enter the Priority Based Queue and be executed based on the weight of the "High" category in the settings.
3.3. Web Jobs in queue
To view the web jobs currently in queue, you can open the "Webjob Queue" page:
- To access the Webjob Queue page, either expand the "Job Queue Manager" section in the left side panel of the WorkPoint 365 administration and select "Webjob Queue", or click the "Webjob Queue" link on the "Webjob Queue Manager Settings" page.
The Web Jobs in queue section contains a list of all current web jobs being processed or jobs which have recently completed or failed.
The following image shows an example of the list containing a couple of jobs:
Note that jobs which are in the "Pending" status can have their priorities changed. This effectively moves them from their predefined prioritization category into the selected on. A pending job in the "Management" queue and e.g., be changed to the Priority based queue by putting it in the "Critical" weight category, and therefore have it's priority changed.
An example is shown in the following image:
In the image above, we have selected a Full Security Replication job and opened the "For selected jobs" menu. Here we can select a new category for the selected jobs in the queue.
To move the selected jobs to the selected category, we select it and click the "Change priority" button.
To cancel one or more selected jobs, click the "Cancel" button(s).
4. Notes
4.1. Job Queue Manager Settings across solutions
The settings in the Job Queue Manager Settings page are common to all WorkPoint solution within the same tenant. This means that if your organization has multiple WorkPoint solutions within the same Microsoft tenant, and you alter the Job Queue Manager settings in one, those changes will also apply to the other solutions on the same tenant.
4.2. Ensuring user triggers jobs run
Jobs which are typically started by users, such as Site Creation- or Aggregation jobs are put into the Priority based job queue. If you want to ensure that these jobs are always run when needed, one way to ensure this is to put low numbers for the allowed concurrent jobs in the other queues.
4.3. Consequences of poorly configured settings
Please note that if you make changes tot he Job Queue Manager settings without consideration of the possible consequences, the end result may be that some job won't run due to excessive queuing.
Comments
0 comments
Please sign in to leave a comment.